From: Ada Couprie Diaz <ada.coupriediaz@arm.com> driver inclusion category: bugfix bugzilla: https://atomgit.com/openeuler/kernel/issues/10041 ---------------------------------------------------------------------- The arm64 hibernate code manages the exception masking in an unsound way, leading to potential crashes and/or warnings during resume. When a hibernation image is saved in `swsusp_arch_suspend()`, all DAIF exceptions are masked (by virtue of `local_daif_save()`), and the suspended image is saved assuming that all DAIF exceptions will remain masked when the image is restored. When a hibernation image is resumed by `swsusp_arch_resume()`, only interrupts are masked (by virtue of `local_irq_disable()` in `resume_target_kernel()`). When pseudo-NMI is enabled the DAIF.IF bits will be clear, and regardless of pseudo-NMI the DAIF.DA bits will be clear. This means that there are two problems: (1) It is possible to take Debug, SError, or pseudo-NMI exceptions during the resume process. This is unsafe, as during the resume process both the old ane new kernels will tranisently be in an inconsistent state, and swsusp_arch_suspend_exit() won't retain an executable mapping of any exception vectors. Any exception taken here will be fatal and silent. (2) When re-entering the resumed kernel, some DAIF bits will be clear unexpectedly. This permits Debug, SError, or pseudo-NMI exceptions to be taken for a short period while the resumed kernel is not yet in a consistent state. This is detected by CONFIG_ARM64_DEBUG_PRIORITY_MASKING. Avoid these issues by masking all DAIF exceptions during resume. Fixes: 82869ac57b5d ("arm64: kernel: Add support for hibernate/suspend-to-disk") Signed-off-by: Ada Couprie Diaz <ada.coupriediaz@arm.com> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com> Signed-off-by: Qinxin Xia <xiaqinxin@huawei.com> Signed-off-by: Hongye Lin <linhongye@h-partners.com> --- arch/arm64/kernel/hibernate.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index 17c436387f1b..165f61b07da1 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -459,9 +459,21 @@ int __nocfi swsusp_arch_resume(void) if (el2_reset_needed()) __hyp_set_vectors(el2_vectors); + /* + * It is necessary to mask all DAIF exceptions here as: + * + * - The copy of swsusp_arch_suspend_exit() in the hibernation + * text cannot handle taking any exceptions. + * + * - The suspended kernel masked all DAIF exceptions in + * swsusp_arch_resume(), and expects to be re-entered in the + * same state : with all DAIF exceptions masked. + */ + local_daif_save(); hibernate_exit(virt_to_phys(tmp_pg_dir), resume_hdr.ttbr1_el1, resume_hdr.reenter_kernel, restore_pblist, resume_hdr.__hyp_stub_vectors, virt_to_phys(zero_page)); + unreachable(); return 0; } -- 2.25.1

From: Ada Couprie Diaz <ada.coupriediaz@arm.com> driver inclusion category: bugfix bugzilla: https://atomgit.com/openeuler/kernel/issues/10041 ---------------------------------------------------------------------- As we clear SCTLR_EL1.SPINTMASK when enabling FEAT_NMI, ALLINT masks IRQs and FIQs regardless of superpriority. Update irqflags.h and `regs_irqs_disabled()` to take it into account, so we properly keep track of the local IRQ masking state. We have documentation at the top of irqflags.h which explains the DAIF masking. Since the additional masking with NMIs is related and also covers the IF in DAIF, extend the comment to note what's going on with NMIs. Fixes: a5f61cc636f4 ("arm64: irqflags: use alternative branches for pseudo-NMI logic") Co-developed-by: Mark Brown <broonie@kernel.org> Signed-off-by: Mark Brown <broonie@kernel.org> Signed-off-by: Ada Couprie Diaz <ada.coupriediaz@arm.com> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com> Signed-off-by: Qinxin Xia <xiaqinxin@huawei.com> Signed-off-by: Hongye Lin <linhongye@h-partners.com> --- arch/arm64/include/asm/irqflags.h | 21 +++++++++++++++++---- arch/arm64/include/asm/ptrace.h | 6 ++++-- arch/arm64/include/uapi/asm/ptrace.h | 1 + 3 files changed, 22 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h index a9f117fb43c6..0477ecca2d8a 100644 --- a/arch/arm64/include/asm/irqflags.h +++ b/arch/arm64/include/asm/irqflags.h @@ -27,8 +27,8 @@ * masking both superpriority interrupts and IRQ/FIQ regardless of the * I and F settings. Since these superpriority interrupts are being * used as NMIs we do not include them in the interrupt masking here, - * anything that requires that NMIs be masked needs to explicitly do - * so. + * anything that requires that NMIs be masked needs to explicitly do so, + * but we do check for ALLINT masking IRQs/FIQs. */ static __always_inline bool __irqflags_uses_pmr(void) @@ -109,16 +109,29 @@ static __always_inline unsigned long __pmr_local_save_flags(void) */ static inline unsigned long arch_local_save_flags(void) { + unsigned long flags; + if (__irqflags_uses_pmr()) { return __pmr_local_save_flags(); } else { - return __daif_local_save_flags(); + flags = __daif_local_save_flags(); + if (system_uses_nmi() && + (read_sysreg_s(SYS_ALLINT) & ALLINT_ALLINT)) + flags |= PSR_ALLINT_BIT; + return flags; } } static __always_inline bool __daif_irqs_disabled_flags(unsigned long flags) { - return flags & PSR_I_BIT; + if (flags & PSR_I_BIT) + return true; + + /* SCTLR_EL1.SPINTMASK is clear, so ALLINT masks *all* IRQs/FIQs. */ + if (system_uses_nmi()) + return !!(flags & PSR_ALLINT_BIT); + + return false; } static __always_inline bool __pmr_irqs_disabled_flags(unsigned long flags) diff --git a/arch/arm64/include/asm/ptrace.h b/arch/arm64/include/asm/ptrace.h index e5d22061c4d2..36f56c6036e8 100644 --- a/arch/arm64/include/asm/ptrace.h +++ b/arch/arm64/include/asm/ptrace.h @@ -243,8 +243,10 @@ static inline void forget_syscall(struct pt_regs *regs) (regs)->pmr_save == GIC_PRIO_IRQON : \ true) -#define interrupts_enabled(regs) \ - (!((regs)->pstate & PSR_I_BIT) && irqs_priority_unmasked(regs)) +#define interrupts_enabled(regs) \ + (!((regs)->pstate & PSR_I_BIT) && \ + !(system_uses_nmi() && ((regs)->pstate & PSR_ALLINT_BIT)) && \ + irqs_priority_unmasked(regs)) #define fast_interrupts_enabled(regs) \ (!(regs)->pstate & PSR_F_BIT) diff --git a/arch/arm64/include/uapi/asm/ptrace.h b/arch/arm64/include/uapi/asm/ptrace.h index 7fa2f7036aa7..2b61a53b8593 100644 --- a/arch/arm64/include/uapi/asm/ptrace.h +++ b/arch/arm64/include/uapi/asm/ptrace.h @@ -46,6 +46,7 @@ #define PSR_I_BIT 0x00000080 #define PSR_A_BIT 0x00000100 #define PSR_D_BIT 0x00000200 +#define PSR_ALLINT_BIT 0x00002000 #define PSR_BTYPE_MASK 0x00000c00 #define PSR_SSBS_BIT 0x00001000 #define PSR_PAN_BIT 0x00400000 -- 2.25.1

From: Vladimir Murzin <vladimir.murzin@arm.com> driver inclusion category: bugfix bugzilla: https://atomgit.com/openeuler/kernel/issues/10041 ---------------------------------------------------------------------- Kprobes masks all DAIF exceptions to keep things simple and avoid nested exceptions. With FEAT_NMI, that is no longer enough. Mask ALLINT as well to ensure things stay simple. Fixes: b3980e48528c ("arm64: kprobes: Recover pstate.D in single-step exception handler") Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com> Reviewed-by: Jinjie Ruan <ruanjinjie@huawei.com> Signed-off-by: Qinxin Xia <xiaqinxin@huawei.com> Signed-off-by: Hongye Lin <linhongye@h-partners.com> --- arch/arm64/kernel/probes/kprobes.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/arch/arm64/kernel/probes/kprobes.c b/arch/arm64/kernel/probes/kprobes.c index 70b91a8c6bb3..97b094900fe5 100644 --- a/arch/arm64/kernel/probes/kprobes.c +++ b/arch/arm64/kernel/probes/kprobes.c @@ -184,16 +184,18 @@ static void __kprobes set_current_kprobe(struct kprobe *p) * the kprobe state is per-CPU and doesn't get migrated. */ static void __kprobes kprobes_save_local_irqflag(struct kprobe_ctlblk *kcb, - struct pt_regs *regs) + struct pt_regs *regs) { - kcb->saved_irqflag = regs->pstate & DAIF_MASK; - regs->pstate |= DAIF_MASK; + kcb->saved_irqflag = regs->pstate & (DAIF_MASK | PSR_ALLINT_BIT); + + regs->pstate |= DAIF_MASK | PSR_ALLINT_BIT; } static void __kprobes kprobes_restore_local_irqflag(struct kprobe_ctlblk *kcb, - struct pt_regs *regs) + struct pt_regs *regs) { - regs->pstate &= ~DAIF_MASK; + regs->pstate &= ~(DAIF_MASK | PSR_ALLINT_BIT); + regs->pstate |= kcb->saved_irqflag; } -- 2.25.1