[PATCH 0/3] arm64: Fix NMI/DAIF handling in hibernate, suspend and kprobes
arm64: Fix NMI/DAIF handling in hibernate, suspend and kprobes Ada Couprie Diaz (1): arm64: hibernate: mask DAIF before restoring hibernated kernel Vladimir Murzin (2): arm64: suspend: Always initialise PSTATE.ALLINT arm64: kprobes: Disable NMIs arch/arm64/include/asm/daifflags.h | 1 + arch/arm64/include/asm/ptrace.h | 2 +- arch/arm64/kernel/hibernate.c | 12 ++++++++++++ arch/arm64/kernel/probes/kprobes.c | 12 +++++++----- 4 files changed, 21 insertions(+), 6 deletions(-) -- 2.33.0
From: Ada Couprie Diaz <ada.coupriediaz@arm.com> driver inclusion category: bugfix bugzilla: https://atomgit.com/openeuler/kernel/issues/10041 ---------------------------------------------------------------------- The arm64 hibernate code manages the exception masking in an unsound way, leading to potential crashes and/or warnings during resume. When a hibernation image is saved in `swsusp_arch_suspend()`, all DAIF exceptions are masked (by virtue of `local_daif_save()`), and the suspended image is saved assuming that all DAIF exceptions will remain masked when the image is restored. When a hibernation image is resumed by `swsusp_arch_resume()`, only interrupts are masked (by virtue of `local_irq_disable()` in `resume_target_kernel()`). When pseudo-NMI is enabled the DAIF.IF bits will be clear, and regardless of pseudo-NMI the DAIF.DA bits will be clear. This means that there are two problems: (1) It is possible to take Debug, SError, or pseudo-NMI exceptions during the resume process. This is unsafe, as during the resume process both the old ane new kernels will tranisently be in an inconsistent state, and swsusp_arch_suspend_exit() won't retain an executable mapping of any exception vectors. Any exception taken here will be fatal and silent. (2) When re-entering the resumed kernel, some DAIF bits will be clear unexpectedly. This permits Debug, SError, or pseudo-NMI exceptions to be taken for a short period while the resumed kernel is not yet in a consistent state. This is detected by CONFIG_ARM64_DEBUG_PRIORITY_MASKING. Avoid these issues by masking all DAIF exceptions during resume. Fixes: 82869ac57b5d ("arm64: kernel: Add support for hibernate/suspend-to-disk") Signed-off-by: Ada Couprie Diaz <ada.coupriediaz@arm.com> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com> Signed-off-by: Qinxin Xia <xiaqinxin@huawei.com> Signed-off-by: Hongye Lin <linhongye@h-partners.com> --- arch/arm64/kernel/hibernate.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index 42fe868ca7f1..6a357dfd19ce 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -461,9 +461,21 @@ int __nocfi swsusp_arch_resume(void) if (el2_reset_needed()) __hyp_set_vectors(el2_vectors); + /* + * It is necessary to mask all DAIF exceptions here as: + * + * - The copy of swsusp_arch_suspend_exit() in the hibernation + * text cannot handle taking any exceptions. + * + * - The suspended kernel masked all DAIF exceptions in + * swsusp_arch_resume(), and expects to be re-entered in the + * same state : with all DAIF exceptions masked. + */ + local_daif_save(); hibernate_exit(virt_to_phys(tmp_pg_dir), resume_hdr.ttbr1_el1, resume_hdr.reenter_kernel, restore_pblist, resume_hdr.__hyp_stub_vectors, virt_to_phys(zero_page)); + unreachable(); return 0; } -- 2.33.0
From: Vladimir Murzin <vladimir.murzin@arm.com> driver inclusion category: bugfix bugzilla: https://atomgit.com/openeuler/kernel/issues/10041 ---------------------------------------------------------------------- PSTATE.ALLINT is always set to the inverse of SCTLR_ELx.SPINTMASK, regardless of the value of SCTLR_ELx.NMI. SCTLR_ELx.NMI is initialised to 0 by default, so PSTATE.ALLINT does not cause any interrupt masking. With upcoming FEAT_NMI support, SCTLR_ELx.NMI will be set as part of the enable sequence. However, during CPU suspend/resume, we reinitialise PSTATE to INIT_PSTATE_EL1. INIT_PSTATE_EL1 currently does not set PSR_ALLINT_BIT, but SCTLR_ELx.NMI is restored from the saved value. Since all exceptions are masked during CPU suspend/resume, avoid a state where SCTLR_ELx.NMI is restored to 1 but PSTATE.ALLINT is clear, since that would permit an NMI during resume. Include PSR_ALLINT_BIT in INIT_PSTATE_EL1 so PSTATE.ALLINT is always set. Fixes: d87a8e65b510 ("arm64: head.S: always initialize PSTATE") Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com> Reviewed-by: Jinjie Ruan <ruanjinjie@huawei.com> Signed-off-by: Qinxin Xia <xiaqinxin@huawei.com> Signed-off-by: Hongye Lin <linhongye@h-partners.com> --- arch/arm64/include/asm/ptrace.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/ptrace.h b/arch/arm64/include/asm/ptrace.h index e5d22061c4d2..aae2c19ed62c 100644 --- a/arch/arm64/include/asm/ptrace.h +++ b/arch/arm64/include/asm/ptrace.h @@ -17,7 +17,7 @@ #define CurrentEL_EL2 (2 << 2) #define INIT_PSTATE_EL1 \ - (PSR_D_BIT | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT | PSR_MODE_EL1h) + (PSR_ALLINT_BIT | PSR_D_BIT | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT | PSR_MODE_EL1h) #define INIT_PSTATE_EL2 \ (PSR_D_BIT | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT | PSR_MODE_EL2h) -- 2.33.0
From: Vladimir Murzin <vladimir.murzin@arm.com> driver inclusion category: bugfix bugzilla: https://atomgit.com/openeuler/kernel/issues/10041 ---------------------------------------------------------------------- Kprobes masks all DAIF exceptions to keep things simple and avoid nested exceptions. With FEAT_NMI, that is no longer enough. Mask ALLINT as well to ensure things stay simple. Fixes: b3980e48528c ("arm64: kprobes: Recover pstate.D in single-step exception handler") Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com> Reviewed-by: Jinjie Ruan <ruanjinjie@huawei.com> Signed-off-by: Qinxin Xia <xiaqinxin@huawei.com> Signed-off-by: Hongye Lin <linhongye@h-partners.com> --- arch/arm64/include/asm/daifflags.h | 1 + arch/arm64/kernel/probes/kprobes.c | 12 +++++++----- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/arch/arm64/include/asm/daifflags.h b/arch/arm64/include/asm/daifflags.h index 2417cc6b1631..bbb9e7da0e35 100644 --- a/arch/arm64/include/asm/daifflags.h +++ b/arch/arm64/include/asm/daifflags.h @@ -17,6 +17,7 @@ #define DAIF_PROCCTX_NOIRQ (PSR_I_BIT | PSR_F_BIT) #define DAIF_ERRCTX (PSR_A_BIT | PSR_I_BIT | PSR_F_BIT) #define DAIF_MASK (PSR_D_BIT | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT) +#define DAIF_ALLINT_MASK (system_uses_nmi() ? (PSR_ALLINT_BIT | DAIF_MASK) : DAIF_MASK) /* mask/save/unmask/restore all exceptions, including interrupts. */ diff --git a/arch/arm64/kernel/probes/kprobes.c b/arch/arm64/kernel/probes/kprobes.c index 70b91a8c6bb3..b2a0be8aa10d 100644 --- a/arch/arm64/kernel/probes/kprobes.c +++ b/arch/arm64/kernel/probes/kprobes.c @@ -184,16 +184,18 @@ static void __kprobes set_current_kprobe(struct kprobe *p) * the kprobe state is per-CPU and doesn't get migrated. */ static void __kprobes kprobes_save_local_irqflag(struct kprobe_ctlblk *kcb, - struct pt_regs *regs) + struct pt_regs *regs) { - kcb->saved_irqflag = regs->pstate & DAIF_MASK; - regs->pstate |= DAIF_MASK; + kcb->saved_irqflag = regs->pstate & DAIF_ALLINT_MASK; + + regs->pstate |= DAIF_ALLINT_MASK; } static void __kprobes kprobes_restore_local_irqflag(struct kprobe_ctlblk *kcb, - struct pt_regs *regs) + struct pt_regs *regs) { - regs->pstate &= ~DAIF_MASK; + regs->pstate &= ~DAIF_ALLINT_MASK; + regs->pstate |= kcb->saved_irqflag; } -- 2.33.0
participants (1)
-
Qinxin Xia