[PATCH OLK-6.6 v3 0/7] arm64: Add support to turn an IPI as NMI
With pseudo NMIs support available its possible to configure SGIs to be triggered as pseudo NMIs running in NMI context. And kernel features such as: - NMI backtrace can leverage IPI turned as NMI to get a backtrace of CPU stuck in hard lockup using magic SYSRQ. - kgdb relies on NMI support to round up CPUs which are stuck in hard lockup state with interrupts disabled. This patch-set adds framework to turn an IPI as NMI which can be triggered as a pseudo NMI which in turn invokes registered NMI handlers. After this patch-set we should be able to get a backtrace for a CPU stuck in HARDLOCKUP. Sumit Garg (7): arm64: Add framework to turn IPI as NMI irqchip/gic-v3: Enable support for SGIs to act as NMIs arm64: smp: Assign and setup an IPI as NMI nmi: backtrace: Allow runtime arch specific override arm64: ipi_nmi: Add support for NMI backtrace kgdb: Expose default CPUs roundup fallback mechanism arm64: kgdb: Roundup cpus using IPI as NMI arch/arm/include/asm/irq.h | 2 +- arch/arm/kernel/smp.c | 3 +- arch/arm64/include/asm/irq.h | 6 +++ arch/arm64/include/asm/nmi.h | 17 +++++++ arch/arm64/kernel/Makefile | 2 +- arch/arm64/kernel/ipi_nmi.c | 84 ++++++++++++++++++++++++++++++++ arch/arm64/kernel/kgdb.c | 18 +++++++ arch/arm64/kernel/smp.c | 8 +++ arch/mips/include/asm/irq.h | 2 +- arch/mips/kernel/process.c | 3 +- arch/powerpc/include/asm/irq.h | 2 +- arch/powerpc/kernel/stacktrace.c | 3 +- arch/sparc/include/asm/irq_64.h | 2 +- arch/sparc/kernel/process_64.c | 4 +- arch/x86/include/asm/irq.h | 2 +- arch/x86/kernel/apic/hw_nmi.c | 3 +- drivers/irqchip/irq-gic-v3.c | 29 ++++++++--- include/linux/kgdb.h | 12 +++++ include/linux/nmi.h | 12 ++--- kernel/debug/debug_core.c | 8 ++- 20 files changed, 194 insertions(+), 28 deletions(-) create mode 100644 arch/arm64/include/asm/nmi.h create mode 100644 arch/arm64/kernel/ipi_nmi.c -- 2.34.1
From: Sumit Garg <sumit.garg@linaro.org> maillist inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I7R4EN CVE: NA Reference: https://lore.kernel.org/all/1604317487-14543-1-git-send-email-sumit.garg@lin... ------------------------------------------------- Introduce framework to turn an IPI as NMI using pseudo NMIs. The main motivation for this feature is to have an IPI that can be leveraged to invoke NMI functions on other CPUs. And current prospective users are NMI backtrace and KGDB CPUs round-up whose support is added via future patches. Signed-off-by: Sumit Garg <sumit.garg@linaro.org> Signed-off-by: Wei Li <liwei391@huawei.com> Reviewed-by: Xie XiuQi <xiexiuqi@huawei.com> Signed-off-by: Zheng Zengkai <zhengzengkai@huawei.com> Signed-off-by: Ruan Jinjie <ruanjinjie@huawei.com> Signed-off-by: Liao Chen <liaochen4@huawei.com> --- arch/arm64/include/asm/nmi.h | 17 ++++++++++ arch/arm64/kernel/ipi_nmi.c | 65 ++++++++++++++++++++++++++++++++++++ 2 files changed, 82 insertions(+) create mode 100644 arch/arm64/include/asm/nmi.h create mode 100644 arch/arm64/kernel/ipi_nmi.c diff --git a/arch/arm64/include/asm/nmi.h b/arch/arm64/include/asm/nmi.h new file mode 100644 index 000000000000..4cd14b6af88b --- /dev/null +++ b/arch/arm64/include/asm/nmi.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_NMI_H +#define __ASM_NMI_H + +#ifndef __ASSEMBLER__ + +#include <linux/cpumask.h> + +extern bool arm64_supports_nmi(void); +extern void arm64_send_nmi(cpumask_t *mask); + +void set_smp_dynamic_ipi(int ipi); +void dynamic_ipi_setup(int cpu); +void dynamic_ipi_teardown(int cpu); + +#endif /* !__ASSEMBLER__ */ +#endif diff --git a/arch/arm64/kernel/ipi_nmi.c b/arch/arm64/kernel/ipi_nmi.c new file mode 100644 index 000000000000..a945dcf8015f --- /dev/null +++ b/arch/arm64/kernel/ipi_nmi.c @@ -0,0 +1,65 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * NMI support for IPIs + * + * Copyright (C) 2020 Linaro Limited + * Author: Sumit Garg <sumit.garg@linaro.org> + */ + +#include <linux/interrupt.h> +#include <linux/irq.h> +#include <linux/smp.h> + +#include <asm/nmi.h> + +static struct irq_desc *ipi_nmi_desc __read_mostly; +static int ipi_nmi_id __read_mostly; + +bool arm64_supports_nmi(void) +{ + if (ipi_nmi_desc) + return true; + + return false; +} + +void arm64_send_nmi(cpumask_t *mask) +{ + if (WARN_ON_ONCE(!ipi_nmi_desc)) + return; + + __ipi_send_mask(ipi_nmi_desc, mask); +} + +static irqreturn_t ipi_nmi_handler(int irq, void *data) +{ + /* nop, NMI handlers for special features can be added here. */ + + return IRQ_NONE; +} + +void dynamic_ipi_setup(int cpu) +{ + if (!ipi_nmi_desc) + return; + + if (!prepare_percpu_nmi(ipi_nmi_id)) + enable_percpu_nmi(ipi_nmi_id, IRQ_TYPE_NONE); +} + +void dynamic_ipi_teardown(int cpu) +{ + if (!ipi_nmi_desc) + return; + + disable_percpu_nmi(ipi_nmi_id); + teardown_percpu_nmi(ipi_nmi_id); +} + +void __init set_smp_dynamic_ipi(int ipi) +{ + if (!request_percpu_nmi(ipi, ipi_nmi_handler, "IPI", &cpu_number)) { + ipi_nmi_desc = irq_to_desc(ipi); + ipi_nmi_id = ipi; + } +} -- 2.34.1
From: Sumit Garg <sumit.garg@linaro.org> maillist inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I7R4EN CVE: NA Reference: https://lore.kernel.org/all/1604317487-14543-3-git-send-email-sumit.garg@lin... ------------------------------------------------- Add support to handle SGIs as pseudo NMIs. As SGIs or IPIs default to a special flow handler: handle_percpu_devid_fasteoi_ipi(), so skip NMI handler update in case of SGIs. Also, enable NMI support prior to gic_smp_init() as allocation of SGIs as IRQs/NMIs happen as part of this routine. Signed-off-by: Sumit Garg <sumit.garg@linaro.org> Signed-off-by: Wei Li <liwei391@huawei.com> Reviewed-by: Xie XiuQi <xiexiuqi@huawei.com> Signed-off-by: Zheng Zengkai <zhengzengkai@huawei.com> Signed-off-by: Ruan Jinjie <ruanjinjie@huawei.com> Signed-off-by: Liao Chen <liaochen4@huawei.com> --- drivers/irqchip/irq-gic-v3.c | 29 +++++++++++++++++++++-------- 1 file changed, 21 insertions(+), 8 deletions(-) diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c index f59ac9586b7b..874fbc4a2da6 100644 --- a/drivers/irqchip/irq-gic-v3.c +++ b/drivers/irqchip/irq-gic-v3.c @@ -527,6 +527,7 @@ static u32 gic_get_ppi_index(struct irq_data *d) static int gic_irq_nmi_setup(struct irq_data *d) { struct irq_desc *desc = irq_to_desc(d->irq); + u32 idx; if (!gic_supports_nmi()) return -EINVAL; @@ -544,16 +545,22 @@ static int gic_irq_nmi_setup(struct irq_data *d) return -EINVAL; /* desc lock should already be held */ - if (gic_irq_in_rdist(d)) { - u32 idx = gic_get_ppi_index(d); + switch (get_intid_range(d)) { + case SGI_RANGE: + break; + case PPI_RANGE: + case EPPI_RANGE: + idx = gic_get_ppi_index(d); /* Setting up PPI as NMI, only switch handler for first NMI */ if (!refcount_inc_not_zero(&ppi_nmi_refs[idx])) { refcount_set(&ppi_nmi_refs[idx], 1); desc->handle_irq = handle_percpu_devid_fasteoi_nmi; } - } else { + break; + default: desc->handle_irq = handle_fasteoi_nmi; + break; } gic_irq_set_prio(d, GICD_INT_NMI_PRI); @@ -564,6 +571,7 @@ static int gic_irq_nmi_setup(struct irq_data *d) static void gic_irq_nmi_teardown(struct irq_data *d) { struct irq_desc *desc = irq_to_desc(d->irq); + u32 idx; if (WARN_ON(!gic_supports_nmi())) return; @@ -581,14 +589,20 @@ static void gic_irq_nmi_teardown(struct irq_data *d) return; /* desc lock should already be held */ - if (gic_irq_in_rdist(d)) { - u32 idx = gic_get_ppi_index(d); + switch (get_intid_range(d)) { + case SGI_RANGE: + break; + case PPI_RANGE: + case EPPI_RANGE: + idx = gic_get_ppi_index(d); /* Tearing down NMI, only switch handler for last NMI */ if (refcount_dec_and_test(&ppi_nmi_refs[idx])) desc->handle_irq = handle_percpu_devid_irq; - } else { + break; + default: desc->handle_irq = handle_fasteoi_irq; + break; } gic_irq_set_prio(d, GICD_INT_DEF_PRI); @@ -2074,6 +2088,7 @@ static int __init gic_init_bases(phys_addr_t dist_phys_base, gic_dist_init(); gic_cpu_init(); + gic_enable_nmi_support(); gic_smp_init(); gic_cpu_pm_init(); @@ -2086,8 +2101,6 @@ static int __init gic_init_bases(phys_addr_t dist_phys_base, gicv2m_init(handle, gic_data.domain); } - gic_enable_nmi_support(); - return 0; out_free: -- 2.34.1
From: Sumit Garg <sumit.garg@linaro.org> maillist inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I7R4EN CVE: NA Reference: https://lore.kernel.org/all/20201102161908.ayn7vff7tyduzlj2@gabell/ ------------------------------------------------- Assign an unused IPI which can be turned as NMI using ipi_nmi framework. Also, invoke corresponding dynamic IPI setup/teardown APIs. Signed-off-by: Sumit Garg <sumit.garg@linaro.org> Signed-off-by: Wei Li <liwei391@huawei.com> Reviewed-by: Xie XiuQi <xiexiuqi@huawei.com> Signed-off-by: Zheng Zengkai <zhengzengkai@huawei.com> Signed-off-by: Ruan Jinjie <ruanjinjie@huawei.com> Signed-off-by: Liao Chen <liaochen4@huawei.com> --- arch/arm64/kernel/smp.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index 960b98b43506..8288549ca5f7 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -43,6 +43,7 @@ #include <asm/daifflags.h> #include <asm/kvm_mmu.h> #include <asm/mmu_context.h> +#include <asm/nmi.h> #include <asm/numa.h> #include <asm/processor.h> #include <asm/smp_plat.h> @@ -935,6 +936,8 @@ static void ipi_setup(int cpu) for (i = 0; i < nr_ipi; i++) enable_percpu_irq(ipi_irq_base + i, 0); + + dynamic_ipi_setup(cpu); } #ifdef CONFIG_HOTPLUG_CPU @@ -947,6 +950,8 @@ static void ipi_teardown(int cpu) for (i = 0; i < nr_ipi; i++) disable_percpu_irq(ipi_irq_base + i); + + dynamic_ipi_teardown(cpu); } #endif @@ -968,6 +973,9 @@ void __init set_smp_ipi_range(int ipi_base, int n) irq_set_status_flags(ipi_base + i, IRQ_HIDDEN); } + if (n > nr_ipi) + set_smp_dynamic_ipi(ipi_base + nr_ipi); + ipi_irq_base = ipi_base; /* Setup the boot CPU immediately */ -- 2.34.1
From: Sumit Garg <sumit.garg@linaro.org> maillist inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I7R4EN CVE: NA Reference: https://lore.kernel.org/all/1604317487-14543-5-git-send-email-sumit.garg@lin... ------------------------------------------------- Add a boolean return to arch_trigger_cpumask_backtrace() to support a use-case where a particular architecture detects at runtime if it supports NMI backtrace or it would like to fallback to default implementation using SMP cross-calls. Currently such an architecture example is arm64 supporting pseudo NMIs feature which is only available on platforms which have support for GICv3 or later version. Signed-off-by: Sumit Garg <sumit.garg@linaro.org> Signed-off-by: Wei Li <liwei391@huawei.com> Reviewed-by: Xie XiuQi <xiexiuqi@huawei.com> Signed-off-by: Zheng Zengkai <zhengzengkai@huawei.com> Signed-off-by: Ruan Jinjie <ruanjinjie@huawei.com> Signed-off-by: Liao Chen <liaochen4@huawei.com> --- arch/arm/include/asm/irq.h | 2 +- arch/arm/kernel/smp.c | 3 ++- arch/arm64/kernel/Makefile | 2 +- arch/mips/include/asm/irq.h | 2 +- arch/mips/kernel/process.c | 3 ++- arch/powerpc/include/asm/irq.h | 2 +- arch/powerpc/kernel/stacktrace.c | 3 ++- arch/sparc/include/asm/irq_64.h | 2 +- arch/sparc/kernel/process_64.c | 4 +++- arch/x86/include/asm/irq.h | 2 +- arch/x86/kernel/apic/hw_nmi.c | 3 ++- include/linux/nmi.h | 12 ++++-------- 12 files changed, 21 insertions(+), 19 deletions(-) diff --git a/arch/arm/include/asm/irq.h b/arch/arm/include/asm/irq.h index 26c1d2ced4ce..d7635f746100 100644 --- a/arch/arm/include/asm/irq.h +++ b/arch/arm/include/asm/irq.h @@ -31,7 +31,7 @@ void handle_IRQ(unsigned int, struct pt_regs *); #ifdef CONFIG_SMP #include <linux/cpumask.h> -extern void arch_trigger_cpumask_backtrace(const cpumask_t *mask, +extern bool arch_trigger_cpumask_backtrace(const cpumask_t *mask, int exclude_cpu); #define arch_trigger_cpumask_backtrace arch_trigger_cpumask_backtrace #endif diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c index 3431c0553f45..a4704d53f791 100644 --- a/arch/arm/kernel/smp.c +++ b/arch/arm/kernel/smp.c @@ -846,7 +846,8 @@ static void raise_nmi(cpumask_t *mask) __ipi_send_mask(ipi_desc[IPI_CPU_BACKTRACE], mask); } -void arch_trigger_cpumask_backtrace(const cpumask_t *mask, int exclude_cpu) +bool arch_trigger_cpumask_backtrace(const cpumask_t *mask, int exclude_cpu) { nmi_trigger_cpumask_backtrace(mask, exclude_cpu, raise_nmi); + return true; } diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile index 59fa5d6e6f3e..187685f57980 100644 --- a/arch/arm64/kernel/Makefile +++ b/arch/arm64/kernel/Makefile @@ -34,7 +34,7 @@ obj-y := debug-monitors.o entry.o irq.o fpsimd.o \ cpufeature.o alternative.o cacheinfo.o \ smp.o smp_spin_table.o topology.o smccc-call.o \ syscall.o proton-pack.o idreg-override.o idle.o \ - patching.o + patching.o ipi_nmi.o obj-$(CONFIG_AARCH32_EL0) += binfmt_elf32.o sys32.o signal32.o \ sys_compat.o diff --git a/arch/mips/include/asm/irq.h b/arch/mips/include/asm/irq.h index 3a848e7e69f7..7923ba290e2d 100644 --- a/arch/mips/include/asm/irq.h +++ b/arch/mips/include/asm/irq.h @@ -76,7 +76,7 @@ extern int cp0_fdc_irq; extern int get_c0_fdc_int(void); -void arch_trigger_cpumask_backtrace(const struct cpumask *mask, +bool arch_trigger_cpumask_backtrace(const struct cpumask *mask, int exclude_cpu); #define arch_trigger_cpumask_backtrace arch_trigger_cpumask_backtrace diff --git a/arch/mips/kernel/process.c b/arch/mips/kernel/process.c index 5387ed0a5186..e2e57dde2570 100644 --- a/arch/mips/kernel/process.c +++ b/arch/mips/kernel/process.c @@ -750,9 +750,10 @@ static void raise_backtrace(cpumask_t *mask) } } -void arch_trigger_cpumask_backtrace(const cpumask_t *mask, int exclude_cpu) +bool arch_trigger_cpumask_backtrace(const cpumask_t *mask, int exclude_cpu) { nmi_trigger_cpumask_backtrace(mask, exclude_cpu, raise_backtrace); + return true; } int mips_get_process_fp_mode(struct task_struct *task) diff --git a/arch/powerpc/include/asm/irq.h b/arch/powerpc/include/asm/irq.h index ba1a5974e714..3acfa52e436b 100644 --- a/arch/powerpc/include/asm/irq.h +++ b/arch/powerpc/include/asm/irq.h @@ -54,7 +54,7 @@ void __do_IRQ(struct pt_regs *regs); int irq_choose_cpu(const struct cpumask *mask); #if defined(CONFIG_PPC_BOOK3S_64) && defined(CONFIG_NMI_IPI) -extern void arch_trigger_cpumask_backtrace(const cpumask_t *mask, +extern bool arch_trigger_cpumask_backtrace(const cpumask_t *mask, int exclude_cpu); #define arch_trigger_cpumask_backtrace arch_trigger_cpumask_backtrace #endif diff --git a/arch/powerpc/kernel/stacktrace.c b/arch/powerpc/kernel/stacktrace.c index e6a958a5da27..3d1c373c1cee 100644 --- a/arch/powerpc/kernel/stacktrace.c +++ b/arch/powerpc/kernel/stacktrace.c @@ -204,8 +204,9 @@ static void raise_backtrace_ipi(cpumask_t *mask) } } -void arch_trigger_cpumask_backtrace(const cpumask_t *mask, int exclude_cpu) +bool arch_trigger_cpumask_backtrace(const cpumask_t *mask, int exclude_cpu) { nmi_trigger_cpumask_backtrace(mask, exclude_cpu, raise_backtrace_ipi); + return true; } #endif /* defined(CONFIG_PPC_BOOK3S_64) && defined(CONFIG_NMI_IPI) */ diff --git a/arch/sparc/include/asm/irq_64.h b/arch/sparc/include/asm/irq_64.h index 8c4c0c87f998..4daf26697100 100644 --- a/arch/sparc/include/asm/irq_64.h +++ b/arch/sparc/include/asm/irq_64.h @@ -86,7 +86,7 @@ static inline unsigned long get_softint(void) return retval; } -void arch_trigger_cpumask_backtrace(const struct cpumask *mask, +bool arch_trigger_cpumask_backtrace(const struct cpumask *mask, int exclude_cpu); #define arch_trigger_cpumask_backtrace arch_trigger_cpumask_backtrace diff --git a/arch/sparc/kernel/process_64.c b/arch/sparc/kernel/process_64.c index 529adfecd58c..5bcb86ec0c07 100644 --- a/arch/sparc/kernel/process_64.c +++ b/arch/sparc/kernel/process_64.c @@ -236,7 +236,7 @@ static void __global_reg_poll(struct global_reg_snapshot *gp) } } -void arch_trigger_cpumask_backtrace(const cpumask_t *mask, int exclude_cpu) +bool arch_trigger_cpumask_backtrace(const cpumask_t *mask, int exclude_cpu) { struct thread_info *tp = current_thread_info(); struct pt_regs *regs = get_irq_regs(); @@ -291,6 +291,8 @@ void arch_trigger_cpumask_backtrace(const cpumask_t *mask, int exclude_cpu) memset(global_cpu_snapshot, 0, sizeof(global_cpu_snapshot)); spin_unlock_irqrestore(&global_cpu_snapshot_lock, flags); + + return true; } #ifdef CONFIG_MAGIC_SYSRQ diff --git a/arch/x86/include/asm/irq.h b/arch/x86/include/asm/irq.h index 836c170d3087..e1cadd9a439d 100644 --- a/arch/x86/include/asm/irq.h +++ b/arch/x86/include/asm/irq.h @@ -41,7 +41,7 @@ extern void __handle_irq(struct irq_desc *desc, struct pt_regs *regs); extern void init_ISA_irqs(void); #ifdef CONFIG_X86_LOCAL_APIC -void arch_trigger_cpumask_backtrace(const struct cpumask *mask, +bool arch_trigger_cpumask_backtrace(const struct cpumask *mask, int exclude_cpu); #define arch_trigger_cpumask_backtrace arch_trigger_cpumask_backtrace diff --git a/arch/x86/kernel/apic/hw_nmi.c b/arch/x86/kernel/apic/hw_nmi.c index 45af535c44a0..3c7ebee0d491 100644 --- a/arch/x86/kernel/apic/hw_nmi.c +++ b/arch/x86/kernel/apic/hw_nmi.c @@ -36,10 +36,11 @@ static void nmi_raise_cpu_backtrace(cpumask_t *mask) __apic_send_IPI_mask(mask, NMI_VECTOR); } -void arch_trigger_cpumask_backtrace(const cpumask_t *mask, int exclude_cpu) +bool arch_trigger_cpumask_backtrace(const cpumask_t *mask, int exclude_cpu) { nmi_trigger_cpumask_backtrace(mask, exclude_cpu, nmi_raise_cpu_backtrace); + return true; } static int nmi_cpu_backtrace_handler(unsigned int cmd, struct pt_regs *regs) diff --git a/include/linux/nmi.h b/include/linux/nmi.h index 7bd446acad24..be4236ec811f 100644 --- a/include/linux/nmi.h +++ b/include/linux/nmi.h @@ -157,26 +157,22 @@ static inline void touch_nmi_watchdog(void) #ifdef arch_trigger_cpumask_backtrace static inline bool trigger_all_cpu_backtrace(void) { - arch_trigger_cpumask_backtrace(cpu_online_mask, -1); - return true; + return arch_trigger_cpumask_backtrace(cpu_online_mask, false); } static inline bool trigger_allbutcpu_cpu_backtrace(int exclude_cpu) { - arch_trigger_cpumask_backtrace(cpu_online_mask, exclude_cpu); - return true; + return arch_trigger_cpumask_backtrace(cpu_online_mask, true); } static inline bool trigger_cpumask_backtrace(struct cpumask *mask) { - arch_trigger_cpumask_backtrace(mask, -1); - return true; + return arch_trigger_cpumask_backtrace(mask, false); } static inline bool trigger_single_cpu_backtrace(int cpu) { - arch_trigger_cpumask_backtrace(cpumask_of(cpu), -1); - return true; + return arch_trigger_cpumask_backtrace(cpumask_of(cpu), false); } /* generic implementation */ -- 2.34.1
From: Sumit Garg <sumit.garg@linaro.org> maillist inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I7R4EN CVE: NA Reference: https://lore.kernel.org/all/1604317487-14543-6-git-send-email-sumit.garg@lin... ------------------------------------------------- Enable NMI backtrace support on arm64 using IPI turned as an NMI leveraging pseudo NMIs support. It is now possible for users to get a backtrace of a CPU stuck in hard-lockup using magic SYSRQ. Signed-off-by: Sumit Garg <sumit.garg@linaro.org> Signed-off-by: Wei Li <liwei391@huawei.com> Reviewed-by: Xie XiuQi <xiexiuqi@huawei.com> Signed-off-by: Zheng Zengkai <zhengzengkai@huawei.com> Signed-off-by: Ruan Jinjie <ruanjinjie@huawei.com> Signed-off-by: Liao Chen <liaochen4@huawei.com> --- arch/arm64/include/asm/irq.h | 6 ++++++ arch/arm64/kernel/ipi_nmi.c | 18 ++++++++++++++++-- 2 files changed, 22 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/irq.h b/arch/arm64/include/asm/irq.h index fac08e18bcd5..9e926b4f7d47 100644 --- a/arch/arm64/include/asm/irq.h +++ b/arch/arm64/include/asm/irq.h @@ -6,6 +6,12 @@ #include <asm-generic/irq.h> +#ifdef CONFIG_SMP +extern bool arch_trigger_cpumask_backtrace(const cpumask_t *mask, + bool exclude_self); +#define arch_trigger_cpumask_backtrace arch_trigger_cpumask_backtrace +#endif + struct pt_regs; int set_handle_irq(void (*handle_irq)(struct pt_regs *)); diff --git a/arch/arm64/kernel/ipi_nmi.c b/arch/arm64/kernel/ipi_nmi.c index a945dcf8015f..597dcf7fb59c 100644 --- a/arch/arm64/kernel/ipi_nmi.c +++ b/arch/arm64/kernel/ipi_nmi.c @@ -8,6 +8,7 @@ #include <linux/interrupt.h> #include <linux/irq.h> +#include <linux/nmi.h> #include <linux/smp.h> #include <asm/nmi.h> @@ -31,11 +32,24 @@ void arm64_send_nmi(cpumask_t *mask) __ipi_send_mask(ipi_nmi_desc, mask); } +bool arch_trigger_cpumask_backtrace(const cpumask_t *mask, bool exclude_self) +{ + if (!ipi_nmi_desc) + return false; + + nmi_trigger_cpumask_backtrace(mask, exclude_self, arm64_send_nmi); + + return true; +} + static irqreturn_t ipi_nmi_handler(int irq, void *data) { - /* nop, NMI handlers for special features can be added here. */ + irqreturn_t ret = IRQ_NONE; + + if (nmi_cpu_backtrace(get_irq_regs())) + ret = IRQ_HANDLED; - return IRQ_NONE; + return ret; } void dynamic_ipi_setup(int cpu) -- 2.34.1
From: Sumit Garg <sumit.garg@linaro.org> maillist inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I7R4EN CVE: NA Reference: https://lore.kernel.org/all/1604317487-14543-7-git-send-email-sumit.garg@lin... ------------------------------------------------- Add a new API kgdb_smp_call_nmi_hook() to expose default CPUs roundup mechanism to a particular archichecture as a runtime fallback if it detects to not support NMI roundup. Currently such an architecture example is arm64 supporting pseudo NMIs feature which is only available on platforms which have support for GICv3 or later version. Signed-off-by: Sumit Garg <sumit.garg@linaro.org> Signed-off-by: Wei Li <liwei391@huawei.com> Reviewed-by: Xie XiuQi <xiexiuqi@huawei.com> Signed-off-by: Zheng Zengkai <zhengzengkai@huawei.com> Signed-off-by: Ruan Jinjie <ruanjinjie@huawei.com> Signed-off-by: Liao Chen <liaochen4@huawei.com> --- include/linux/kgdb.h | 12 ++++++++++++ kernel/debug/debug_core.c | 8 +++++++- 2 files changed, 19 insertions(+), 1 deletion(-) diff --git a/include/linux/kgdb.h b/include/linux/kgdb.h index 76e891ee9e37..f28bea121891 100644 --- a/include/linux/kgdb.h +++ b/include/linux/kgdb.h @@ -199,6 +199,18 @@ kgdb_arch_handle_qxfer_pkt(char *remcom_in_buffer, extern void kgdb_call_nmi_hook(void *ignored); +/** + * kgdb_smp_call_nmi_hook - Provide default fallback mechanism to + * round-up CPUs + * + * If you're using the default implementation of kgdb_roundup_cpus() + * this function will be called. And if an arch detects at runtime to + * not support NMI based roundup then it can fallback to default + * mechanism using this API. + */ + +extern void kgdb_smp_call_nmi_hook(void); + /** * kgdb_roundup_cpus - Get other CPUs into a holding pattern * diff --git a/kernel/debug/debug_core.c b/kernel/debug/debug_core.c index ce1bb2301c06..46aa536205bc 100644 --- a/kernel/debug/debug_core.c +++ b/kernel/debug/debug_core.c @@ -238,7 +238,7 @@ NOKPROBE_SYMBOL(kgdb_call_nmi_hook); static DEFINE_PER_CPU(call_single_data_t, kgdb_roundup_csd) = CSD_INIT(kgdb_call_nmi_hook, NULL); -void __weak kgdb_roundup_cpus(void) +void kgdb_smp_call_nmi_hook(void) { call_single_data_t *csd; int this_cpu = raw_smp_processor_id(); @@ -269,6 +269,12 @@ void __weak kgdb_roundup_cpus(void) kgdb_info[cpu].rounding_up = false; } } +NOKPROBE_SYMBOL(kgdb_smp_call_nmi_hook); + +void __weak kgdb_roundup_cpus(void) +{ + kgdb_smp_call_nmi_hook(); +} NOKPROBE_SYMBOL(kgdb_roundup_cpus); #endif -- 2.34.1
From: Sumit Garg <sumit.garg@linaro.org> maillist inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I7R4EN CVE: NA Reference: https://lore.kernel.org/all/1604317487-14543-8-git-send-email-sumit.garg@lin... ------------------------------------------------- arm64 platforms with GICv3 or later supports pseudo NMIs which can be leveraged to roundup CPUs which are stuck in hard lockup state with interrupts disabled that wouldn't be possible with a normal IPI. So instead switch to roundup CPUs using IPI turned as NMI. And in case a particular arm64 platform doesn't supports pseudo NMIs, it will switch back to default kgdb CPUs roundup mechanism. Signed-off-by: Sumit Garg <sumit.garg@linaro.org> Signed-off-by: Wei Li <liwei391@huawei.com> Reviewed-by: Xie XiuQi <xiexiuqi@huawei.com> Signed-off-by: Zheng Zengkai <zhengzengkai@huawei.com> Signed-off-by: Ruan Jinjie <ruanjinjie@huawei.com> Signed-off-by: Liao Chen <liaochen4@huawei.com> --- arch/arm/kernel/smp.c | 2 +- arch/arm64/kernel/ipi_nmi.c | 5 +++++ arch/arm64/kernel/kgdb.c | 18 ++++++++++++++++++ 3 files changed, 24 insertions(+), 1 deletion(-) diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c index a4704d53f791..9ffc6c2536fa 100644 --- a/arch/arm/kernel/smp.c +++ b/arch/arm/kernel/smp.c @@ -849,5 +849,5 @@ static void raise_nmi(cpumask_t *mask) bool arch_trigger_cpumask_backtrace(const cpumask_t *mask, int exclude_cpu) { nmi_trigger_cpumask_backtrace(mask, exclude_cpu, raise_nmi); - return true; + return true; } diff --git a/arch/arm64/kernel/ipi_nmi.c b/arch/arm64/kernel/ipi_nmi.c index 597dcf7fb59c..898d69c37429 100644 --- a/arch/arm64/kernel/ipi_nmi.c +++ b/arch/arm64/kernel/ipi_nmi.c @@ -8,6 +8,7 @@ #include <linux/interrupt.h> #include <linux/irq.h> +#include <linux/kgdb.h> #include <linux/nmi.h> #include <linux/smp.h> @@ -45,10 +46,14 @@ bool arch_trigger_cpumask_backtrace(const cpumask_t *mask, bool exclude_self) static irqreturn_t ipi_nmi_handler(int irq, void *data) { irqreturn_t ret = IRQ_NONE; + unsigned int cpu = smp_processor_id(); if (nmi_cpu_backtrace(get_irq_regs())) ret = IRQ_HANDLED; + if (!kgdb_nmicallback(cpu, get_irq_regs())) + ret = IRQ_HANDLED; + return ret; } diff --git a/arch/arm64/kernel/kgdb.c b/arch/arm64/kernel/kgdb.c index 4e1f983df3d1..98d8c1027f06 100644 --- a/arch/arm64/kernel/kgdb.c +++ b/arch/arm64/kernel/kgdb.c @@ -17,6 +17,7 @@ #include <asm/debug-monitors.h> #include <asm/insn.h> +#include <asm/nmi.h> #include <asm/patching.h> #include <asm/traps.h> @@ -356,3 +357,20 @@ int kgdb_arch_remove_breakpoint(struct kgdb_bkpt *bpt) return aarch64_insn_write((void *)bpt->bpt_addr, *(u32 *)bpt->saved_instr); } + +void kgdb_roundup_cpus(void) +{ + struct cpumask mask; + + if (!arm64_supports_nmi()) { + kgdb_smp_call_nmi_hook(); + return; + } + + cpumask_copy(&mask, cpu_online_mask); + cpumask_clear_cpu(raw_smp_processor_id(), &mask); + if (cpumask_empty(&mask)) + return; + + arm64_send_nmi(&mask); +} -- 2.34.1
反馈： 您发送到kernel@openeuler.org的补丁/补丁集，已成功转换为PR！ PR链接地址： https://gitee.com/openeuler/kernel/pulls/3527 邮件列表地址：https://mailweb.openeuler.org/hyperkitty/list/kernel@openeuler.org/message/N... FeedBack: The patch(es) which you have sent to kernel@openeuler.org mailing list has been converted to a pull request successfully! Pull request link: https://gitee.com/openeuler/kernel/pulls/3527 Mailing list address: https://mailweb.openeuler.org/hyperkitty/list/kernel@openeuler.org/message/N...
participants (2)
-
Liao Chen -
patchwork bot