[PATCH v4 OLK-6.6 0/4] arm64: Allow CPU0 to be nohz_full
test ignore kabi Xiongfeng Wang (4): isolation: Do not check whether housekeeping CPUs are present isolation: Check whether there exists a housekeeping CPU online arm64: Kconfig: select ARCH_SUSPEND_NONZERO_CPU by default arm64: defconfig: Enable CONFIG_PM_SLEEP_SMP_NONZERO_CPU and CONFIG_ARCH_SUSPEND_NONZERO_CPU arch/arm64/Kconfig | 3 ++ arch/arm64/configs/openeuler_defconfig | 2 + arch/arm64/kernel/process.c | 19 ++++++++- include/linux/cpu.h | 3 +- include/linux/sched/isolation.h | 2 + include/linux/tick.h | 2 + kernel/cpu.c | 6 +++ kernel/sched/isolation.c | 56 ++++++++++++++++++++++---- kernel/smp.c | 4 ++ kernel/time/tick-sched.c | 11 ++++- 10 files changed, 97 insertions(+), 11 deletions(-) -- 2.34.1
From: Xiongfeng Wang <wangxiongfeng2@huawei.com> hulk inclusion category: feature bugzilla: https://atomgit.com/openeuler/kernel/issues/9856 -------------------------------- If we set isolcpus including CPU0, it will print the following error: [ 0.000000] Housekeeping: must include one present CPU, using boot CPU:0 It is because when string 'isolcpus=xxx' is parsed, only CPU0 is set as present for arch arm64. Unlike X86 and PowerPC, the present_cpu_mask is set rather late in smp_prepare_cpus() for arm64. In CPU hotplug situation, some possible CPUs are not marked as present, only CPUs with its GICC marked as ACPI_MADT_ENABLED are marked as present. So we cannot set preset_cpu_mask for all possible CPUs at early stage. Only check cpu_present_mask cannot assure that the CPUs will be broughtup online, as described in the below Link1. So do not check whether housekeeping CPUs are present when parsing 'isolcpus'. A following commit will add check after CPU online process is finished, as suggested in below Link2. With the boot CPU allowed to be nohz_full, housekeeping_any_cpu() can return an invalid CPU number at boot time before start_secondary() brings the 1st housekeeping CPU up, which crashes the kernel when the value is used as a per-CPU index by get_nohz_timer_target()'s callers (timer.c/hrtimer.c get_target_base()). This is the code part of the upstream fix 5097cbcb38e6 ("sched/isolation: Prevent boot crash when the boot CPU is nohz_full"), which was stripped from the LTS backport already merged into this tree (only the documentation part was taken). Include it here since this series makes 'nohz_full=' masks including the boot CPU reachable on arm64. Link: https://lore.kernel.org/all/20190504002733.GB19076@lenoir/ Link: https://lore.kernel.org/all/1557186148.ocs72ssdjc.astroid@bobo.none/ Signed-off-by: Xiongfeng Wang <wangxiongfeng2@huawei.com> Signed-off-by: Xia Fukun <xiafukun@huawei.com> --- include/linux/tick.h | 2 ++ kernel/sched/isolation.c | 37 +++++++++++++++++++++++++++++-------- kernel/time/tick-sched.c | 9 +++++++++ 3 files changed, 40 insertions(+), 8 deletions(-) diff --git a/include/linux/tick.h b/include/linux/tick.h index 39f01f0bef76..5c929c429007 100644 --- a/include/linux/tick.h +++ b/include/linux/tick.h @@ -42,8 +42,10 @@ extern void tick_irq_enter(void); # ifndef arch_needs_cpu # define arch_needs_cpu() (0) # endif +extern bool support_cpu0_nohz_full; # else static inline void tick_irq_enter(void) { } +#define support_cpu0_nohz_full 0 #endif #if defined(CONFIG_GENERIC_CLOCKEVENTS_BROADCAST) && defined(CONFIG_TICK_ONESHOT) diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c index 1179dc21873e..e2d6a88611dc 100644 --- a/kernel/sched/isolation.c +++ b/kernel/sched/isolation.c @@ -46,7 +46,16 @@ int housekeeping_any_cpu(enum hk_type type) if (cpu < nr_cpu_ids) return cpu; - return cpumask_any_and(housekeeping.cpumasks[type], cpu_online_mask); + cpu = cpumask_any_and(housekeeping.cpumasks[type], cpu_online_mask); + if (likely(cpu < nr_cpu_ids)) + return cpu; + /* + * Unless we have another problem this can only happen + * at boot time before start_secondary() brings the 1st + * housekeeping CPU up. + */ + WARN_ON_ONCE(system_state == SYSTEM_RUNNING || + type != HK_TYPE_TIMER); } } return smp_processor_id(); @@ -130,13 +139,25 @@ static int __init housekeeping_setup(char *str, unsigned long flags) cpumask_andnot(housekeeping_staging, cpu_possible_mask, non_housekeeping_mask); - first_cpu = cpumask_first_and(cpu_present_mask, housekeeping_staging); - if (first_cpu >= nr_cpu_ids || first_cpu >= setup_max_cpus) { - __cpumask_set_cpu(smp_processor_id(), housekeeping_staging); - __cpumask_clear_cpu(smp_processor_id(), non_housekeeping_mask); - if (!housekeeping.flags) { - pr_warn("Housekeeping: must include one present CPU, " - "using boot CPU:%d\n", smp_processor_id()); + if (support_cpu0_nohz_full) { + if (cpumask_empty(housekeeping_staging)) { + pr_warn("Housekeeping cpumask is NULL, using boot CPU\n"); + __cpumask_set_cpu(smp_processor_id(), housekeeping_staging); + /* Update non_housekeeping_mask to reflect this change */ + cpumask_andnot(non_housekeeping_mask, + cpu_possible_mask, housekeeping_staging); + } + /* No need to check against cpu_present_mask here */ + first_cpu = smp_processor_id(); + } else { + first_cpu = cpumask_first_and(cpu_present_mask, housekeeping_staging); + if (first_cpu >= nr_cpu_ids || first_cpu >= setup_max_cpus) { + __cpumask_set_cpu(smp_processor_id(), housekeeping_staging); + __cpumask_clear_cpu(smp_processor_id(), non_housekeeping_mask); + if (!housekeeping.flags) { + pr_warn("Housekeeping: must include one present CPU, " + "using boot CPU:%d\n", smp_processor_id()); + } } } diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index 55cbc49f70d1..33a471692def 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -38,6 +38,8 @@ */ static DEFINE_PER_CPU(struct tick_sched, tick_cpu_sched); +bool support_cpu0_nohz_full; + struct tick_sched *tick_get_tick_sched(int cpu) { return &per_cpu(tick_cpu_sched, cpu); @@ -1615,3 +1617,10 @@ int tick_check_oneshot_change(int allow_nohz) tick_nohz_switch_to_nohz(); return 0; } + +static int __init support_cpu0_nohz_full_setup(char *str) +{ + support_cpu0_nohz_full = true; + return 0; +} +early_param("support_cpu0_nohz_full", support_cpu0_nohz_full_setup); -- 2.34.1
From: Xiongfeng Wang <wangxiongfeng2@huawei.com> hulk inclusion category: feature bugzilla: https://atomgit.com/openeuler/kernel/issues/9856 -------------------------------- We need to make sure that the housekeeper, one of the housekeeping CPUs, is online, as describe in the below Link. We add this check after secondary CPUs online is finished. Link: https://lore.kernel.org/all/20190504002733.GB19076@lenoir/ Signed-off-by: Xiongfeng Wang <wangxiongfeng2@huawei.com> Signed-off-by: Xia Fukun <xiafukun@huawei.com> --- include/linux/sched/isolation.h | 2 ++ kernel/sched/isolation.c | 19 +++++++++++++++++++ kernel/smp.c | 4 ++++ 3 files changed, 25 insertions(+) diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h index 3894e74e8dc5..7addf79326df 100644 --- a/include/linux/sched/isolation.h +++ b/include/linux/sched/isolation.h @@ -27,6 +27,7 @@ extern bool housekeeping_enabled(enum hk_type type); extern void housekeeping_affine(struct task_struct *t, enum hk_type type); extern bool housekeeping_test_cpu(int cpu, enum hk_type type); extern void __init housekeeping_init(void); +extern void check_housekeeping_cpu_online(void); #else @@ -56,6 +57,7 @@ static inline bool housekeeping_test_cpu(int cpu, enum hk_type type) } static inline void housekeeping_init(void) { } +static inline void check_housekeeping_cpu_online(void) { } #endif /* CONFIG_CPU_ISOLATION */ static inline bool housekeeping_cpu(int cpu, enum hk_type type) diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c index e2d6a88611dc..52f25e78ef45 100644 --- a/kernel/sched/isolation.c +++ b/kernel/sched/isolation.c @@ -273,3 +273,22 @@ static int __init enhanced_isolcpus_setup(char *str) return 0; } __setup("enhanced_isolcpus", enhanced_isolcpus_setup); + +void check_housekeeping_cpu_online(void) +{ + enum hk_type type; + + if (!support_cpu0_nohz_full) + return; + if (!housekeeping.flags) + return; + for_each_set_bit(type, &housekeeping.flags, HK_TYPE_MAX) { + if (!cpumask_intersects(housekeeping.cpumasks[type], + cpu_online_mask)) { + pr_err("All housekeeping CPUs for type %d are offline!\n", type); + pr_err("please modify the kernel parameter !\n"); + /* BUG() here, otherwise we may BUG() in sched_init_smp(). */ + BUG(); + } + } +} diff --git a/kernel/smp.c b/kernel/smp.c index 13b376c7296a..551702122fcb 100644 --- a/kernel/smp.c +++ b/kernel/smp.c @@ -20,6 +20,7 @@ #include <linux/cpu.h> #include <linux/sched.h> #include <linux/sched/idle.h> +#include <linux/sched/isolation.h> #include <linux/hypervisor.h> #include <linux/sched/clock.h> #include <linux/nmi.h> @@ -986,6 +987,9 @@ void __init smp_init(void) /* Any cleanup work */ smp_cpus_done(setup_max_cpus); + + /* Check whether there exists a housekeeping CPU online */ + check_housekeeping_cpu_online(); } /* -- 2.34.1
From: Xiongfeng Wang <wangxiongfeng2@huawei.com> hulk inclusion category: feature bugzilla: https://atomgit.com/openeuler/kernel/issues/9856 -------------------------------- Commit 08ae95f4fd3b ("nohz_full: Allow the boot CPU to be nohz_full") allows the boot CPU to be a nohz_full CPU, handing over its housekeeping duty. But it is only supported when CONFIG_PM_SLEEP_SMP is not configured, or when it is configured and the arch allows suspend on non-zero CPUs. We cannot disable CONFIG_PM_SLEEP_SMP because freeze_secondary_cpus(), which is used in kexec, is included in the config. For arm64, there is no restriction in kernel to prevent nonzero cpu entering suspend using PSCI interface. when ARCH_SUSPEND_NONZERO_CPU is enabled, PM_SLEEP_SMP_NONZERO_CPU will be also enabled. The only influence is that suspend_disable_secondary_cpus() will use a housekeeping CPU to enter suspend. If pm sleep/hibernate is not involved, only kexec will use this function, which is tested on an arm64 server, and the vmcore can be created successfully. We add 'support_cpu0_nohz_full' in the CONFIG_PM_SLEEP_SMP_NONZERO_CPU check branch. When 'support_cpu0_nohz_full' is zero. There is no functional change. Besides, the normal kexec path (kexec -e, not the crash path) calls machine_shutdown() -> smp_shutdown_nonboot_cpus(reboot_cpu), with reboot_cpu being 0. After this patch makes 'nohz_full=' masks including CPU0 possible, tick_do_timer_cpu is handed over to a non-zero housekeeping CPU, and the offline of that CPU is rejected by tick_nohz_cpu_down() with -EBUSY. smp_shutdown_nonboot_cpus() then hits its unconditional BUG_ON(num_online_cpus() > 1) and kexec fails. The vmcore test above does not cover this path (the crash path goes through machine_crash_shutdown() instead). Fix it by extending smp_shutdown_nonboot_cpus() to accept primary_cpu == -1, which selects a housekeeping TIMER CPU as the reboot CPU the same way freeze_secondary_cpus() does, and make arm64's machine_shutdown() pass -1 when 'support_cpu0_nohz_full' is given and CPU0 is in the nohz_full mask. Since the reboot task was pinned to reboot_cpu (CPU0) by migrate_to_reboot_cpu() and CPU0 is now offlined by the shutdown loop, also widen its affinity to the housekeeping TIMER cpumask before calling smp_shutdown_nonboot_cpus(), so the task which executes machine_kexec() ends up on the surviving reboot CPU, as it did before. Without the boot parameter, reboot_cpu keeps being used and the behavior is unchanged. Signed-off-by: Xiongfeng Wang <wangxiongfeng2@huawei.com> Signed-off-by: Xia Fukun <xiafukun@huawei.com> --- arch/arm64/Kconfig | 3 +++ arch/arm64/kernel/process.c | 19 ++++++++++++++++++- include/linux/cpu.h | 3 ++- kernel/cpu.c | 6 ++++++ kernel/time/tick-sched.c | 2 +- 5 files changed, 30 insertions(+), 3 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 182a6a06b299..2f6f5931f718 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -2870,6 +2870,9 @@ config ARCH_HIBERNATION_HEADER config ARCH_SUSPEND_POSSIBLE def_bool y +config ARCH_SUSPEND_NONZERO_CPU + def_bool y + config ARCH_CPUIDLE_HALTPOLL bool "Enable selection of the cpuidle-haltpoll driver" help diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index e8e0e11b1c4d..8eae04523777 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -11,6 +11,7 @@ #include <linux/elf.h> #include <linux/export.h> #include <linux/sched.h> +#include <linux/sched/isolation.h> #include <linux/sched/debug.h> #include <linux/sched/task.h> #include <linux/sched/task_stack.h> @@ -87,7 +88,23 @@ void __noreturn arch_cpu_idle_dead(void) */ void machine_shutdown(void) { - smp_shutdown_nonboot_cpus(reboot_cpu); + unsigned int primary_cpu = reboot_cpu; + + if (support_cpu0_nohz_full && tick_nohz_full_cpu(0)) { + primary_cpu = -1; + /* + * The reboot task was pinned to reboot_cpu (CPU0) by + * migrate_to_reboot_cpu(), but CPU0 will be offlined by + * smp_shutdown_nonboot_cpus() below. Move it to a + * housekeeping TIMER CPU first, so that the task which + * executes machine_kexec() stays on the reboot CPU, + * as it did before. + */ + set_cpus_allowed_ptr(current, + housekeeping_cpumask(HK_TYPE_TIMER)); + } + + smp_shutdown_nonboot_cpus(primary_cpu); } /* diff --git a/include/linux/cpu.h b/include/linux/cpu.h index 98e06cf4b49d..58b184e03016 100644 --- a/include/linux/cpu.h +++ b/include/linux/cpu.h @@ -20,6 +20,7 @@ #include <linux/cpuhotplug.h> #include <linux/cpuhplock.h> #include <linux/cpu_smt.h> +#include <linux/tick.h> struct device; struct device_node; @@ -143,7 +144,7 @@ static inline int suspend_disable_secondary_cpus(void) { int cpu = 0; - if (IS_ENABLED(CONFIG_PM_SLEEP_SMP_NONZERO_CPU)) + if (support_cpu0_nohz_full && IS_ENABLED(CONFIG_PM_SLEEP_SMP_NONZERO_CPU)) cpu = -1; return freeze_secondary_cpus(cpu); diff --git a/kernel/cpu.c b/kernel/cpu.c index 8a0955256578..cefddfbab780 100644 --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -1530,6 +1530,12 @@ void smp_shutdown_nonboot_cpus(unsigned int primary_cpu) cpu_maps_update_begin(); + if (primary_cpu == -1) { + primary_cpu = cpumask_first(cpu_online_mask); + if (!housekeeping_cpu(primary_cpu, HK_TYPE_TIMER)) + primary_cpu = housekeeping_any_cpu(HK_TYPE_TIMER); + } + /* * Make certain the cpu I'm about to reboot on is online. * diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index 33a471692def..549160ecfd88 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -582,7 +582,7 @@ void __init tick_nohz_init(void) } if (IS_ENABLED(CONFIG_PM_SLEEP_SMP) && - !IS_ENABLED(CONFIG_PM_SLEEP_SMP_NONZERO_CPU)) { + (!support_cpu0_nohz_full || !IS_ENABLED(CONFIG_PM_SLEEP_SMP_NONZERO_CPU))) { cpu = smp_processor_id(); if (cpumask_test_cpu(cpu, tick_nohz_full_mask)) { -- 2.34.1
From: Xiongfeng Wang <wangxiongfeng2@huawei.com> hulk inclusion category: feature bugzilla: https://atomgit.com/openeuler/kernel/issues/9856 -------------------------------- Enable CONFIG_PM_SLEEP_SMP_NONZERO_CPU and CONFIG_ARCH_SUSPEND_NONZERO_CPU. Signed-off-by: Xiongfeng Wang <wangxiongfeng2@huawei.com> Signed-off-by: Xia Fukun <xiafukun@huawei.com> --- arch/arm64/configs/openeuler_defconfig | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/arm64/configs/openeuler_defconfig b/arch/arm64/configs/openeuler_defconfig index 007158aed18e..19e44f0b84cc 100644 --- a/arch/arm64/configs/openeuler_defconfig +++ b/arch/arm64/configs/openeuler_defconfig @@ -661,6 +661,7 @@ CONFIG_HIBERNATION_SNAPSHOT_DEV=y CONFIG_PM_STD_PARTITION="" CONFIG_PM_SLEEP=y CONFIG_PM_SLEEP_SMP=y +CONFIG_PM_SLEEP_SMP_NONZERO_CPU=y # CONFIG_PM_AUTOSLEEP is not set # CONFIG_PM_USERSPACE_AUTOSLEEP is not set # CONFIG_PM_WAKELOCKS is not set @@ -680,6 +681,7 @@ CONFIG_CPU_PM=y CONFIG_ARCH_HIBERNATION_POSSIBLE=y CONFIG_ARCH_HIBERNATION_HEADER=y CONFIG_ARCH_SUSPEND_POSSIBLE=y +CONFIG_ARCH_SUSPEND_NONZERO_CPU=y CONFIG_ARCH_CPUIDLE_HALTPOLL=y # end of Power management options -- 2.34.1
反馈： 您发送到kernel@openeuler.org的补丁/补丁集，已成功转换为PR！ PR链接地址： https://atomgit.com/openeuler/kernel/merge_requests/30243 邮件列表地址：https://mailweb.openeuler.org/archives/list/kernel@openeuler.org/message/HCJ... FeedBack: The patch(es) which you have sent to kernel@openeuler.org mailing list has been converted to a pull request successfully! Pull request link: https://atomgit.com/openeuler/kernel/merge_requests/30243 Mailing list address: https://mailweb.openeuler.org/archives/list/kernel@openeuler.org/message/HCJ...
participants (2)
-
patchwork bot -
Xia Fukun