[PATCH OLK-6.6 0/4] arm64: Allow CPU0 to be nohz_full
The upstream commits 2f1a6fbbef77, 9ca12ac04bb7, 9219565aa890 and 08ae95f4fd3b provide the generic framework to allow the boot CPU to be nohz_full. x86 and PowerPC already benefit from it. However, on arm64, CPU0 still cannot be put into the nohz_full/isolcpus range due to two obstacles: 1. The cpu_present_mask of arm64 is populated rather late (in smp_prepare_cpus()), because only CPUs with GICC entries marked as ACPI_MADT_ENABLED can be marked present. When 'isolcpus=' or 'nohz_full=' is parsed during early boot, only CPU0 is present, so the present check in housekeeping_setup() forcibly pulls CPU0 back into the housekeeping mask, which breaks the isolation. 2. The framework requires the arch to declare ARCH_SUSPEND_NONZERO_CPU so that a non-zero CPU can act as the primary CPU for suspend/hibernate/kexec. arm64 did not define it. This series removes the two obstacles: Patch 1/4 skips the unreliable early present check (present does not guarantee the CPU will be brought up online) when the new boot param 'support_cpu0_nohz_full' is given, keeping only the "housekeeping mask must not be empty" fallback. Patch 2/4 adds check_housekeeping_cpu_online() at the end of smp_init() as the safety net: after all CPUs are onlined, it verifies that every configured housekeeping type has at least one CPU online, and BUG()s with a clear message otherwise ("early release, late check"). Patch 3/4 declares ARCH_SUSPEND_NONZERO_CPU for arm64 (PSCI does not restrict which CPU enters suspend), and gates the behavior change with the 'support_cpu0_nohz_full' boot param, so the default behavior is unchanged. Patch 4/4 enables the two configs in openeuler_defconfig explicitly. With "support_cpu0_nohz_full isolcpus=0-N nohz_full=0-N" on the kernel command line, all CPUs including CPU0 can be isolated for latency critical workloads, while the timekeeping, RCU and workqueue duties converge on the remaining housekeeping CPUs. This series is the OLK-6.6 adaptation of the same feature merged in OLK-5.10 (original author Xiongfeng Wang). Xiongfeng Wang (4): isolation: Do not check whether housekeeping CPUs are present isolation: Check whether there exists a housekeeping CPU online arm64: Kconfig: select ARCH_SUSPEND_NONZERO_CPU by default arm64: defconfig: Enable CONFIG_PM_SLEEP_SMP_NONZERO_CPU and CONFIG_ARCH_SUSPEND_NONZERO_CPU arch/arm64/Kconfig | 3 ++ arch/arm64/configs/openeuler_defconfig | 2 ++ include/linux/cpu.h | 5 ++- include/linux/sched/isolation.h | 2 ++ include/linux/tick.h | 2 ++ kernel/sched/isolation.c | 45 ++++++++++++++++++++++---- kernel/smp.c | 4 +++ kernel/time/tick-sched.c | 11 ++++++- 8 files changed, 65 insertions(+), 9 deletions(-) -- 2.34.1
From: Xiongfeng Wang <wangxiongfeng2@huawei.com> hulk inclusion category: feature bugzilla: https://atomgit.com/openeuler/kernel/issues/9856 -------------------------------- If we set isolcpus including CPU0, it will print the following error: [ 0.000000] Housekeeping: must include one present CPU, using boot CPU:0 It is because when string 'isolcpus=xxx' is parsed, only CPU0 is set as present for arch arm64. Unlike X86 and PowerPC, the present_cpu_mask is set rather late in smp_prepare_cpus() for arm64. In CPU hotplug situation, some possible CPUs are not marked as present, only CPUs with its GICC marked as ACPI_MADT_ENABLED are marked as present. So we cannot set preset_cpu_mask for all possible CPUs at early stage. Only check cpu_present_mask cannot assure that the CPUs will be broughtup online, as described in the below Link1. So do not check whether housekeeping CPUs are present when parsing 'isolcpus'. A following commit will add check after CPU online process is finished, as suggested in below Link2. Link: https://lore.kernel.org/all/20190504002733.GB19076@lenoir/ Link: https://lore.kernel.org/all/1557186148.ocs72ssdjc.astroid@bobo.none/ Signed-off-by: Xiongfeng Wang <wangxiongfeng2@huawei.com> Signed-off-by: Xia Fukun <xiafukun@huawei.com> --- include/linux/tick.h | 2 ++ kernel/sched/isolation.c | 26 +++++++++++++++++++------- kernel/time/tick-sched.c | 9 +++++++++ 3 files changed, 30 insertions(+), 7 deletions(-) diff --git a/include/linux/tick.h b/include/linux/tick.h index 39f01f0bef76..5c929c429007 100644 --- a/include/linux/tick.h +++ b/include/linux/tick.h @@ -42,8 +42,10 @@ extern void tick_irq_enter(void); # ifndef arch_needs_cpu # define arch_needs_cpu() (0) # endif +extern bool support_cpu0_nohz_full; # else static inline void tick_irq_enter(void) { } +#define support_cpu0_nohz_full 0 #endif #if defined(CONFIG_GENERIC_CLOCKEVENTS_BROADCAST) && defined(CONFIG_TICK_ONESHOT) diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c index 1179dc21873e..7f7b8017b38f 100644 --- a/kernel/sched/isolation.c +++ b/kernel/sched/isolation.c @@ -130,13 +130,25 @@ static int __init housekeeping_setup(char *str, unsigned long flags) cpumask_andnot(housekeeping_staging, cpu_possible_mask, non_housekeeping_mask); - first_cpu = cpumask_first_and(cpu_present_mask, housekeeping_staging); - if (first_cpu >= nr_cpu_ids || first_cpu >= setup_max_cpus) { - __cpumask_set_cpu(smp_processor_id(), housekeeping_staging); - __cpumask_clear_cpu(smp_processor_id(), non_housekeeping_mask); - if (!housekeeping.flags) { - pr_warn("Housekeeping: must include one present CPU, " - "using boot CPU:%d\n", smp_processor_id()); + if (support_cpu0_nohz_full) { + if (cpumask_empty(housekeeping_staging)) { + pr_warn("Housekeeping cpumask is NULL, using boot CPU\n"); + __cpumask_set_cpu(smp_processor_id(), housekeeping_staging); + /* Update non_housekeeping_mask to reflect this change */ + cpumask_andnot(non_housekeeping_mask, + cpu_possible_mask, housekeeping_staging); + } + /* No need to check against cpu_present_mask here */ + first_cpu = smp_processor_id(); + } else { + first_cpu = cpumask_first_and(cpu_present_mask, housekeeping_staging); + if (first_cpu >= nr_cpu_ids || first_cpu >= setup_max_cpus) { + __cpumask_set_cpu(smp_processor_id(), housekeeping_staging); + __cpumask_clear_cpu(smp_processor_id(), non_housekeeping_mask); + if (!housekeeping.flags) { + pr_warn("Housekeeping: must include one present CPU, " + "using boot CPU:%d\n", smp_processor_id()); + } } } diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index 55cbc49f70d1..33a471692def 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -38,6 +38,8 @@ */ static DEFINE_PER_CPU(struct tick_sched, tick_cpu_sched); +bool support_cpu0_nohz_full; + struct tick_sched *tick_get_tick_sched(int cpu) { return &per_cpu(tick_cpu_sched, cpu); @@ -1615,3 +1617,10 @@ int tick_check_oneshot_change(int allow_nohz) tick_nohz_switch_to_nohz(); return 0; } + +static int __init support_cpu0_nohz_full_setup(char *str) +{ + support_cpu0_nohz_full = true; + return 0; +} +early_param("support_cpu0_nohz_full", support_cpu0_nohz_full_setup); -- 2.34.1
From: Xiongfeng Wang <wangxiongfeng2@huawei.com> hulk inclusion category: feature bugzilla: https://atomgit.com/openeuler/kernel/issues/9856 -------------------------------- We need to make sure that the housekeeper, one of the housekeeping CPUs, is online, as describe in the below Link. We add this check after secondary CPUs online is finished. Link: https://lore.kernel.org/all/20190504002733.GB19076@lenoir/ Signed-off-by: Xiongfeng Wang <wangxiongfeng2@huawei.com> Signed-off-by: Xia Fukun <xiafukun@huawei.com> --- include/linux/sched/isolation.h | 2 ++ kernel/sched/isolation.c | 19 +++++++++++++++++++ kernel/smp.c | 4 ++++ 3 files changed, 25 insertions(+) diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h index 3894e74e8dc5..7addf79326df 100644 --- a/include/linux/sched/isolation.h +++ b/include/linux/sched/isolation.h @@ -27,6 +27,7 @@ extern bool housekeeping_enabled(enum hk_type type); extern void housekeeping_affine(struct task_struct *t, enum hk_type type); extern bool housekeeping_test_cpu(int cpu, enum hk_type type); extern void __init housekeeping_init(void); +extern void check_housekeeping_cpu_online(void); #else @@ -56,6 +57,7 @@ static inline bool housekeeping_test_cpu(int cpu, enum hk_type type) } static inline void housekeeping_init(void) { } +static inline void check_housekeeping_cpu_online(void) { } #endif /* CONFIG_CPU_ISOLATION */ static inline bool housekeeping_cpu(int cpu, enum hk_type type) diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c index 7f7b8017b38f..899363c5449b 100644 --- a/kernel/sched/isolation.c +++ b/kernel/sched/isolation.c @@ -264,3 +264,22 @@ static int __init enhanced_isolcpus_setup(char *str) return 0; } __setup("enhanced_isolcpus", enhanced_isolcpus_setup); + +void check_housekeeping_cpu_online(void) +{ + enum hk_type type; + + if (!support_cpu0_nohz_full) + return; + if (!housekeeping.flags) + return; + for_each_set_bit(type, &housekeeping.flags, HK_TYPE_MAX) { + if (!cpumask_intersects(housekeeping.cpumasks[type], + cpu_online_mask)) { + pr_err("All housekeeping CPUs for type %d are offline!\n", type); + pr_err("please modify the kernel parameter !\n"); + /* BUG() here, otherwise we may BUG() in sched_init_smp(). */ + BUG(); + } + } +} diff --git a/kernel/smp.c b/kernel/smp.c index 13b376c7296a..551702122fcb 100644 --- a/kernel/smp.c +++ b/kernel/smp.c @@ -20,6 +20,7 @@ #include <linux/cpu.h> #include <linux/sched.h> #include <linux/sched/idle.h> +#include <linux/sched/isolation.h> #include <linux/hypervisor.h> #include <linux/sched/clock.h> #include <linux/nmi.h> @@ -986,6 +987,9 @@ void __init smp_init(void) /* Any cleanup work */ smp_cpus_done(setup_max_cpus); + + /* Check whether there exists a housekeeping CPU online */ + check_housekeeping_cpu_online(); } /* -- 2.34.1
From: Xiongfeng Wang <wangxiongfeng2@huawei.com> hulk inclusion category: feature bugzilla: https://atomgit.com/openeuler/kernel/issues/9856 -------------------------------- Commit 08ae95f4fd3b ("nohz_full: Allow the boot CPU to be nohz_full") allow CPU0 to be housekeeping CPU. But it is only supported when CONFIG_PM_SLEEP_SMP is not configured, or when it is configured and the arch allows suspend on non-zero CPUs. We cannot disable CONFIG_PM_SLEEP_SMP because freeze_secondary_cpus(), which is used in kexec, is included in the config. For arm64, there is no restriction in kernel to prevent nonzero cpu entering suspend using PSCI interface. when ARCH_SUSPEND_NONZERO_CPU is enabled, PM_SLEEP_SMP_NONZERO_CPU will be also enabled. The only influence is that suspend_disable_secondary_cpus() will use a housekeeping CPU to enter suspend. If pm sleep/hibernate is not involved, only kexec will use this function, which is tested on a arm64 server, and the vmcore can be created successfully. We add 'support_cpu0_nohz_full' in the CONFIG_PM_SLEEP_SMP_NONZERO_CPU check branch. When 'support_cpu0_nohz_full' is zero. There is no functional change. Signed-off-by: Xiongfeng Wang <wangxiongfeng2@huawei.com> Signed-off-by: Xia Fukun <xiafukun@huawei.com> --- arch/arm64/Kconfig | 3 +++ include/linux/cpu.h | 5 ++++- kernel/time/tick-sched.c | 2 +- 3 files changed, 8 insertions(+), 2 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 50146c8f8617..18080b5f0edf 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -2871,6 +2871,9 @@ config ARCH_HIBERNATION_HEADER config ARCH_SUSPEND_POSSIBLE def_bool y +config ARCH_SUSPEND_NONZERO_CPU + def_bool y + config ARCH_CPUIDLE_HALTPOLL bool "Enable selection of the cpuidle-haltpoll driver" help diff --git a/include/linux/cpu.h b/include/linux/cpu.h index 98e06cf4b49d..e7ec449d9134 100644 --- a/include/linux/cpu.h +++ b/include/linux/cpu.h @@ -20,6 +20,9 @@ #include <linux/cpuhotplug.h> #include <linux/cpuhplock.h> #include <linux/cpu_smt.h> +#ifndef __GENKSYMS__ +#include <linux/tick.h> +#endif struct device; struct device_node; @@ -143,7 +146,7 @@ static inline int suspend_disable_secondary_cpus(void) { int cpu = 0; - if (IS_ENABLED(CONFIG_PM_SLEEP_SMP_NONZERO_CPU)) + if (support_cpu0_nohz_full && IS_ENABLED(CONFIG_PM_SLEEP_SMP_NONZERO_CPU)) cpu = -1; return freeze_secondary_cpus(cpu); diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index 33a471692def..549160ecfd88 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -582,7 +582,7 @@ void __init tick_nohz_init(void) } if (IS_ENABLED(CONFIG_PM_SLEEP_SMP) && - !IS_ENABLED(CONFIG_PM_SLEEP_SMP_NONZERO_CPU)) { + (!support_cpu0_nohz_full || !IS_ENABLED(CONFIG_PM_SLEEP_SMP_NONZERO_CPU))) { cpu = smp_processor_id(); if (cpumask_test_cpu(cpu, tick_nohz_full_mask)) { -- 2.34.1
From: Xiongfeng Wang <wangxiongfeng2@huawei.com> hulk inclusion category: feature bugzilla: https://atomgit.com/openeuler/kernel/issues/9856 -------------------------------- Enable CONFIG_PM_SLEEP_SMP_NONZERO_CPU and CONFIG_ARCH_SUSPEND_NONZERO_CPU. Signed-off-by: Xiongfeng Wang <wangxiongfeng2@huawei.com> Signed-off-by: Xia Fukun <xiafukun@huawei.com> --- arch/arm64/configs/openeuler_defconfig | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/arm64/configs/openeuler_defconfig b/arch/arm64/configs/openeuler_defconfig index edf2ed90efb8..913a926fe370 100644 --- a/arch/arm64/configs/openeuler_defconfig +++ b/arch/arm64/configs/openeuler_defconfig @@ -661,6 +661,7 @@ CONFIG_HIBERNATION_SNAPSHOT_DEV=y CONFIG_PM_STD_PARTITION="" CONFIG_PM_SLEEP=y CONFIG_PM_SLEEP_SMP=y +CONFIG_PM_SLEEP_SMP_NONZERO_CPU=y # CONFIG_PM_AUTOSLEEP is not set # CONFIG_PM_USERSPACE_AUTOSLEEP is not set # CONFIG_PM_WAKELOCKS is not set @@ -680,6 +681,7 @@ CONFIG_CPU_PM=y CONFIG_ARCH_HIBERNATION_POSSIBLE=y CONFIG_ARCH_HIBERNATION_HEADER=y CONFIG_ARCH_SUSPEND_POSSIBLE=y +CONFIG_ARCH_SUSPEND_NONZERO_CPU=y CONFIG_ARCH_CPUIDLE_HALTPOLL=y # end of Power management options -- 2.34.1
反馈： 您发送到kernel@openeuler.org的补丁/补丁集，已成功转换为PR！ PR链接地址： https://atomgit.com/openeuler/kernel/merge_requests/30127 邮件列表地址：https://mailweb.openeuler.org/archives/list/kernel@openeuler.org/message/7FO... FeedBack: The patch(es) which you have sent to kernel@openeuler.org mailing list has been converted to a pull request successfully! Pull request link: https://atomgit.com/openeuler/kernel/merge_requests/30127 Mailing list address: https://mailweb.openeuler.org/archives/list/kernel@openeuler.org/message/7FO...
participants (2)
-
patchwork bot -
Xia Fukun