Xiongfeng Wang (3): Revert "sched/isolation: Require a present CPU in housekeeping mask" isolation: Check whether all the housekeeping CPUs are online arm64: Kconfig: select ARCH_SUSPEND_NONZERO_CPU by default
arch/arm64/Kconfig | 3 +++ include/linux/sched/isolation.h | 2 ++ kernel/sched/isolation.c | 32 ++++++++++++++++++++------------ kernel/smp.c | 4 ++++ 4 files changed, 29 insertions(+), 12 deletions(-)
反馈: 您发送到kernel@openeuler.org的补丁/补丁集，转换为PR失败！ 邮件列表地址：https://mailweb.openeuler.org/hyperkitty/list/kernel@openeuler.org/message/K... 失败原因：应用补丁/补丁集失败，Patch failed at 0002 isolation: Check whether all the housekeeping CPUs are online 建议解决方法：请查看失败原因， 确认补丁是否可以应用在当前期望分支的最新代码上
FeedBack: The patch(es) which you have sent to kernel@openeuler.org has been converted to PR failed! Mailing list address: https://mailweb.openeuler.org/hyperkitty/list/kernel@openeuler.org/message/K... Failed Reason: apply patch(es) failed, Patch failed at 0002 isolation: Check whether all the housekeeping CPUs are online Suggest Solution: please checkout if the failed patch(es) can work on the newest codes in expected branch
hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I9NR7Q CVE: NA
--------------------------------
This reverts commit 9219565aa89033a9cfdae788c1940473a1253d6c.
If we set isolcpus without CPU0, it will print the following error: [ 0.000000] Housekeeping: must include one present CPU, using boot CPU:0 It is because when string 'isolcpus=xxx' is parsed, only CPU0 is set as present for arch arm64.
Unlike X86 and PowerPC, the present_cpu_mask is set rather late in smp_prepare_cpus() for arm64. In CPU hotplug situation, some possible CPUs are not marked as present, only CPUs with its GICC marked as ACPI_MADT_ENABLED are marked as present. So we cannot set preset_cpu_mask for all possible CPUs at early stage.
Only check cpu_present_mask cannot assure that the CPUs will be broughtup online, as described in the below Link1. So revert this commit. A following commit will add check after CPU online process is finished, as suggested in below Link2.
Link1: https://lore.kernel.org/all/20190504002733.GB19076@lenoir/ Link2: https://lore.kernel.org/all/1557186148.ocs72ssdjc.astroid@bobo.none/ Signed-off-by: Xiongfeng Wang wangxiongfeng2@huawei.com --- kernel/sched/isolation.c | 18 +++++------------- 1 file changed, 5 insertions(+), 13 deletions(-)
diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c index 5a6ea03f9882..2ff78176127d 100644 --- a/kernel/sched/isolation.c +++ b/kernel/sched/isolation.c @@ -80,7 +80,6 @@ void __init housekeeping_init(void) static int __init housekeeping_setup(char *str, enum hk_flags flags) { cpumask_var_t non_housekeeping_mask; - cpumask_var_t tmp; int err;
alloc_bootmem_cpumask_var(&non_housekeeping_mask); @@ -91,23 +90,16 @@ static int __init housekeeping_setup(char *str, enum hk_flags flags) return 0; }
- alloc_bootmem_cpumask_var(&tmp); if (!housekeeping_flags) { alloc_bootmem_cpumask_var(&housekeeping_mask); cpumask_andnot(housekeeping_mask, cpu_possible_mask, non_housekeeping_mask); - - cpumask_andnot(tmp, cpu_present_mask, non_housekeeping_mask); - if (cpumask_empty(tmp)) { - pr_warn("Housekeeping: must include one present CPU, " - "using boot CPU:%d\n", smp_processor_id()); + if (cpumask_empty(housekeeping_mask)) __cpumask_set_cpu(smp_processor_id(), housekeeping_mask); - __cpumask_clear_cpu(smp_processor_id(), non_housekeeping_mask); - } } else { - cpumask_andnot(tmp, cpu_present_mask, non_housekeeping_mask); - if (cpumask_empty(tmp)) - __cpumask_clear_cpu(smp_processor_id(), non_housekeeping_mask); + cpumask_var_t tmp; + + alloc_bootmem_cpumask_var(&tmp); cpumask_andnot(tmp, cpu_possible_mask, non_housekeeping_mask); if (!cpumask_equal(tmp, housekeeping_mask)) { pr_warn("Housekeeping: nohz_full= must match isolcpus=\n"); @@ -115,8 +107,8 @@ static int __init housekeeping_setup(char *str, enum hk_flags flags) free_bootmem_cpumask_var(non_housekeeping_mask); return 0; } + free_bootmem_cpumask_var(tmp); } - free_bootmem_cpumask_var(tmp);
if ((flags & HK_FLAG_TICK) && !(housekeeping_flags & HK_FLAG_TICK)) { if (IS_ENABLED(CONFIG_NO_HZ_FULL)) {
hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I9NR7Q CVE: NA
--------------------------------
All the housekeeping CPUs are need to be brought up online, as describe in the below Link. We add this check after secondary CPUs online is finished.
Link: https://lore.kernel.org/all/20190504002733.GB19076@lenoir/ Signed-off-by: Xiongfeng Wang wangxiongfeng2@huawei.com --- include/linux/sched/isolation.h | 2 ++ kernel/sched/isolation.c | 18 +++++++++++++++++- kernel/smp.c | 4 ++++ 3 files changed, 23 insertions(+), 1 deletion(-)
diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h index cc9f393e2a70..c53de8a44988 100644 --- a/include/linux/sched/isolation.h +++ b/include/linux/sched/isolation.h @@ -25,6 +25,7 @@ extern bool housekeeping_enabled(enum hk_flags flags); extern void housekeeping_affine(struct task_struct *t, enum hk_flags flags); extern bool housekeeping_test_cpu(int cpu, enum hk_flags flags); extern void __init housekeeping_init(void); +extern void check_housekeeping_cpus_online(void);
#else
@@ -46,6 +47,7 @@ static inline bool housekeeping_enabled(enum hk_flags flags) static inline void housekeeping_affine(struct task_struct *t, enum hk_flags flags) { } static inline void housekeeping_init(void) { } +static inline void check_housekeeping_cpus_online(void) { } #endif /* CONFIG_CPU_ISOLATION */
static inline bool housekeeping_cpu(int cpu, enum hk_flags flags) diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c index 2ff78176127d..744a167e0c0e 100644 --- a/kernel/sched/isolation.c +++ b/kernel/sched/isolation.c @@ -94,8 +94,13 @@ static int __init housekeeping_setup(char *str, enum hk_flags flags) alloc_bootmem_cpumask_var(&housekeeping_mask); cpumask_andnot(housekeeping_mask, cpu_possible_mask, non_housekeeping_mask); - if (cpumask_empty(housekeeping_mask)) + if (cpumask_empty(housekeeping_mask)) { __cpumask_set_cpu(smp_processor_id(), housekeeping_mask); + /* update non_housekeeping_mask because it will be used below + in tick_nohz_full_setup() */ + cpumask_andnot(non_housekeeping_mask, + cpu_possible_mask, housekeeping_mask); + } } else { cpumask_var_t tmp;
@@ -190,3 +195,14 @@ static int __init housekeeping_isolcpus_setup(char *str) return housekeeping_setup(str, flags); } __setup("isolcpus=", housekeeping_isolcpus_setup); + +void check_housekeeping_cpus_online(void) +{ + if (!housekeeping_flags) + return; + if (!cpumask_subset(housekeeping_mask, cpu_online_mask)) { + pr_err("Not all the housekeeping CPUs are online, please modify the kernel parameter !\n"); + /* BUG_ON here, otherwise there may exist other potential error */ + BUG_ON(1); + } +} diff --git a/kernel/smp.c b/kernel/smp.c index 114776d0d11e..a009af8087fa 100644 --- a/kernel/smp.c +++ b/kernel/smp.c @@ -20,6 +20,7 @@ #include <linux/cpu.h> #include <linux/sched.h> #include <linux/sched/idle.h> +#include <linux/sched/isolation.h> #include <linux/hypervisor.h> #include <linux/sched/clock.h> #include <linux/nmi.h> @@ -875,6 +876,9 @@ void __init smp_init(void)
/* Any cleanup work */ smp_cpus_done(setup_max_cpus); + + /* Check whether all the housekeeping CPUs are online */ + check_housekeeping_cpus_online(); }
/*
hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I9NR7Q CVE: NA
--------------------------------
Commit 08ae95f4fd3b ("nohz_full: Allow the boot CPU to be nohz_full") allow CPU0 to be housekeeping CPU. But it is only supproted when CONFIG_PM_SLEEP_SMP is not configured, or when it is configured and the arch allows suspend on non-zero CPUs.
We cannot disable CONFIG_PM_SLEEP_SMP because freeze_secondary_cpus(), which is used in kexec, is included in the config. For arm64, there is not restriction in kernel to prevent nonzero cpu entering suspend using PSCI interface. when ARCH_SUSPEND_NONZERO_CPU is enabled, PM_SLEEP_SMP_NONZERO_CPU will be also enabled. The only influence is that suspend_disable_secondary_cpus() will use a housekeeping CPU to enter suspend. If pm sleep/hibernate is not involved, only kexec will use this function, which is tested on a arm64 server, and the vmcore can be created successfully.
Signed-off-by: Xiongfeng Wang wangxiongfeng2@huawei.com --- arch/arm64/Kconfig | 3 +++ 1 file changed, 3 insertions(+)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 3427154e218d..4469751d03d2 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -2199,6 +2199,9 @@ config ARCH_HIBERNATION_HEADER config ARCH_SUSPEND_POSSIBLE def_bool y
+config ARCH_SUSPEND_NONZERO_CPU + def_bool y + endmenu
menu "CPU Power Management"
-
patchwork bot -
Xiongfeng Wang