hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I9OJK9 CVE: NA ---------------------------------------- Here are many scenarios we tested with smart_grid, we found that the first domain level is key to the benchmark. The reason is that there are many things such as interrupt affinity, memory affinity factor that can have a big impact on the test. Before this patch, the first domain level is unchangeable after creation. This patch introduce the 'cpu.rebuild_affinity_domain' to dynamically reconfigure all domain levels. Typical use cases: echo $cpu_id > cpu.rebuild_affinity_domain The cpu_id means which cpu we want to set first level. If we set cpu_id = 34, we can see some change like: ---------------- ----------------- | level 0 (0-31) | | level 0 (32-63) | ---------------- ----------------- v v ------------------- ------------------ | level 1 (0-63) | | level 1 (0-63) | ------------------- ------------------ v --> v --------------------- -------------------- | level 2 (0-95) | | level 2 (0-95) | --------------------- -------------------- v v ------------------------ ---------------------- | level 3 (0-127) | | level 3 (0-127) | ------------------------ ---------------------- There are number of constraints on the rebuild feature: 1. Only rebuild domain while auto mode disabled. (cpu.dynamic_affinity_mode == 1) 2. Only rebuild on active and housekeeping cpu. (Offline and isolate CPUs are forbidden) 3. This file is write only. Signed-off-by: Yipeng Zou <zouyipeng@huawei.com> --- kernel/sched/core.c | 13 +++++++++++++ kernel/sched/fair.c | 43 +++++++++++++++++++++++++++++++++++++++++++ kernel/sched/sched.h | 1 + 3 files changed, 57 insertions(+) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 700dbd19eb44..b9b3f536959d 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -11555,6 +11555,15 @@ static int cpu_affinity_stat_show(struct seq_file *sf, void *v) return 0; } + +static int cpu_rebuild_affinity_domain_u64(struct cgroup_subsys_state *css, + struct cftype *cftype, + u64 cpu) +{ + struct task_group *tg = css_tg(css); + + return tg_rebuild_affinity_domains(cpu, tg->auto_affinity); +} #endif /* CONFIG_QOS_SCHED_SMART_GRID */ #ifdef CONFIG_QOS_SCHED @@ -11735,6 +11744,10 @@ static struct cftype cpu_legacy_files[] = { .name = "affinity_stat", .seq_show = cpu_affinity_stat_show, }, + { + .name = "rebuild_affinity_domain", + .write_u64 = cpu_rebuild_affinity_domain_u64, + }, #endif #ifdef CONFIG_CFS_BANDWIDTH { diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index d2efd40fb784..07a7c6bf7b1e 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -7252,6 +7252,49 @@ static void destroy_auto_affinity(struct task_group *tg) kfree(tg->auto_affinity); tg->auto_affinity = NULL; } + +int tg_rebuild_affinity_domains(int cpu, struct auto_affinity *auto_affi) +{ + int ret = 0; + int level = 0; + struct sched_domain *tmp; + + if (unlikely(!auto_affi)) + return -EPERM; + + mutex_lock(&smart_grid_used_mutex); + raw_spin_lock_irq(&auto_affi->lock); + /* Only build domain while auto mode disabled */ + if (auto_affi->mode) { + ret = -EPERM; + goto unlock_all; + } + + /* Only build on active and housekeeping cpu */ + if (!cpu_active(cpu) || !housekeeping_cpu(cpu, HK_TYPE_DOMAIN)) { + ret = -EINVAL; + goto unlock_all; + } + + for_each_domain(cpu, tmp) { + if (!auto_affi->ad.domains[level] || !auto_affi->ad.domains_orig[level]) + continue; + + /* rebuild domain[,_orig] and reset schedstat counter */ + cpumask_copy(auto_affi->ad.domains[level], sched_domain_span(tmp)); + cpumask_copy(auto_affi->ad.domains_orig[level], auto_affi->ad.domains[level]); + __schedstat_set(auto_affi->ad.stay_cnt[level], 0); + level++; + } + + /* trigger to update smart grid zone */ + sched_grid_zone_update(false); + +unlock_all: + raw_spin_unlock_irq(&auto_affi->lock); + mutex_unlock(&smart_grid_used_mutex); + return ret; +} #else static void __maybe_unused destroy_auto_affinity(struct task_group *tg) {} diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 9de2bac6444f..79b93b0a2ef0 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -551,6 +551,7 @@ extern void start_auto_affinity(struct auto_affinity *auto_affi); extern void stop_auto_affinity(struct auto_affinity *auto_affi); extern int init_auto_affinity(struct task_group *tg); extern void tg_update_affinity_domains(int cpu, int online); +extern int tg_rebuild_affinity_domains(int cpu, struct auto_affinity *auto_affi); #else static inline int init_auto_affinity(struct task_group *tg) -- 2.34.1