November 2023 - Kernel - mailweb.openeuler.org

[PATCH OLK-5.10 v2 0/2] cpufreq: Abort show()/store() for half-initialized policies
by Jinjie Ruan 07 Nov '23

07 Nov '23

Fix cpu offline but cat scaling_cur_freq is the same as online bug. Schspa Shi (2): cpufreq: Abort show()/store() for half-initialized policies cpufreq: make interface functions and lock holding state clear drivers/cpufreq/cpufreq.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) -- 2.34.1

2 3

[PATCH openEuler-1.0-LTS v2 0/2] cpufreq: Abort show()/store() for half-initialized policies
by Jinjie Ruan 07 Nov '23

07 Nov '23

Fix cpu offline but cat scaling_cur_freq is the same as online bug. Changes in v2: - Fix the CHECKPATCH warnings. Schspa Shi (2): cpufreq: Abort show()/store() for half-initialized policies cpufreq: make interface functions and lock holding state clear drivers/cpufreq/cpufreq.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) -- 2.34.1

2 3

[PATCH OLK-5.10 0/2] cpufreq: Abort show()/store() for half-initialized policies
by Jinjie Ruan 07 Nov '23

07 Nov '23

Fix cpu offline but cat scaling_cur_freq is the same as online bug. Schspa Shi (2): cpufreq: Abort show()/store() for half-initialized policies cpufreq: make interface functions and lock holding state clear drivers/cpufreq/cpufreq.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) -- 2.34.1

2 3

[PATCH openEuler-1.0-LTS 0/2] cpufreq: Abort show()/store() for half-initialized policies
by Jinjie Ruan 07 Nov '23

07 Nov '23

Fix cpu offline but cat scaling_cur_freq is the same as online bug. Schspa Shi (2): cpufreq: Abort show()/store() for half-initialized policies cpufreq: make interface functions and lock holding state clear drivers/cpufreq/cpufreq.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) -- 2.34.1

2 3

[PATCH v2 openEuler-1.0-LTS 0/3] handle uninitialized numa nodes gracefully
by Wupeng Ma 07 Nov '23

07 Nov '23

From: Ma Wupeng <mawupeng1(a)huawei.com> handle uninitialized numa nodes gracefully. Changelog since v1: - add bugfix patch #3. Michal Hocko (2): mm, memory_hotplug: make arch_alloc_nodedata independent on CONFIG_MEMORY_HOTPLUG mm: handle uninitialized numa nodes gracefully Oscar Salvador (1): arch/x86/mm/numa: Do not initialize nodes twice arch/ia64/mm/discontig.c | 6 +- arch/x86/mm/numa.c | 26 +++---- include/linux/memory_hotplug.h | 120 ++++++++++++++++----------------- mm/internal.h | 2 + mm/memory_hotplug.c | 24 +++---- mm/page_alloc.c | 45 +++++++++++-- 6 files changed, 123 insertions(+), 100 deletions(-) -- 2.25.1

2 4

[PATCH OLK-5.10 v3] blk-mq: avoid housekeeping CPUs scheduling a worker on a non-housekeeping CPU
by Xiongfeng Wang 07 Nov '23

07 Nov '23

hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I7ZBTV CVE: N/A ---------------------------------------------------- When NOHZ_FULL is enabled, such as in HPC situation, CPUs are divided into housekeeping CPUs and non-housekeeping CPUs. Non-housekeeping CPUs are NOHZ_FULL CPUs and are often monopolized by the userspace process, such HPC application process. Any sort of interruption is not expected. blk_mq_hctx_next_cpu() selects each cpu in 'hctx->cpumask' alternately to schedule the work thread blk_mq_run_work_fn(). When 'hctx->cpumask' contains housekeeping CPU and non-housekeeping CPU at the same time, a housekeeping CPU, which want to request a IO, may schedule a worker on a non-housekeeping CPU. This may affect the performance of the userspace application running on non-housekeeping CPUs. So let's just schedule the worker thread on the current CPU when the current CPU is housekeeping CPU. Signed-off-by: Xiongfeng Wang <wangxiongfeng2(a)huawei.com> --- block/blk-mq.c | 12 +++++++++++- include/linux/sched/isolation.h | 2 ++ kernel/sched/isolation.c | 8 ++++++++ 3 files changed, 21 insertions(+), 1 deletion(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 1d1200afb771..3245809453b8 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -23,6 +23,9 @@ #include <linux/sched/sysctl.h> #include <linux/sched/topology.h> #include <linux/sched/signal.h> +#ifndef __GENKSYMS__ +#include <linux/sched/isolation.h> +#endif #include <linux/delay.h> #include <linux/crash_dump.h> #include <linux/prefetch.h> @@ -1676,6 +1679,8 @@ static int blk_mq_hctx_next_cpu(struct blk_mq_hw_ctx *hctx) static void __blk_mq_delay_run_hw_queue(struct blk_mq_hw_ctx *hctx, bool async, unsigned long msecs) { + int work_cpu; + if (unlikely(blk_mq_hctx_stopped(hctx))) return; @@ -1697,7 +1702,12 @@ static void __blk_mq_delay_run_hw_queue(struct blk_mq_hw_ctx *hctx, bool async, if (!percpu_ref_tryget(&hctx->queue->q_usage_counter)) return; - kblockd_mod_delayed_work_on(blk_mq_hctx_next_cpu(hctx), &hctx->run_work, + if (enhanced_isolcpus && tick_nohz_full_enabled() && + housekeeping_cpu(raw_smp_processor_id(), HK_FLAG_WQ)) + work_cpu = raw_smp_processor_id(); + else + work_cpu = blk_mq_hctx_next_cpu(hctx); + kblockd_mod_delayed_work_on(work_cpu, &hctx->run_work, msecs_to_jiffies(msecs)); percpu_ref_put(&hctx->queue->q_usage_counter); } diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h index cc9f393e2a70..2f93081ad7a0 100644 --- a/include/linux/sched/isolation.h +++ b/include/linux/sched/isolation.h @@ -18,6 +18,7 @@ enum hk_flags { }; #ifdef CONFIG_CPU_ISOLATION +extern bool enhanced_isolcpus; DECLARE_STATIC_KEY_FALSE(housekeeping_overridden); extern int housekeeping_any_cpu(enum hk_flags flags); extern const struct cpumask *housekeeping_cpumask(enum hk_flags flags); @@ -28,6 +29,7 @@ extern void __init housekeeping_init(void); #else +#define enhanced_isolcpus 0 static inline int housekeeping_any_cpu(enum hk_flags flags) { return smp_processor_id(); diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c index 5a6ea03f9882..785ef5201116 100644 --- a/kernel/sched/isolation.c +++ b/kernel/sched/isolation.c @@ -198,3 +198,11 @@ static int __init housekeeping_isolcpus_setup(char *str) return housekeeping_setup(str, flags); } __setup("isolcpus=", housekeeping_isolcpus_setup); + +bool enhanced_isolcpus; +static int __init enhanced_isolcpus_setup(char *str) +{ + enhanced_isolcpus = true; + return 0; +} +__setup("enhanced_isolcpus", enhanced_isolcpus_setup); -- 2.20.1

2 1

[PATCH OLK-5.10 00/67] backport SME feature from mainline
by Wang ShaoBo 07 Nov '23

07 Nov '23

From: w00502114 <bobo.shaobowang(a)huawei.com> backport SME feature from mainline Alexandru Elisei (1): arm64: Do not trap PMSNEVFR_EL1 Marc Zyngier (10): KVM: arm64: Let vcpu_sve_pffr() handle HYP VAs KVM: arm64: Provide KVM's own save/restore SVE primitives KVM: arm64: Use {read,write}_sysreg_el1 to access ZCR_EL1 KVM: arm64: Introduce vcpu_sve_vq() helper arm64: sve: Provide a conditional update accessor for ZCR_ELx KVM: arm64: Map SVE context at EL2 when available KVM: arm64: Rework SVE host-save/guest-restore KVM: arm64: Save guest's ZCR_EL1 before saving the FPSIMD state KVM: arm64: Save/restore SVE state for nVHE KVM: arm64: Always start with clearing SME flag on load Mark Brown (54): arm64/sve: Remove redundant system_supports_sve() tests arm64/sve: Add compile time checks for SVE hooks in generic functions arm64/sve: Rework SVE access trap to convert state in registers arm64/sve: Split _sve_flush macro into separate Z and predicate flushes arm64/sve: Skip flushing Z registers with 128 bit vectors arm64/sve: Use the sve_flush macros in sve_load_from_fpsimd_state() arm64/sve: Remove sve_load_from_fpsimd_state() arm64/sve: Better handle failure to allocate SVE register storage arm64/fp: Reindent fpsimd_save() arm64/sve: Make access to FFR optional arm64/sve: Rename find_supported_vector_length() arm64/sve: Use accessor functions for vector lengths in thread_struct arm64/sve: Put system wide vector length information into structs arm64/sve: Explicitly load vector length when restoring SVE state arm64/sve: Track vector lengths for tasks in an array arm64/sve: Make sysctl interface for SVE reusable by SME arm64/sve: Generalise vector length configuration prctl() for SME arm64/sve: Minor clarification of ABI documentation arm64: cpufeature: Always specify and use a field width for capabilities arm64/sme: Provide ABI documentation for SME arm64/sme: System register and exception syndrome definitions arm64/sme: Manually encode SME instructions arm64: Disable fine grained traps on boot arm64/sme: Early CPU setup for SME arm64/sme: Basic enumeration support arm64/sme: Identify supported SME vector lengths at boot arm64/sme: Implement sysctl to set the default vector length arm64/sme: Implement vector length configuration prctl()s arm64/sme: Implement support for TPIDR2 arm64/sme: Implement SVCR context switching arm64/sme: Implement streaming SVE context switching arm64/sme: Implement ZA context switching arm64/sme: Implement traps and syscall handling for SME arm64/sme: Disable ZA and streaming mode when handling signals arm64/sme: Implement streaming SVE signal handling arm64/sme: Implement ZA signal handling arm64/sme: Implement ptrace support for streaming mode SVE registers arm64/sme: Add ptrace support for ZA arm64/sme: Disable streaming mode and ZA when flushing CPU state arm64/sme: Save and restore streaming mode over EFI runtime calls arm64/sme: Provide Kconfig for SME arm64/sme: Add ID_AA64SMFR0_EL1 to __read_sysreg_by_encoding() arm64/sme: More sensibly define the size for the ZA register set KVM: arm64: Hide SME system registers from guests KVM: arm64: Trap SME usage in guest KVM: arm64: Handle SME host state when running guests arm64/fp: Make SVE and SME length register definition match architecture arm64/fp: Rename SVE and SME LEN field name to _WIDTH arm64/sme: Drop SYS_ from SMIDR_EL1 defines arm64/sme: Standardise bitfield names for SVCR arm64/sme: Remove _EL0 from name of SVCR - FIXME sysreg.h arm64/sme: Fix tests for 0b1111 value ID registers arm64/sme: Fix SVE/SME typo in ABI documentation arm64/sme: Fix EFI save/restore Wan Jiabing (1): arm64/sme: Fix NULL check after kzalloc Xiaofei Tan (1): arm64: sve: Provide sve_cond_update_zcr_vq fallback when !ARM64_SVE Documentation/arm64/elf_hwcaps.rst | 34 + Documentation/arm64/index.rst | 1 + Documentation/arm64/sme.rst | 428 ++++++++++ Documentation/arm64/sve.rst | 72 +- arch/arm64/Kconfig | 11 + arch/arm64/include/asm/cpu.h | 4 + arch/arm64/include/asm/cpucaps.h | 2 + arch/arm64/include/asm/cpufeature.h | 25 + arch/arm64/include/asm/esr.h | 13 +- arch/arm64/include/asm/exception.h | 1 + arch/arm64/include/asm/fpsimd.h | 262 ++++++- arch/arm64/include/asm/fpsimdmacros.h | 108 ++- arch/arm64/include/asm/hwcap.h | 8 + arch/arm64/include/asm/kvm_arm.h | 1 + arch/arm64/include/asm/kvm_host.h | 14 +- arch/arm64/include/asm/kvm_hyp.h | 2 + arch/arm64/include/asm/processor.h | 78 +- arch/arm64/include/asm/signal_common.h | 19 +- arch/arm64/include/asm/sysreg.h | 79 +- arch/arm64/include/asm/thread_info.h | 4 +- arch/arm64/include/uapi/asm/hwcap.h | 8 + arch/arm64/include/uapi/asm/ptrace.h | 69 +- arch/arm64/include/uapi/asm/sigcontext.h | 55 +- arch/arm64/kernel/cpufeature.c | 282 +++++-- arch/arm64/kernel/cpuinfo.c | 13 + arch/arm64/kernel/entry-common.c | 11 + arch/arm64/kernel/entry-fpsimd.S | 72 +- arch/arm64/kernel/fpsimd.c | 944 ++++++++++++++++++----- arch/arm64/kernel/head.S | 87 +++ arch/arm64/kernel/process.c | 45 +- arch/arm64/kernel/ptrace.c | 373 +++++++-- arch/arm64/kernel/signal.c | 175 ++++- arch/arm64/kernel/syscall.c | 29 +- arch/arm64/kernel/traps.c | 1 + arch/arm64/kvm/fpsimd.c | 70 +- arch/arm64/kvm/guest.c | 6 +- arch/arm64/kvm/hyp/fpsimd.S | 12 + arch/arm64/kvm/hyp/include/hyp/switch.h | 75 +- arch/arm64/kvm/hyp/nvhe/switch.c | 38 +- arch/arm64/kvm/hyp/vhe/switch.c | 10 +- arch/arm64/kvm/reset.c | 14 +- arch/arm64/kvm/sys_regs.c | 8 +- include/uapi/linux/elf.h | 2 + include/uapi/linux/prctl.h | 9 + kernel/sys.c | 12 + 45 files changed, 3126 insertions(+), 460 deletions(-) create mode 100644 Documentation/arm64/sme.rst -- 2.25.1

1 67

[PATCH OLK-5.10 00/23] arm64/perf: Enable branch stack sampling
by Junhao He 07 Nov '23

07 Nov '23

This series enables perf branch stack sampling support on arm64 platform via a new arch feature called Branch Record Buffer Extension (BRBE). All relevant register definitions could be accessed here. Anshuman Khandual (16): perf tools: Add missing branch_sample_type to perf_event_attr__fprintf() perf: Add irq and exception return branch types perf: Add system error and not in transaction branch types perf: Extend branch type classification perf: Capture branch privilege information perf: Add PERF_BR_NEW_ARCH_[N] map for BRBE on arm64 platform perf branch: Add system error and not in transaction branch types perf branch: Extend branch type classification perf branch: Add branch privilege information request flag perf branch: Add PERF_BR_NEW_ARCH_[N] map for BRBE on arm64 platform perf: Consolidate branch sample filter helpers perf record: Add remaining branch filters: "no_cycles", "no_flags" & "hw_index" arm64/sysreg: Add BRBE registers and fields drivers: perf: arm_pmu: Add new sched_task() callback drivers: perf: arm_pmuv3: Enable branch stack sampling framework drivers: perf: arm_pmuv3: Enable branch stack sampling via FEAT_BRBE James Clark (5): perf evsel: Add error message for unsupported branch stack cases perf session: Print branch stack entry type in --dump-raw-trace perf script: Refactor branch stack printing perf script: Output branch sample type perf branch: Fix interpretation of branch records Mark Brown (1): arm64/sysreg: Introduce helpers for access to sysreg fields Sandipan Das (1): perf/core: Add speculation info to branch entries Documentation/arch/arm64/brbe.rst | 145 ++++ arch/arm64/include/asm/perf_event.h | 60 ++ arch/arm64/include/asm/sysreg.h | 499 ++++++++++++++ arch/arm64/kernel/head.S | 48 ++ arch/arm64/kernel/perf_event.c | 90 ++- arch/x86/events/intel/lbr.c | 4 +- drivers/perf/Kconfig | 11 + drivers/perf/Makefile | 1 + drivers/perf/arm_brbe.c | 779 ++++++++++++++++++++++ drivers/perf/arm_brbe.h | 261 ++++++++ drivers/perf/arm_pmu.c | 34 +- include/linux/perf/arm_pmu.h | 28 +- include/linux/perf_event.h | 27 + include/uapi/linux/perf_event.h | 51 +- kernel/events/core.c | 9 +- tools/include/uapi/linux/perf_event.h | 38 +- tools/perf/builtin-script.c | 28 +- tools/perf/util/branch.c | 72 +- tools/perf/util/branch.h | 8 +- tools/perf/util/evsel.c | 4 + tools/perf/util/parse-branch-options.c | 4 + tools/perf/util/perf_event_attr_fprintf.c | 2 +- tools/perf/util/session.c | 5 +- 23 files changed, 2172 insertions(+), 36 deletions(-) create mode 100644 Documentation/arch/arm64/brbe.rst create mode 100644 drivers/perf/arm_brbe.c create mode 100644 drivers/perf/arm_brbe.h -- 2.33.0

2 24

[PATCH OLK-5.10 00/67] backport SME feature from mainline
by w00502114 07 Nov '23

07 Nov '23

backport SME feature from mainline Alexandru Elisei (1): arm64: Do not trap PMSNEVFR_EL1 Marc Zyngier (10): KVM: arm64: Let vcpu_sve_pffr() handle HYP VAs KVM: arm64: Provide KVM's own save/restore SVE primitives KVM: arm64: Use {read,write}_sysreg_el1 to access ZCR_EL1 KVM: arm64: Introduce vcpu_sve_vq() helper arm64: sve: Provide a conditional update accessor for ZCR_ELx KVM: arm64: Map SVE context at EL2 when available KVM: arm64: Rework SVE host-save/guest-restore KVM: arm64: Save guest's ZCR_EL1 before saving the FPSIMD state KVM: arm64: Save/restore SVE state for nVHE KVM: arm64: Always start with clearing SME flag on load Mark Brown (54): arm64/sve: Remove redundant system_supports_sve() tests arm64/sve: Add compile time checks for SVE hooks in generic functions arm64/sve: Rework SVE access trap to convert state in registers arm64/sve: Split _sve_flush macro into separate Z and predicate flushes arm64/sve: Skip flushing Z registers with 128 bit vectors arm64/sve: Use the sve_flush macros in sve_load_from_fpsimd_state() arm64/sve: Remove sve_load_from_fpsimd_state() arm64/sve: Better handle failure to allocate SVE register storage arm64/fp: Reindent fpsimd_save() arm64/sve: Make access to FFR optional arm64/sve: Rename find_supported_vector_length() arm64/sve: Use accessor functions for vector lengths in thread_struct arm64/sve: Put system wide vector length information into structs arm64/sve: Explicitly load vector length when restoring SVE state arm64/sve: Track vector lengths for tasks in an array arm64/sve: Make sysctl interface for SVE reusable by SME arm64/sve: Generalise vector length configuration prctl() for SME arm64/sve: Minor clarification of ABI documentation arm64: cpufeature: Always specify and use a field width for capabilities arm64/sme: Provide ABI documentation for SME arm64/sme: System register and exception syndrome definitions arm64/sme: Manually encode SME instructions arm64: Disable fine grained traps on boot arm64/sme: Early CPU setup for SME arm64/sme: Basic enumeration support arm64/sme: Identify supported SME vector lengths at boot arm64/sme: Implement sysctl to set the default vector length arm64/sme: Implement vector length configuration prctl()s arm64/sme: Implement support for TPIDR2 arm64/sme: Implement SVCR context switching arm64/sme: Implement streaming SVE context switching arm64/sme: Implement ZA context switching arm64/sme: Implement traps and syscall handling for SME arm64/sme: Disable ZA and streaming mode when handling signals arm64/sme: Implement streaming SVE signal handling arm64/sme: Implement ZA signal handling arm64/sme: Implement ptrace support for streaming mode SVE registers arm64/sme: Add ptrace support for ZA arm64/sme: Disable streaming mode and ZA when flushing CPU state arm64/sme: Save and restore streaming mode over EFI runtime calls arm64/sme: Provide Kconfig for SME arm64/sme: Add ID_AA64SMFR0_EL1 to __read_sysreg_by_encoding() arm64/sme: More sensibly define the size for the ZA register set KVM: arm64: Hide SME system registers from guests KVM: arm64: Trap SME usage in guest KVM: arm64: Handle SME host state when running guests arm64/fp: Make SVE and SME length register definition match architecture arm64/fp: Rename SVE and SME LEN field name to _WIDTH arm64/sme: Drop SYS_ from SMIDR_EL1 defines arm64/sme: Standardise bitfield names for SVCR arm64/sme: Remove _EL0 from name of SVCR - FIXME sysreg.h arm64/sme: Fix tests for 0b1111 value ID registers arm64/sme: Fix SVE/SME typo in ABI documentation arm64/sme: Fix EFI save/restore Wan Jiabing (1): arm64/sme: Fix NULL check after kzalloc Xiaofei Tan (1): arm64: sve: Provide sve_cond_update_zcr_vq fallback when !ARM64_SVE Documentation/arm64/elf_hwcaps.rst | 34 + Documentation/arm64/index.rst | 1 + Documentation/arm64/sme.rst | 428 ++++++++++ Documentation/arm64/sve.rst | 72 +- arch/arm64/Kconfig | 11 + arch/arm64/include/asm/cpu.h | 4 + arch/arm64/include/asm/cpucaps.h | 2 + arch/arm64/include/asm/cpufeature.h | 25 + arch/arm64/include/asm/esr.h | 13 +- arch/arm64/include/asm/exception.h | 1 + arch/arm64/include/asm/fpsimd.h | 262 ++++++- arch/arm64/include/asm/fpsimdmacros.h | 108 ++- arch/arm64/include/asm/hwcap.h | 8 + arch/arm64/include/asm/kvm_arm.h | 1 + arch/arm64/include/asm/kvm_host.h | 14 +- arch/arm64/include/asm/kvm_hyp.h | 2 + arch/arm64/include/asm/processor.h | 78 +- arch/arm64/include/asm/signal_common.h | 19 +- arch/arm64/include/asm/sysreg.h | 79 +- arch/arm64/include/asm/thread_info.h | 4 +- arch/arm64/include/uapi/asm/hwcap.h | 8 + arch/arm64/include/uapi/asm/ptrace.h | 69 +- arch/arm64/include/uapi/asm/sigcontext.h | 55 +- arch/arm64/kernel/cpufeature.c | 282 +++++-- arch/arm64/kernel/cpuinfo.c | 13 + arch/arm64/kernel/entry-common.c | 11 + arch/arm64/kernel/entry-fpsimd.S | 72 +- arch/arm64/kernel/fpsimd.c | 944 ++++++++++++++++++----- arch/arm64/kernel/head.S | 87 +++ arch/arm64/kernel/process.c | 45 +- arch/arm64/kernel/ptrace.c | 373 +++++++-- arch/arm64/kernel/signal.c | 175 ++++- arch/arm64/kernel/syscall.c | 29 +- arch/arm64/kernel/traps.c | 1 + arch/arm64/kvm/fpsimd.c | 70 +- arch/arm64/kvm/guest.c | 6 +- arch/arm64/kvm/hyp/fpsimd.S | 12 + arch/arm64/kvm/hyp/include/hyp/switch.h | 75 +- arch/arm64/kvm/hyp/nvhe/switch.c | 38 +- arch/arm64/kvm/hyp/vhe/switch.c | 10 +- arch/arm64/kvm/reset.c | 14 +- arch/arm64/kvm/sys_regs.c | 8 +- include/uapi/linux/elf.h | 2 + include/uapi/linux/prctl.h | 9 + kernel/sys.c | 12 + 45 files changed, 3126 insertions(+), 460 deletions(-) create mode 100644 Documentation/arm64/sme.rst -- 2.25.1

1 67

[PATCH OLK-5.10 v2] blk-mq: avoid housekeeping CPUs scheduling a worker on a non-housekeeping CPU
by Xiongfeng Wang 07 Nov '23

07 Nov '23

hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I7ZBTV CVE: N/A ---------------------------------------------------- When NOHZ_FULL is enabled, such as in HPC situation, CPUs are divided into housekeeping CPUs and non-housekeeping CPUs. Non-housekeeping CPUs are NOHZ_FULL CPUs and are often monopolized by the userspace process, such HPC application process. Any sort of interruption is not expected. blk_mq_hctx_next_cpu() selects each cpu in 'hctx->cpumask' alternately to schedule the work thread blk_mq_run_work_fn(). When 'hctx->cpumask' contains housekeeping CPU and non-housekeeping CPU at the same time, a housekeeping CPU, which want to request a IO, may schedule a worker on a non-housekeeping CPU. This may affect the performance of the userspace application running on non-housekeeping CPUs. So let's just schedule the worker thread on the current CPU when the current CPU is housekeeping CPU. Signed-off-by: Xiongfeng Wang <wangxiongfeng2(a)huawei.com> --- block/blk-mq.c | 13 ++++++++++++- include/linux/sched/isolation.h | 2 ++ kernel/sched/isolation.c | 8 ++++++++ 3 files changed, 22 insertions(+), 1 deletion(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 1d1200afb771..b81d4a69056c 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -23,6 +23,9 @@ #include <linux/sched/sysctl.h> #include <linux/sched/topology.h> #include <linux/sched/signal.h> +#ifndef __GENKSYMS__ +#include <linux/sched/isolation.h> +#endif #include <linux/delay.h> #include <linux/crash_dump.h> #include <linux/prefetch.h> @@ -1676,6 +1679,8 @@ static int blk_mq_hctx_next_cpu(struct blk_mq_hw_ctx *hctx) static void __blk_mq_delay_run_hw_queue(struct blk_mq_hw_ctx *hctx, bool async, unsigned long msecs) { + int work_cpu; + if (unlikely(blk_mq_hctx_stopped(hctx))) return; @@ -1697,7 +1702,13 @@ static void __blk_mq_delay_run_hw_queue(struct blk_mq_hw_ctx *hctx, bool async, if (!percpu_ref_tryget(&hctx->queue->q_usage_counter)) return; - kblockd_mod_delayed_work_on(blk_mq_hctx_next_cpu(hctx), &hctx->run_work, + if (enhanced_isolcpus && tick_nohz_full_enabled() && + housekeeping_cpu(raw_smp_processor_id(), HK_FLAG_WQ)) + work_cpu = smp_processor_id(); + else + work_cpu = blk_mq_hctx_next_cpu(hctx); + + kblockd_mod_delayed_work_on(work_cpu, &hctx->run_work, msecs_to_jiffies(msecs)); percpu_ref_put(&hctx->queue->q_usage_counter); } diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h index cc9f393e2a70..2f93081ad7a0 100644 --- a/include/linux/sched/isolation.h +++ b/include/linux/sched/isolation.h @@ -18,6 +18,7 @@ enum hk_flags { }; #ifdef CONFIG_CPU_ISOLATION +extern bool enhanced_isolcpus; DECLARE_STATIC_KEY_FALSE(housekeeping_overridden); extern int housekeeping_any_cpu(enum hk_flags flags); extern const struct cpumask *housekeeping_cpumask(enum hk_flags flags); @@ -28,6 +29,7 @@ extern void __init housekeeping_init(void); #else +#define enhanced_isolcpus 0 static inline int housekeeping_any_cpu(enum hk_flags flags) { return smp_processor_id(); diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c index 5a6ea03f9882..785ef5201116 100644 --- a/kernel/sched/isolation.c +++ b/kernel/sched/isolation.c @@ -198,3 +198,11 @@ static int __init housekeeping_isolcpus_setup(char *str) return housekeeping_setup(str, flags); } __setup("isolcpus=", housekeeping_isolcpus_setup); + +bool enhanced_isolcpus; +static int __init enhanced_isolcpus_setup(char *str) +{ + enhanced_isolcpus = true; + return 0; +} +__setup("enhanced_isolcpus", enhanced_isolcpus_setup); -- 2.20.1

2 1

2024

2023

2022

2021

2020

2019

Kernel November 2023