- Kernel - mailweb.openeuler.org

[PATCH OLK-6.6 0/9] mm: PCP high auto-tuning
by Ze Zuo 29 Nov '23

29 Nov '23

The page allocation performance requirements of different workloads are often different. So, we need to tune the PCP (Per-CPU Pageset) high on each CPU automatically to optimize the page allocation performance. The list of patches in series is as follows, [1/9] mm, pcp: avoid to drain PCP when process exit [2/9] cacheinfo: calculate per-CPU data cache size [3/9] mm, pcp: reduce lock contention for draining high-order pages [4/9] mm: restrict the pcp batch scale factor to avoid too long latency [5/9] mm, page_alloc: scale the number of pages that are batch allocated [6/9] mm: add framework for PCP high auto-tuning [7/9] mm: tune PCP high automatically [8/9] mm, pcp: decrease PCP high if free pages < high watermark [9/9] mm, pcp: reduce detecting time of consecutive high order page freeing Patch [1/9], [2/9], [3/9] optimize the PCP draining for consecutive high-order pages freeing. Patch [4/9], [5/9] optimize batch freeing and allocating. Patch [6/9], [7/9], [8/9] implement and optimize a PCP high auto-tuning method. Patch [9/9] optimize the PCP draining for consecutive high order page freeing based on PCP high auto-tuning. The test results for patches with performance impact are as follows, kbuild ====== On a 2-socket Intel server with 224 logical CPU, we run 8 kbuild instances in parallel (each with `make -j 28`) in 8 cgroup. This simulates the kbuild server that is used by 0-Day kbuild service. build time lock contend% free_high alloc_zone ---------- ---------- --------- ---------- base 100.0 14.0 100.0 100.0 patch1 99.5 12.8 19.5 95.6 patch3 99.4 12.6 7.1 95.6 patch5 98.6 11.0 8.1 97.1 patch7 95.1 0.5 2.8 15.6 patch9 95.0 1.0 8.8 20.0 The PCP draining optimization (patch [1/9], [3/9]) and PCP batch allocation optimization (patch [5/9]) reduces zone lock contention a little. The PCP high auto-tuning (patch [7/9], [9/9]) reduces build time visibly. Where the tuning target: the number of pages allocated from zone reduces greatly. So, the zone contention cycles% reduces greatly. With PCP tuning patches (patch [7/9], [9/9]), the average used memory during test increases up to 18.4% because more pages are cached in PCP. But at the end of the test, the number of the used memory decreases to the same level as that of the base patch. That is, the pages cached in PCP will be released to zone after not being used actively. netperf SCTP_STREAM_MANY ======================== On a 2-socket Intel server with 128 logical CPU, we tested SCTP_STREAM_MANY test case of netperf test suite with 64-pair processes. score lock contend% free_high alloc_zone cache miss rate% ----- ---------- --------- ---------- ---------------- base 100.0 2.1 100.0 100.0 1.3 patch1 99.4 2.1 99.4 99.4 1.3 patch3 106.4 1.3 13.3 106.3 1.3 patch5 106.0 1.2 13.2 105.9 1.3 patch7 103.4 1.9 6.7 90.3 7.6 patch9 108.6 1.3 13.7 108.6 1.3 The PCP draining optimization (patch [1/9]+[3/9]) improves performance. The PCP high auto-tuning (patch [7/9]) reduces performance a little because PCP draining cannot be triggered in time sometimes. So, the cache miss rate% increases. The further PCP draining optimization (patch [9/9]) based on PCP tuning restore the performance. lmbench3 UNIX (AF_UNIX) ======================= On a 2-socket Intel server with 128 logical CPU, we tested UNIX (AF_UNIX socket) test case of lmbench3 test suite with 16-pair processes. score lock contend% free_high alloc_zone cache miss rate% ----- ---------- --------- ---------- ---------------- base 100.0 51.4 100.0 100.0 0.2 patch1 116.8 46.1 69.5 104.3 0.2 patch3 199.1 21.3 7.0 104.9 0.2 patch5 200.0 20.8 7.1 106.9 0.3 patch7 191.6 19.9 6.8 103.8 2.8 patch9 193.4 21.7 7.0 104.7 2.1 The PCP draining optimization (patch [1/9], [3/9]) improves performance much. The PCP tuning (patch [7/9]) reduces performance a little because PCP draining cannot be triggered in time sometimes. The further PCP draining optimization (patch [9/9]) based on PCP tuning restores the performance partly. The patchset adds several fields in struct per_cpu_pages. The struct layout before/after the patchset is as follows, base ==== struct per_cpu_pages { spinlock_t lock; /* 0 4 */ int count; /* 4 4 */ int high; /* 8 4 */ int batch; /* 12 4 */ short int free_factor; /* 16 2 */ short int expire; /* 18 2 */ /* XXX 4 bytes hole, try to pack */ struct list_head lists[13]; /* 24 208 */ /* size: 256, cachelines: 4, members: 7 */ /* sum members: 228, holes: 1, sum holes: 4 */ /* padding: 24 */ } __attribute__((__aligned__(64))); patched ======= struct per_cpu_pages { spinlock_t lock; /* 0 4 */ int count; /* 4 4 */ int high; /* 8 4 */ int high_min; /* 12 4 */ int high_max; /* 16 4 */ int batch; /* 20 4 */ u8 flags; /* 24 1 */ u8 alloc_factor; /* 25 1 */ u8 expire; /* 26 1 */ /* XXX 1 byte hole, try to pack */ short int free_count; /* 28 2 */ /* XXX 2 bytes hole, try to pack */ struct list_head lists[13]; /* 32 208 */ /* size: 256, cachelines: 4, members: 11 */ /* sum members: 237, holes: 2, sum holes: 3 */ /* padding: 16 */ } __attribute__((__aligned__(64))); The size of the struct doesn't changed with the patchset. Huang Ying (9): mm, pcp: avoid to drain PCP when process exit cacheinfo: calculate size of per-CPU data cache slice mm, pcp: reduce lock contention for draining high-order pages mm: restrict the pcp batch scale factor to avoid too long latency mm, page_alloc: scale the number of pages that are batch allocated mm: add framework for PCP high auto-tuning mm: tune PCP high automatically mm, pcp: decrease PCP high if free pages < high watermark mm, pcp: reduce detecting time of consecutive high order page freeing drivers/base/cacheinfo.c | 51 ++++++- include/linux/cacheinfo.h | 1 + include/linux/gfp.h | 2 + include/linux/mmzone.h | 27 +++- mm/Kconfig | 11 ++ mm/page_alloc.c | 310 +++++++++++++++++++++++++++++++------- mm/vmstat.c | 8 +- 7 files changed, 345 insertions(+), 65 deletions(-) -- 2.25.1

1 9

[PACH OLK-5.10 00/11] KVM: arm64: Add host/guest KVM-PTP support
by Dongxu Sun 29 Nov '23

29 Nov '23

Jianyong Wu (4): ptp: Reorganize ptp_kvm.c to make it arch-independent clocksource: Add clocksource id for arm arch counter KVM: arm64: Add support for the KVM PTP service ptp: arm/arm64: Enable ptp_kvm for arm/arm64 Jon Hunter (1): ptp: Don't print an error if ptp_kvm is not supported Kele Huang (1): ptp: fix error print of ptp_kvm on X86_64 platform Thomas Gleixner (1): time: Add mechanism to recognize clocksource in time_get_snapshot Will Deacon (2): arm/arm64: Probe for the presence of KVM hypervisor KVM: arm64: Advertise KVM UID to guests via SMCCC Yanan Wang (1): KVM: arm64: fix compile error/warning Zenghui Yu (1): KVM: arm64: Fix Function ID typo for PTP_KVM service Documentation/virt/kvm/api.rst | 9 ++ Documentation/virt/kvm/arm/index.rst | 1 + Documentation/virt/kvm/arm/ptp_kvm.rst | 25 ++++++ arch/arm/include/asm/hypervisor.h | 3 + arch/arm64/include/asm/hypervisor.h | 3 + arch/arm64/include/asm/kvm_para.h | 2 +- arch/arm64/include/uapi/asm/Kbuild | 2 +- arch/arm64/include/uapi/asm/kvm_para.h | 7 ++ arch/arm64/kvm/arm.c | 1 + arch/arm64/kvm/hypercalls.c | 94 ++++++++++++++++---- drivers/clocksource/arm_arch_timer.c | 36 ++++++++ drivers/firmware/psci/psci.c | 2 + drivers/firmware/smccc/Makefile | 2 +- drivers/firmware/smccc/kvm_guest.c | 50 +++++++++++ drivers/firmware/smccc/smccc.c | 1 + drivers/ptp/Kconfig | 2 +- drivers/ptp/Makefile | 2 + drivers/ptp/ptp_kvm_arm.c | 28 ++++++ drivers/ptp/{ptp_kvm.c => ptp_kvm_common.c} | 85 +++++------------- drivers/ptp/ptp_kvm_x86.c | 97 +++++++++++++++++++++ include/linux/arm-smccc.h | 41 +++++++++ include/linux/clocksource.h | 6 ++ include/linux/clocksource_ids.h | 12 +++ include/linux/ptp_kvm.h | 19 ++++ include/linux/timekeeping.h | 12 +-- include/uapi/linux/kvm.h | 1 + kernel/time/clocksource.c | 2 + kernel/time/timekeeping.c | 1 + 28 files changed, 459 insertions(+), 87 deletions(-) create mode 100644 Documentation/virt/kvm/arm/ptp_kvm.rst create mode 100644 arch/arm64/include/uapi/asm/kvm_para.h create mode 100644 drivers/firmware/smccc/kvm_guest.c create mode 100644 drivers/ptp/ptp_kvm_arm.c rename drivers/ptp/{ptp_kvm.c => ptp_kvm_common.c} (60%) create mode 100644 drivers/ptp/ptp_kvm_x86.c create mode 100644 include/linux/clocksource_ids.h create mode 100644 include/linux/ptp_kvm.h -- 2.33.0

1 11

[PACH OLK-5.10 00/11] KVM: arm64: Add host/guest KVM-PTP support
by Dongxu Sun 29 Nov '23

29 Nov '23

Jianyong Wu (4): ptp: Reorganize ptp_kvm.c to make it arch-independent clocksource: Add clocksource id for arm arch counter KVM: arm64: Add support for the KVM PTP service ptp: arm/arm64: Enable ptp_kvm for arm/arm64 Jon Hunter (1): ptp: Don't print an error if ptp_kvm is not supported Kele Huang (1): ptp: fix error print of ptp_kvm on X86_64 platform Thomas Gleixner (1): time: Add mechanism to recognize clocksource in time_get_snapshot Will Deacon (2): arm/arm64: Probe for the presence of KVM hypervisor KVM: arm64: Advertise KVM UID to guests via SMCCC Yanan Wang (1): KVM: arm64: fix compile error/warning Zenghui Yu (1): KVM: arm64: Fix Function ID typo for PTP_KVM service Documentation/virt/kvm/api.rst | 9 ++ Documentation/virt/kvm/arm/index.rst | 1 + Documentation/virt/kvm/arm/ptp_kvm.rst | 25 ++++++ arch/arm/include/asm/hypervisor.h | 3 + arch/arm64/include/asm/hypervisor.h | 3 + arch/arm64/include/asm/kvm_para.h | 2 +- arch/arm64/include/uapi/asm/Kbuild | 2 +- arch/arm64/include/uapi/asm/kvm_para.h | 7 ++ arch/arm64/kvm/arm.c | 1 + arch/arm64/kvm/hypercalls.c | 94 ++++++++++++++++---- drivers/clocksource/arm_arch_timer.c | 36 ++++++++ drivers/firmware/psci/psci.c | 2 + drivers/firmware/smccc/Makefile | 2 +- drivers/firmware/smccc/kvm_guest.c | 50 +++++++++++ drivers/firmware/smccc/smccc.c | 1 + drivers/ptp/Kconfig | 2 +- drivers/ptp/Makefile | 2 + drivers/ptp/ptp_kvm_arm.c | 28 ++++++ drivers/ptp/{ptp_kvm.c => ptp_kvm_common.c} | 85 +++++------------- drivers/ptp/ptp_kvm_x86.c | 97 +++++++++++++++++++++ include/linux/arm-smccc.h | 41 +++++++++ include/linux/clocksource.h | 6 ++ include/linux/clocksource_ids.h | 12 +++ include/linux/ptp_kvm.h | 19 ++++ include/linux/timekeeping.h | 12 +-- include/uapi/linux/kvm.h | 1 + kernel/time/clocksource.c | 2 + kernel/time/timekeeping.c | 1 + 28 files changed, 459 insertions(+), 87 deletions(-) create mode 100644 Documentation/virt/kvm/arm/ptp_kvm.rst create mode 100644 arch/arm64/include/uapi/asm/kvm_para.h create mode 100644 drivers/firmware/smccc/kvm_guest.c create mode 100644 drivers/ptp/ptp_kvm_arm.c rename drivers/ptp/{ptp_kvm.c => ptp_kvm_common.c} (60%) create mode 100644 drivers/ptp/ptp_kvm_x86.c create mode 100644 include/linux/clocksource_ids.h create mode 100644 include/linux/ptp_kvm.h -- 2.33.0

1 4

[PATCH OLK-5.10] drivers: perf: Add feature flag check in armpmu_add/del
by Junhao He 29 Nov '23

29 Nov '23

driver inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I8EC9K CVE: NA -------------------------------- Function armpmu_add will call armv8pmu_branch_reset() to reset BRBE. The BRBE feature depend on ARMV8 PMU hardware support for branch stack sampling. On older platforms the kernel panics. Internal error: Oops - Undefined instruction: 0000000002000000 [#1] SMP Call trace: armv8pmu_branch_reset+0xc/0x20 event_sched_in+0xc8/0x1a0 merge_sched_in+0x16c/0x41c Fixes: 2690e65ac9f9 ("drivers: perf: arm_pmuv3: Enable branch stack sampling via FEAT_BRBE") Signed-off-by: Junhao He <hejunhao3(a)huawei.com> --- drivers/perf/arm_pmu.c | 34 +++++++++++++++++++--------------- 1 file changed, 19 insertions(+), 15 deletions(-) diff --git a/drivers/perf/arm_pmu.c b/drivers/perf/arm_pmu.c index d02a85c0bedc..4ef8aee84eea 100644 --- a/drivers/perf/arm_pmu.c +++ b/drivers/perf/arm_pmu.c @@ -313,10 +313,12 @@ armpmu_del(struct perf_event *event, int flags) struct hw_perf_event *hwc = &event->hw; int idx = hwc->idx; - WARN_ON_ONCE(!hw_events->brbe_users); - hw_events->brbe_users--; - if (!hw_events->brbe_users) - hw_events->brbe_context = NULL; + if (has_branch_stack(event)) { + WARN_ON_ONCE(!hw_events->brbe_users); + hw_events->brbe_users--; + if (!hw_events->brbe_users) + hw_events->brbe_context = NULL; + } armpmu_stop(event, PERF_EF_UPDATE); hw_events->events[idx] = NULL; @@ -334,18 +336,20 @@ armpmu_add(struct perf_event *event, int flags) struct hw_perf_event *hwc = &event->hw; int idx; - /* - * Reset branch records buffer if a new task event gets - * scheduled on a PMU which might have existing records. - * Otherwise older branch records present in the buffer - * might leak into the new task event. - */ - if (event->ctx->task && hw_events->brbe_context != event->ctx) { - hw_events->brbe_context = event->ctx; - if (armpmu->branch_reset) - armpmu->branch_reset(); + if (has_branch_stack(event)) { + /* + * Reset branch records buffer if a new task event gets + * scheduled on a PMU which might have existing records. + * Otherwise older branch records present in the buffer + * might leak into the new task event. + */ + if (event->ctx->task && hw_events->brbe_context != event->ctx) { + hw_events->brbe_context = event->ctx; + if (armpmu->branch_reset) + armpmu->branch_reset(); + } + hw_events->brbe_users++; } - hw_events->brbe_users++; /* An event following a process won't be stopped earlier */ if (!cpumask_test_cpu(smp_processor_id(), &armpmu->supported_cpus)) -- 2.33.0

2 1

[PATCH OLK-6.6 v2 0/6] Handle more faults under the VMA lock
by Tong Tiangen 29 Nov '23

29 Nov '23

At this point, we're handling the majority of file-backed page faults under the VMA lock, using the ->map_pages entry point. This patch set attempts to expand that for the following siutations: - We have to do a read. This could be because we've hit the point in the readahead window where we need to kick off the next readahead, or because the page is simply not present in cache. - We're handling a write fault. Most applications don't do I/O by writes to shared mmaps for very good reasons, but some do, and it'd be nice to not make that slow unnecessarily. - We're doing a COW of a private mapping (both PTE already present and PTE not-present). These are two different codepaths and I handle both of them in this patch set. There is no support in this patch set for drivers to mark themselves as being VMA lock friendly; they could implement the ->map_pages vm_operation, but if they do, they would be the first. This is probably something we want to change at some point in the future, and I've marked where to make that change in the code. There is very little performance change in the benchmarks we've run; mostly because the vast majority of page faults are handled through the other paths. I still think this patch series is useful for workloads that may take these paths more often, and just for cleaning up the fault path in general (it's now clearer why we have to retry in these cases). Matthew Wilcox (Oracle) (6): mm: make lock_folio_maybe_drop_mmap() VMA lock aware mm: call wp_page_copy() under the VMA lock mm: handle shared faults under the VMA lock mm: handle COW faults under the VMA lock mm: handle read faults under the VMA lock mm: handle write faults to RO pages under the VMA lock mm/filemap.c | 13 ++++---- mm/memory.c | 93 ++++++++++++++++++++++++++++++++-------------------- 2 files changed, 65 insertions(+), 41 deletions(-) -- 2.25.1

2 7

[PATCH OLK-6.6] tcp/dccp: Add another way to allocate local ports in connect()
by Zhengchao Shao 29 Nov '23

29 Nov '23

From: Lu Wei <luwei32(a)huawei.com> hulk inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I8KBYH -------------------------------- Commit 07f4c90062f8 ("tcp/dccp: try to not exhaust ip_local_port_range in connect()") allocates even ports for connect() first while leaving odd ports for bind() and this works well in busy servers. But this strategy causes severe performance degradation in busy clients. when a client has used more than half of the local ports setted in proc/sys/net/ipv4/ip_local_port_range, if this client trys to connect to a server again, the connect time increases rapidly since it will traverse all the even ports though they are exhausted. So this path provides another strategy by introducing a system option: local_port_allocation. If it is a busy client, users should set it to 1 to use sequential allocation while it should be set to 0 in other situations. Its default value is 0. Signed-off-by: Lu Wei <luwei32(a)huawei.com> Signed-off-by: Liu Jian <liujian56(a)huawei.com> Conflicts: include/net/tcp.h net/ipv4/inet_hashtables.c net/ipv4/sysctl_net_ipv4.c Signed-off-by: Zhengchao Shao <shaozhengchao(a)huawei.com> --- include/net/tcp.h | 1 + net/ipv4/inet_hashtables.c | 11 ++++++++--- net/ipv4/sysctl_net_ipv4.c | 8 ++++++++ 3 files changed, 17 insertions(+), 3 deletions(-) diff --git a/include/net/tcp.h b/include/net/tcp.h index 0239e815edf7..e9d387fffe22 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -258,6 +258,7 @@ DECLARE_PER_CPU(int, tcp_memory_per_cpu_fw_alloc); extern struct percpu_counter tcp_sockets_allocated; extern unsigned long tcp_memory_pressure; +extern int sysctl_local_port_allocation; /* optimized version of sk_under_memory_pressure() for TCP sockets */ static inline bool tcp_under_memory_pressure(const struct sock *sk) diff --git a/net/ipv4/inet_hashtables.c b/net/ipv4/inet_hashtables.c index 598c1b114d2c..919f0f869118 100644 --- a/net/ipv4/inet_hashtables.c +++ b/net/ipv4/inet_hashtables.c @@ -1011,7 +1011,7 @@ int __inet_hash_connect(struct inet_timewait_death_row *death_row, struct inet_bind_bucket *tb; bool tb_created = false; u32 remaining, offset; - int ret, i, low, high; + int ret, i, low, high, span_size; int l3mdev; u32 index; @@ -1021,6 +1021,11 @@ int __inet_hash_connect(struct inet_timewait_death_row *death_row, local_bh_enable(); return ret; } + /* local_port_allocation 0 means even and odd port allocation strategy + * will be applied, so span size is 2; otherwise sequential allocation + * will be used and span size is 1. Default value is 0. + */ + span_size = sysctl_local_port_allocation ? 1 : 2; l3mdev = inet_sk_bound_l3mdev(sk); @@ -1043,7 +1048,7 @@ int __inet_hash_connect(struct inet_timewait_death_row *death_row, offset &= ~1U; other_parity_scan: port = low + offset; - for (i = 0; i < remaining; i += 2, port += 2) { + for (i = 0; i < remaining; i += span_size, port += span_size) { if (unlikely(port >= high)) port -= remaining; if (inet_is_local_reserved_port(net, port)) @@ -1084,7 +1089,7 @@ int __inet_hash_connect(struct inet_timewait_death_row *death_row, } offset++; - if ((offset & 1) && remaining > 1) + if ((offset & 1) && remaining > 1 && span_size == 2) goto other_parity_scan; return -EADDRNOTAVAIL; diff --git a/net/ipv4/sysctl_net_ipv4.c b/net/ipv4/sysctl_net_ipv4.c index 6ac890b4073f..b17eb28a9690 100644 --- a/net/ipv4/sysctl_net_ipv4.c +++ b/net/ipv4/sysctl_net_ipv4.c @@ -39,6 +39,7 @@ static unsigned long ip_ping_group_range_min[] = { 0, 0 }; static unsigned long ip_ping_group_range_max[] = { GID_T_MAX, GID_T_MAX }; static u32 u32_max_div_HZ = UINT_MAX / HZ; static int one_day_secs = 24 * 3600; +int sysctl_local_port_allocation; static u32 fib_multipath_hash_fields_all_mask __maybe_unused = FIB_MULTIPATH_HASH_FIELD_ALL_MASK; static unsigned int tcp_child_ehash_entries_max = 16 * 1024 * 1024; @@ -579,6 +580,13 @@ static struct ctl_table ipv4_table[] = { .extra1 = &sysctl_fib_sync_mem_min, .extra2 = &sysctl_fib_sync_mem_max, }, + { + .procname = "local_port_allocation", + .data = &sysctl_local_port_allocation, + .maxlen = sizeof(int), + .mode = 0644, + .proc_handler = proc_dointvec, + }, { } }; -- 2.34.1

2 1

[PACH OLK-5.10 v5 00/20] psi irq in cgroupv1 and psi fine grained
by Lu Jialin 29 Nov '23

29 Nov '23

Chen Wandun (3): mm: disable psi cgroup v1 by default mm: add config isolation for psi under cgroup v1 psi: dont alloc memory for psi by default Chengming Zhou (3): sched/psi: Fix periodic aggregation shut off sched/psi: Optimize task switch inside shared cgroups again sched/psi: Add PSI_IRQ to track IRQ/SOFTIRQ pressure Hao Jia (1): sched/psi: Zero the memory of struct psi_group Johannes Weiner (1): sched/psi: Remove NR_ONCPU task accounting Lu Jialin (11): psi: support irq.pressure under cgroup v1 psi: enable CONFIG_PSI_CGROUP_V1 in openeuler_defconfig psi: update psi irqtime when the irq delta is nozero memcg: split async memcg reclaim from reclaim_high psi: add struct psi_group_ext PSI: Introduce fine grained stall time collect for cgroup reclaim PSI: Introduce avgs and total calculation for cgroup reclaim PSI: Introduce pressure.stat in psi PSI: add more memory fine grained stall tracking in pressure.stat add cpu fine grained stall tracking in pressure.stat PSI: enable CONFIG_PSI_FINE_GRAINED in openeuler_defconfig Suren Baghdasaryan (1): psi: Fix "defined but not used" warnings when CONFIG_PROC_FS=n Documentation/admin-guide/cgroup-v2.rst | 6 + arch/arm64/configs/openeuler_defconfig | 2 + arch/x86/configs/openeuler_defconfig | 2 + block/blk-cgroup.c | 2 +- block/blk-core.c | 2 +- include/linux/cgroup-defs.h | 6 +- include/linux/cgroup.h | 2 +- include/linux/psi.h | 9 +- include/linux/psi_types.h | 99 +++- include/linux/sched.h | 4 + init/Kconfig | 20 + kernel/cgroup/cgroup.c | 64 ++- kernel/sched/core.c | 2 + kernel/sched/cpuacct.c | 6 +- kernel/sched/fair.c | 6 - kernel/sched/psi.c | 691 ++++++++++++++++++++---- kernel/sched/stats.h | 9 + mm/compaction.c | 2 +- mm/filemap.c | 4 +- mm/memcontrol.c | 92 ++-- mm/page_alloc.c | 6 + mm/page_io.c | 3 + mm/vmscan.c | 5 +- 23 files changed, 873 insertions(+), 171 deletions(-) -- 2.34.1

1 20

[PATCH OLK-5.10] livepatch/powerpc: Fix issue that miss one layer on stack checking
by Zheng Yejian 29 Nov '23

29 Nov '23

hulk inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I7BMC9 CVE: NA -------------------------------- Suppose task 'T' is running in function 'B', and user is doing livepatch for function 'A', and function 'A' is calling 'B'. Then task 'T' will be interrupted by stop_machine, and suppose that 'B' has not saved the link register (which point to address in 'A') into stack memory, and data in link register position may be invalid, then 'A' would be missed on stack checking. To find a solution, we list following cases (suppose calling chain: P -> A -> B, A runs `bl B`, 'nip' means instruction pointer found in interrupt frame, 'lr' means link register found in interrupt frame, 'frame->pc' means link register saved in first stack frame): 1. 'nip' is in 'A', `bl B` is not executed, then: 'lr' is in 'P', 'frame->pc' should be ignored; 2. 'nip' is in some ppc64_stub or ppc32_plt which also not save link register, then: 'lr' is in 'A', 'frame->pc' should be ignored; 3. 'nip' is in 'B', but 'B' has not saved link register, then: 'lr' is in 'A', 'frame->pc' should be ignored; 4. 'nip' is in 'B', 'B' has saved link register, then: 'lr' is in 'A', 'frame->pc' is also in 'A'; 5. 'nip' is in 'A', 'B' has returned but its stack pointer is not moved, then 'lr' is in 'A', 'frame->pc' is also in 'A'; 6. 'nip' is in 'A', 'B' has returned and its stack pointer is moved, then 'lr' is in 'A', 'frame->pc' is in 'P'. As a conclusion, we need to: 1. check 'nip' or 'lr' or both if they are not in same function; 2. ignore 'frame->pc' if 'nip' and 'lr' are not in same function. Signed-off-by: Zheng Yejian <zhengyejian1(a)huawei.com> --- arch/powerpc/include/asm/livepatch.h | 11 ++++++++++ arch/powerpc/kernel/livepatch.c | 28 +++++++++++++++++++++++-- arch/powerpc/kernel/livepatch_32.c | 31 +++++++++++++++++++--------- arch/powerpc/kernel/livepatch_64.c | 31 +++++++++++++++++++--------- 4 files changed, 79 insertions(+), 22 deletions(-) diff --git a/arch/powerpc/include/asm/livepatch.h b/arch/powerpc/include/asm/livepatch.h index ae674ea59ab3..285602e637f1 100644 --- a/arch/powerpc/include/asm/livepatch.h +++ b/arch/powerpc/include/asm/livepatch.h @@ -121,9 +121,20 @@ struct arch_klp_data { #define KLP_MAX_REPLACE_SIZE sizeof_field(struct arch_klp_data, old_insns) struct stackframe { + /* stack frame to be unwinded */ unsigned long sp; + /* link register saved in last stack frame */ unsigned long pc; + /* instruction register saved in pt_regs */ unsigned long nip; + /* link register saved in pt_regs */ + unsigned long link; + /* stack frame pointer (r1 register) saved in pt_regs */ + unsigned long sfp; + /* check if nip and link are in same function */ + unsigned int nip_link_in_same_func; + /* check if it is top frame before interrupt */ + unsigned int is_top_frame; }; #ifdef PPC64_ELF_ABI_v1 diff --git a/arch/powerpc/kernel/livepatch.c b/arch/powerpc/kernel/livepatch.c index d568e8c8b16b..5ba38c2c7c5c 100644 --- a/arch/powerpc/kernel/livepatch.c +++ b/arch/powerpc/kernel/livepatch.c @@ -23,6 +23,7 @@ #include <linux/module.h> #include <linux/ftrace.h> #include <linux/livepatch.h> +#include <linux/kallsyms.h> #include <asm/probes.h> #include <asm/livepatch.h> #include <asm/code-patching.h> @@ -67,6 +68,22 @@ int klp_brk_handler(struct pt_regs *regs) return 1; } +static int check_addr_in_same_func(unsigned long addr1, unsigned long addr2) +{ + unsigned long size = 0; + unsigned long offset = 0; + unsigned long start; + + if (addr1 == 0 || addr2 == 0) + return 0; + if (addr1 == addr2) + return 1; + if (!kallsyms_lookup_size_offset(addr1, &size, &offset)) + return 0; + start = addr1 - offset; + return (addr2 >= start) && (addr2 - start < size); +} + int klp_unwind_frame(struct task_struct *tsk, struct stackframe *frame) { unsigned long *stack; @@ -79,7 +96,10 @@ int klp_unwind_frame(struct task_struct *tsk, struct stackframe *frame) if (frame->nip != 0) frame->nip = 0; + if (frame->link != 0) + frame->link = 0; + frame->is_top_frame = (frame->sfp == frame->sp); stack = (unsigned long *)frame->sp; /* @@ -94,10 +114,14 @@ int klp_unwind_frame(struct task_struct *tsk, struct stackframe *frame) struct pt_regs *regs = (struct pt_regs *) (frame->sp + STACK_FRAME_OVERHEAD); frame->nip = regs->nip; - pr_debug("--- interrupt: task = %d/%s, trap %lx at NIP=x%lx/%pS, LR=0x%lx/%pS\n", + frame->link = regs->link; + frame->sfp = regs->gpr[PT_R1]; + frame->nip_link_in_same_func = check_addr_in_same_func(frame->nip, frame->link); + pr_debug("--- interrupt: task = %d/%s, trap %lx at NIP=0x%lx/%pS, LR=0x%lx/%pS, SFP=0x%lx, nip_link_in_same_func=%u\n", tsk->pid, tsk->comm, regs->trap, regs->nip, (void *)regs->nip, - regs->link, (void *)regs->link); + regs->link, (void *)regs->link, + frame->sfp, frame->nip_link_in_same_func); } frame->sp = stack[0]; diff --git a/arch/powerpc/kernel/livepatch_32.c b/arch/powerpc/kernel/livepatch_32.c index 3fe4f3c5790b..134b08e12e74 100644 --- a/arch/powerpc/kernel/livepatch_32.c +++ b/arch/powerpc/kernel/livepatch_32.c @@ -272,13 +272,21 @@ static int klp_check_jump_func(struct stackframe *frame, void *data) struct walk_stackframe_args *args = data; struct klp_func_list *check_funcs = args->check_funcs; - /* check the PC first */ - if (!check_func_list(check_funcs, &args->ret, frame->pc)) - return args->ret; - /* check NIP when the exception stack switching */ if (frame->nip && !check_func_list(check_funcs, &args->ret, frame->nip)) return args->ret; + if (frame->link && !frame->nip_link_in_same_func && + !check_func_list(check_funcs, &args->ret, frame->link)) + return args->ret; + /* + * There are two cases that frame->pc is reliable: + * 1. frame->pc is not in top frame before interrupt; + * 2. nip and link are in same function; + */ + if (!frame->is_top_frame || frame->nip_link_in_same_func) { + if (!check_func_list(check_funcs, &args->ret, frame->pc)) + return args->ret; + } return 0; } @@ -298,10 +306,11 @@ static int do_check_calltrace(struct walk_stackframe_args *args, int (*fn)(struct stackframe *, void *)) { struct task_struct *g, *t; - struct stackframe frame; unsigned long *stack; for_each_process_thread(g, t) { + struct stackframe frame = { 0 }; + if (t == current) { /* * Handle the current carefully on each CPUs, we shouldn't @@ -329,7 +338,6 @@ static int do_check_calltrace(struct walk_stackframe_args *args, frame.sp = (unsigned long)stack; frame.pc = stack[STACK_FRAME_LR_SAVE]; - frame.nip = 0; klp_walk_stackframe(&frame, fn, t, args); if (args->ret) { pr_info("PID: %d Comm: %.20s\n", t->pid, t->comm); @@ -368,13 +376,16 @@ static int check_module_calltrace(struct stackframe *frame, void *data) { struct walk_stackframe_args *args = data; - /* check the PC first */ - if (within_module_core(frame->pc, args->mod)) - goto err_out; - /* check NIP when the exception stack switching */ if (frame->nip && within_module_core(frame->nip, args->mod)) goto err_out; + if (frame->link && !frame->nip_link_in_same_func && + within_module_core(frame->link, args->mod)) + goto err_out; + if (!frame->is_top_frame || frame->nip_link_in_same_func) { + if (within_module_core(frame->pc, args->mod)) + goto err_out; + } return 0; diff --git a/arch/powerpc/kernel/livepatch_64.c b/arch/powerpc/kernel/livepatch_64.c index a2ec7c8c1bad..b33839b5916a 100644 --- a/arch/powerpc/kernel/livepatch_64.c +++ b/arch/powerpc/kernel/livepatch_64.c @@ -273,13 +273,21 @@ static int klp_check_jump_func(struct stackframe *frame, void *data) struct walk_stackframe_args *args = data; struct klp_func_list *check_funcs = args->check_funcs; - /* check the PC first */ - if (!check_func_list(check_funcs, &args->ret, frame->pc)) - return args->ret; - /* check NIP when the exception stack switching */ if (frame->nip && !check_func_list(check_funcs, &args->ret, frame->nip)) return args->ret; + if (frame->link && !frame->nip_link_in_same_func && + !check_func_list(check_funcs, &args->ret, frame->link)) + return args->ret; + /* + * There are two cases that frame->pc is reliable: + * 1. frame->pc is not in top frame before interrupt; + * 2. nip and link are in same function; + */ + if (!frame->is_top_frame || frame->nip_link_in_same_func) { + if (!check_func_list(check_funcs, &args->ret, frame->pc)) + return args->ret; + } return 0; } @@ -299,10 +307,11 @@ static int do_check_calltrace(struct walk_stackframe_args *args, int (*fn)(struct stackframe *, void *)) { struct task_struct *g, *t; - struct stackframe frame; unsigned long *stack; for_each_process_thread(g, t) { + struct stackframe frame = { 0 }; + if (t == current) { /* * Handle the current carefully on each CPUs, @@ -332,7 +341,6 @@ static int do_check_calltrace(struct walk_stackframe_args *args, frame.sp = (unsigned long)stack; frame.pc = stack[STACK_FRAME_LR_SAVE]; - frame.nip = 0; klp_walk_stackframe(&frame, fn, t, args); if (args->ret) { pr_debug("%s FAILED when %s\n", __func__, @@ -373,13 +381,16 @@ static int check_module_calltrace(struct stackframe *frame, void *data) { struct walk_stackframe_args *args = data; - /* check the PC first */ - if (within_module_core(frame->pc, args->mod)) - goto err_out; - /* check NIP when the exception stack switching */ if (frame->nip && within_module_core(frame->nip, args->mod)) goto err_out; + if (frame->link && !frame->nip_link_in_same_func && + within_module_core(frame->link, args->mod)) + goto err_out; + if (!frame->is_top_frame || frame->nip_link_in_same_func) { + if (within_module_core(frame->pc, args->mod)) + goto err_out; + } return 0; -- 2.25.1

2 1

[PACH OLK-5.10 v5 00/20] psi irq in cgroupv1 and psi fine grained
by Lu Jialin 29 Nov '23

29 Nov '23

Chen Wandun (3): mm: disable psi cgroup v1 by default mm: add config isolation for psi under cgroup v1 psi: dont alloc memory for psi by default Chengming Zhou (3): sched/psi: Fix periodic aggregation shut off sched/psi: Optimize task switch inside shared cgroups again sched/psi: Add PSI_IRQ to track IRQ/SOFTIRQ pressure Hao Jia (1): sched/psi: Zero the memory of struct psi_group Johannes Weiner (1): sched/psi: Remove NR_ONCPU task accounting Lu Jialin (11): psi: support irq.pressure under cgroup v1 psi: enable CONFIG_PSI_CGROUP_V1 in openeuler_defconfig psi: update psi irqtime when the irq delta is nozero memcg: split async memcg reclaim from reclaim_high psi: add struct psi_group_ext PSI: Introduce fine grained stall time collect for cgroup reclaim PSI: Introduce avgs and total calculation for cgroup reclaim PSI: Introduce pressure.stat in psi PSI: add more memory fine grained stall tracking in pressure.stat add cpu fine grained stall tracking in pressure.stat PSI: enable CONFIG_PSI_FINE_GRAINED in openeuler_defconfig Suren Baghdasaryan (1): psi: Fix "defined but not used" warnings when CONFIG_PROC_FS=n Documentation/admin-guide/cgroup-v2.rst | 6 + arch/arm64/configs/openeuler_defconfig | 2 + arch/x86/configs/openeuler_defconfig | 2 + block/blk-cgroup.c | 2 +- block/blk-core.c | 2 +- include/linux/cgroup-defs.h | 6 +- include/linux/cgroup.h | 2 +- include/linux/psi.h | 9 +- include/linux/psi_types.h | 99 +++- include/linux/sched.h | 4 + init/Kconfig | 20 + kernel/cgroup/cgroup.c | 64 ++- kernel/sched/core.c | 2 + kernel/sched/cpuacct.c | 6 +- kernel/sched/fair.c | 6 - kernel/sched/psi.c | 691 ++++++++++++++++++++---- kernel/sched/stats.h | 9 + mm/compaction.c | 2 +- mm/filemap.c | 4 +- mm/memcontrol.c | 92 ++-- mm/page_alloc.c | 6 + mm/page_io.c | 3 + mm/vmscan.c | 5 +- 23 files changed, 873 insertions(+), 171 deletions(-) -- 2.34.1

1 20

[PATCH v2 OLK-5.10 0/3] support page eject
by Wupeng Ma 29 Nov '23

29 Nov '23

From: Ma Wupeng <mawupeng1(a)huawei.com> Page ejection is achieved by soft_offline_page and unpoison_memory. A linked list is maintained to log the pfns which are offlined by this module. Only the pfns present in the list is allowed to go online. The follow sysfs is introduced to online/offline page via this module, since there is a internal linked list to record all offlined pages, remove_page is introduce to remove this page from list. - /sys/kernel/page_eject/online_page - /sys/kernel/page_eject/offline_page - /sys/kernel/page_eject/remove_page Changelog since v1: - remove mce related patches temporarily. Ma Wupeng (3): mm/hwpoison: Export symbol soft_offline_page mm: page_eject: Introuduce page ejection config: update defconfig for PAGE_EJECT arch/arm64/configs/openeuler_defconfig | 1 + arch/x86/configs/openeuler_defconfig | 1 + drivers/ras/Kconfig | 10 + drivers/ras/Makefile | 1 + drivers/ras/page_eject.c | 248 +++++++++++++++++++++++++ mm/memory-failure.c | 1 + 6 files changed, 262 insertions(+) create mode 100644 drivers/ras/page_eject.c -- 2.25.1

2 4

2024

2023

2022

2021

2020

2019

Kernel