- Kernel - mailweb.openeuler.org

[PATCH OLK-6.6 V2 00/12] infrastructure for scheduler bpf
by Guan Jing 08 Jan '24

08 Jan '24

infrastructure for scheduler bpf Guan Jing (12): sched: programmable: Introduce bpf sched sched: programmable: Add a tag for the task sched: programmable: Add a tag for the task group sched: programmable: Add user interface of task group tag sched: programmable: Add user interface of task tag sched: basic infrastructure for scheduler bpf sched: introduce bpf_sched_enable() libbpf: add support for scheduler bpf programs bpftool: recognize scheduler programs sched: Add helper functions to get cpu statistics sched: programmable: Add hook in select_task_rq_fair() sched: programmable: Add hook in can_migrate_task() arch/arm64/configs/openeuler_defconfig | 1 + arch/x86/configs/openeuler_defconfig | 1 + fs/proc/base.c | 64 ++++++++++++++ include/linux/bpf_sched.h | 50 +++++++++++ include/linux/bpf_types.h | 4 + include/linux/sched.h | 39 +++++++++ include/linux/sched_hook_defs.h | 4 + include/uapi/linux/bpf.h | 9 ++ init/init_task.c | 3 + kernel/bpf/Kconfig | 12 +++ kernel/bpf/btf.c | 1 + kernel/bpf/syscall.c | 27 ++++++ kernel/bpf/trampoline.c | 1 + kernel/bpf/verifier.c | 11 ++- kernel/sched/bpf_sched.c | 102 +++++++++++++++++++++ kernel/sched/build_utility.c | 4 + kernel/sched/core.c | 117 +++++++++++++++++++++++++ kernel/sched/fair.c | 46 ++++++++++ kernel/sched/sched.h | 10 +++ scripts/bpf_doc.py | 4 + tools/include/uapi/linux/bpf.h | 9 ++ tools/lib/bpf/bpf.c | 1 + tools/lib/bpf/libbpf.c | 23 ++++- tools/lib/bpf/libbpf.h | 2 + tools/lib/bpf/libbpf.map | 1 + 25 files changed, 544 insertions(+), 2 deletions(-) create mode 100644 include/linux/bpf_sched.h create mode 100644 include/linux/sched_hook_defs.h create mode 100644 kernel/sched/bpf_sched.c -- 2.34.1

1 12

[PATCH OLK-5.10 0/4] ext4: fix some ext4_lblk_t overflow issues
by Baokun Li 08 Jan '24

08 Jan '24

Baokun Li (4): ext4: add two helper functions extent_logical_end() and pa_logical_end() ext4: fix BUG in ext4_mb_new_inode_pa() due to overflow ext4: avoid overlapping preallocations due to overflow ext4: prevent the normalized size from exceeding EXT_MAX_BLOCKS fs/ext4/mballoc.c | 55 ++++++++++++++++++++++------------------------- fs/ext4/mballoc.h | 14 ++++++++++++ 2 files changed, 40 insertions(+), 29 deletions(-) -- 2.31.1

1 4

[PATCH OLK-6.6 v2 0/2] ext4: fix slab-out-of-bounds in ext4_find_extent()
by Baokun Li 08 Jan '24

08 Jan '24

yangerkun (2): ext4: avoid recheck extent for EXT4_EX_FORCE_CACHE ext4: check magic even the extent block bh is verified fs/ext4/extents.c | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-) -- 2.31.1

2 3

[PATCH OLK-6.6 v2] aio: add timeout validity check for io_[p]getevents
by Baokun Li 08 Jan '24

08 Jan '24

From: "zhangyi (F)" <yi.zhang(a)huawei.com> maillist inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I8U48M CVE: NA Reference: https://lore.kernel.org/lkml/1564451504-27906-1-git-send-email-yi.zhang@hua… --------------------------- io_[p]getevents syscall should return -EINVAL if timeout is out of range, add this validity check. Signed-off-by: zhangyi (F) <yi.zhang(a)huawei.com> Reviewed-by: Jeff Moyer <jmoyer(a)redhat.com> Cc: Jeff Moyer <jmoyer(a)redhat.com> Cc: Arnd Bergmann <arnd(a)arndb.de> Cc: Deepa Dinamani <deepa.kernel(a)gmail.com> Signed-off-by: Baokun Li <libaokun1(a)huawei.com> --- fs/aio.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/fs/aio.c b/fs/aio.c index f8589caef9c1..73555729bc36 100644 --- a/fs/aio.c +++ b/fs/aio.c @@ -2170,10 +2170,17 @@ static long do_io_getevents(aio_context_t ctx_id, struct io_event __user *events, struct timespec64 *ts) { - ktime_t until = ts ? timespec64_to_ktime(*ts) : KTIME_MAX; - struct kioctx *ioctx = lookup_ioctx(ctx_id); + ktime_t until = KTIME_MAX; + struct kioctx *ioctx = NULL; long ret = -EINVAL; + if (ts) { + if (!timespec64_valid(ts)) + return ret; + until = timespec64_to_ktime(*ts); + } + + ioctx = lookup_ioctx(ctx_id); if (likely(ioctx)) { if (likely(min_nr <= nr && min_nr >= 0)) ret = read_events(ioctx, min_nr, nr, events, until); -- 2.31.1

2 1

[PATCH OLK-6.6 00/10] Backport etmem feature to OLK 6.6
by Yuchen Tang 08 Jan '24

08 Jan '24

v2 -> v3 - fix git message style - fix etmem scan and swap config v1 -> v2 - make independent patches to export symbols we need - make independent patches for etmem scan and swap config - fix indentation and code style problems Yuchen Tang (10): etmem: a tiered memory manager etmem: etmem scan, the page hotness tracker mm: Export symbol __pud_trans_huge_lock mm: Export symbol walk_page_range etmem: add etmem scan feature to openEuler mm: Export symbol reclaim_pages etmem: add etmem swap feature to openEuler etmem: add original kernel swap enabled options etmem: add swapcache reclaim to etmem etmem: enable etmem configurations arch/arm64/configs/openeuler_defconfig | 3 + arch/x86/configs/openeuler_defconfig | 3 + fs/proc/Makefile | 3 + fs/proc/base.c | 8 + fs/proc/etmem_proc.c | 216 ++++ fs/proc/etmem_scan.c | 1383 ++++++++++++++++++++++++ fs/proc/etmem_scan.h | 149 +++ fs/proc/etmem_swap.c | 283 +++++ fs/proc/internal.h | 4 + include/linux/etmem.h | 75 ++ include/linux/mm.h | 4 + include/linux/mm_types.h | 5 + include/linux/swap.h | 1 + include/uapi/asm-generic/mman-common.h | 4 + mm/Kconfig | 33 + mm/Makefile | 1 + mm/etmem.c | 384 +++++++ mm/huge_memory.c | 1 + mm/internal.h | 1 - mm/madvise.c | 16 + mm/pagewalk.c | 1 + mm/swap_state.c | 4 + mm/vmscan.c | 16 + virt/kvm/kvm_main.c | 8 + 24 files changed, 2605 insertions(+), 1 deletion(-) create mode 100644 fs/proc/etmem_proc.c create mode 100644 fs/proc/etmem_scan.c create mode 100644 fs/proc/etmem_scan.h create mode 100644 fs/proc/etmem_swap.c create mode 100644 include/linux/etmem.h create mode 100644 mm/etmem.c -- 2.33.0

1 10

[PATCH OLK-6.6 v2] pipe: Fix endless sleep problem due to the out-of-order
by Baokun Li 08 Jan '24

08 Jan '24

From: yangerkun <yangerkun(a)huawei.com> hulk inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I8U3TT --------------------------- Thers is a out-of-order access problem which would cause endless sleep when we use pipe with epoll. The story is following, we assume the ring size is 2, the ring head is 1, the ring tail is 0, task0 is write task, task1 is read task, task2 is write task. Task0 Task1 Task2 epoll_ctl(fd, EPOLL_CTL_ADD, ...) pipe_poll() poll_wait() tail = READ_ONCE(pipe->tail); // Re-order and get tail=0 pipe_read tail++ //tail=1 pipe_write head++ //head=2 head = READ_ONCE(pipe->head); // head = 2 check ring is full by head - tail Task0 get head = 2 and tail = 0, so it mistake that the pipe ring is full, then task0 is not add into ready list. If the ring is not full anymore, task0 would not be woken up forever. The reason of this problem is that we got inconsistent head/tail value of the pipe ring, so we fix the problem by getting them protected. Signed-off-by: yangerkun <yangerkun(a)huawei.com> Signed-off-by: Baokun Li <libaokun1(a)huawei.com> --- fs/pipe.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/pipe.c b/fs/pipe.c index 139190165a1c..34752ba942ab 100644 --- a/fs/pipe.c +++ b/fs/pipe.c @@ -672,8 +672,10 @@ pipe_poll(struct file *filp, poll_table *wait) * if something changes and you got it wrong, the poll * table entry will wake you up and fix it. */ + spin_lock_irq(&pipe->rd_wait.lock); head = READ_ONCE(pipe->head); tail = READ_ONCE(pipe->tail); + spin_unlock_irq(&pipe->rd_wait.lock); mask = 0; if (filp->f_mode & FMODE_READ) { -- 2.31.1

2 1

[PATCH openEuler-1.0-LTS] nvmet: nul-terminate the NQNs passed in the connect command
by Cheng Yu 08 Jan '24

08 Jan '24

From: Christoph Hellwig <hch(a)lst.de> mainline inclusion from mainline-v6.7-rc3 commit 1c22e0295a5eb571c27b53c7371f95699ef705ff category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/I8H4UJ CVE: CVE-2023-6121 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?… -------------------------------- The host and subsystem NQNs are passed in the connect command payload and interpreted as nul-terminated strings. Ensure they actually are nul-terminated before using them. Fixes: a07b4970f464 "nvmet: add a generic NVMe target") Reported-by: Alon Zahavi <zahavi.alon(a)gmail.com> Reviewed-by: Chaitanya Kulkarni <kch(a)nvidia.com> Signed-off-by: Christoph Hellwig <hch(a)lst.de> Signed-off-by: Keith Busch <kbusch(a)kernel.org> Signed-off-by: Cheng Yu <serein.chengyu(a)huawei.com> --- drivers/nvme/target/fabrics-cmd.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/nvme/target/fabrics-cmd.c b/drivers/nvme/target/fabrics-cmd.c index d84ae004cb85..637160deb3ac 100644 --- a/drivers/nvme/target/fabrics-cmd.c +++ b/drivers/nvme/target/fabrics-cmd.c @@ -153,6 +153,8 @@ static void nvmet_execute_admin_connect(struct nvmet_req *req) goto out; } + d->subsysnqn[NVMF_NQN_FIELD_LEN - 1] = '\0'; + d->hostnqn[NVMF_NQN_FIELD_LEN - 1] = '\0'; status = nvmet_alloc_ctrl(d->subsysnqn, d->hostnqn, req, le32_to_cpu(c->kato), &ctrl); if (status) @@ -203,6 +205,8 @@ static void nvmet_execute_io_connect(struct nvmet_req *req) goto out; } + d->subsysnqn[NVMF_NQN_FIELD_LEN - 1] = '\0'; + d->hostnqn[NVMF_NQN_FIELD_LEN - 1] = '\0'; status = nvmet_ctrl_find_get(d->subsysnqn, d->hostnqn, le16_to_cpu(d->cntlid), req, &ctrl); -- 2.25.1

2 1

[PATCH OLK-6.6 0/2] ext4: fix slab-out-of-bounds in ext4_find_extent()
by Baokun Li 08 Jan '24

08 Jan '24

yangerkun (2): ext4: avoid recheck extent for EXT4_EX_FORCE_CACHE ext4: check magic even the extent block bh is verified fs/ext4/extents.c | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-) -- 2.31.1

2 3

[PATCH openEuler-1.0-LTS] netfilter: ctnetlink: fix possible refcount leak in ctnetlink_create_conntrack()
by Dong Chenchen 08 Jan '24

08 Jan '24

From: Hangyu Hua <hbh25y(a)gmail.com> stable inclusion from stable-v4.19.275 commit 43b9a9c78e37a5532c2a9260dff9d9989f2bbb23 category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/I8SL4W CVE: CVE-2023-7192 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id… -------------------------------- [ Upstream commit ac4893980bbe79ce383daf9a0885666a30fe4c83 ] nf_ct_put() needs to be called to put the refcount got by nf_conntrack_find_get() to avoid refcount leak when nf_conntrack_hash_check_insert() fails. Fixes: 7d367e06688d ("netfilter: ctnetlink: fix soft lockup when netlink adds new entries (v2)") Signed-off-by: Hangyu Hua <hbh25y(a)gmail.com> Acked-by: Florian Westphal <fw(a)strlen.de> Signed-off-by: Pablo Neira Ayuso <pablo(a)netfilter.org> Signed-off-by: Sasha Levin <sashal(a)kernel.org> Signed-off-by: Dong Chenchen <dongchenchen2(a)huawei.com> --- net/netfilter/nf_conntrack_netlink.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/net/netfilter/nf_conntrack_netlink.c b/net/netfilter/nf_conntrack_netlink.c index 2850a638401d..58bba2e2691f 100644 --- a/net/netfilter/nf_conntrack_netlink.c +++ b/net/netfilter/nf_conntrack_netlink.c @@ -2056,12 +2056,15 @@ ctnetlink_create_conntrack(struct net *net, err = nf_conntrack_hash_check_insert(ct); if (err < 0) - goto err2; + goto err3; rcu_read_unlock(); return ct; +err3: + if (ct->master) + nf_ct_put(ct->master); err2: rcu_read_unlock(); err1: -- 2.25.1

2 1

[PATCH openEuler-1.0-LTS V1] nvmet: nul-terminate the NQNs passed in the connect command
by Cheng Yu 08 Jan '24

08 Jan '24

From: Christoph Hellwig <hch(a)lst.de> mainline inclusion from mainline-v6.7-rc3 commit 1c22e0295a5eb571c27b53c7371f95699ef705ff category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I8U4HU CVE: CVE-2023-6121 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?… -------------------------------- The host and subsystem NQNs are passed in the connect command payload and interpreted as nul-terminated strings. Ensure they actually are nul-terminated before using them. Fixes: a07b4970f464 "nvmet: add a generic NVMe target") Reported-by: Alon Zahavi <zahavi.alon(a)gmail.com> Reviewed-by: Chaitanya Kulkarni <kch(a)nvidia.com> Signed-off-by: Christoph Hellwig <hch(a)lst.de> Signed-off-by: Keith Busch <kbusch(a)kernel.org> Signed-off-by: Cheng Yu <serein.chengyu(a)huawei.com> --- drivers/nvme/target/fabrics-cmd.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/nvme/target/fabrics-cmd.c b/drivers/nvme/target/fabrics-cmd.c index d84ae004cb85..637160deb3ac 100644 --- a/drivers/nvme/target/fabrics-cmd.c +++ b/drivers/nvme/target/fabrics-cmd.c @@ -153,6 +153,8 @@ static void nvmet_execute_admin_connect(struct nvmet_req *req) goto out; } + d->subsysnqn[NVMF_NQN_FIELD_LEN - 1] = '\0'; + d->hostnqn[NVMF_NQN_FIELD_LEN - 1] = '\0'; status = nvmet_alloc_ctrl(d->subsysnqn, d->hostnqn, req, le32_to_cpu(c->kato), &ctrl); if (status) @@ -203,6 +205,8 @@ static void nvmet_execute_io_connect(struct nvmet_req *req) goto out; } + d->subsysnqn[NVMF_NQN_FIELD_LEN - 1] = '\0'; + d->hostnqn[NVMF_NQN_FIELD_LEN - 1] = '\0'; status = nvmet_ctrl_find_get(d->subsysnqn, d->hostnqn, le16_to_cpu(d->cntlid), req, &ctrl); -- 2.25.1

2 1

[PATCH OLK-6.6] cgroup: Export cgroup.stat from cgroupv2 to cgroupv1
by Lu Jialin 07 Jan '24

07 Jan '24

hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I8U6EM -------------------------------- Export cgroup.stat feature from cgroupv2 to cgroupv1. Therefore, user can get dying descendants cgroup number and descendants number in cgroupv1. --- arch/arm64/configs/openeuler_defconfig | 1 + arch/x86/configs/openeuler_defconfig | 1 + init/Kconfig | 5 +++++ kernel/cgroup/cgroup-internal.h | 1 + kernel/cgroup/cgroup-v1.c | 6 ++++++ kernel/cgroup/cgroup.c | 2 +- 6 files changed, 15 insertions(+), 1 deletion(-) diff --git a/arch/arm64/configs/openeuler_defconfig b/arch/arm64/configs/openeuler_defconfig index 9a053bc885dd..1fc527e975be 100644 --- a/arch/arm64/configs/openeuler_defconfig +++ b/arch/arm64/configs/openeuler_defconfig @@ -188,6 +188,7 @@ CONFIG_CGROUP_BPF=y # CONFIG_CGROUP_DEBUG is not set CONFIG_SOCK_CGROUP_DATA=y CONFIG_CGROUP_V1_KILL=y +CONFIG_CGROUP_V1_STAT=y CONFIG_CGROUP_FILES=y CONFIG_NAMESPACES=y CONFIG_UTS_NS=y diff --git a/arch/x86/configs/openeuler_defconfig b/arch/x86/configs/openeuler_defconfig index 4bf011ed6307..0980126766cc 100644 --- a/arch/x86/configs/openeuler_defconfig +++ b/arch/x86/configs/openeuler_defconfig @@ -209,6 +209,7 @@ CONFIG_CGROUP_BPF=y # CONFIG_CGROUP_DEBUG is not set CONFIG_SOCK_CGROUP_DATA=y CONFIG_CGROUP_V1_KILL=y +CONFIG_CGROUP_V1_STAT=y CONFIG_CGROUP_FILES=y CONFIG_NAMESPACES=y CONFIG_UTS_NS=y diff --git a/init/Kconfig b/init/Kconfig index 869eea4108d0..fad38e359428 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -1254,6 +1254,11 @@ config CGROUP_V1_KILL default n depends on CGROUPS +config CGROUP_V1_STAT + bool "Getting Cgroup Stat in Cgroup V1" + default n + depends on CGROUPS + config CGROUP_FILES bool "Files Resource Controller for Control Groups" select PAGE_COUNTER diff --git a/kernel/cgroup/cgroup-internal.h b/kernel/cgroup/cgroup-internal.h index d5a197d4b0ec..96a9bd2c26f0 100644 --- a/kernel/cgroup/cgroup-internal.h +++ b/kernel/cgroup/cgroup-internal.h @@ -266,6 +266,7 @@ int cgroup_task_count(const struct cgroup *cgrp); ssize_t cgroup_kill_write(struct kernfs_open_file *of, char *buf, size_t nbytes, loff_t off); +int cgroup_stat_show(struct seq_file *seq, void *v); /* * rstat.c diff --git a/kernel/cgroup/cgroup-v1.c b/kernel/cgroup/cgroup-v1.c index 622398d492b6..ece2f4e17197 100644 --- a/kernel/cgroup/cgroup-v1.c +++ b/kernel/cgroup/cgroup-v1.c @@ -666,6 +666,12 @@ struct cftype cgroup1_base_files[] = { .flags = CFTYPE_NOT_ON_ROOT, .write = cgroup_kill_write, }, +#endif +#ifdef CONFIG_CGROUP_V1_STAT + { + .name = "cgroup.stat", + .seq_show = cgroup_stat_show, + }, #endif { } /* terminate */ }; diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c index cfae217e6e7f..36707df23cfd 100644 --- a/kernel/cgroup/cgroup.c +++ b/kernel/cgroup/cgroup.c @@ -3657,7 +3657,7 @@ static int cgroup_events_show(struct seq_file *seq, void *v) return 0; } -static int cgroup_stat_show(struct seq_file *seq, void *v) +int cgroup_stat_show(struct seq_file *seq, void *v) { struct cgroup *cgroup = seq_css(seq)->cgroup; -- 2.34.1

2 1

[PATCH hulk-4.19 V1] nvmet: nul-terminate the NQNs passed in the connect command
by Cheng Yu 06 Jan '24

06 Jan '24

From: Christoph Hellwig <hch(a)lst.de> mainline inclusion from mainline-v6.7-rc3 commit 1c22e0295a5eb571c27b53c7371f95699ef705ff category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I8U4HU CVE: CVE-2023-6121 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?… -------------------------------- The host and subsystem NQNs are passed in the connect command payload and interpreted as nul-terminated strings. Ensure they actually are nul-terminated before using them. Fixes: a07b4970f464 "nvmet: add a generic NVMe target") Reported-by: Alon Zahavi <zahavi.alon(a)gmail.com> Reviewed-by: Chaitanya Kulkarni <kch(a)nvidia.com> Signed-off-by: Christoph Hellwig <hch(a)lst.de> Signed-off-by: Keith Busch <kbusch(a)kernel.org> Signed-off-by: Cheng Yu <serein.chengyu(a)huawei.com> --- drivers/nvme/target/fabrics-cmd.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/nvme/target/fabrics-cmd.c b/drivers/nvme/target/fabrics-cmd.c index d84ae004cb85..637160deb3ac 100644 --- a/drivers/nvme/target/fabrics-cmd.c +++ b/drivers/nvme/target/fabrics-cmd.c @@ -153,6 +153,8 @@ static void nvmet_execute_admin_connect(struct nvmet_req *req) goto out; } + d->subsysnqn[NVMF_NQN_FIELD_LEN - 1] = '\0'; + d->hostnqn[NVMF_NQN_FIELD_LEN - 1] = '\0'; status = nvmet_alloc_ctrl(d->subsysnqn, d->hostnqn, req, le32_to_cpu(c->kato), &ctrl); if (status) @@ -203,6 +205,8 @@ static void nvmet_execute_io_connect(struct nvmet_req *req) goto out; } + d->subsysnqn[NVMF_NQN_FIELD_LEN - 1] = '\0'; + d->hostnqn[NVMF_NQN_FIELD_LEN - 1] = '\0'; status = nvmet_ctrl_find_get(d->subsysnqn, d->hostnqn, le16_to_cpu(d->cntlid), req, &ctrl); -- 2.25.1

1 0

[PATCH OLK-6.6 0/4] xfs: fix block space problems
by Long Li 06 Jan '24

06 Jan '24

*** BLURB HERE *** This patch set fix block space problems Gao Xiang (1): xfs: account extra freespace btree splits for multiple allocations Guo Xuenan (1): xfs: set minleft correctly for randomly sparse inode allocations yangerkun (2): xfs: fix xfs shutdown since we reserve more blocks in agfl fixup xfs: longest free extent no need consider postalloc fs/xfs/libxfs/xfs_alloc.c | 49 ++++++++++++++++++++++++++++++++------ fs/xfs/libxfs/xfs_alloc.h | 1 + fs/xfs/libxfs/xfs_bmap.c | 1 + fs/xfs/libxfs/xfs_ialloc.c | 3 +++ fs/xfs/xfs_mount.c | 9 +++++++ 5 files changed, 56 insertions(+), 7 deletions(-) -- 2.31.1

2 5

[PATCH hulk-4.19 V1] nvmet: nul-terminate the NQNs passed in the connect command
by Cheng Yu 06 Jan '24

06 Jan '24

From: Christoph Hellwig <hch(a)lst.de> mainline inclusion from mainline-v6.7-rc3 commit 1c22e0295a5eb571c27b53c7371f95699ef705ff category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I8U4HU CVE: CVE-2023-6121 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?… -------------------------------- The host and subsystem NQNs are passed in the connect command payload and interpreted as nul-terminated strings. Ensure they actually are nul-terminated before using them. Fixes: a07b4970f464 "nvmet: add a generic NVMe target") Reported-by: Alon Zahavi <zahavi.alon(a)gmail.com> Reviewed-by: Chaitanya Kulkarni <kch(a)nvidia.com> Signed-off-by: Christoph Hellwig <hch(a)lst.de> Signed-off-by: Keith Busch <kbusch(a)kernel.org> Signed-off-by: Cheng Yu <serein.chengyu(a)huawei.com> --- drivers/nvme/target/fabrics-cmd.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/nvme/target/fabrics-cmd.c b/drivers/nvme/target/fabrics-cmd.c index d84ae004cb85..637160deb3ac 100644 --- a/drivers/nvme/target/fabrics-cmd.c +++ b/drivers/nvme/target/fabrics-cmd.c @@ -153,6 +153,8 @@ static void nvmet_execute_admin_connect(struct nvmet_req *req) goto out; } + d->subsysnqn[NVMF_NQN_FIELD_LEN - 1] = '\0'; + d->hostnqn[NVMF_NQN_FIELD_LEN - 1] = '\0'; status = nvmet_alloc_ctrl(d->subsysnqn, d->hostnqn, req, le32_to_cpu(c->kato), &ctrl); if (status) @@ -203,6 +205,8 @@ static void nvmet_execute_io_connect(struct nvmet_req *req) goto out; } + d->subsysnqn[NVMF_NQN_FIELD_LEN - 1] = '\0'; + d->hostnqn[NVMF_NQN_FIELD_LEN - 1] = '\0'; status = nvmet_ctrl_find_get(d->subsysnqn, d->hostnqn, le16_to_cpu(d->cntlid), req, &ctrl); -- 2.25.1

1 0

[PATCH OLK-6.6] aio: add timeout validity check for io_[p]getevents
by Baokun Li 06 Jan '24

06 Jan '24

From: "zhangyi (F)" <yi.zhang(a)huawei.com> maillist inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I8U48M CVE: NA Reference: https://lore.kernel.org/lkml/1564451504-27906-1-git-send-email-yi.zhang@hua… --------------------------- io_[p]getevents syscall should return -EINVAL if timeout is out of range, add this validity check. Signed-off-by: zhangyi (F) <yi.zhang(a)huawei.com> Reviewed-by: Jeff Moyer <jmoyer(a)redhat.com> Cc: Jeff Moyer <jmoyer(a)redhat.com> Cc: Arnd Bergmann <arnd(a)arndb.de> Cc: Deepa Dinamani <deepa.kernel(a)gmail.com> Signed-off-by: yangerkun <yangerkun(a)huawei.com> Reviewed-by: zhangyi (F) <yi.zhang(a)huawei.com> Signed-off-by: Zheng Zengkai <zhengzengkai(a)huawei.com> Signed-off-by: Baokun Li <libaokun1(a)huawei.com> --- fs/aio.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/fs/aio.c b/fs/aio.c index f8589caef9c1..73555729bc36 100644 --- a/fs/aio.c +++ b/fs/aio.c @@ -2170,10 +2170,17 @@ static long do_io_getevents(aio_context_t ctx_id, struct io_event __user *events, struct timespec64 *ts) { - ktime_t until = ts ? timespec64_to_ktime(*ts) : KTIME_MAX; - struct kioctx *ioctx = lookup_ioctx(ctx_id); + ktime_t until = KTIME_MAX; + struct kioctx *ioctx = NULL; long ret = -EINVAL; + if (ts) { + if (!timespec64_valid(ts)) + return ret; + until = timespec64_to_ktime(*ts); + } + + ioctx = lookup_ioctx(ctx_id); if (likely(ioctx)) { if (likely(min_nr <= nr && min_nr >= 0)) ret = read_events(ioctx, min_nr, nr, events, until); -- 2.31.1

2 1

[PATCH OLK-6.6] pipe: Fix endless sleep problem due to the out-of-order
by Baokun Li 06 Jan '24

06 Jan '24

From: yangerkun <yangerkun(a)huawei.com> hulk inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I8U3TT --------------------------- Thers is a out-of-order access problem which would cause endless sleep when we use pipe with epoll. The story is following, we assume the ring size is 2, the ring head is 1, the ring tail is 0, task0 is write task, task1 is read task, task2 is write task. Task0 Task1 Task2 epoll_ctl(fd, EPOLL_CTL_ADD, ...) pipe_poll() poll_wait() tail = READ_ONCE(pipe->tail); // Re-order and get tail=0 pipe_read tail++ //tail=1 pipe_write head++ //head=2 head = READ_ONCE(pipe->head); // head = 2 check ring is full by head - tail Task0 get head = 2 and tail = 0, so it mistake that the pipe ring is full, then task0 is not add into ready list. If the ring is not full anymore, task0 would not be woken up forever. The reason of this problem is that we got inconsistent head/tail value of the pipe ring, so we fix the problem by getting them protected. Signed-off-by: yangerkun <yangerkun(a)huawei.com> Signed-off-by: Zhang Xiaoxu <zhangxiaoxu5(a)huawei.com> Reviewed-by: Zhang Yi <yi.zhang(a)huawei.com> Signed-off-by: Chen Jun <chenjun102(a)huawei.com> Signed-off-by: Zheng Zengkai <zhengzengkai(a)huawei.com> Signed-off-by: Baokun Li <libaokun1(a)huawei.com> --- fs/pipe.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/pipe.c b/fs/pipe.c index 139190165a1c..34752ba942ab 100644 --- a/fs/pipe.c +++ b/fs/pipe.c @@ -672,8 +672,10 @@ pipe_poll(struct file *filp, poll_table *wait) * if something changes and you got it wrong, the poll * table entry will wake you up and fix it. */ + spin_lock_irq(&pipe->rd_wait.lock); head = READ_ONCE(pipe->head); tail = READ_ONCE(pipe->tail); + spin_unlock_irq(&pipe->rd_wait.lock); mask = 0; if (filp->f_mode & FMODE_READ) { -- 2.31.1

2 1

[PATCH OLK-6.6 v2] openeuler_defconfig: enable erofs ondemand for x86 and arm64
by Baokun Li 06 Jan '24

06 Jan '24

hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I8RP48 -------------------------------- To support erofs over fscache mode, the following config is enabled by default on x86 and arm64: CONFIG_CACHEFILES_ONDEMAND=y CONFIG_EROFS_FS=m CONFIG_EROFS_FS_XATTR=y CONFIG_EROFS_FS_POSIX_ACL=y CONFIG_EROFS_FS_SECURITY=y CONFIG_EROFS_FS_ONDEMAND=y Signed-off-by: Baokun Li <libaokun1(a)huawei.com> --- arch/arm64/configs/openeuler_defconfig | 10 ++++++++-- arch/x86/configs/openeuler_defconfig | 10 ++++++++-- 2 files changed, 16 insertions(+), 4 deletions(-) diff --git a/arch/arm64/configs/openeuler_defconfig b/arch/arm64/configs/openeuler_defconfig index 9a053bc885dd..af539123c619 100644 --- a/arch/arm64/configs/openeuler_defconfig +++ b/arch/arm64/configs/openeuler_defconfig @@ -6874,7 +6874,7 @@ CONFIG_FSCACHE_STATS=y CONFIG_CACHEFILES=m # CONFIG_CACHEFILES_DEBUG is not set # CONFIG_CACHEFILES_ERROR_INJECTION is not set -# CONFIG_CACHEFILES_ONDEMAND is not set +CONFIG_CACHEFILES_ONDEMAND=y # end of Caches # @@ -6980,7 +6980,13 @@ CONFIG_PSTORE_RAM=m # CONFIG_PSTORE_BLK is not set # CONFIG_SYSV_FS is not set # CONFIG_UFS_FS is not set -# CONFIG_EROFS_FS is not set +CONFIG_EROFS_FS=m +# CONFIG_EROFS_FS_DEBUG is not set +CONFIG_EROFS_FS_XATTR=y +CONFIG_EROFS_FS_POSIX_ACL=y +CONFIG_EROFS_FS_SECURITY=y +# CONFIG_EROFS_FS_ZIP is not set +CONFIG_EROFS_FS_ONDEMAND=y CONFIG_NETWORK_FILESYSTEMS=y CONFIG_NFS_FS=m CONFIG_NFS_V2=m diff --git a/arch/x86/configs/openeuler_defconfig b/arch/x86/configs/openeuler_defconfig index 4bf011ed6307..3adfcf262f54 100644 --- a/arch/x86/configs/openeuler_defconfig +++ b/arch/x86/configs/openeuler_defconfig @@ -8068,7 +8068,7 @@ CONFIG_FSCACHE_STATS=y CONFIG_CACHEFILES=m # CONFIG_CACHEFILES_DEBUG is not set # CONFIG_CACHEFILES_ERROR_INJECTION is not set -# CONFIG_CACHEFILES_ONDEMAND is not set +CONFIG_CACHEFILES_ONDEMAND=y # end of Caches # @@ -8177,7 +8177,13 @@ CONFIG_PSTORE_RAM=m # CONFIG_PSTORE_BLK is not set # CONFIG_SYSV_FS is not set # CONFIG_UFS_FS is not set -# CONFIG_EROFS_FS is not set +CONFIG_EROFS_FS=m +# CONFIG_EROFS_FS_DEBUG is not set +CONFIG_EROFS_FS_XATTR=y +CONFIG_EROFS_FS_POSIX_ACL=y +CONFIG_EROFS_FS_SECURITY=y +# CONFIG_EROFS_FS_ZIP is not set +CONFIG_EROFS_FS_ONDEMAND=y CONFIG_NETWORK_FILESYSTEMS=y CONFIG_NFS_FS=m # CONFIG_NFS_V2 is not set -- 2.31.1

2 1

[PATCH OLK-6.6 00/55] backport mainline md patch
by Li Nan 06 Jan '24

06 Jan '24

v2: - drop lts patches which have been merged. Alex Lyakas (1): md: Whenassemble the array, consult the superblock of the freshest device Denis Plotnikov (1): md-cluster: check for timeout while a new disk adding Gou Hao (1): md/raid1: remove unnecessary null checking Junxiao Bi (2): md: bypass block throttle for superblock update Revert "md/raid5: Wait for MD_SB_CHANGE_PENDING in raid5d" Justin Stitt (1): md: replace deprecated strncpy with memcpy Li Nan (2): md: factor out a helper exceed_read_errors() to check read_errors md/raid1: support read error check Mariusz Tkaczyk (1): md: do not require mddev_lock() for all options in array_state_store() Yu Kuai (46): md: use separate work_struct for md_start_sync() md: factor out a helper to choose sync action from md_check_recovery() md: delay choosing sync action to md_start_sync() md: factor out a helper rdev_removeable() from remove_and_add_spares() md: factor out a helper rdev_is_spare() from remove_and_add_spares() md: factor out a helper rdev_addable() from remove_and_add_spares() md: delay remove_and_add_spares() for read only array to md_start_sync() md: initialize 'active_io' while allocating mddev md: initialize 'writes_pending' while allocating mddev md-bitmap: remove the checking of 'pers->quiesce' from location_store() md-bitmap: suspend array earlier in location_store() md: don't check 'mddev->pers' from suspend_hi_store() md: don't check 'mddev->pers' and 'pers->quiesce' from suspend_lo_store() md: factor out a helper from mddev_put() md: simplify md_seq_ops md/raid1: don't split discard io for write behind md: use READ_ONCE/WRITE_ONCE for 'suspend_lo' and 'suspend_hi' md/raid5-cache: use READ_ONCE/WRITE_ONCE for 'conf->log' md: replace is_md_suspended() with 'mddev->suspended' in md_check_recovery() md: add new helpers to suspend/resume array md: add new helpers to suspend/resume and lock/unlock array md/dm-raid: use new apis to suspend array md/md-bitmap: use new apis to suspend array for location_store() md/raid5-cache: use new apis to suspend array md/raid5: use new apis to suspend array md: use new apis to suspend array for sysfs apis md: use new apis to suspend array for adding/removing rdev from state_store() md: use new apis to suspend array for ioctls involed array reconfiguration md: use new apis to suspend array before mddev_create/destroy_serial_pool md: cleanup mddev_create/destroy_serial_pool() md/md-linear: cleanup linear_add() md/raid5: replace suspend with quiesce() callback md: suspend array in md_start_sync() if array need reconfiguration md: remove old apis to suspend the array md: rename __mddev_suspend/resume() back to mddev_suspend/resume() md: cleanup pers->prepare_suspend() md: remove flag RemoveSynchronized md/raid10: remove rcu protection to access rdev from conf md/raid1: remove rcu protection to access rdev from conf md/raid5: remove rcu protection to access rdev from conf md/md-multipath: remove rcu protection to access rdev from conf md: synchronize flush io with array reconfiguration md: fix missing flush of sync_work md: fix stopping sync thread md: split MD_RECOVERY_NEEDED out of mddev_resume dm-raid: delay flushing event_work() after reconfig_mutex is released drivers/md/md.h | 75 +-- drivers/md/raid5.h | 4 +- drivers/md/dm-raid.c | 20 +- drivers/md/md-autodetect.c | 4 +- drivers/md/md-bitmap.c | 61 +-- drivers/md/md-cluster.c | 15 +- drivers/md/md-linear.c | 2 - drivers/md/md-multipath.c | 32 +- drivers/md/md.c | 1065 ++++++++++++++++++++---------------- drivers/md/raid1-10.c | 54 ++ drivers/md/raid1.c | 97 ++-- drivers/md/raid10.c | 274 +++------- drivers/md/raid5-cache.c | 75 ++- drivers/md/raid5-ppl.c | 16 +- drivers/md/raid5.c | 306 +++-------- 15 files changed, 980 insertions(+), 1120 deletions(-) -- 2.39.2

2 56

[PATCH OLK-5.10] ext4: fix kernel BUG in 'ext4_write_inline_data_end()'
by Zhihao Cheng 06 Jan '24

06 Jan '24

From: Ye Bin <yebin10(a)huawei.com> mainline inclusion from mainline-v6.2-rc1 commit 5c099c4fdc438014d5893629e70a8ba934433ee8 category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/I8U2KA CVE: NA Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?… -------------------------------- Syzbot report follow issue: ------------[ cut here ]------------ kernel BUG at fs/ext4/inline.c:227! invalid opcode: 0000 [#1] PREEMPT SMP KASAN CPU: 1 PID: 3629 Comm: syz-executor212 Not tainted 6.1.0-rc5-syzkaller-00018-g59d0d52c30d4 #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/26/2022 RIP: 0010:ext4_write_inline_data+0x344/0x3e0 fs/ext4/inline.c:227 RSP: 0018:ffffc90003b3f368 EFLAGS: 00010293 RAX: 0000000000000000 RBX: ffff8880704e16c0 RCX: 0000000000000000 RDX: ffff888021763a80 RSI: ffffffff821e31a4 RDI: 0000000000000006 RBP: 000000000006818e R08: 0000000000000006 R09: 0000000000068199 R10: 0000000000000079 R11: 0000000000000000 R12: 000000000000000b R13: 0000000000068199 R14: ffffc90003b3f408 R15: ffff8880704e1c82 FS: 000055555723e3c0(0000) GS:ffff8880b9b00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007fffe8ac9080 CR3: 0000000079f81000 CR4: 0000000000350ee0 Call Trace: <TASK> ext4_write_inline_data_end+0x2a3/0x12f0 fs/ext4/inline.c:768 ext4_write_end+0x242/0xdd0 fs/ext4/inode.c:1313 ext4_da_write_end+0x3ed/0xa30 fs/ext4/inode.c:3063 generic_perform_write+0x316/0x570 mm/filemap.c:3764 ext4_buffered_write_iter+0x15b/0x460 fs/ext4/file.c:285 ext4_file_write_iter+0x8bc/0x16e0 fs/ext4/file.c:700 call_write_iter include/linux/fs.h:2191 [inline] do_iter_readv_writev+0x20b/0x3b0 fs/read_write.c:735 do_iter_write+0x182/0x700 fs/read_write.c:861 vfs_iter_write+0x74/0xa0 fs/read_write.c:902 iter_file_splice_write+0x745/0xc90 fs/splice.c:686 do_splice_from fs/splice.c:764 [inline] direct_splice_actor+0x114/0x180 fs/splice.c:931 splice_direct_to_actor+0x335/0x8a0 fs/splice.c:886 do_splice_direct+0x1ab/0x280 fs/splice.c:974 do_sendfile+0xb19/0x1270 fs/read_write.c:1255 __do_sys_sendfile64 fs/read_write.c:1323 [inline] __se_sys_sendfile64 fs/read_write.c:1309 [inline] __x64_sys_sendfile64+0x1d0/0x210 fs/read_write.c:1309 do_syscall_x64 arch/x86/entry/common.c:50 [inline] do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80 entry_SYSCALL_64_after_hwframe+0x63/0xcd ---[ end trace 0000000000000000 ]--- Above issue may happens as follows: ext4_da_write_begin ext4_da_write_inline_data_begin ext4_da_convert_inline_data_to_extent ext4_clear_inode_state(inode, EXT4_STATE_MAY_INLINE_DATA); ext4_da_write_end ext4_run_li_request ext4_mb_prefetch ext4_read_block_bitmap_nowait ext4_validate_block_bitmap ext4_mark_group_bitmap_corrupted(sb, block_group, EXT4_GROUP_INFO_BBITMAP_CORRUPT) percpu_counter_sub(&sbi->s_freeclusters_counter,grp->bb_free); -> sbi->s_freeclusters_counter become zero ext4_da_write_begin if (ext4_nonda_switch(inode->i_sb)) -> As freeclusters_counter is zero will return true *fsdata = (void *)FALL_BACK_TO_NONDELALLOC; ext4_write_begin ext4_da_write_end if (write_mode == FALL_BACK_TO_NONDELALLOC) ext4_write_end if (inline_data) ext4_write_inline_data_end ext4_write_inline_data BUG_ON(pos + len > EXT4_I(inode)->i_inline_size); -> As inode is already convert to extent, so 'pos + len' > inline_size -> then trigger BUG. To solve this issue, instead of checking ext4_has_inline_data() which is only cleared after data has been written back, check the EXT4_STATE_MAY_INLINE_DATA flag in ext4_write_end(). Fixes: f19d5870cbf7 ("ext4: add normal write support for inline data") Reported-by: syzbot+4faa160fa96bfba639f8(a)syzkaller.appspotmail.com Reported-by: Jun Nie <jun.nie(a)linaro.org> Signed-off-by: Ye Bin <yebin10(a)huawei.com> Link: https://lore.kernel.org/r/20221206144134.1919987-1-yebin@huaweicloud.com Signed-off-by: Theodore Ts'o <tytso(a)mit.edu> Cc: stable(a)kernel.org Signed-off-by: Zhihao Cheng <chengzhihao1(a)huawei.com> --- fs/ext4/inode.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c index 0757f6a6455f..243a2a23015e 100644 --- a/fs/ext4/inode.c +++ b/fs/ext4/inode.c @@ -1294,7 +1294,8 @@ static int ext4_write_end(struct file *file, trace_ext4_write_end(inode, pos, len, copied); - if (ext4_has_inline_data(inode)) + if (ext4_has_inline_data(inode) && + ext4_test_inode_state(inode, EXT4_STATE_MAY_INLINE_DATA)) return ext4_write_inline_data_end(inode, pos, len, copied, page); copied = block_write_end(file, mapping, pos, len, copied, page, fsdata); -- 2.31.1

2 1

[PATCH OLK-6.6 v2] block: support to account io_ticks precisely
by Yu Kuai 06 Jan '24

06 Jan '24

hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I8RIME CVE: NA ----------------------------------------------- Currently, io_ticks is accounted based on sampling, specifically update_io_ticks() will always account io_ticks by 1 jiffies from bdev_start_io_acct()/blk_account_io_start(), and the result can be inaccurate, for example(HZ is 250): Test script: fio -filename=/dev/sda -bs=4k -rw=write -direct=1 -name=test -thinktime=4ms Test result: util is about 90%, while the disk is really idle. In order to account io_ticks precisely, update_io_ticks() must know if there are IO inflight already, and this requires overhead slightly, hence precise io accounting is disabled by default, and user can enable it through sysfs entry. Noted that for rq-based devcie, part_stat_local_inc/dec() and part_in_flight() is used to track inflight instead of iterating tags, which is not supposed to be used in fast path because 'tags->lock' is grabbed in blk_mq_find_and_get_req(). Signed-off-by: Yu Kuai <yukuai3(a)huawei.com> --- Documentation/ABI/stable/sysfs-block | 8 ++++-- block/blk-core.c | 27 ++++++++++++++++-- block/blk-merge.c | 3 ++ block/blk-mq-debugfs.c | 2 ++ block/blk-mq.c | 12 ++++++-- block/blk-sysfs.c | 42 ++++++++++++++++++++++++++-- block/blk.h | 1 + block/genhd.c | 2 +- include/linux/blk-mq.h | 1 + include/linux/blkdev.h | 3 ++ 10 files changed, 91 insertions(+), 10 deletions(-) diff --git a/Documentation/ABI/stable/sysfs-block b/Documentation/ABI/stable/sysfs-block index 1fe9a553c37b..e5fedecf7bdf 100644 --- a/Documentation/ABI/stable/sysfs-block +++ b/Documentation/ABI/stable/sysfs-block @@ -358,8 +358,12 @@ What: /sys/block/<disk>/queue/iostats Date: January 2009 Contact: linux-block(a)vger.kernel.org Description: - [RW] This file is used to control (on/off) the iostats - accounting of the disk. + [RW] This file is used to control the iostats accounting of the + disk. If this value is 0, iostats accounting is disabled; If + this value is 1, iostats accounting is enabled, but io_ticks is + accounted by sampling and the result is not accurate; If this + value is 2, iostats accounting is enabled and io_ticks is + accounted precisely, but there will be slightly overhead. What: /sys/block/<disk>/queue/logical_block_size diff --git a/block/blk-core.c b/block/blk-core.c index fdf25b8d6e78..13a27d1d8540 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -71,6 +71,21 @@ static struct kmem_cache *blk_requestq_cachep; */ static struct workqueue_struct *kblockd_workqueue; +static bool precise_iostat; + +static int __init precise_iostat_setup(char *str) +{ + bool precise; + + if (!strtobool(str, &precise)) { + precise_iostat = precise; + pr_info("precise iostat %d\n", precise_iostat); + } + + return 1; +} +__setup("precise_iostat=", precise_iostat_setup); + /** * blk_queue_flag_set - atomically set a queue flag * @flag: flag to be set @@ -441,6 +456,8 @@ struct request_queue *blk_alloc_queue(int node_id) blk_set_default_limits(&q->limits); q->nr_requests = BLKDEV_DEFAULT_RQ; + if (precise_iostat) + blk_queue_flag_set(QUEUE_FLAG_PRECISE_IO_STAT, q); return q; @@ -938,11 +955,15 @@ EXPORT_SYMBOL_GPL(iocb_bio_iopoll); void update_io_ticks(struct block_device *part, unsigned long now, bool end) { unsigned long stamp; + bool precise = blk_queue_precise_io_stat(part->bd_queue); again: stamp = READ_ONCE(part->bd_stamp); - if (unlikely(time_after(now, stamp))) { - if (likely(try_cmpxchg(&part->bd_stamp, &stamp, now))) - __part_stat_add(part, io_ticks, end ? now - stamp : 1); + if (unlikely(time_after(now, stamp)) && + likely(try_cmpxchg(&part->bd_stamp, &stamp, now))) { + if (end || (precise && part_in_flight(part))) + __part_stat_add(part, io_ticks, now - stamp); + else if (!precise) + __part_stat_add(part, io_ticks, 1); } if (part->bd_partno) { part = bdev_whole(part); diff --git a/block/blk-merge.c b/block/blk-merge.c index 65e75efa9bd3..5db8228c46fc 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@ -783,6 +783,9 @@ static void blk_account_io_merge_request(struct request *req) if (blk_do_io_stat(req)) { part_stat_lock(); part_stat_inc(req->part, merges[op_stat_group(req_op(req))]); + if (req->rq_flags & RQF_PRECISE_IO_STAT) + part_stat_local_dec(req->part, + in_flight[op_is_write(req_op(req))]); part_stat_unlock(); } } diff --git a/block/blk-mq-debugfs.c b/block/blk-mq-debugfs.c index aa7a1357c3e1..9ad108307344 100644 --- a/block/blk-mq-debugfs.c +++ b/block/blk-mq-debugfs.c @@ -86,6 +86,7 @@ static const char *const blk_queue_flag_name[] = { QUEUE_FLAG_NAME(FAIL_IO), QUEUE_FLAG_NAME(NONROT), QUEUE_FLAG_NAME(IO_STAT), + QUEUE_FLAG_NAME(PRECISE_IO_STAT), QUEUE_FLAG_NAME(NOXMERGES), QUEUE_FLAG_NAME(ADD_RANDOM), QUEUE_FLAG_NAME(SYNCHRONOUS), @@ -254,6 +255,7 @@ static const char *const rqf_name[] = { RQF_NAME(FAILED), RQF_NAME(QUIET), RQF_NAME(IO_STAT), + RQF_NAME(PRECISE_IO_STAT), RQF_NAME(PM), RQF_NAME(HASHED), RQF_NAME(STATS), diff --git a/block/blk-mq.c b/block/blk-mq.c index dd7c9e3eca1b..9c894e0d0b19 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -360,8 +360,11 @@ static struct request *blk_mq_rq_ctx_init(struct blk_mq_alloc_data *data, if (data->flags & BLK_MQ_REQ_PM) data->rq_flags |= RQF_PM; - if (blk_queue_io_stat(q)) + if (blk_queue_io_stat(q)) { data->rq_flags |= RQF_IO_STAT; + if (blk_queue_precise_io_stat(q)) + data->rq_flags |= RQF_PRECISE_IO_STAT; + } rq->rq_flags = data->rq_flags; if (data->rq_flags & RQF_SCHED_TAGS) { @@ -994,6 +997,9 @@ static inline void blk_account_io_done(struct request *req, u64 now) update_io_ticks(req->part, jiffies, true); part_stat_inc(req->part, ios[sgrp]); part_stat_add(req->part, nsecs[sgrp], now - req->start_time_ns); + if (req->rq_flags & RQF_PRECISE_IO_STAT) + part_stat_local_dec(req->part, + in_flight[op_is_write(req_op(req))]); part_stat_unlock(); } } @@ -1001,7 +1007,6 @@ static inline void blk_account_io_done(struct request *req, u64 now) static inline void blk_account_io_start(struct request *req) { trace_block_io_start(req); - if (blk_do_io_stat(req)) { /* * All non-passthrough requests are created from a bio with one @@ -1016,6 +1021,9 @@ static inline void blk_account_io_start(struct request *req) part_stat_lock(); update_io_ticks(req->part, jiffies, false); + if (req->rq_flags & RQF_PRECISE_IO_STAT) + part_stat_local_inc(req->part, + in_flight[op_is_write(req_op(req))]); part_stat_unlock(); } } diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c index 63e481262336..d498cf0942d3 100644 --- a/block/blk-sysfs.c +++ b/block/blk-sysfs.c @@ -303,7 +303,6 @@ queue_##name##_store(struct request_queue *q, const char *page, size_t count) \ QUEUE_SYSFS_BIT_FNS(nonrot, NONROT, 1); QUEUE_SYSFS_BIT_FNS(random, ADD_RANDOM, 0); -QUEUE_SYSFS_BIT_FNS(iostats, IO_STAT, 0); QUEUE_SYSFS_BIT_FNS(stable_writes, STABLE_WRITES, 0); #undef QUEUE_SYSFS_BIT_FNS @@ -473,6 +472,45 @@ static ssize_t queue_dax_show(struct request_queue *q, char *page) return queue_var_show(blk_queue_dax(q), page); } +static ssize_t queue_iostats_show(struct request_queue *q, char *page) +{ + int val = 0; + + if (blk_queue_io_stat(q)) + val = blk_queue_precise_io_stat(q) ? 2 : 1; + + return sprintf(page, "%u\n", val); +} + +static ssize_t +queue_iostats_store(struct request_queue *q, const char *page, size_t count) +{ + unsigned long nr; + int ret = queue_var_store(&nr, page, count); + + if (ret < 0) + return ret; + + switch (nr) { + case 0: + blk_queue_flag_clear(QUEUE_FLAG_IO_STAT, q); + blk_queue_flag_clear(QUEUE_FLAG_PRECISE_IO_STAT, q); + break; + case 1: + blk_queue_flag_set(QUEUE_FLAG_IO_STAT, q); + blk_queue_flag_clear(QUEUE_FLAG_PRECISE_IO_STAT, q); + break; + case 2: + blk_queue_flag_set(QUEUE_FLAG_IO_STAT, q); + blk_queue_flag_set(QUEUE_FLAG_PRECISE_IO_STAT, q); + break; + default: + return -EINVAL; + } + + return count; +} + #define QUEUE_RO_ENTRY(_prefix, _name) \ static struct queue_sysfs_entry _prefix##_entry = { \ .attr = { .name = _name, .mode = 0444 }, \ @@ -494,6 +532,7 @@ QUEUE_RO_ENTRY(queue_max_segments, "max_segments"); QUEUE_RO_ENTRY(queue_max_integrity_segments, "max_integrity_segments"); QUEUE_RO_ENTRY(queue_max_segment_size, "max_segment_size"); QUEUE_RW_ENTRY(elv_iosched, "scheduler"); +QUEUE_RW_ENTRY(queue_iostats, "iostats"); QUEUE_RO_ENTRY(queue_logical_block_size, "logical_block_size"); QUEUE_RO_ENTRY(queue_physical_block_size, "physical_block_size"); @@ -539,7 +578,6 @@ static struct queue_sysfs_entry queue_hw_sector_size_entry = { }; QUEUE_RW_ENTRY(queue_nonrot, "rotational"); -QUEUE_RW_ENTRY(queue_iostats, "iostats"); QUEUE_RW_ENTRY(queue_random, "add_random"); QUEUE_RW_ENTRY(queue_stable_writes, "stable_writes"); diff --git a/block/blk.h b/block/blk.h index 08a358bc0919..67915b04b3c1 100644 --- a/block/blk.h +++ b/block/blk.h @@ -344,6 +344,7 @@ static inline bool blk_do_io_stat(struct request *rq) } void update_io_ticks(struct block_device *part, unsigned long now, bool end); +unsigned int part_in_flight(struct block_device *part); static inline void req_set_nomerge(struct request_queue *q, struct request *req) { diff --git a/block/genhd.c b/block/genhd.c index 4a16a424f57d..b3ff653f3e50 100644 --- a/block/genhd.c +++ b/block/genhd.c @@ -118,7 +118,7 @@ static void part_stat_read_all(struct block_device *part, } } -static unsigned int part_in_flight(struct block_device *part) +unsigned int part_in_flight(struct block_device *part) { unsigned int inflight = 0; int cpu; diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index 3d74f3e5b995..9dfee0644f9c 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -46,6 +46,7 @@ typedef __u32 __bitwise req_flags_t; #define RQF_QUIET ((__force req_flags_t)(1 << 11)) /* account into disk and partition IO statistics */ #define RQF_IO_STAT ((__force req_flags_t)(1 << 13)) +#define RQF_PRECISE_IO_STAT ((__force req_flags_t)(1 << 14)) /* runtime pm request */ #define RQF_PM ((__force req_flags_t)(1 << 15)) /* on IO scheduler merge hash */ diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 9f3bcbcb156d..bea0b5fdac74 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -536,6 +536,7 @@ struct request_queue { #define QUEUE_FLAG_NONROT 6 /* non-rotational device (SSD) */ #define QUEUE_FLAG_VIRT QUEUE_FLAG_NONROT /* paravirt device */ #define QUEUE_FLAG_IO_STAT 7 /* do disk/partitions IO accounting */ +#define QUEUE_FLAG_PRECISE_IO_STAT 8 /* do disk/partitions IO accounting precisely */ #define QUEUE_FLAG_NOXMERGES 9 /* No extended merges */ #define QUEUE_FLAG_ADD_RANDOM 10 /* Contributes to random pool */ #define QUEUE_FLAG_SYNCHRONOUS 11 /* always completes in submit context */ @@ -576,6 +577,8 @@ bool blk_queue_flag_test_and_set(unsigned int flag, struct request_queue *q); #define blk_queue_stable_writes(q) \ test_bit(QUEUE_FLAG_STABLE_WRITES, &(q)->queue_flags) #define blk_queue_io_stat(q) test_bit(QUEUE_FLAG_IO_STAT, &(q)->queue_flags) +#define blk_queue_precise_io_stat(q) \ + test_bit(QUEUE_FLAG_PRECISE_IO_STAT, &(q)->queue_flags) #define blk_queue_add_random(q) test_bit(QUEUE_FLAG_ADD_RANDOM, &(q)->queue_flags) #define blk_queue_zone_resetall(q) \ test_bit(QUEUE_FLAG_ZONE_RESETALL, &(q)->queue_flags) -- 2.39.2

2 1

[PATCH OLK-6.6 v2] block: support to account io_ticks precisely
by Yu Kuai 06 Jan '24

06 Jan '24

Currently, io_ticks is accounted based on sampling, specifically update_io_ticks() will always account io_ticks by 1 jiffies from bdev_start_io_acct()/blk_account_io_start(), and the result can be inaccurate, for example(HZ is 250): Test script: fio -filename=/dev/sda -bs=4k -rw=write -direct=1 -name=test -thinktime=4ms Test result: util is about 90%, while the disk is really idle. In order to account io_ticks precisely, update_io_ticks() must know if there are IO inflight already, and this requires overhead slightly, hence precise io accounting is disabled by default, and user can enable it through sysfs entry. Noted that for rq-based devcie, part_stat_local_inc/dec() and part_in_flight() is used to track inflight instead of iterating tags, which is not supposed to be used in fast path because 'tags->lock' is grabbed in blk_mq_find_and_get_req(). Signed-off-by: Yu Kuai <yukuai3(a)huawei.com> --- Documentation/ABI/stable/sysfs-block | 8 ++++-- block/blk-core.c | 27 ++++++++++++++++-- block/blk-merge.c | 3 ++ block/blk-mq-debugfs.c | 2 ++ block/blk-mq.c | 12 ++++++-- block/blk-sysfs.c | 42 ++++++++++++++++++++++++++-- block/blk.h | 1 + block/genhd.c | 2 +- include/linux/blk-mq.h | 1 + include/linux/blkdev.h | 3 ++ 10 files changed, 91 insertions(+), 10 deletions(-) diff --git a/Documentation/ABI/stable/sysfs-block b/Documentation/ABI/stable/sysfs-block index 1fe9a553c37b..e5fedecf7bdf 100644 --- a/Documentation/ABI/stable/sysfs-block +++ b/Documentation/ABI/stable/sysfs-block @@ -358,8 +358,12 @@ What: /sys/block/<disk>/queue/iostats Date: January 2009 Contact: linux-block(a)vger.kernel.org Description: - [RW] This file is used to control (on/off) the iostats - accounting of the disk. + [RW] This file is used to control the iostats accounting of the + disk. If this value is 0, iostats accounting is disabled; If + this value is 1, iostats accounting is enabled, but io_ticks is + accounted by sampling and the result is not accurate; If this + value is 2, iostats accounting is enabled and io_ticks is + accounted precisely, but there will be slightly overhead. What: /sys/block/<disk>/queue/logical_block_size diff --git a/block/blk-core.c b/block/blk-core.c index fdf25b8d6e78..13a27d1d8540 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -71,6 +71,21 @@ static struct kmem_cache *blk_requestq_cachep; */ static struct workqueue_struct *kblockd_workqueue; +static bool precise_iostat; + +static int __init precise_iostat_setup(char *str) +{ + bool precise; + + if (!strtobool(str, &precise)) { + precise_iostat = precise; + pr_info("precise iostat %d\n", precise_iostat); + } + + return 1; +} +__setup("precise_iostat=", precise_iostat_setup); + /** * blk_queue_flag_set - atomically set a queue flag * @flag: flag to be set @@ -441,6 +456,8 @@ struct request_queue *blk_alloc_queue(int node_id) blk_set_default_limits(&q->limits); q->nr_requests = BLKDEV_DEFAULT_RQ; + if (precise_iostat) + blk_queue_flag_set(QUEUE_FLAG_PRECISE_IO_STAT, q); return q; @@ -938,11 +955,15 @@ EXPORT_SYMBOL_GPL(iocb_bio_iopoll); void update_io_ticks(struct block_device *part, unsigned long now, bool end) { unsigned long stamp; + bool precise = blk_queue_precise_io_stat(part->bd_queue); again: stamp = READ_ONCE(part->bd_stamp); - if (unlikely(time_after(now, stamp))) { - if (likely(try_cmpxchg(&part->bd_stamp, &stamp, now))) - __part_stat_add(part, io_ticks, end ? now - stamp : 1); + if (unlikely(time_after(now, stamp)) && + likely(try_cmpxchg(&part->bd_stamp, &stamp, now))) { + if (end || (precise && part_in_flight(part))) + __part_stat_add(part, io_ticks, now - stamp); + else if (!precise) + __part_stat_add(part, io_ticks, 1); } if (part->bd_partno) { part = bdev_whole(part); diff --git a/block/blk-merge.c b/block/blk-merge.c index 65e75efa9bd3..5db8228c46fc 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@ -783,6 +783,9 @@ static void blk_account_io_merge_request(struct request *req) if (blk_do_io_stat(req)) { part_stat_lock(); part_stat_inc(req->part, merges[op_stat_group(req_op(req))]); + if (req->rq_flags & RQF_PRECISE_IO_STAT) + part_stat_local_dec(req->part, + in_flight[op_is_write(req_op(req))]); part_stat_unlock(); } } diff --git a/block/blk-mq-debugfs.c b/block/blk-mq-debugfs.c index aa7a1357c3e1..9ad108307344 100644 --- a/block/blk-mq-debugfs.c +++ b/block/blk-mq-debugfs.c @@ -86,6 +86,7 @@ static const char *const blk_queue_flag_name[] = { QUEUE_FLAG_NAME(FAIL_IO), QUEUE_FLAG_NAME(NONROT), QUEUE_FLAG_NAME(IO_STAT), + QUEUE_FLAG_NAME(PRECISE_IO_STAT), QUEUE_FLAG_NAME(NOXMERGES), QUEUE_FLAG_NAME(ADD_RANDOM), QUEUE_FLAG_NAME(SYNCHRONOUS), @@ -254,6 +255,7 @@ static const char *const rqf_name[] = { RQF_NAME(FAILED), RQF_NAME(QUIET), RQF_NAME(IO_STAT), + RQF_NAME(PRECISE_IO_STAT), RQF_NAME(PM), RQF_NAME(HASHED), RQF_NAME(STATS), diff --git a/block/blk-mq.c b/block/blk-mq.c index dd7c9e3eca1b..9c894e0d0b19 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -360,8 +360,11 @@ static struct request *blk_mq_rq_ctx_init(struct blk_mq_alloc_data *data, if (data->flags & BLK_MQ_REQ_PM) data->rq_flags |= RQF_PM; - if (blk_queue_io_stat(q)) + if (blk_queue_io_stat(q)) { data->rq_flags |= RQF_IO_STAT; + if (blk_queue_precise_io_stat(q)) + data->rq_flags |= RQF_PRECISE_IO_STAT; + } rq->rq_flags = data->rq_flags; if (data->rq_flags & RQF_SCHED_TAGS) { @@ -994,6 +997,9 @@ static inline void blk_account_io_done(struct request *req, u64 now) update_io_ticks(req->part, jiffies, true); part_stat_inc(req->part, ios[sgrp]); part_stat_add(req->part, nsecs[sgrp], now - req->start_time_ns); + if (req->rq_flags & RQF_PRECISE_IO_STAT) + part_stat_local_dec(req->part, + in_flight[op_is_write(req_op(req))]); part_stat_unlock(); } } @@ -1001,7 +1007,6 @@ static inline void blk_account_io_done(struct request *req, u64 now) static inline void blk_account_io_start(struct request *req) { trace_block_io_start(req); - if (blk_do_io_stat(req)) { /* * All non-passthrough requests are created from a bio with one @@ -1016,6 +1021,9 @@ static inline void blk_account_io_start(struct request *req) part_stat_lock(); update_io_ticks(req->part, jiffies, false); + if (req->rq_flags & RQF_PRECISE_IO_STAT) + part_stat_local_inc(req->part, + in_flight[op_is_write(req_op(req))]); part_stat_unlock(); } } diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c index 63e481262336..d498cf0942d3 100644 --- a/block/blk-sysfs.c +++ b/block/blk-sysfs.c @@ -303,7 +303,6 @@ queue_##name##_store(struct request_queue *q, const char *page, size_t count) \ QUEUE_SYSFS_BIT_FNS(nonrot, NONROT, 1); QUEUE_SYSFS_BIT_FNS(random, ADD_RANDOM, 0); -QUEUE_SYSFS_BIT_FNS(iostats, IO_STAT, 0); QUEUE_SYSFS_BIT_FNS(stable_writes, STABLE_WRITES, 0); #undef QUEUE_SYSFS_BIT_FNS @@ -473,6 +472,45 @@ static ssize_t queue_dax_show(struct request_queue *q, char *page) return queue_var_show(blk_queue_dax(q), page); } +static ssize_t queue_iostats_show(struct request_queue *q, char *page) +{ + int val = 0; + + if (blk_queue_io_stat(q)) + val = blk_queue_precise_io_stat(q) ? 2 : 1; + + return sprintf(page, "%u\n", val); +} + +static ssize_t +queue_iostats_store(struct request_queue *q, const char *page, size_t count) +{ + unsigned long nr; + int ret = queue_var_store(&nr, page, count); + + if (ret < 0) + return ret; + + switch (nr) { + case 0: + blk_queue_flag_clear(QUEUE_FLAG_IO_STAT, q); + blk_queue_flag_clear(QUEUE_FLAG_PRECISE_IO_STAT, q); + break; + case 1: + blk_queue_flag_set(QUEUE_FLAG_IO_STAT, q); + blk_queue_flag_clear(QUEUE_FLAG_PRECISE_IO_STAT, q); + break; + case 2: + blk_queue_flag_set(QUEUE_FLAG_IO_STAT, q); + blk_queue_flag_set(QUEUE_FLAG_PRECISE_IO_STAT, q); + break; + default: + return -EINVAL; + } + + return count; +} + #define QUEUE_RO_ENTRY(_prefix, _name) \ static struct queue_sysfs_entry _prefix##_entry = { \ .attr = { .name = _name, .mode = 0444 }, \ @@ -494,6 +532,7 @@ QUEUE_RO_ENTRY(queue_max_segments, "max_segments"); QUEUE_RO_ENTRY(queue_max_integrity_segments, "max_integrity_segments"); QUEUE_RO_ENTRY(queue_max_segment_size, "max_segment_size"); QUEUE_RW_ENTRY(elv_iosched, "scheduler"); +QUEUE_RW_ENTRY(queue_iostats, "iostats"); QUEUE_RO_ENTRY(queue_logical_block_size, "logical_block_size"); QUEUE_RO_ENTRY(queue_physical_block_size, "physical_block_size"); @@ -539,7 +578,6 @@ static struct queue_sysfs_entry queue_hw_sector_size_entry = { }; QUEUE_RW_ENTRY(queue_nonrot, "rotational"); -QUEUE_RW_ENTRY(queue_iostats, "iostats"); QUEUE_RW_ENTRY(queue_random, "add_random"); QUEUE_RW_ENTRY(queue_stable_writes, "stable_writes"); diff --git a/block/blk.h b/block/blk.h index 08a358bc0919..67915b04b3c1 100644 --- a/block/blk.h +++ b/block/blk.h @@ -344,6 +344,7 @@ static inline bool blk_do_io_stat(struct request *rq) } void update_io_ticks(struct block_device *part, unsigned long now, bool end); +unsigned int part_in_flight(struct block_device *part); static inline void req_set_nomerge(struct request_queue *q, struct request *req) { diff --git a/block/genhd.c b/block/genhd.c index 4a16a424f57d..b3ff653f3e50 100644 --- a/block/genhd.c +++ b/block/genhd.c @@ -118,7 +118,7 @@ static void part_stat_read_all(struct block_device *part, } } -static unsigned int part_in_flight(struct block_device *part) +unsigned int part_in_flight(struct block_device *part) { unsigned int inflight = 0; int cpu; diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index 3d74f3e5b995..9dfee0644f9c 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -46,6 +46,7 @@ typedef __u32 __bitwise req_flags_t; #define RQF_QUIET ((__force req_flags_t)(1 << 11)) /* account into disk and partition IO statistics */ #define RQF_IO_STAT ((__force req_flags_t)(1 << 13)) +#define RQF_PRECISE_IO_STAT ((__force req_flags_t)(1 << 14)) /* runtime pm request */ #define RQF_PM ((__force req_flags_t)(1 << 15)) /* on IO scheduler merge hash */ diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 9f3bcbcb156d..bea0b5fdac74 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -536,6 +536,7 @@ struct request_queue { #define QUEUE_FLAG_NONROT 6 /* non-rotational device (SSD) */ #define QUEUE_FLAG_VIRT QUEUE_FLAG_NONROT /* paravirt device */ #define QUEUE_FLAG_IO_STAT 7 /* do disk/partitions IO accounting */ +#define QUEUE_FLAG_PRECISE_IO_STAT 8 /* do disk/partitions IO accounting precisely */ #define QUEUE_FLAG_NOXMERGES 9 /* No extended merges */ #define QUEUE_FLAG_ADD_RANDOM 10 /* Contributes to random pool */ #define QUEUE_FLAG_SYNCHRONOUS 11 /* always completes in submit context */ @@ -576,6 +577,8 @@ bool blk_queue_flag_test_and_set(unsigned int flag, struct request_queue *q); #define blk_queue_stable_writes(q) \ test_bit(QUEUE_FLAG_STABLE_WRITES, &(q)->queue_flags) #define blk_queue_io_stat(q) test_bit(QUEUE_FLAG_IO_STAT, &(q)->queue_flags) +#define blk_queue_precise_io_stat(q) \ + test_bit(QUEUE_FLAG_PRECISE_IO_STAT, &(q)->queue_flags) #define blk_queue_add_random(q) test_bit(QUEUE_FLAG_ADD_RANDOM, &(q)->queue_flags) #define blk_queue_zone_resetall(q) \ test_bit(QUEUE_FLAG_ZONE_RESETALL, &(q)->queue_flags) -- 2.39.2

2 1

[PATCH OLK-6.6 0/2] support ext3/ext4 netlink error report.
by Baokun Li 06 Jan '24

06 Jan '24

Zhao Minmin (1): ext4: report error to userspace by netlink Zhihao Cheng (1): Add new config 'CONFIG_EXT4_ERROR_REPORT' to control ext3/4 error reporting arch/arm64/configs/openeuler_defconfig | 1 + arch/x86/configs/openeuler_defconfig | 1 + fs/ext4/Kconfig | 8 +++ fs/ext4/ext4.h | 11 ++++ fs/ext4/super.c | 71 ++++++++++++++++++++++++++ include/uapi/linux/netlink.h | 1 + 6 files changed, 93 insertions(+) -- 2.31.1

2 3

[PATCH OLK-6.6 0/3] cgroup kabi
by Chen Ridong 05 Jan '24

05 Jan '24

Chen Ridong (3): cgroup: kabi: reserve space for cgroup frame namespace: kabi: reserve for future namespace development cgroup/psi: reserve kabi for future psi development include/linux/cgroup-defs.h | 32 ++++++++++++++++++++++++++++++++ include/linux/ipc_namespace.h | 6 ++++++ include/linux/nsproxy.h | 9 +++++++++ include/linux/pid_namespace.h | 2 ++ include/linux/psi_types.h | 16 ++++++++++++++++ include/linux/user_namespace.h | 33 +++++++++++++++++++++++++++++++++ kernel/ucount.c | 18 ++++++++++++++++++ 7 files changed, 116 insertions(+) -- 2.34.1

2 4

[PATCH OLK-6.6] arm64: cpufeature: add cmdline option for disable lse
by Wei Li 05 Jan '24

05 Jan '24

From: Xie XiuQi <xiexiuqi(a)huawei.com> hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I4JBSJ CVE: NA ---------------------------------------- LSE atomic instruction might introduce performance regression on specific benchmark or business. So add a cmdline option to disable/enable it. "lse=off" cmdline option means disable LSE atomic instruction. Signed-off-by: Xie XiuQi <xiexiuqi(a)huawei.com> [liwei: Fix compile warning with CONFIG_ARM64_LSE_ATOMICS=n] Signed-off-by: Wei Li <liwei391(a)huawei.com> --- arch/arm64/kernel/cpufeature.c | 32 +++++++++++++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 3517dd2b0358..6d12bbc54728 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -1431,6 +1431,22 @@ u64 __read_sysreg_by_encoding(u32 sys_id) #include <linux/irqchip/arm-gic-v3.h> +#ifdef CONFIG_ARM64_LSE_ATOMICS +static bool lse_disabled; + +static int __init parse_lse(char *str) +{ + if (str == NULL) + return 1; + + if (!strncmp(str, "off", 3)) + lse_disabled = true; + + return 0; +} +early_param("lse", parse_lse); +#endif + static bool has_always(const struct arm64_cpu_capabilities *entry, int scope) { @@ -1539,6 +1555,20 @@ static bool has_32bit_el0(const struct arm64_cpu_capabilities *entry, int scope) return true; } +#ifdef CONFIG_ARM64_LSE_ATOMICS +static bool has_cpuid_feature_lse(const struct arm64_cpu_capabilities *entry, + int scope) +{ + if (lse_disabled) { + pr_info_once("%s forced OFF by command line option\n", + entry->desc); + return false; + } + + return has_cpuid_feature(entry, scope); +} +#endif + static bool has_useable_gicv3_cpuif(const struct arm64_cpu_capabilities *entry, int scope) { bool has_sre; @@ -2316,7 +2346,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .desc = "LSE atomic instructions", .capability = ARM64_HAS_LSE_ATOMICS, .type = ARM64_CPUCAP_SYSTEM_FEATURE, - .matches = has_cpuid_feature, + .matches = has_cpuid_feature_lse, ARM64_CPUID_FIELDS(ID_AA64ISAR0_EL1, ATOMIC, IMP) }, #endif /* CONFIG_ARM64_LSE_ATOMICS */ -- 2.25.1

2 1

[PATCH OLK-6.6 0/4] JFFS2: Fix the race issues caused by the GC of jffs2
by ZhaoLong Wang 05 Jan '24

05 Jan '24

Hou Tao (4): jffs2: protect no-raw-node-ref check of inocache by erase_completion_lock jffs2: handle INO_STATE_CLEARING in jffs2_do_read_inode() jffs2: make the overwritten xattr invisible after remount jffs2: reset pino_nlink to 0 when inode creation failed fs/jffs2/dir.c | 28 ++++++++++++++++++---- fs/jffs2/nodelist.c | 7 ++++++ fs/jffs2/readinode.c | 11 ++++++++- fs/jffs2/xattr.c | 55 +++++++++++++++++++++++++++++++++++++++----- 4 files changed, 90 insertions(+), 11 deletions(-) -- 2.39.2

2 5

[PATCH OLK-6.6 0/4] JFFS2: Fix the race issues caused by the GC of jffs2
by ZhaoLong Wang 05 Jan '24

05 Jan '24

Hou Tao (4): jffs2: protect no-raw-node-ref check of inocache by erase_completion_lock jffs2: handle INO_STATE_CLEARING in jffs2_do_read_inode() jffs2: make the overwritten xattr invisible after remount jffs2: reset pino_nlink to 0 when inode creation failed fs/jffs2/dir.c | 28 ++++++++++++++++++---- fs/jffs2/nodelist.c | 7 ++++++ fs/jffs2/readinode.c | 11 ++++++++- fs/jffs2/xattr.c | 55 +++++++++++++++++++++++++++++++++++++++----- 4 files changed, 90 insertions(+), 11 deletions(-) -- 2.39.2

2 6

[PATCH OLK-6.6] kernel: add OPENEULER_VERSION_CODE to version.h
by Wei Li 05 Jan '24

05 Jan '24

From: Xie XiuQi <xiexiuqi(a)huawei.com> hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I7LEXY --------------------------------------------- Add OPENEULER_VERSION_CODE to version.h, so modules or applications can identify the specific kernel version. In most cases, the out-of-tree module needs to identify the release version of the openEuler for interface adaptation. The existing OPENEULER_VERSION() and OPENEULER_VERSION_CODE() cannot distinguish between LTS versions and innovative versions. Therefore, a new macro OPENEULER_LTS is introduced. The version information is generated in "linux/version.h", as bellow: include/generated/uapi/linux/version.h ``` #define OPENEULER_LTS 0 // 1: LTS; 0: none LTS #define OPENEULER_MAJOR 9999 // OS Major version, 9999: none released #define OPENEULER_MINOR 0 // SP release version #define OPENEULER_VERSION(a,b) (((a) << 8) + (b)) #define OPENEULER_VERSION_CODE 2559744 ``` Use Makefile.oever to avoid future merge conflicts. Link:https://gitee.com/openeuler/kernel/issues/I575IT Link:https://gitee.com/openeuler/kernel/issues/I7761D Link:https://gitee.com/openeuler/kernel/pulls/1442 Link:https://gitee.com/openeuler/kernel/pulls/1219 Link:https://gitee.com/openeuler/kernel/pulls/195 Signed-off-by: Xie XiuQi <xiexiuqi(a)huawei.com> Signed-off-by: Wei Li <liwei391(a)huawei.com> --- Makefile | 11 ++++++++++- Makefile.oever | 4 ++++ 2 files changed, 14 insertions(+), 1 deletion(-) create mode 100644 Makefile.oever diff --git a/Makefile b/Makefile index 5c418efbe89b..e458c8265d75 100644 --- a/Makefile +++ b/Makefile @@ -1226,6 +1226,9 @@ uapi-asm-generic: # Generate some files # --------------------------------------------------------------------------- +# openEuler version variables +include $(srctree)/Makefile.oever + # KERNELRELEASE can change from a few different places, meaning version.h # needs to be updated, so this check is forced on all builds @@ -1250,7 +1253,13 @@ define filechk_version.h ((c) > 255 ? 255 : (c)))'; \ echo \#define LINUX_VERSION_MAJOR $(VERSION); \ echo \#define LINUX_VERSION_PATCHLEVEL $(PATCHLEVEL); \ - echo \#define LINUX_VERSION_SUBLEVEL $(SUBLEVEL) + echo \#define LINUX_VERSION_SUBLEVEL $(SUBLEVEL); \ + echo \#define OPENEULER_LTS $(OPENEULER_LTS); \ + echo \#define OPENEULER_MAJOR $(OPENEULER_MAJOR); \ + echo \#define OPENEULER_MINOR $(OPENEULER_MINOR); \ + echo '#define OPENEULER_VERSION(a,b) (((a) << 8) + (b))'; \ + echo \#define OPENEULER_VERSION_CODE $(shell \ + expr $(OPENEULER_MAJOR) \* 256 + $(OPENEULER_MINOR)) endef $(version_h): PATCHLEVEL := $(or $(PATCHLEVEL), 0) diff --git a/Makefile.oever b/Makefile.oever new file mode 100644 index 000000000000..96beccf3569c --- /dev/null +++ b/Makefile.oever @@ -0,0 +1,4 @@ +# SPDX-License-Identifier: GPL-2.0 +OPENEULER_LTS = 0 +OPENEULER_MAJOR = 9999 +OPENEULER_MINOR = 0 -- 2.25.1

2 1

[PATCH OLK-6.6 00/10] Backport etmem feature to OLK 6.6
by Yuchen Tang 05 Jan '24

05 Jan '24

v1 -> v2 - make independent patches to export symbols we need - make independent patches for etmem scan and swap config - fix indentation and code style problems Yuchen Tang (10): etmem: a tiered memory manager etmem: etmem scan, the page hotness tracker mm: Export symbol __pud_trans_huge_lock mm: Export symbol walk_page_range etmem: add etmem scan feature to openEuler mm: Export symbol reclaim_pages etmem: add etmem swap feature to openEuler etmem: add original kernel swap enabled options etmem: add swapcache reclaim to etmem etmem: enable etmem configurations arch/arm64/configs/openeuler_defconfig | 3 + arch/x86/configs/openeuler_defconfig | 3 + fs/proc/Makefile | 3 + fs/proc/base.c | 8 + fs/proc/etmem_proc.c | 216 ++++ fs/proc/etmem_scan.c | 1383 ++++++++++++++++++++++++ fs/proc/etmem_scan.h | 149 +++ fs/proc/etmem_swap.c | 283 +++++ fs/proc/internal.h | 4 + include/linux/etmem.h | 75 ++ include/linux/mm.h | 4 + include/linux/mm_types.h | 5 + include/linux/swap.h | 1 + include/uapi/asm-generic/mman-common.h | 4 + mm/Kconfig | 33 + mm/Makefile | 1 + mm/etmem.c | 384 +++++++ mm/huge_memory.c | 1 + mm/internal.h | 1 - mm/madvise.c | 16 + mm/pagewalk.c | 1 + mm/swap_state.c | 4 + mm/vmscan.c | 16 + virt/kvm/kvm_main.c | 8 + 24 files changed, 2605 insertions(+), 1 deletion(-) create mode 100644 fs/proc/etmem_proc.c create mode 100644 fs/proc/etmem_scan.c create mode 100644 fs/proc/etmem_scan.h create mode 100644 fs/proc/etmem_swap.c create mode 100644 include/linux/etmem.h create mode 100644 mm/etmem.c -- 2.33.0

2 11

[PATCH OLK-6.6 v6 0/2] optimize inlining
by Yuntao Liu 05 Jan '24

05 Jan '24

Guo Xuenan (2): Revert "compiler: remove CONFIG_OPTIMIZE_INLINING entirely" make OPTIMIZE_INLINING config editable arch/arm64/kvm/sys_regs.h | 7 +++++- arch/x86/configs/i386_defconfig | 1 + arch/x86/configs/x86_64_defconfig | 1 + .../net/ethernet/mellanox/mlx5/core/health.c | 24 +++++++++++++++++++ .../ethernet/mellanox/mlx5/core/mlx5_core.h | 24 ------------------- .../pci/hive_isp_css_include/print_support.h | 4 ++++ include/linux/compiler_types.h | 8 +++++++ include/trace/trace_events.h | 2 +- kernel/configs/tiny.config | 1 + lib/Kconfig.debug | 13 ++++++++++ 10 files changed, 59 insertions(+), 26 deletions(-) -- 2.34.1

2 3

[PATCH OLK-6.6 v3 0/8] introduce smart_grid zone
by Yipeng Zou 05 Jan '24

05 Jan '24

The patch sets include two parts: 1. patch 1~3: Rebase smart_grid from OLK-5.10 to OLK-6.6 2. patch 4~7: introduce smart_grid zone qos and cpufreq Since V2: 1. move new sysctl form sysctl.c to fair.c 2. add smart_grid cmdline Since V1: 1. Don't init smart_grid cpufreq with dynamic_affinity disable. 2. Forbid smart_grid cgroup interface with dynamic_affinity disable. Hui Tang (1): sched: Introduce smart grid scheduling strategy for cfs Wang ShaoBo (2): sched: smart grid: init sched_grid_qos structure on QOS purpose config: enable CONFIG_QOS_SCHED_SMART_GRID by default Yipeng Zou (5): sched: introduce smart grid qos zone smart_grid: introduce /proc/pid/smart_grid_level smart_grid: introduce smart_grid_strategy_ctrl sysctl smart_grid: cpufreq: introduce smart_grid cpufreq control smart_grid: introduce smart_grid cmdline arch/arm64/configs/openeuler_defconfig | 1 + arch/x86/configs/openeuler_defconfig | 1 + drivers/cpufreq/cpufreq.c | 244 +++++++++++ fs/proc/array.c | 16 + fs/proc/base.c | 84 ++++ include/linux/cpufreq.h | 11 + include/linux/sched.h | 31 ++ include/linux/sched/grid_qos.h | 135 +++++++ init/Kconfig | 13 + kernel/fork.c | 14 + kernel/sched/Makefile | 1 + kernel/sched/core.c | 181 +++++++++ kernel/sched/fair.c | 538 ++++++++++++++++++++++++- kernel/sched/grid/Makefile | 2 + kernel/sched/grid/internal.h | 6 + kernel/sched/grid/power.c | 27 ++ kernel/sched/grid/qos.c | 279 +++++++++++++ kernel/sched/grid/stat.c | 47 +++ kernel/sched/sched.h | 48 ++- mm/mempolicy.c | 11 +- 20 files changed, 1679 insertions(+), 11 deletions(-) create mode 100644 include/linux/sched/grid_qos.h create mode 100644 kernel/sched/grid/Makefile create mode 100644 kernel/sched/grid/internal.h create mode 100644 kernel/sched/grid/power.c create mode 100644 kernel/sched/grid/qos.c create mode 100644 kernel/sched/grid/stat.c -- 2.34.1

2 9

[openEuler-23.09] livepatch/core: Fix miss disable ro for MOD_RO_AFTER_INIT memory
by Zheng Yejian 05 Jan '24

05 Jan '24

hulk inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I8HNHM -------------------------------- It was missed to set MOD_RO_AFTER_INIT type memory to be writable when porting e1e064674b6b ("livepatch/core: Revert module_enable_ro and module_disable_ro"). Thanks the report from laokz in https://gitee.com/laokz/klpmake/issues/I8HFMR. Fixes: e1e064674b6b ("livepatch/core: Revert module_enable_ro and module_disable_ro") Signed-off-by: Zheng Yejian <zhengyejian1(a)huawei.com> --- kernel/module/strict_rwx.c | 1 + 1 file changed, 1 insertion(+) diff --git a/kernel/module/strict_rwx.c b/kernel/module/strict_rwx.c index 397e18f50517..575bf99c723a 100644 --- a/kernel/module/strict_rwx.c +++ b/kernel/module/strict_rwx.c @@ -46,6 +46,7 @@ void module_disable_ro(const struct module *mod) module_set_memory(mod, MOD_INIT_TEXT, set_memory_rw); module_set_memory(mod, MOD_RODATA, set_memory_rw); module_set_memory(mod, MOD_INIT_RODATA, set_memory_rw); + module_set_memory(mod, MOD_RO_AFTER_INIT, set_memory_rw); } #endif /* CONFIG_LIVEPATCH_WO_FTRACE */ -- 2.25.1

1 0

[PATCH OLK-6.6 v5 0/2] optimize inlining
by Yuntao Liu 05 Jan '24

05 Jan '24

*** BLURB HERE *** Guo Xuenan (2): Revert "compiler: remove CONFIG_OPTIMIZE_INLINING entirely" make OPTIMIZE_INLINING config editable arch/arm64/kvm/sys_regs.h | 7 +++++- arch/x86/configs/i386_defconfig | 1 + arch/x86/configs/x86_64_defconfig | 1 + .../net/ethernet/mellanox/mlx5/core/health.c | 24 +++++++++++++++++++ .../ethernet/mellanox/mlx5/core/mlx5_core.h | 24 ------------------- .../pci/hive_isp_css_include/print_support.h | 4 ++++ include/linux/compiler_types.h | 8 +++++++ include/trace/trace_events.h | 2 +- kernel/configs/tiny.config | 1 + lib/Kconfig.debug | 13 ++++++++++ 10 files changed, 59 insertions(+), 26 deletions(-) -- 2.34.1

2 3

[PATCH V3 OLK-6.6 00/12] Backport vDPA migration support patches
by Jiang Dongxu 05 Jan '24

05 Jan '24

From: jiangdongxu <jiangdongxu1(a)huawei.com> Backport vDPA migration support patches merged on branch OLK-5.10 V3: fix compile error of kernel-6.6 new vdpa driver pds_vdpa V2: fix compile error of kernel-6.6 new vdpa driver vduse and snet_vdpa jiangdongxu (12): vdpa: add log operations vhost-vdpa: add uAPI for logging vdpa: add device state operations vhost-vdpa: add uAPI for device buffer vdpa: add vdpa device migration status ops vhost-vdpa: add uAPI for device migration status vhost: add VHOST feature VHOST_BACKEND_F_BYTEMAPLOG vhost-vdpa: Allow transparent MSI IOV vhost-vdpa: fix msi irq request err vhost-vdpa: allow set feature VHOST_F_LOG_ALL when been negotiated. vhost-vdpa: add reset state params to indicate reset level vdpa: add vmstate header file drivers/vdpa/ifcvf/ifcvf_main.c | 2 +- drivers/vdpa/mlx5/net/mlx5_vnet.c | 2 +- drivers/vdpa/pds/vdpa_dev.c | 2 +- drivers/vdpa/solidrun/snet_main.c | 2 +- drivers/vdpa/vdpa_sim/vdpa_sim.c | 2 +- drivers/vdpa/vdpa_user/vduse_dev.c | 2 +- drivers/vdpa/virtio_pci/vp_vdpa.c | 2 +- drivers/vhost/vdpa.c | 232 ++++++++++++++++++++++++++--- drivers/virtio/virtio_vdpa.c | 2 +- include/linux/vdpa.h | 56 ++++++- include/linux/vdpa_vmstate.h | 182 ++++++++++++++++++++++ include/uapi/linux/vhost.h | 12 ++ include/uapi/linux/vhost_types.h | 19 +++ 13 files changed, 486 insertions(+), 31 deletions(-) create mode 100644 include/linux/vdpa_vmstate.h -- 2.27.0

2 13

[PATCH OLK-6.6 v4 0/2] optimize inlining
by Yuntao Liu 05 Jan '24

05 Jan '24

*** BLURB HERE *** Guo Xuenan (2): Revert "compiler: remove CONFIG_OPTIMIZE_INLINING entirely" make OPTIMIZE_INLINING config editable arch/arm64/kvm/sys_regs.h | 7 +++++- arch/x86/configs/i386_defconfig | 1 + arch/x86/configs/x86_64_defconfig | 1 + .../net/ethernet/mellanox/mlx5/core/health.c | 24 +++++++++++++++++++ .../ethernet/mellanox/mlx5/core/mlx5_core.h | 24 ------------------- .../pci/hive_isp_css_include/print_support.h | 4 ++++ include/linux/compiler_types.h | 8 +++++++ kernel/configs/tiny.config | 1 + lib/Kconfig.debug | 13 ++++++++++ 9 files changed, 58 insertions(+), 25 deletions(-) -- 2.34.1

2 3

[PATCH openEuler-1.0-LTS] net/sched: cbs: Fix not adding cbs instance to list
by Lu Wei 05 Jan '24

05 Jan '24

From: Vinicius Costa Gomes <vinicius.gomes(a)intel.com> mainline inclusion from mainline-v5.4-rc1 commit 3e8b9bfa110896f95d602d8c98d5f9d67e41d78c category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/I8TS78 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?… -------------------------------- When removing a cbs instance when offloading is enabled, the crash below can be observed. The problem happens because that when offloading is enabled, the cbs instance is not added to the list. Also, the current code doesn't handle correctly the case when offload is disabled without removing the qdisc: if the link speed changes the credit calculations will be wrong. When we create the cbs instance with offloading enabled, it's not added to the notification list, when later we disable offloading, it's not in the list, so link speed changes will not affect it. The solution for both issues is the same, add the cbs instance being created unconditionally to the global list, even if the link state notification isn't useful "right now". Crash log: [518758.189866] BUG: kernel NULL pointer dereference, address: 0000000000000000 [518758.189870] #PF: supervisor read access in kernel mode [518758.189871] #PF: error_code(0x0000) - not-present page [518758.189872] PGD 0 P4D 0 [518758.189874] Oops: 0000 [#1] SMP PTI [518758.189876] CPU: 3 PID: 4825 Comm: tc Not tainted 5.2.9 #1 [518758.189877] Hardware name: Gigabyte Technology Co., Ltd. Z390 AORUS ULTRA/Z390 AORUS ULTRA-CF, BIOS F7 03/14/2019 [518758.189881] RIP: 0010:__list_del_entry_valid+0x29/0xa0 [518758.189883] Code: 90 48 b8 00 01 00 00 00 00 ad de 55 48 8b 17 4c 8b 47 08 48 89 e5 48 39 c2 74 27 48 b8 00 02 00 00 00 00 ad de 49 39 c0 74 2d <49> 8b 30 48 39 fe 75 3d 48 8b 52 08 48 39 f2 75 4c b8 01 00 00 00 [518758.189885] RSP: 0018:ffffa27e43903990 EFLAGS: 00010207 [518758.189887] RAX: dead000000000200 RBX: ffff8bce69f0f000 RCX: 0000000000000000 [518758.189888] RDX: 0000000000000000 RSI: ffff8bce69f0f064 RDI: ffff8bce69f0f1e0 [518758.189890] RBP: ffffa27e43903990 R08: 0000000000000000 R09: ffff8bce69e788c0 [518758.189891] R10: ffff8bce62acd400 R11: 00000000000003cb R12: ffff8bce69e78000 [518758.189892] R13: ffff8bce69f0f140 R14: 0000000000000000 R15: 0000000000000000 [518758.189894] FS: 00007fa1572c8f80(0000) GS:ffff8bce6e0c0000(0000) knlGS:0000000000000000 [518758.189895] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [518758.189896] CR2: 0000000000000000 CR3: 000000040a398006 CR4: 00000000003606e0 [518758.189898] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [518758.189899] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [518758.189900] Call Trace: [518758.189904] cbs_destroy+0x32/0xa0 [sch_cbs] [518758.189906] qdisc_destroy+0x45/0x120 [518758.189907] qdisc_put+0x25/0x30 [518758.189908] qdisc_graft+0x2c1/0x450 [518758.189910] tc_get_qdisc+0x1c8/0x310 [518758.189912] ? get_page_from_freelist+0x91a/0xcb0 [518758.189914] rtnetlink_rcv_msg+0x293/0x360 [518758.189916] ? kmem_cache_alloc_node_trace+0x178/0x260 [518758.189918] ? __kmalloc_node_track_caller+0x38/0x50 [518758.189920] ? rtnl_calcit.isra.0+0xf0/0xf0 [518758.189922] netlink_rcv_skb+0x48/0x110 [518758.189923] rtnetlink_rcv+0x10/0x20 [518758.189925] netlink_unicast+0x15b/0x1d0 [518758.189926] netlink_sendmsg+0x1ea/0x380 [518758.189929] sock_sendmsg+0x2f/0x40 [518758.189930] ___sys_sendmsg+0x295/0x2f0 [518758.189932] ? ___sys_recvmsg+0x151/0x1e0 [518758.189933] ? do_wp_page+0x7e/0x450 [518758.189935] __sys_sendmsg+0x48/0x80 [518758.189937] __x64_sys_sendmsg+0x1a/0x20 [518758.189939] do_syscall_64+0x53/0x1f0 [518758.189941] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [518758.189942] RIP: 0033:0x7fa15755169a [518758.189944] Code: 48 c7 c0 ff ff ff ff eb be 0f 1f 80 00 00 00 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 18 b8 2e 00 00 00 c5 fc 77 0f 05 <48> 3d 00 f0 ff ff 77 5e c3 0f 1f 44 00 00 48 83 ec 28 89 54 24 1c [518758.189946] RSP: 002b:00007ffda58b60b8 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [518758.189948] RAX: ffffffffffffffda RBX: 000055e4b836d9a0 RCX: 00007fa15755169a [518758.189949] RDX: 0000000000000000 RSI: 00007ffda58b6128 RDI: 0000000000000003 [518758.189951] RBP: 00007ffda58b6190 R08: 0000000000000001 R09: 000055e4b9d848a0 [518758.189952] R10: 0000000000000000 R11: 0000000000000246 R12: 000000005d654b49 [518758.189953] R13: 0000000000000000 R14: 00007ffda58b6230 R15: 00007ffda58b6210 [518758.189955] Modules linked in: sch_cbs sch_etf sch_mqprio netlink_diag unix_diag e1000e igb intel_pch_thermal thermal video backlight pcc_cpufreq [518758.189960] CR2: 0000000000000000 [518758.189961] ---[ end trace 6a13f7aaf5376019 ]--- [518758.189963] RIP: 0010:__list_del_entry_valid+0x29/0xa0 [518758.189964] Code: 90 48 b8 00 01 00 00 00 00 ad de 55 48 8b 17 4c 8b 47 08 48 89 e5 48 39 c2 74 27 48 b8 00 02 00 00 00 00 ad de 49 39 c0 74 2d <49> 8b 30 48 39 fe 75 3d 48 8b 52 08 48 39 f2 75 4c b8 01 00 00 00 [518758.189967] RSP: 0018:ffffa27e43903990 EFLAGS: 00010207 [518758.189968] RAX: dead000000000200 RBX: ffff8bce69f0f000 RCX: 0000000000000000 [518758.189969] RDX: 0000000000000000 RSI: ffff8bce69f0f064 RDI: ffff8bce69f0f1e0 [518758.189971] RBP: ffffa27e43903990 R08: 0000000000000000 R09: ffff8bce69e788c0 [518758.189972] R10: ffff8bce62acd400 R11: 00000000000003cb R12: ffff8bce69e78000 [518758.189973] R13: ffff8bce69f0f140 R14: 0000000000000000 R15: 0000000000000000 [518758.189975] FS: 00007fa1572c8f80(0000) GS:ffff8bce6e0c0000(0000) knlGS:0000000000000000 [518758.189976] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [518758.189977] CR2: 0000000000000000 CR3: 000000040a398006 CR4: 00000000003606e0 [518758.189979] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [518758.189980] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Fixes: e0a7683d30e9 ("net/sched: cbs: fix port_rate miscalculation") Signed-off-by: Vinicius Costa Gomes <vinicius.gomes(a)intel.com> Acked-by: Cong Wang <xiyou.wangcong(a)gmail.com> Signed-off-by: David S. Miller <davem(a)davemloft.net> conflict: net/sched/sch_cbs.c Signed-off-by: Lu Wei <luwei32(a)huawei.com> --- net/sched/sch_cbs.c | 30 +++++++++++++----------------- 1 file changed, 13 insertions(+), 17 deletions(-) diff --git a/net/sched/sch_cbs.c b/net/sched/sch_cbs.c index 72111842d46a..8d63b48d74f7 100644 --- a/net/sched/sch_cbs.c +++ b/net/sched/sch_cbs.c @@ -406,7 +406,6 @@ static int cbs_init(struct Qdisc *sch, struct nlattr *opt, { struct cbs_sched_data *q = qdisc_priv(sch); struct net_device *dev = qdisc_dev(sch); - int err; if (!opt) { NL_SET_ERR_MSG(extack, "Missing CBS qdisc options which are mandatory"); @@ -418,6 +417,10 @@ static int cbs_init(struct Qdisc *sch, struct nlattr *opt, if (!q->qdisc) return -ENOMEM; + spin_lock(&cbs_list_lock); + list_add(&q->cbs_list, &cbs_list); + spin_unlock(&cbs_list_lock); + qdisc_hash_add(q->qdisc, false); q->queue = sch->dev_queue - netdev_get_tx_queue(dev, 0); @@ -427,17 +430,7 @@ static int cbs_init(struct Qdisc *sch, struct nlattr *opt, qdisc_watchdog_init(&q->watchdog, sch); - err = cbs_change(sch, opt, extack); - if (err) - return err; - - if (!q->offload) { - spin_lock(&cbs_list_lock); - list_add(&q->cbs_list, &cbs_list); - spin_unlock(&cbs_list_lock); - } - - return 0; + return cbs_change(sch, opt, extack); } static void cbs_destroy(struct Qdisc *sch) @@ -445,15 +438,18 @@ static void cbs_destroy(struct Qdisc *sch) struct cbs_sched_data *q = qdisc_priv(sch); struct net_device *dev = qdisc_dev(sch); - spin_lock(&cbs_list_lock); - list_del(&q->cbs_list); - spin_unlock(&cbs_list_lock); + /* Nothing to do if we couldn't create the underlying qdisc */ + if (!q->qdisc) + return; qdisc_watchdog_cancel(&q->watchdog); cbs_disable_offload(dev, q); - if (q->qdisc) - qdisc_destroy(q->qdisc); + spin_lock(&cbs_list_lock); + list_del(&q->cbs_list); + spin_unlock(&cbs_list_lock); + + qdisc_destroy(q->qdisc); } static int cbs_dump(struct Qdisc *sch, struct sk_buff *skb) -- 2.34.1

2 1

[PATCH V2 OLK-6.6 00/12] Backport vDPA migration support patches
by Jiang Dongxu 05 Jan '24

05 Jan '24

From: jiangdongxu <jiangdongxu1(a)huawei.com> Backport vDPA migration support patches merged on branch OLK-5.10 V2: fix compile error of kernel-6.6 new vdpa driver vduse and snet_vdpa jiangdongxu (12): vdpa: add log operations vhost-vdpa: add uAPI for logging vdpa: add device state operations vhost-vdpa: add uAPI for device buffer vdpa: add vdpa device migration status ops vhost-vdpa: add uAPI for device migration status vhost: add VHOST feature VHOST_BACKEND_F_BYTEMAPLOG vhost-vdpa: Allow transparent MSI IOV vhost-vdpa: fix msi irq request err vhost-vdpa: allow set feature VHOST_F_LOG_ALL when been negotiated. vhost-vdpa: add reset state params to indicate reset level vdpa: add vmstate header file drivers/vdpa/ifcvf/ifcvf_main.c | 2 +- drivers/vdpa/mlx5/net/mlx5_vnet.c | 2 +- drivers/vdpa/solidrun/snet_main.c | 2 +- drivers/vdpa/vdpa_sim/vdpa_sim.c | 2 +- drivers/vdpa/vdpa_user/vduse_dev.c | 2 +- drivers/vdpa/virtio_pci/vp_vdpa.c | 2 +- drivers/vhost/vdpa.c | 232 ++++++++++++++++++++++++++--- drivers/virtio/virtio_vdpa.c | 2 +- include/linux/vdpa.h | 56 ++++++- include/linux/vdpa_vmstate.h | 182 ++++++++++++++++++++++ include/uapi/linux/vhost.h | 12 ++ include/uapi/linux/vhost_types.h | 19 +++ 12 files changed, 485 insertions(+), 30 deletions(-) create mode 100644 include/linux/vdpa_vmstate.h -- 2.27.0

2 13

[PATCH openEuler-22.03-LTS-SP1 0/5] md: protect md_thread with rcu
by Li Lingfeng 05 Jan '24

05 Jan '24

Protect md_thread with rcu. Yu Kuai (5): md: factor out a helper to wake up md_thread directly dm-raid: remove useless checking in raid_message() md/bitmap: always wake up md_thread in timeout_store md/bitmap: factor out a helper to set timeout md: protect md_thread with rcu drivers/md/dm-raid.c | 4 +- drivers/md/md-bitmap.c | 43 +++++++++++-------- drivers/md/md-cluster.c | 17 +++++--- drivers/md/md-multipath.c | 4 +- drivers/md/md.c | 88 +++++++++++++++++++++------------------ drivers/md/md.h | 8 ++-- drivers/md/raid1.c | 7 ++-- drivers/md/raid1.h | 2 +- drivers/md/raid10.c | 20 +++++---- drivers/md/raid10.h | 2 +- drivers/md/raid5-cache.c | 22 ++++++---- drivers/md/raid5.c | 15 +++---- drivers/md/raid5.h | 2 +- 13 files changed, 132 insertions(+), 102 deletions(-) -- 2.31.1

2 6

[PATCH openEuler-22.03-LTS 0/5] md: protect md_thread with rcu
by Li Lingfeng 05 Jan '24

05 Jan '24

Protect md_thread with rcu. Yu Kuai (5): md: factor out a helper to wake up md_thread directly dm-raid: remove useless checking in raid_message() md/bitmap: always wake up md_thread in timeout_store md/bitmap: factor out a helper to set timeout md: protect md_thread with rcu drivers/md/dm-raid.c | 4 +- drivers/md/md-bitmap.c | 43 +++++++++++-------- drivers/md/md-cluster.c | 17 +++++--- drivers/md/md-multipath.c | 4 +- drivers/md/md.c | 88 +++++++++++++++++++++------------------ drivers/md/md.h | 8 ++-- drivers/md/raid1.c | 7 ++-- drivers/md/raid1.h | 2 +- drivers/md/raid10.c | 20 +++++---- drivers/md/raid10.h | 2 +- drivers/md/raid5-cache.c | 22 ++++++---- drivers/md/raid5.c | 15 +++---- drivers/md/raid5.h | 2 +- 13 files changed, 132 insertions(+), 102 deletions(-) -- 2.31.1

2 6

[PATCH OLK-6.6 0/6] Backport etmem feature to OLK 6.6
by Yuchen Tang 04 Jan '24

04 Jan '24

Yuchen Tang (6): etmem: add etmem scan feature to openEuler mm: Export symbol reclaim_pages etmem: add etmem swap feature to openEuler etmem: add original kernel swap enabled options etmem: add swapcache reclaim to etmem etmem: enable etmem configurations arch/arm64/configs/openeuler_defconfig | 3 + arch/x86/configs/openeuler_defconfig | 3 + fs/proc/Makefile | 3 + fs/proc/base.c | 8 + fs/proc/etmem_proc.c | 216 ++++ fs/proc/etmem_scan.c | 1383 ++++++++++++++++++++++++ fs/proc/etmem_scan.h | 149 +++ fs/proc/etmem_swap.c | 283 +++++ fs/proc/internal.h | 4 + include/linux/etmem.h | 75 ++ include/linux/mm.h | 4 + include/linux/mm_types.h | 7 + include/linux/swap.h | 2 +- include/uapi/asm-generic/mman-common.h | 4 + mm/Kconfig | 33 + mm/Makefile | 1 + mm/etmem.c | 384 +++++++ mm/huge_memory.c | 3 + mm/internal.h | 1 - mm/madvise.c | 16 + mm/pagewalk.c | 3 + mm/swap_state.c | 4 + mm/vmscan.c | 20 + virt/kvm/kvm_main.c | 6 + 24 files changed, 2613 insertions(+), 2 deletions(-) create mode 100644 fs/proc/etmem_proc.c create mode 100644 fs/proc/etmem_scan.c create mode 100644 fs/proc/etmem_scan.h create mode 100644 fs/proc/etmem_swap.c create mode 100644 include/linux/etmem.h create mode 100644 mm/etmem.c -- 2.33.0

3 8

[PATCH OLK-5.10 0/3] cppc_cpufreq: use policy->cpu in cppc_set_perf()
by Zeng Heng 04 Jan '24

04 Jan '24

Ionela Voinescu (2): cppc_cpufreq: clean up cpu, cpu_num and cpunum variable use cppc_cpufreq: use policy->cpu as driver of frequency setting Zeng Heng (1): cppc_cpufreq: get cpunum from fb_ctrs instead of smp_processor_id() drivers/cpufreq/cppc_cpufreq.c | 166 +++++++++++++++++---------------- 1 file changed, 85 insertions(+), 81 deletions(-) -- 2.25.1

2 4

[PATCH OLK-6.6] arm64: clear_page: Add cmdline option for clear_page() by STNP
by Wei Li 04 Jan '24

04 Jan '24

hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I3ZN72 CVE: NA --------------------------- Generally, clear_page() clear the page through 'dc zva', while the page may not be used immediately mostly, so the cache flush is in vain. Commit f0616abd4e67 ("arm64: clear_page() shouldn't use DC ZVA when DCZID_EL0.DZP == 1") introduced an alternative implementation by 'stnp', also bring performance profit in some case. Make it switchable through boot cmdline 'mm.use_clearpage_stnp'. In the hugetlb clear test, we gained about 53.7% performance improvement: Set mm.use_clearpage_stnp = 0 | Set mm.use_clearpage_stnp = 1 [root@localhost liwei]# ./a.out 50 20 | [root@localhost liwei]# ./a.out 50 20 size is 50 Gib, test times is 20 | size is 50 Gib, test times is 20 test_time[0] : use 8.438046 sec | test_time[0] : use 3.722682 sec test_time[1] : use 8.028493 sec | test_time[1] : use 3.640274 sec test_time[2] : use 8.646547 sec | test_time[2] : use 4.095052 sec test_time[3] : use 8.122490 sec | test_time[3] : use 3.998446 sec test_time[4] : use 8.053038 sec | test_time[4] : use 4.084259 sec test_time[5] : use 8.843512 sec | test_time[5] : use 3.933871 sec test_time[6] : use 8.308906 sec | test_time[6] : use 3.934334 sec test_time[7] : use 8.093817 sec | test_time[7] : use 3.869142 sec test_time[8] : use 8.303504 sec | test_time[8] : use 3.902916 sec test_time[9] : use 8.178336 sec | test_time[9] : use 3.541885 sec test_time[10] : use 8.003625 sec | test_time[10] : use 3.595554 sec test_time[11] : use 8.163807 sec | test_time[11] : use 3.583813 sec test_time[12] : use 8.267464 sec | test_time[12] : use 3.863033 sec test_time[13] : use 8.055326 sec | test_time[13] : use 3.770953 sec test_time[14] : use 8.246986 sec | test_time[14] : use 3.808006 sec test_time[15] : use 8.546992 sec | test_time[15] : use 3.653194 sec test_time[16] : use 8.727256 sec | test_time[16] : use 3.722395 sec test_time[17] : use 8.288951 sec | test_time[17] : use 3.683508 sec test_time[18] : use 8.019322 sec | test_time[18] : use 4.253087 sec test_time[19] : use 8.250685 sec | test_time[19] : use 4.082845 sec hugetlb test end! | hugetlb test end! Signed-off-by: Wei Li <liwei391(a)huawei.com> --- arch/arm64/kernel/cpufeature.c | 24 ++++++++++++++++++++++++ arch/arm64/lib/clear_page.S | 3 ++- arch/arm64/tools/cpucaps | 1 + 3 files changed, 27 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 455e72ce080a..3517dd2b0358 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -2242,6 +2242,24 @@ cpucap_panic_on_conflict(const struct arm64_cpu_capabilities *cap) return !!(cap->type & ARM64_CPUCAP_PANIC_ON_CONFLICT); } +static bool use_clearpage_stnp; + +static int __init early_use_clearpage_stnp(char *p) +{ + return strtobool(p, &use_clearpage_stnp); +} +early_param("mm.use_clearpage_stnp", early_use_clearpage_stnp); + +static bool is_datazero_prohibited(const struct arm64_cpu_capabilities *entry, + int scope) +{ + /* Active if DC ZVA is prohibited */ + if (read_sysreg(dczid_el0) & BIT(DCZID_EL0_DZP_SHIFT)) + return true; + + return use_clearpage_stnp; +} + static const struct arm64_cpu_capabilities arm64_features[] = { { .capability = ARM64_ALWAYS_BOOT, @@ -2722,6 +2740,12 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .matches = has_cpuid_feature, ARM64_CPUID_FIELDS(ID_AA64MMFR2_EL1, EVT, IMP) }, + { + .desc = "Data Zero Prohibited", + .capability = ARM64_HAS_NO_DCZVA, + .type = ARM64_CPUCAP_SYSTEM_FEATURE, + .matches = is_datazero_prohibited, + }, {}, }; diff --git a/arch/arm64/lib/clear_page.S b/arch/arm64/lib/clear_page.S index ebde40e7fa2b..3884572296d6 100644 --- a/arch/arm64/lib/clear_page.S +++ b/arch/arm64/lib/clear_page.S @@ -7,6 +7,7 @@ #include <linux/const.h> #include <asm/assembler.h> #include <asm/page.h> +#include <asm/alternative.h> /* * Clear page @dest @@ -16,7 +17,7 @@ */ SYM_FUNC_START(__pi_clear_page) mrs x1, dczid_el0 - tbnz x1, #4, 2f /* Branch if DC ZVA is prohibited */ + ALTERNATIVE("nop", "b 2f", ARM64_HAS_NO_DCZVA) and w1, w1, #0xf mov x2, #4 lsl x1, x2, x1 diff --git a/arch/arm64/tools/cpucaps b/arch/arm64/tools/cpucaps index 569ecec76c16..88f1f4022b8f 100644 --- a/arch/arm64/tools/cpucaps +++ b/arch/arm64/tools/cpucaps @@ -39,6 +39,7 @@ HAS_LDAPR HAS_LSE_ATOMICS HAS_MOPS HAS_NESTED_VIRT +HAS_NO_DCZVA HAS_NO_FPSIMD HAS_NO_HW_PREFETCH HAS_PAN -- 2.25.1

2 1

[PATCH OLK-6.6 0/4] KABI reservation for driver
by Yi Yang 04 Jan '24

04 Jan '24

KABI reservation for tty/mtd/mmc/ipmi Yi Yang (4): tty: kabi: KABI reservation for tty mtd: kabi: KABI reservation for mtd mmc: kabi: KABI reservation for mmc ipmi: kabi: KABI reservation for ipmi drivers/char/ipmi/ipmi_msghandler.c | 6 ++++++ include/linux/ipmi.h | 2 ++ include/linux/ipmi_smi.h | 3 +++ include/linux/mmc/card.h | 6 ++++++ include/linux/mmc/host.h | 8 ++++++++ include/linux/mtd/mtd.h | 8 ++++++++ include/linux/mtd/partitions.h | 2 ++ include/linux/tty.h | 7 +++++++ include/linux/tty_buffer.h | 4 ++++ include/linux/tty_driver.h | 3 +++ include/linux/tty_ldisc.h | 3 +++ include/linux/tty_port.h | 3 +++ 12 files changed, 55 insertions(+) -- 2.25.1

2 5

[PATCH OLK-6.6 0/2] kabi reserve for memcg and cgroup bpf
by Lu Jialin 04 Jan '24

04 Jan '24

Lu Jialin (2): memcg/kabi: reserve space for memcg related structures cgroup_bpf/kabi: reserve space for cgroup_bpf releated structure include/linux/bpf-cgroup-defs.h | 28 ++++++++++++++++++++++++++++ include/linux/bpf.h | 11 +++++++++++ include/linux/memcontrol.h | 28 ++++++++++++++++++++++++++++ mm/memcontrol.c | 7 +++++++ 4 files changed, 74 insertions(+) -- 2.34.1

2 3

[PATCH OLK-6.6 00/12] Backport vDPA migration support patches
by Jiang Dongxu 04 Jan '24

04 Jan '24

From: jiangdongxu <jiangdongxu1(a)huawei.com> Backport vDPA migration support patches merged on branch OLK-5.10. jiangdongxu (12): vdpa: add log operations vhost-vdpa: add uAPI for logging vdpa: add device state operations vhost-vdpa: add uAPI for device buffer vdpa: add vdpa device migration status ops vhost-vdpa: add uAPI for device migration status vhost: add VHOST feature VHOST_BACKEND_F_BYTEMAPLOG vhost-vdpa: Allow transparent MSI IOV vhost-vdpa: fix msi irq request err vhost-vdpa: allow set feature VHOST_F_LOG_ALL when been negotiated. vhost-vdpa: add reset state params to indicate reset level vdpa: add vmstate header file drivers/vdpa/ifcvf/ifcvf_main.c | 2 +- drivers/vdpa/mlx5/net/mlx5_vnet.c | 2 +- drivers/vdpa/vdpa_sim/vdpa_sim.c | 2 +- drivers/vdpa/virtio_pci/vp_vdpa.c | 2 +- drivers/vhost/vdpa.c | 233 +++++++++++++++++++++++++++--- drivers/virtio/virtio_vdpa.c | 2 +- include/linux/vdpa.h | 56 ++++++- include/linux/vdpa_vmstate.h | 182 +++++++++++++++++++++++ include/uapi/linux/vhost.h | 12 ++ include/uapi/linux/vhost_types.h | 19 +++ 10 files changed, 484 insertions(+), 28 deletions(-) create mode 100644 include/linux/vdpa_vmstate.h -- 2.27.0

2 13

[PATCH OLK-6.6] scsi: sd: unregister device if device_add_disk() failed in sd_probe()
by Li Nan 04 Jan '24

04 Jan '24

hulk inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I8TPCX ---------------------------------------- "if device_add() succeeds, you should call device_del() when you want to get rid of it." In sd_probe(), device_add_disk() fails when device_add() has already succeeded, so change put_device() to device_unregister() to ensure device resources are released. Fixes: 2a7a891f4c40 ("scsi: sd: Add error handling support for add_disk()") Signed-off-by: Li Nan <linan122(a)huawei.com> --- drivers/scsi/sd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c index 6effa13039f3..72a7e80b6401 100644 --- a/drivers/scsi/sd.c +++ b/drivers/scsi/sd.c @@ -3725,7 +3725,7 @@ static int sd_probe(struct device *dev) error = device_add_disk(dev, gd, NULL); if (error) { - put_device(&sdkp->disk_dev); + device_unregister(&sdkp->disk_dev); put_disk(gd); goto out; } -- 2.39.2

2 1

[PATCH OLK-6.6] ubi: block: fix memleak in ubiblock_create()
by Li Nan 04 Jan '24

04 Jan '24

hulk inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I8P68G ------------------------------- If idr_alloc() fails, dev->gd will be put after goto out_cleanup_disk in ubiblock_create(), but dev->gd has not been assigned yet at this time, and gd will not be put anymore. Fix it by putting gd directly. Signed-off-by: Li Nan <linan122(a)huawei.com> --- drivers/mtd/ubi/block.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/mtd/ubi/block.c b/drivers/mtd/ubi/block.c index 309a42aeaa4c..654bd7372cd8 100644 --- a/drivers/mtd/ubi/block.c +++ b/drivers/mtd/ubi/block.c @@ -434,7 +434,7 @@ int ubiblock_create(struct ubi_volume_info *vi) list_del(&dev->list); idr_remove(&ubiblock_minor_idr, gd->first_minor); out_cleanup_disk: - put_disk(dev->gd); + put_disk(gd); out_free_tags: blk_mq_free_tag_set(&dev->tag_set); out_free_dev: -- 2.39.2

2 1

[PATCH V2 OLK-6.6] fs/dirty_pages: dump the number of dirty pages for each inode
by Zizhi Wo 04 Jan '24

04 Jan '24

From: yu kuai <yukuai3(a)huawei.com> hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I8OHM4?from=project-issue CVE: NA --------------------------- In order to analyse the IO performance when using buffer IO, it's useful to obtain the number of dirty pages for each inode in the filesystem. This feature depends on 'CONFIG_DIRTY_PAGES'. It creates 3 interfaces by using procfs. /proc/dirty/buffer_size for buffer allocation and release, /proc/dirty/page_threshold to filter result and /proc/dirty/dirty_list to get dirty pages. Signed-off-by: yu kuai <yukuai3(a)huawei.com> Signed-off-by: Zizhi Wo <wozizhi(a)huawei.com> --- arch/arm64/configs/openeuler_defconfig | 1 + arch/x86/configs/openeuler_defconfig | 1 + fs/Kconfig | 13 + fs/Makefile | 1 + fs/dirty_pages.c | 380 +++++++++++++++++++++++++ 5 files changed, 396 insertions(+) create mode 100644 fs/dirty_pages.c diff --git a/arch/arm64/configs/openeuler_defconfig b/arch/arm64/configs/openeuler_defconfig index 8e29cb161800..984ad448b5db 100644 --- a/arch/arm64/configs/openeuler_defconfig +++ b/arch/arm64/configs/openeuler_defconfig @@ -6909,6 +6909,7 @@ CONFIG_PROC_PAGE_MONITOR=y CONFIG_PROC_CHILDREN=y CONFIG_KERNFS=y CONFIG_SYSFS=y +CONFIG_DIRTY_PAGES=y CONFIG_TMPFS=y CONFIG_TMPFS_POSIX_ACL=y CONFIG_TMPFS_XATTR=y diff --git a/arch/x86/configs/openeuler_defconfig b/arch/x86/configs/openeuler_defconfig index b3e32559ab62..233b95fa4a84 100644 --- a/arch/x86/configs/openeuler_defconfig +++ b/arch/x86/configs/openeuler_defconfig @@ -8111,6 +8111,7 @@ CONFIG_PROC_PID_ARCH_STATUS=y CONFIG_PROC_CPU_RESCTRL=y CONFIG_KERNFS=y CONFIG_SYSFS=y +CONFIG_DIRTY_PAGES=y CONFIG_TMPFS=y CONFIG_TMPFS_POSIX_ACL=y CONFIG_TMPFS_XATTR=y diff --git a/fs/Kconfig b/fs/Kconfig index aa7e03cc1941..a2280cf98729 100644 --- a/fs/Kconfig +++ b/fs/Kconfig @@ -170,6 +170,19 @@ source "fs/proc/Kconfig" source "fs/kernfs/Kconfig" source "fs/sysfs/Kconfig" +config DIRTY_PAGES + bool "Dumps the number of dirty pages of each file" + depends on PROC_FS + default y + help + This config supports the rendering of dirty page data to the user, + which may be useful to analyze the IO performance when using buffer + IO. + + It create 3 interfaces by using procfs. /proc/dirty/buffer_size for + buffer allocation and release; /proc/dirty/page_threshold to filter + result; /proc/dirty/dirty_list to get dirty pages. + config TMPFS bool "Tmpfs virtual memory file system support (former shm fs)" depends on SHMEM diff --git a/fs/Makefile b/fs/Makefile index bb6164200a39..cc4735c23c32 100644 --- a/fs/Makefile +++ b/fs/Makefile @@ -44,6 +44,7 @@ obj-$(CONFIG_FS_POSIX_ACL) += posix_acl.o obj-$(CONFIG_NFS_COMMON) += nfs_common/ obj-$(CONFIG_COREDUMP) += coredump.o obj-$(CONFIG_SYSCTL) += drop_caches.o sysctls.o +obj-$(CONFIG_DIRTY_PAGES) += dirty_pages.o obj-$(CONFIG_FHANDLE) += fhandle.o obj-$(CONFIG_CGROUP_FILES) += filescontrol.o diff --git a/fs/dirty_pages.c b/fs/dirty_pages.c new file mode 100644 index 000000000000..19d9b0464176 --- /dev/null +++ b/fs/dirty_pages.c @@ -0,0 +1,380 @@ +#include <linux/proc_fs.h> +#include <linux/seq_file.h> +#include <linux/uaccess.h> +#include <linux/pagemap.h> +#include <linux/pagevec.h> +#include <linux/fs.h> +#include <linux/mm.h> +#include <linux/slab.h> +#include <linux/sched.h> +#include <linux/proc_fs.h> +#include <linux/kdev_t.h> +#include <linux/vmalloc.h> +#include <linux/module.h> +#include <linux/kernel.h> +#include <linux/init.h> +#include "internal.h" + +static char *buf_dirty; /* buffer to store number of dirty pages */ +static unsigned long buf_size; /* size of buffer in bytes */ +static unsigned long buff_num; /* size of buffer in number of pages */ +static unsigned long buff_limit; /* filter threshold of dirty pages*/ + +static struct proc_dir_entry *dirty_dir; + +unsigned long dirty_page_open; + +/* proc root directory */ +#define DIRTY_ROOT "dirty" +/* proc file for buffer allocation and release */ +#define DIRTY_SWITCH "buffer_size" +/* proc file to obtain diry pages of each inode */ +#define DIRTY_PAGES "dirty_list" +/* proc file to filter result */ +#define DIRTY_LIMIT "page_threshold" + +#define MAX_BUFF_SIZE 102400 + +static unsigned long dump_dirtypages_inode(struct inode *inode) +{ + XA_STATE(xas, &inode->i_mapping->i_pages, 0); + unsigned long nr_dirtys = 0; + void *page; + + xas_lock_irq(&xas); + xas_for_each_marked(&xas, page, (pgoff_t)-1, PAGECACHE_TAG_DIRTY) { + if (++nr_dirtys % XA_CHECK_SCHED) + continue; + + xas_pause(&xas); + xas_unlock_irq(&xas); + cond_resched(); + xas_lock_irq(&xas); + } + xas_unlock_irq(&xas); + + return nr_dirtys; +} + +static char *inode_filename(struct inode *inode, char *tmpname) +{ + struct dentry *dentry; + char *filename; + + dentry = d_find_alias(inode); + if (!dentry) + return ERR_PTR(-ENOENT); + + tmpname[PATH_MAX-1] = '\0'; + filename = dentry_path_raw(dentry, tmpname, PATH_MAX); + + dput(dentry); + + return filename; +} + +static inline bool is_sb_writable(struct super_block *sb) +{ + if (sb_rdonly(sb)) + return false; + + if (sb->s_writers.frozen == SB_FREEZE_COMPLETE) + return false; + + return true; +} + +/* + * dump_dirtypages_sb - dump the dirty pages of each inode in the sb + * @sb the super block + * @m the seq_file witch is initialized in proc_dpages_open + * + * For each inode in the sb, call dump_dirtypages_pages to get the number + * of dirty pages. And use seq_printf to store the result in the buffer + * if it's not less than the threshold. The inode in unusual state will + * be skipped. + */ +static void dump_dirtypages_sb(struct super_block *sb, struct seq_file *m) +{ + struct inode *inode, *toput_inode = NULL; + unsigned long nr_dirtys; + const char *fstype; + char *filename; + char *tmpname; + unsigned long limit = READ_ONCE(buff_limit); + + if (!is_sb_writable(sb) || (sb->s_iflags & SB_I_NODEV)) + return; + + tmpname = kmalloc(PATH_MAX, GFP_KERNEL); + if (!tmpname) + return; + + spin_lock(&sb->s_inode_list_lock); + list_for_each_entry(inode, &sb->s_inodes, i_sb_list) { + spin_lock(&inode->i_lock); + + /* + * We must skip inodes in unusual state. We may also skip + * inodes without pages but we deliberately won't in case + * we need to reschedule to avoid softlockups. + */ + if ((inode->i_state & (I_FREEING|I_WILL_FREE|I_NEW)) || + (inode->i_mapping->nrpages == 0 && !need_resched()) || + !mapping_tagged(inode->i_mapping, PAGECACHE_TAG_DIRTY)) { + spin_unlock(&inode->i_lock); + continue; + } + __iget(inode); + spin_unlock(&inode->i_lock); + spin_unlock(&sb->s_inode_list_lock); + + cond_resched(); + + nr_dirtys = dump_dirtypages_inode(inode); + if (!nr_dirtys || nr_dirtys < limit) + goto skip; + + filename = inode_filename(inode, tmpname); + if (IS_ERR_OR_NULL(filename)) + filename = "unknown"; + + if (sb->s_type && sb->s_type->name) + fstype = sb->s_type->name; + else + fstype = "unknown"; + + seq_printf(m, "FSType: %s, Dev ID: %u(%u:%u) ino %lu, dirty pages %lu, path %s\n", + fstype, sb->s_dev, MAJOR(sb->s_dev), + MINOR(sb->s_dev), inode->i_ino, + nr_dirtys, filename); + + if (seq_has_overflowed(m)) { + m->size += 13; /* keep size > count to avoid overflow in seq_read_iter() */ + seq_printf(m, "terminated\n"); + iput(inode); + goto done; + } +skip: + iput(toput_inode); + toput_inode = inode; + spin_lock(&sb->s_inode_list_lock); + } + spin_unlock(&sb->s_inode_list_lock); +done: + iput(toput_inode); + kfree(tmpname); +} + +static int proc_dpages_show(struct seq_file *m, void *v) +{ + iterate_supers((void *)dump_dirtypages_sb, (void *)m); + return 0; +} + +static void free_buf_dirty(void) +{ + if (buf_dirty != NULL) { + vfree(buf_dirty); + buf_dirty = NULL; + buf_size = 0; + } +} + +static ssize_t write_proc( + struct file *filp, + const char *buf, + size_t count, + loff_t *offp) +{ + int ret = 0; + unsigned long old_buff_num; + + if (count > PAGE_SIZE) { + ret = -EINVAL; + goto out; + } + + old_buff_num = buff_num; + ret = kstrtoul_from_user(buf, count, 10, &buff_num); + if (ret != 0 || buff_num > MAX_BUFF_SIZE) { + buff_num = old_buff_num; + ret = -EINVAL; + goto out; + } + + ret = count; + if (buff_num == 0) { + free_buf_dirty(); + goto out; + } + if (buff_num == old_buff_num) + goto out; + + free_buf_dirty(); + buf_size = PAGE_SIZE * buff_num; + buf_dirty = vzalloc(buf_size); + if (!buf_dirty) + ret = -ENOMEM; + +out: + return ret; +} + +static int proc_dpages_open(struct inode *inode, struct file *filp) +{ + int ret; + struct seq_file *m; + + if (xchg(&dirty_page_open, 1) == 1) + return -EBUSY; + + if (buf_dirty == NULL || buf_size == 0) { + pr_warn("please allocate buffer before getting dirty pages\n"); + dirty_page_open = 0; + return -ENOMEM; + } + + ret = single_open(filp, proc_dpages_show, NULL); + if (ret) { + dirty_page_open = 0; + return ret; + } + + m = (struct seq_file *)filp->private_data; + memset(buf_dirty, 0, buf_size); + /* if seq_has_overflowed() return true, it need to contain "terminated\n\0" */ + m->size = buf_size - 13; + m->buf = buf_dirty; + + return ret; +} + +static int seq_release_dirty(struct inode *inode, struct file *file) +{ + struct seq_file *m = file->private_data; + + /* we don't want to free the buf */ + m->buf = NULL; + single_release(inode, file); + dirty_page_open = 0; + return 0; +} + +static const struct proc_ops proc_dpages_operations = { + .proc_open = proc_dpages_open, + .proc_read = seq_read, + .proc_release = seq_release_dirty, +}; + +static int proc_switch_show(struct seq_file *m, void *v) +{ + seq_printf(m, "%lu\n", buff_num); + return 0; +} + +static int proc_limit_show(struct seq_file *m, void *v) +{ + seq_printf(m, "%lu\n", READ_ONCE(buff_limit)); + return 0; +} + +static int proc_switch_open(struct inode *inode, struct file *filp) +{ + int ret; + + if (xchg(&dirty_page_open, 1) == 1) + return -EBUSY; + + ret = single_open(filp, proc_switch_show, NULL); + if (!ret) + dirty_page_open = 0; + + return ret; +} + +static int proc_limit_open(struct inode *inode, struct file *filp) +{ + return single_open(filp, proc_limit_show, NULL); +} + +static int proc_switch_release(struct inode *inode, struct file *filp) { + dirty_page_open = 0; + return single_release(inode, filp); +} + +static ssize_t write_limit_proc( + struct file *filp, + const char *buf, + size_t count, + loff_t *offp) +{ + int ret = 0; + unsigned long tmp; + + if (count > PAGE_SIZE) + return -EINVAL; + + ret = kstrtoul_from_user(buf, count, 10, &tmp); + if (ret != 0) + return -EINVAL; + + WRITE_ONCE(buff_limit, tmp); + + return count; +} + + +static const struct proc_ops proc_switch_operations = { + .proc_open = proc_switch_open, + .proc_read = seq_read, + .proc_write = write_proc, + .proc_lseek = seq_lseek, + .proc_release = proc_switch_release, +}; + +static const struct proc_ops proc_limit_operations = { + .proc_open = proc_limit_open, + .proc_read = seq_read, + .proc_write = write_limit_proc, + .proc_lseek = seq_lseek, + .proc_release = single_release, +}; + + +static int __init dpages_proc_init(void) +{ + static struct proc_dir_entry *proc_file; + + dirty_dir = proc_mkdir(DIRTY_ROOT, NULL); + if (!dirty_dir) + goto fail_dir; + + proc_file = proc_create(DIRTY_PAGES, 0440, + dirty_dir, &proc_dpages_operations); + if (!proc_file) + goto fail_pages; + + proc_file = proc_create(DIRTY_SWITCH, 0640, + dirty_dir, &proc_switch_operations); + if (!proc_file) + goto fail_switch; + + proc_file = proc_create(DIRTY_LIMIT, 0640, + dirty_dir, &proc_limit_operations); + if (!proc_file) + goto fail_limit; + + return 0; + +fail_limit: + remove_proc_entry(DIRTY_SWITCH, dirty_dir); +fail_switch: + remove_proc_entry(DIRTY_PAGES, dirty_dir); +fail_pages: + remove_proc_entry(DIRTY_ROOT, NULL); +fail_dir: + return -ENOMEM; +} + +subsys_initcall(dpages_proc_init); -- 2.39.2

2 1

[PATCH OLK-5.10] Revert "sched: clear credit count in error branch"
by Yipeng Zou 04 Jan '24

04 Jan '24

hulk inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I8TNSV CVE: NA -------------------------------- There is no need fix this error branch in OLK-5.10. This reverts commit 2f31dbded7394d42a0e6c7b26ab3613b4b78d238. Signed-off-by: Yipeng Zou <zouyipeng(a)huawei.com> --- kernel/fork.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/fork.c b/kernel/fork.c index 7e31806a7452..0268eb744973 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -2068,7 +2068,7 @@ static __latent_entropy struct task_struct *copy_process( #ifdef CONFIG_QOS_SCHED_DYNAMIC_AFFINITY retval = sched_prefer_cpus_fork(p, current->prefer_cpus); if (retval) - goto bad_fork_cleanup_count; + goto bad_fork_free; #endif lockdep_assert_irqs_enabled(); -- 2.34.1

2 1

[PATCH OLK-6.6] fs/dirty_pages: dump the number of dirty pages for each inode
by Zizhi Wo 04 Jan '24

04 Jan '24

From: yu kuai <yukuai3(a)huawei.com> hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I8OHM4?from=project-issue CVE: NA --------------------------- In order to analyse the IO performance when using buffer IO, it's useful to obtain the number of dirty pages for each inode in the filesystem. This feature depends on 'CONFIG_DIRTY_PAGES'. It creates 3 interfaces by using procfs. /proc/dirty/buffer_size for buffer allocation and release, /proc/dirty/page_threshold to filter result and /proc/dirty/dirty_list to get dirty pages. Signed-off-by: yu kuai <yukuai3(a)huawei.com> Signed-off-by: Zizhi Wo <wozizhi(a)huawei.com> --- arch/arm64/configs/openeuler_defconfig | 1 + arch/x86/configs/openeuler_defconfig | 1 + fs/Kconfig | 13 + fs/Makefile | 1 + fs/dirty_pages.c | 381 +++++++++++++++++++++++++ 5 files changed, 397 insertions(+) create mode 100644 fs/dirty_pages.c diff --git a/arch/arm64/configs/openeuler_defconfig b/arch/arm64/configs/openeuler_defconfig index 8e29cb161800..984ad448b5db 100644 --- a/arch/arm64/configs/openeuler_defconfig +++ b/arch/arm64/configs/openeuler_defconfig @@ -6909,6 +6909,7 @@ CONFIG_PROC_PAGE_MONITOR=y CONFIG_PROC_CHILDREN=y CONFIG_KERNFS=y CONFIG_SYSFS=y +CONFIG_DIRTY_PAGES=y CONFIG_TMPFS=y CONFIG_TMPFS_POSIX_ACL=y CONFIG_TMPFS_XATTR=y diff --git a/arch/x86/configs/openeuler_defconfig b/arch/x86/configs/openeuler_defconfig index b3e32559ab62..233b95fa4a84 100644 --- a/arch/x86/configs/openeuler_defconfig +++ b/arch/x86/configs/openeuler_defconfig @@ -8111,6 +8111,7 @@ CONFIG_PROC_PID_ARCH_STATUS=y CONFIG_PROC_CPU_RESCTRL=y CONFIG_KERNFS=y CONFIG_SYSFS=y +CONFIG_DIRTY_PAGES=y CONFIG_TMPFS=y CONFIG_TMPFS_POSIX_ACL=y CONFIG_TMPFS_XATTR=y diff --git a/fs/Kconfig b/fs/Kconfig index aa7e03cc1941..a2280cf98729 100644 --- a/fs/Kconfig +++ b/fs/Kconfig @@ -170,6 +170,19 @@ source "fs/proc/Kconfig" source "fs/kernfs/Kconfig" source "fs/sysfs/Kconfig" +config DIRTY_PAGES + bool "Dumps the number of dirty pages of each file" + depends on PROC_FS + default y + help + This config supports the rendering of dirty page data to the user, + which may be useful to analyze the IO performance when using buffer + IO. + + It create 3 interfaces by using procfs. /proc/dirty/buffer_size for + buffer allocation and release; /proc/dirty/page_threshold to filter + result; /proc/dirty/dirty_list to get dirty pages. + config TMPFS bool "Tmpfs virtual memory file system support (former shm fs)" depends on SHMEM diff --git a/fs/Makefile b/fs/Makefile index bb6164200a39..cc4735c23c32 100644 --- a/fs/Makefile +++ b/fs/Makefile @@ -44,6 +44,7 @@ obj-$(CONFIG_FS_POSIX_ACL) += posix_acl.o obj-$(CONFIG_NFS_COMMON) += nfs_common/ obj-$(CONFIG_COREDUMP) += coredump.o obj-$(CONFIG_SYSCTL) += drop_caches.o sysctls.o +obj-$(CONFIG_DIRTY_PAGES) += dirty_pages.o obj-$(CONFIG_FHANDLE) += fhandle.o obj-$(CONFIG_CGROUP_FILES) += filescontrol.o diff --git a/fs/dirty_pages.c b/fs/dirty_pages.c new file mode 100644 index 000000000000..49e42df6638d --- /dev/null +++ b/fs/dirty_pages.c @@ -0,0 +1,381 @@ +#include <linux/proc_fs.h> +#include <linux/seq_file.h> +#include <linux/uaccess.h> +#include <linux/pagemap.h> +#include <linux/pagevec.h> +#include <linux/fs.h> +#include <linux/mm.h> +#include <linux/slab.h> +#include <linux/sched.h> +#include <linux/proc_fs.h> +#include <linux/kdev_t.h> +#include <linux/vmalloc.h> +#include <linux/module.h> +#include <linux/kernel.h> +#include <linux/init.h> +#include "internal.h" + +static char *buf_dirty; /* buffer to store number of dirty pages */ +static unsigned long buf_size; /* size of buffer in bytes */ +static unsigned long buff_num; /* size of buffer in number of pages */ +static unsigned long buff_limit; /* filter threshold of dirty pages*/ + +static struct proc_dir_entry *dirty_dir; + +unsigned long dirty_page_open; + +/* proc root directory */ +#define DIRTY_ROOT "dirty" +/* proc file for buffer allocation and release */ +#define DIRTY_SWITCH "buffer_size" +/* proc file to obtain diry pages of each inode */ +#define DIRTY_PAGES "dirty_list" +/* proc file to filter result */ +#define DIRTY_LIMIT "page_threshold" + +#define MAX_BUFF_SIZE 102400 + +static unsigned long dump_dirtypages_inode(struct inode *inode) +{ + XA_STATE(xas, &inode->i_mapping->i_pages, 0); + unsigned long nr_dirtys = 0; + void *page; + + xas_lock_irq(&xas); + xas_for_each_marked(&xas, page, (pgoff_t)-1, PAGECACHE_TAG_DIRTY) { + if (++nr_dirtys % XA_CHECK_SCHED) + continue; + + xas_pause(&xas); + xas_unlock_irq(&xas); + cond_resched(); + xas_lock_irq(&xas); + } + xas_unlock_irq(&xas); + + return nr_dirtys; +} + +static char *inode_filename(struct inode *inode, char *tmpname) +{ + struct dentry *dentry; + char *filename; + + dentry = d_find_alias(inode); + if (!dentry) + return ERR_PTR(-ENOENT); + + tmpname[PATH_MAX-1] = '\0'; + filename = dentry_path_raw(dentry, tmpname, PATH_MAX); + + dput(dentry); + + return filename; +} + +static inline bool is_sb_writable(struct super_block *sb) +{ + if (sb_rdonly(sb)) + return false; + + if (sb->s_writers.frozen == SB_FREEZE_COMPLETE) + return false; + + return true; +} + +/* + * dump_dirtypages_sb - dump the dirty pages of each inode in the sb + * @sb the super block + * @m the seq_file witch is initialized in proc_dpages_open + * + * For each inode in the sb, call dump_dirtypages_pages to get the number + * of dirty pages. And use seq_printf to store the result in the buffer + * if it's not less than the threshold. The inode in unusual state will + * be skipped. + */ +static void dump_dirtypages_sb(struct super_block *sb, struct seq_file *m) +{ + struct inode *inode, *toput_inode = NULL; + unsigned long nr_dirtys; + const char *fstype; + char *filename; + char *tmpname; + unsigned long limit = READ_ONCE(buff_limit); + + if (!is_sb_writable(sb) || (sb->s_iflags & SB_I_NODEV)) + return; + + tmpname = kmalloc(PATH_MAX, GFP_KERNEL); + if (!tmpname) + return; + + spin_lock(&sb->s_inode_list_lock); + list_for_each_entry(inode, &sb->s_inodes, i_sb_list) { + spin_lock(&inode->i_lock); + + /* + * We must skip inodes in unusual state. We may also skip + * inodes without pages but we deliberately won't in case + * we need to reschedule to avoid softlockups. + */ + if ((inode->i_state & (I_FREEING|I_WILL_FREE|I_NEW)) || + (inode->i_mapping->nrpages == 0 && !need_resched()) || + !mapping_tagged(inode->i_mapping, PAGECACHE_TAG_DIRTY)) { + spin_unlock(&inode->i_lock); + continue; + } + __iget(inode); + spin_unlock(&inode->i_lock); + spin_unlock(&sb->s_inode_list_lock); + + cond_resched(); + + nr_dirtys = dump_dirtypages_inode(inode); + if (!nr_dirtys || nr_dirtys < limit) + goto skip; + + filename = inode_filename(inode, tmpname); + if (IS_ERR_OR_NULL(filename)) + filename = "unknown"; + + if (sb->s_type && sb->s_type->name) + fstype = sb->s_type->name; + else + fstype = "unknown"; + + seq_printf(m, "FSType: %s, Dev ID: %u(%u:%u) ino %lu, dirty pages %lu, path %s\n", + fstype, sb->s_dev, MAJOR(sb->s_dev), + MINOR(sb->s_dev), inode->i_ino, + nr_dirtys, filename); + + if (seq_has_overflowed(m)) { + m->size += 13; /* keep size > count to avoid overflow in seq_read_iter() */ + strncpy(m->buf + m->count - 12, "terminated\n\0", 12); + m->count += 12; + iput(inode); + goto done; + } +skip: + iput(toput_inode); + toput_inode = inode; + spin_lock(&sb->s_inode_list_lock); + } + spin_unlock(&sb->s_inode_list_lock); +done: + iput(toput_inode); + kfree(tmpname); +} + +static int proc_dpages_show(struct seq_file *m, void *v) +{ + iterate_supers((void *)dump_dirtypages_sb, (void *)m); + return 0; +} + +static void free_buf_dirty(void) +{ + if (buf_dirty != NULL) { + vfree(buf_dirty); + buf_dirty = NULL; + buf_size = 0; + } +} + +static ssize_t write_proc( + struct file *filp, + const char *buf, + size_t count, + loff_t *offp) +{ + int ret = 0; + unsigned long old_buff_num; + + if (count > PAGE_SIZE) { + ret = -EINVAL; + goto out; + } + + old_buff_num = buff_num; + ret = kstrtoul_from_user(buf, count, 10, &buff_num); + if (ret != 0 || buff_num > MAX_BUFF_SIZE) { + buff_num = old_buff_num; + ret = -EINVAL; + goto out; + } + + ret = count; + if (buff_num == 0) { + free_buf_dirty(); + goto out; + } + if (buff_num == old_buff_num) + goto out; + + free_buf_dirty(); + buf_size = PAGE_SIZE * buff_num; + buf_dirty = vzalloc(buf_size); + if (!buf_dirty) + ret = -ENOMEM; + +out: + return ret; +} + +static int proc_dpages_open(struct inode *inode, struct file *filp) +{ + int ret; + struct seq_file *m; + + if (xchg(&dirty_page_open, 1) == 1) + return -EBUSY; + + if (buf_dirty == NULL || buf_size == 0) { + pr_warn("please allocate buffer before getting dirty pages\n"); + dirty_page_open = 0; + return -ENOMEM; + } + + ret = single_open(filp, proc_dpages_show, NULL); + if (!ret) { + dirty_page_open = 0; + return ret; + } + + m = (struct seq_file *)filp->private_data; + memset(buf_dirty, 0, buf_size); + /* if seq_has_overflowed() return true, it need to contain "terminated\n\0" */ + m->size = buf_size - 13; + m->buf = buf_dirty; + + return ret; +} + +static int seq_release_dirty(struct inode *inode, struct file *file) +{ + struct seq_file *m = file->private_data; + + /* we don't want to free the buf */ + m->buf = NULL; + single_release(inode, file); + dirty_page_open = 0; + return 0; +} + +static const struct proc_ops proc_dpages_operations = { + .proc_open = proc_dpages_open, + .proc_read = seq_read, + .proc_release = seq_release_dirty, +}; + +static int proc_switch_show(struct seq_file *m, void *v) +{ + seq_printf(m, "%lu\n", buff_num); + return 0; +} + +static int proc_limit_show(struct seq_file *m, void *v) +{ + seq_printf(m, "%lu\n", READ_ONCE(buff_limit)); + return 0; +} + +static int proc_switch_open(struct inode *inode, struct file *filp) +{ + int ret; + + if (xchg(&dirty_page_open, 1) == 1) + return -EBUSY; + + ret = single_open(filp, proc_switch_show, NULL); + if (!ret) + dirty_page_open = 0; + + return ret; +} + +static int proc_limit_open(struct inode *inode, struct file *filp) +{ + return single_open(filp, proc_limit_show, NULL); +} + +static int proc_switch_release(struct inode *inode, struct file *filp) { + dirty_page_open = 0; + return single_release(inode, filp); +} + +static ssize_t write_limit_proc( + struct file *filp, + const char *buf, + size_t count, + loff_t *offp) +{ + int ret = 0; + unsigned long tmp; + + if (count > PAGE_SIZE) + return -EINVAL; + + ret = kstrtoul_from_user(buf, count, 10, &tmp); + if (ret != 0) + return -EINVAL; + + WRITE_ONCE(buff_limit, tmp); + + return count; +} + + +static const struct proc_ops proc_switch_operations = { + .proc_open = proc_switch_open, + .proc_read = seq_read, + .proc_write = write_proc, + .proc_lseek = seq_lseek, + .proc_release = proc_switch_release, +}; + +static const struct proc_ops proc_limit_operations = { + .proc_open = proc_limit_open, + .proc_read = seq_read, + .proc_write = write_limit_proc, + .proc_lseek = seq_lseek, + .proc_release = single_release, +}; + + +static int __init dpages_proc_init(void) +{ + static struct proc_dir_entry *proc_file; + + dirty_dir = proc_mkdir(DIRTY_ROOT, NULL); + if (!dirty_dir) + goto fail_dir; + + proc_file = proc_create(DIRTY_PAGES, 0440, + dirty_dir, &proc_dpages_operations); + if (!proc_file) + goto fail_pages; + + proc_file = proc_create(DIRTY_SWITCH, 0640, + dirty_dir, &proc_switch_operations); + if (!proc_file) + goto fail_switch; + + proc_file = proc_create(DIRTY_LIMIT, 0640, + dirty_dir, &proc_limit_operations); + if (!proc_file) + goto fail_limit; + + return 0; + +fail_limit: + remove_proc_entry(DIRTY_SWITCH, dirty_dir); +fail_switch: + remove_proc_entry(DIRTY_PAGES, dirty_dir); +fail_pages: + remove_proc_entry(DIRTY_ROOT, NULL); +fail_dir: + return -ENOMEM; +} + +subsys_initcall(dpages_proc_init); -- 2.39.2

2 1

[PATCH OLK-6.6] fs/dirty_pages: dump the number of dirty pages for each inode
by Zizhi Wo 04 Jan '24

04 Jan '24

From: yu kuai <yukuai3(a)huawei.com> hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I8OHM4?from=project-issue CVE: NA --------------------------- In order to analyse the IO performance when using buffer IO, it's useful to obtain the number of dirty pages for each inode in the filesystem. This feature depends on 'CONFIG_DIRTY_PAGES'. It creates 3 interfaces by using procfs. /proc/dirty/buffer_size for buffer allocation and release, /proc/dirty/page_threshold to filter result and /proc/dirty/dirty_list to get dirty pages. Signed-off-by: yu kuai <yukuai3(a)huawei.com> Signed-off-by: Zizhi Wo <wozizhi(a)huawei.com> --- arch/arm64/configs/openeuler_defconfig | 1 + arch/x86/configs/openeuler_defconfig | 1 + fs/Kconfig | 13 + fs/Makefile | 1 + fs/dirty_pages.c | 380 +++++++++++++++++++++++++ 5 files changed, 396 insertions(+) create mode 100644 fs/dirty_pages.c diff --git a/arch/arm64/configs/openeuler_defconfig b/arch/arm64/configs/openeuler_defconfig index 039f3496af78..66b4812050df 100644 --- a/arch/arm64/configs/openeuler_defconfig +++ b/arch/arm64/configs/openeuler_defconfig @@ -6883,6 +6883,7 @@ CONFIG_PROC_PAGE_MONITOR=y CONFIG_PROC_CHILDREN=y CONFIG_KERNFS=y CONFIG_SYSFS=y +CONFIG_DIRTY_PAGES=y CONFIG_TMPFS=y CONFIG_TMPFS_POSIX_ACL=y CONFIG_TMPFS_XATTR=y diff --git a/arch/x86/configs/openeuler_defconfig b/arch/x86/configs/openeuler_defconfig index f481c1c8b4d1..6133f66fde61 100644 --- a/arch/x86/configs/openeuler_defconfig +++ b/arch/x86/configs/openeuler_defconfig @@ -8090,6 +8090,7 @@ CONFIG_PROC_PID_ARCH_STATUS=y CONFIG_PROC_CPU_RESCTRL=y CONFIG_KERNFS=y CONFIG_SYSFS=y +CONFIG_DIRTY_PAGES=y CONFIG_TMPFS=y CONFIG_TMPFS_POSIX_ACL=y CONFIG_TMPFS_XATTR=y diff --git a/fs/Kconfig b/fs/Kconfig index aa7e03cc1941..a2280cf98729 100644 --- a/fs/Kconfig +++ b/fs/Kconfig @@ -170,6 +170,19 @@ source "fs/proc/Kconfig" source "fs/kernfs/Kconfig" source "fs/sysfs/Kconfig" +config DIRTY_PAGES + bool "Dumps the number of dirty pages of each file" + depends on PROC_FS + default y + help + This config supports the rendering of dirty page data to the user, + which may be useful to analyze the IO performance when using buffer + IO. + + It create 3 interfaces by using procfs. /proc/dirty/buffer_size for + buffer allocation and release; /proc/dirty/page_threshold to filter + result; /proc/dirty/dirty_list to get dirty pages. + config TMPFS bool "Tmpfs virtual memory file system support (former shm fs)" depends on SHMEM diff --git a/fs/Makefile b/fs/Makefile index f9541f40be4e..6246e173e1e4 100644 --- a/fs/Makefile +++ b/fs/Makefile @@ -44,6 +44,7 @@ obj-$(CONFIG_FS_POSIX_ACL) += posix_acl.o obj-$(CONFIG_NFS_COMMON) += nfs_common/ obj-$(CONFIG_COREDUMP) += coredump.o obj-$(CONFIG_SYSCTL) += drop_caches.o sysctls.o +obj-$(CONFIG_DIRTY_PAGES) += dirty_pages.o obj-$(CONFIG_FHANDLE) += fhandle.o obj-y += iomap/ diff --git a/fs/dirty_pages.c b/fs/dirty_pages.c new file mode 100644 index 000000000000..0e29bb50358d --- /dev/null +++ b/fs/dirty_pages.c @@ -0,0 +1,380 @@ +#include <linux/proc_fs.h> +#include <linux/seq_file.h> +#include <linux/uaccess.h> +#include <linux/pagemap.h> +#include <linux/pagevec.h> +#include <linux/fs.h> +#include <linux/mm.h> +#include <linux/slab.h> +#include <linux/sched.h> +#include <linux/proc_fs.h> +#include <linux/kdev_t.h> +#include <linux/vmalloc.h> +#include <linux/module.h> +#include <linux/kernel.h> +#include <linux/init.h> +#include "internal.h" + +static char *buf_dirty; /* buffer to store number of dirty pages */ +static unsigned long buf_size; /* size of buffer in bytes */ +static unsigned long buff_num; /* size of buffer in number of pages */ +static unsigned long buff_limit; /* filter threshold of dirty pages*/ + +static struct proc_dir_entry *dirty_dir; + +unsigned long dirty_page_open; + +/* proc root directory */ +#define DIRTY_ROOT "dirty" +/* proc file for buffer allocation and release */ +#define DIRTY_SWITCH "buffer_size" +/* proc file to obtain diry pages of each inode */ +#define DIRTY_PAGES "dirty_list" +/* proc file to filter result */ +#define DIRTY_LIMIT "page_threshold" + +#define MAX_BUFF_SIZE 102400 + +static unsigned long dump_dirtypages_inode(struct inode *inode) +{ + XA_STATE(xas, &inode->i_mapping->i_pages, 0); + unsigned long nr_dirtys = 0; + void *page; + + xas_lock_irq(&xas); + xas_for_each_marked(&xas, page, (pgoff_t)-1, PAGECACHE_TAG_DIRTY) { + if (++nr_dirtys % XA_CHECK_SCHED) + continue; + + xas_pause(&xas); + xas_unlock_irq(&xas); + cond_resched(); + xas_lock_irq(&xas); + } + xas_unlock_irq(&xas); + + return nr_dirtys; +} + +static char *inode_filename(struct inode *inode, char *tmpname) +{ + struct dentry *dentry; + char *filename; + + dentry = d_find_alias(inode); + if (!dentry) + return ERR_PTR(-ENOENT); + + tmpname[PATH_MAX-1] = '\0'; + filename = dentry_path_raw(dentry, tmpname, PATH_MAX); + + dput(dentry); + + return filename; +} + +static inline bool is_sb_writable(struct super_block *sb) +{ + if (sb_rdonly(sb)) + return false; + + if (sb->s_writers.frozen == SB_FREEZE_COMPLETE) + return false; + + return true; +} + +/* + * dump_dirtypages_sb - dump the dirty pages of each inode in the sb + * @sb the super block + * @m the seq_file witch is initialized in proc_dpages_open + * + * For each inode in the sb, call dump_dirtypages_pages to get the number + * of dirty pages. And use seq_printf to store the result in the buffer + * if it's not less than the threshold. The inode in unusual state will + * be skipped. + */ +static void dump_dirtypages_sb(struct super_block *sb, struct seq_file *m) +{ + struct inode *inode, *toput_inode = NULL; + unsigned long nr_dirtys; + const char *fstype; + char *filename; + char *tmpname; + unsigned long limit = READ_ONCE(buff_limit); + + if (!is_sb_writable(sb) || (sb->s_iflags & SB_I_NODEV)) + return; + + tmpname = kmalloc(PATH_MAX, GFP_KERNEL); + if (!tmpname) + return; + + spin_lock(&sb->s_inode_list_lock); + list_for_each_entry(inode, &sb->s_inodes, i_sb_list) { + spin_lock(&inode->i_lock); + + /* + * We must skip inodes in unusual state. We may also skip + * inodes without pages but we deliberately won't in case + * we need to reschedule to avoid softlockups. + */ + if ((inode->i_state & (I_FREEING|I_WILL_FREE|I_NEW)) || + (inode->i_mapping->nrpages == 0 && !need_resched()) || + !mapping_tagged(inode->i_mapping, PAGECACHE_TAG_DIRTY)) { + spin_unlock(&inode->i_lock); + continue; + } + __iget(inode); + spin_unlock(&inode->i_lock); + spin_unlock(&sb->s_inode_list_lock); + + cond_resched(); + + nr_dirtys = dump_dirtypages_inode(inode); + if (!nr_dirtys || nr_dirtys < limit) + goto skip; + + filename = inode_filename(inode, tmpname); + if (IS_ERR_OR_NULL(filename)) + filename = "unknown"; + + if (sb->s_type && sb->s_type->name) + fstype = sb->s_type->name; + else + fstype = "unknown"; + + seq_printf(m, "FSType: %s, Dev ID: %u(%u:%u) ino %lu, dirty pages %lu, path %s\n", + fstype, sb->s_dev, MAJOR(sb->s_dev), + MINOR(sb->s_dev), inode->i_ino, + nr_dirtys, filename); + + if (seq_has_overflowed(m)) { + m->size += 13; /* keep size > count to avoid overflow in seq_read_iter() */ + seq_printf(m, "terminated\n\0"); + iput(inode); + goto done; + } +skip: + iput(toput_inode); + toput_inode = inode; + spin_lock(&sb->s_inode_list_lock); + } + spin_unlock(&sb->s_inode_list_lock); +done: + iput(toput_inode); + kfree(tmpname); +} + +static int proc_dpages_show(struct seq_file *m, void *v) +{ + iterate_supers((void *)dump_dirtypages_sb, (void *)m); + return 0; +} + +static void free_buf_dirty(void) +{ + if (buf_dirty != NULL) { + vfree(buf_dirty); + buf_dirty = NULL; + buf_size = 0; + } +} + +static ssize_t write_proc( + struct file *filp, + const char *buf, + size_t count, + loff_t *offp) +{ + int ret = 0; + unsigned long old_buff_num; + + if (count > PAGE_SIZE) { + ret = -EINVAL; + goto out; + } + + old_buff_num = buff_num; + ret = kstrtoul_from_user(buf, count, 10, &buff_num); + if (ret != 0 || buff_num > MAX_BUFF_SIZE) { + buff_num = old_buff_num; + ret = -EINVAL; + goto out; + } + + ret = count; + if (buff_num == 0) { + free_buf_dirty(); + goto out; + } + if (buff_num == old_buff_num) + goto out; + + free_buf_dirty(); + buf_size = PAGE_SIZE * buff_num; + buf_dirty = vzalloc(buf_size); + if (!buf_dirty) + ret = -ENOMEM; + +out: + return ret; +} + +static int proc_dpages_open(struct inode *inode, struct file *filp) +{ + int ret; + struct seq_file *m; + + if (xchg(&dirty_page_open, 1) == 1) + return -EBUSY; + + if (buf_dirty == NULL || buf_size == 0) { + pr_warn("please allocate buffer before getting dirty pages\n"); + dirty_page_open = 0; + return -ENOMEM; + } + + ret = single_open(filp, proc_dpages_show, NULL); + if (!ret) { + dirty_page_open = 0; + return ret; + } + + m = (struct seq_file *)filp->private_data; + memset(buf_dirty, 0, buf_size); + /* if seq_has_overflowed() return true, it need to contain "terminated\n\0" */ + m->size = buf_size - 13; + m->buf = buf_dirty; + + return ret; +} + +static int seq_release_dirty(struct inode *inode, struct file *file) +{ + struct seq_file *m = file->private_data; + + /* we don't want to free the buf */ + m->buf = NULL; + single_release(inode, file); + dirty_page_open = 0; + return 0; +} + +static const struct proc_ops proc_dpages_operations = { + .proc_open = proc_dpages_open, + .proc_read = seq_read, + .proc_release = seq_release_dirty, +}; + +static int proc_switch_show(struct seq_file *m, void *v) +{ + seq_printf(m, "%lu\n", buff_num); + return 0; +} + +static int proc_limit_show(struct seq_file *m, void *v) +{ + seq_printf(m, "%lu\n", READ_ONCE(buff_limit)); + return 0; +} + +static int proc_switch_open(struct inode *inode, struct file *filp) +{ + int ret; + + if (xchg(&dirty_page_open, 1) == 1) + return -EBUSY; + + ret = single_open(filp, proc_switch_show, NULL); + if (!ret) + dirty_page_open = 0; + + return ret; +} + +static int proc_limit_open(struct inode *inode, struct file *filp) +{ + return single_open(filp, proc_limit_show, NULL); +} + +static int proc_switch_release(struct inode *inode, struct file *filp) { + dirty_page_open = 0; + return single_release(inode, filp); +} + +static ssize_t write_limit_proc( + struct file *filp, + const char *buf, + size_t count, + loff_t *offp) +{ + int ret = 0; + unsigned long tmp; + + if (count > PAGE_SIZE) + return -EINVAL; + + ret = kstrtoul_from_user(buf, count, 10, &tmp); + if (ret != 0) + return -EINVAL; + + WRITE_ONCE(buff_limit, tmp); + + return count; +} + + +static const struct proc_ops proc_switch_operations = { + .proc_open = proc_switch_open, + .proc_read = seq_read, + .proc_write = write_proc, + .proc_lseek = seq_lseek, + .proc_release = proc_switch_release, +}; + +static const struct proc_ops proc_limit_operations = { + .proc_open = proc_limit_open, + .proc_read = seq_read, + .proc_write = write_limit_proc, + .proc_lseek = seq_lseek, + .proc_release = single_release, +}; + + +static int __init dpages_proc_init(void) +{ + static struct proc_dir_entry *proc_file; + + dirty_dir = proc_mkdir(DIRTY_ROOT, NULL); + if (!dirty_dir) + goto fail_dir; + + proc_file = proc_create(DIRTY_PAGES, 0440, + dirty_dir, &proc_dpages_operations); + if (!proc_file) + goto fail_pages; + + proc_file = proc_create(DIRTY_SWITCH, 0640, + dirty_dir, &proc_switch_operations); + if (!proc_file) + goto fail_switch; + + proc_file = proc_create(DIRTY_LIMIT, 0640, + dirty_dir, &proc_limit_operations); + if (!proc_file) + goto fail_limit; + + return 0; + +fail_limit: + remove_proc_entry(DIRTY_SWITCH, dirty_dir); +fail_switch: + remove_proc_entry(DIRTY_PAGES, dirty_dir); +fail_pages: + remove_proc_entry(DIRTY_ROOT, NULL); +fail_dir: + return -ENOMEM; +} + +subsys_initcall(dpages_proc_init); -- 2.39.2

2 1