- Kernel - mailweb.openeuler.org

[PATCH openEuler-23.09 v5 00/19] Add sharepool support
by Wang Wensheng 05 Sep '23

05 Sep '23

This patch series migrates share pool features from OLK-5.10 to openEuler-23.09. We don't want to do this work patch by patch since there are hundreds of patches in OLK-5.10 including bugfix and small features generation. Instead we just take the final version and split it into a few small patches, each of which contains one feature at the final version. The share pool features is a big feature, it is mainly used to share user virtual memory for different processes in the same group. It could be used by this steps: 1. Process A create a new group which is owned by process A. 2. Process A add process B to the group. 3. Process A add process C to the same group. 4. Process B alloc a new memory VA, and write something in it. 5. The VA was send to the process C by IPC, then process C got it. 6. The process C access the VA and got the data directly. 7. The process A could add more processes in the group to share the memory. 8. Fix the memory by use the free function or exit the group. The new features is enabled both by CONFIG_ASCEND_SHARE_POOL and the enable_ascend_share_pool bootarg, it would not affect anything if disabled. v1->v2: Fix some compile issues. v2->v3: Fix some checkpatch issues. v3->v4: Move the kconfigs assocated with mm to mm/Kconfig. v4->v5: Change the description for some commits. Wang Wensheng (19): mm/hugetlb: Introduce hugetlb_insert_hugepage_pte[_by_pa] mm/vmalloc: Extend vmalloc usage about hugepage mm: Extend mmap assocated functions to accept mm_struct mm/sharepool: Add base framework for share_pool mm/sharepool: Reserve the va space for share_pool mm/sharepool: Add sp_area management code mm/sharepool: Implement mg_sp_make_share_u2k() mm/sharepool: Implement mg_sp_unshare_kva mm/sharepool: Implement mg_sp_walk_page_range() mm/sharepool: Implement mg_sp_free() mm/sharepool: Implement mg_sp_alloc() mm/sharepool: Implement mg_sp_make_share_k2u() mm/sharepool: Implement mg_sp_group_add_task() mm/sharepool: Implement mg_sp_group_id_by_pid() mm/sharepool: Implement mg_sp_id_of_current() mm/sharepool: Implement mg_sp_config_dvpp_range() mm/sharepool: Add proc interfaces to show sp info mm/sharepool: support fork() and exit() to handle the mm mm/sharepool: Adjust __unmap_hugepage_range() for k2u include/linux/hugetlb.h | 32 + include/linux/mempolicy.h | 11 + include/linux/mm.h | 17 + include/linux/mm_types.h | 7 + include/linux/share_pool.h | 274 +++ include/linux/vmalloc.h | 18 + kernel/fork.c | 3 + mm/Kconfig | 34 + mm/Makefile | 1 + mm/gup.c | 23 +- mm/hugetlb.c | 78 + mm/mempolicy.c | 12 +- mm/mmap.c | 43 +- mm/share_pool.c | 3434 ++++++++++++++++++++++++++++++++++++ mm/util.c | 4 + mm/vmalloc.c | 205 ++- 16 files changed, 4172 insertions(+), 24 deletions(-) create mode 100644 include/linux/share_pool.h create mode 100644 mm/share_pool.c -- 2.17.1

2 20

[PATCH v2 openEuler-23.09 0/2] enable cgroup1 writeback
by Lu Jialin 05 Sep '23

05 Sep '23

v2->v1: fix some code style problem Lu Jialin (2): cgroup: Add cgroup1_get_from_id() cgroup: support cgroup writeback on cgroupv1 arch/arm64/configs/openeuler_defconfig | 1 + arch/x86/configs/openeuler_defconfig | 1 + block/blk-cgroup.c | 3 + block/blk-cgroup.h | 3 + include/linux/backing-dev.h | 29 +++++- include/linux/cgroup-defs.h | 2 + include/linux/cgroup.h | 3 + include/linux/memcontrol.h | 5 ++ init/Kconfig | 5 ++ kernel/cgroup/cgroup.c | 46 +++++++++- mm/backing-dev.c | 118 ++++++++++++++++++++++++- mm/memcontrol.c | 84 +++++++++++++++++- 12 files changed, 291 insertions(+), 9 deletions(-) -- 2.34.1

2 3

[PATCH v2 openEuler-23.09 0/9] add kabi macros helpers
by Yu Liao 05 Sep '23

05 Sep '23

Add kabi macros helpers. Li Lingfeng (1): kabi: deduplication friendly structs Wei Li (2): kabi: fix build error when CONFIG_KABI_RESERVE=n kabi: cleanup config entries of kabi Xie XiuQi (3): kabi: add kabi helper macros kabi: add KABI_SIZE_ALIGN_CHECKS for more stringent kabi checks kabi: enables more stringent kabi checks Zheng Zengkai (3): KABI: Add CONFIG_KABI_RESERVE to control KABI padding reserve openeuler_defconfig: Enable CONFIG_KABI_RESERVE for x86 and arm64 kabi: Generalize naming of kabi helper macros arch/arm64/configs/openeuler_defconfig | 2 + arch/x86/configs/openeuler_defconfig | 2 + include/linux/kabi.h | 509 +++++++++++++++++++++++++ init/Kconfig | 16 + 4 files changed, 529 insertions(+) create mode 100644 include/linux/kabi.h -- 2.33.0

2 10

[PATCH v2 openEuler-23.09 0/9] add kabi macros helpers
by Yu Liao 05 Sep '23

05 Sep '23

Add kabi macros helpers. Li Lingfeng (1): kabi: deduplication friendly structs Wei Li (2): kabi: fix build error when CONFIG_KABI_RESERVE=n kabi: cleanup config entries of kabi Xie XiuQi (3): kabi: add kabi helper macros kabi: add KABI_SIZE_ALIGN_CHECKS for more stringent kabi checks kabi: enables more stringent kabi checks Zheng Zengkai (3): KABI: Add CONFIG_KABI_RESERVE to control KABI padding reserve openeuler_defconfig: Enable CONFIG_KABI_RESERVE for x86 and arm64 kabi: Generalize naming of kabi helper macros arch/arm64/configs/openeuler_defconfig | 2 + arch/x86/configs/openeuler_defconfig | 2 + include/linux/kabi.h | 509 +++++++++++++++++++++++++ init/Kconfig | 16 + 4 files changed, 529 insertions(+) create mode 100644 include/linux/kabi.h -- 2.33.0

2 10

[PATCH openEuler-23.09 v4 00/19] Add sharepool support
by Wang Wensheng 05 Sep '23

05 Sep '23

This patch series migrates share pool features from OLK-5.10 to openEuler-23.09. We don't want to do this work patch by patch since there are hundreds of patches in OLK-5.10 including bugfix and small features generation. Instead we just take the final version and split it into a few small patches, each of which contains one feature at the final version. The share pool features is a big feature, it is mainly used to share user virtual memory for different processes in the same group. It could be used by this steps: 1. Process A create a new group which is owned by process A. 2. Process A add process B to the group. 3. Process A add process C to the same group. 4. Process B alloc a new memory VA, and write something in it. 5. The VA was send to the process C by IPC, then process C got it. 6. The process C access the VA and got the data directly. 7. The process A could add more processes in the group to share the memory. 8. Fix the memory by use the free function or exit the group. The new features is enabled both by CONFIG_ASCEND_SHARE_POOL and the enable_ascend_share_pool bootarg, it would not affect anything if disabled. v1->v2: Fix some compile issues. v2->v3: Fix some checkpatch issues. v3->v4: Move the kconfigs assocated with mm to mm/Kconfig. Wang Wensheng (19): mm/hugetlb: Extend hugetlb usage for ascend mm/vmalloc: Extend vmalloc usage for ascend mm: Extend mmap assocated functions to accept mm_struct mm/sharepool: Add base framework for share_pool mm/sharepool: Reserve the va space for share_pool mm/sharepool: Add sp_area management code mm/sharepool: Implement mg_sp_make_share_u2k() mm/sharepool: Implement mg_sp_unshare_kva mm/sharepool: Implement mg_sp_walk_page_range() mm/sharepool: Implement mg_sp_free() mm/sharepool: Implement mg_sp_alloc() mm/sharepool: Implement mg_sp_make_share_k2u() mm/sharepool: Implement mg_sp_group_add_task() mm/sharepool: Implement mg_sp_group_id_by_pid() mm/sharepool: Implement mg_sp_id_of_current() mm/sharepool: Implement mg_sp_config_dvpp_range() mm/sharepool: Add proc interfaces to show sp info mm/sharepool: support fork() and exit() to handle the mm mm/sharepool: Adjust __unmap_hugepage_range() for k2u arch/arm64/Kconfig | 12 + include/linux/hugetlb.h | 32 + include/linux/mempolicy.h | 11 + include/linux/mm.h | 17 + include/linux/mm_types.h | 7 + include/linux/share_pool.h | 274 +++ include/linux/vmalloc.h | 18 + kernel/fork.c | 3 + mm/Kconfig | 22 + mm/Makefile | 1 + mm/gup.c | 23 +- mm/hugetlb.c | 78 + mm/mempolicy.c | 12 +- mm/mmap.c | 43 +- mm/share_pool.c | 3434 ++++++++++++++++++++++++++++++++++++ mm/util.c | 4 + mm/vmalloc.c | 205 ++- 17 files changed, 4172 insertions(+), 24 deletions(-) create mode 100644 include/linux/share_pool.h create mode 100644 mm/share_pool.c -- 2.17.1

2 20

[PATCH v3 openEuler-23.09 00/21]Introduce programmable base on bpf for scheduler
by Guan Jing 05 Sep '23

05 Sep '23

Chen Hui (8): sched: programmable: Add user interface of task tag sched: programmable: add bpf_sched_task_tag_of helper function bpf: sched: Add helper functions to get cpu statistics sched: programmable: Add helper function for cpu topology. sched: programmable: Add convenient helper functions to convert sched entity bpf:programmable: Add cpumask ops collection sched: programmable: Add three hooks in select_task_rq_fair() sched: programmable: Add lib for sched programmable Guan Jing (10): bpf: sched: basic infrastructure for scheduler bpf bpf: sched: introduce bpf_sched_enable() libbpf: add support for scheduler bpf programs bpftool: recognize scheduler programs sched: programmable: Add a tag for the task sched: programmable: Add a tag for the task group sched: programmable: Add user interface of task group tag sched: programmable: add bpf_sched_tg_tag_of helper function sched: cfs: add bpf hooks to control wakeup and tick preemption sched: programmable: Add hook for pick next task Hui Tang (1): bpf:programmable: Add helper func to check cpu share cache Ren Zhijie (2): sched: programmable: Add helpers to set tag of task or task_group sched: programmable: add bpf hooks to update rq and task state in enqueue_task/deqeue_task of CFS fs/proc/base.c | 65 ++++ include/linux/bpf_sched.h | 50 ++++ include/linux/bpf_topology.h | 46 +++ include/linux/bpf_types.h | 4 + include/linux/sched.h | 83 ++++++ include/linux/sched_hook_defs.h | 12 + include/uapi/linux/bpf.h | 125 ++++++++ init/init_task.c | 3 + kernel/bpf/Kconfig | 11 + kernel/bpf/btf.c | 1 + kernel/bpf/helpers.c | 12 + kernel/bpf/syscall.c | 24 ++ kernel/bpf/trampoline.c | 1 + kernel/bpf/verifier.c | 11 +- kernel/sched/bpf_sched.c | 395 +++++++++++++++++++++++++ kernel/sched/bpf_topology.c | 87 ++++++ kernel/sched/build_utility.c | 5 + kernel/sched/core.c | 114 +++++++ kernel/sched/fair.c | 121 ++++++++ kernel/sched/sched.h | 6 + scripts/bpf_doc.py | 18 ++ tools/include/uapi/linux/bpf.h | 125 ++++++++ tools/lib/bpf/Makefile | 2 +- tools/lib/bpf/bpf.c | 1 + tools/lib/bpf/libbpf.c | 23 +- tools/lib/bpf/libbpf.h | 2 + tools/lib/bpf/libbpf.map | 1 + tools/lib/bpf/libbpf_sched.h | 507 ++++++++++++++++++++++++++++++++ 28 files changed, 1852 insertions(+), 3 deletions(-) create mode 100644 include/linux/bpf_sched.h create mode 100644 include/linux/bpf_topology.h create mode 100644 include/linux/sched_hook_defs.h create mode 100644 kernel/sched/bpf_sched.c create mode 100644 kernel/sched/bpf_topology.c create mode 100644 tools/lib/bpf/libbpf_sched.h -- 2.17.1

2 22

[PATCH openEuler-23.09 0/2] enable cgroup1 writeback
by Lu Jialin 05 Sep '23

05 Sep '23

Lu Jialin (2): cgroup: Add cgroup1_get_from_id() cgroup: support cgroup writeback on cgroupv1 arch/arm64/configs/openeuler_defconfig | 1 + arch/x86/configs/openeuler_defconfig | 1 + block/blk-cgroup.c | 3 + block/blk-cgroup.h | 3 + include/linux/backing-dev.h | 29 +++++- include/linux/cgroup-defs.h | 2 + include/linux/cgroup.h | 3 + include/linux/memcontrol.h | 5 ++ init/Kconfig | 5 ++ kernel/cgroup/cgroup.c | 46 +++++++++- mm/backing-dev.c | 118 ++++++++++++++++++++++++- mm/memcontrol.c | 84 +++++++++++++++++- 12 files changed, 291 insertions(+), 9 deletions(-) -- 2.34.1

2 3

[PATCH v3 openEuler-23.09 00/21] Introduce programmable base on bpf for scheduler
by Guan Jing 05 Sep '23

05 Sep '23

Chen Hui (8): sched: programmable: Add user interface of task tag sched: programmable: add bpf_sched_task_tag_of helper function bpf: sched: Add helper functions to get cpu statistics sched: programmable: Add helper function for cpu topology. sched: programmable: Add convenient helper functions to convert sched entity bpf:programmable: Add cpumask ops collection sched: programmable: Add three hooks in select_task_rq_fair() sched: programmable: Add lib for sched programmable Guan Jing (10): bpf: sched: basic infrastructure for scheduler bpf bpf: sched: introduce bpf_sched_enable() libbpf: add support for scheduler bpf programs bpftool: recognize scheduler programs sched: programmable: Add a tag for the task sched: programmable: Add a tag for the task group sched: programmable: Add user interface of task group tag sched: programmable: add bpf_sched_tg_tag_of helper function sched: cfs: add bpf hooks to control wakeup and tick preemption sched: programmable: Add hook for pick next task Hui Tang (1): bpf:programmable: Add helper func to check cpu share cache Ren Zhijie (2): sched: programmable: Add helpers to set tag of task or task_group sched: programmable: add bpf hooks to update rq and task state in enqueue_task/deqeue_task of CFS fs/proc/base.c | 65 ++++ include/linux/bpf_sched.h | 50 ++++ include/linux/bpf_topology.h | 46 +++ include/linux/bpf_types.h | 4 + include/linux/sched.h | 83 ++++++ include/linux/sched_hook_defs.h | 12 + include/uapi/linux/bpf.h | 125 ++++++++ init/init_task.c | 3 + kernel/bpf/Kconfig | 11 + kernel/bpf/btf.c | 1 + kernel/bpf/helpers.c | 12 + kernel/bpf/syscall.c | 24 ++ kernel/bpf/trampoline.c | 1 + kernel/bpf/verifier.c | 11 +- kernel/sched/bpf_sched.c | 395 +++++++++++++++++++++++++ kernel/sched/bpf_topology.c | 87 ++++++ kernel/sched/build_utility.c | 5 + kernel/sched/core.c | 114 +++++++ kernel/sched/fair.c | 121 ++++++++ kernel/sched/sched.h | 6 + scripts/bpf_doc.py | 18 ++ tools/include/uapi/linux/bpf.h | 125 ++++++++ tools/lib/bpf/Makefile | 2 +- tools/lib/bpf/bpf.c | 1 + tools/lib/bpf/libbpf.c | 23 +- tools/lib/bpf/libbpf.h | 2 + tools/lib/bpf/libbpf.map | 1 + tools/lib/bpf/libbpf_sched.h | 507 ++++++++++++++++++++++++++++++++ 28 files changed, 1852 insertions(+), 3 deletions(-) create mode 100644 include/linux/bpf_sched.h create mode 100644 include/linux/bpf_topology.h create mode 100644 include/linux/sched_hook_defs.h create mode 100644 kernel/sched/bpf_sched.c create mode 100644 kernel/sched/bpf_topology.c create mode 100644 tools/lib/bpf/libbpf_sched.h -- 2.17.1

2 22

[PATCH openEuler-22.03-LTS-SP1] block: don't get gendisk if queue has not been registered
by Li Lingfeng 05 Sep '23

05 Sep '23

hulk inclusion category: bugfix bugzilla: 189153, https://gitee.com/openeuler/kernel/issues/I7XVQD CVE: NA ---------------------------------------- Creating and using dm device at same time may trigger null-ptr-deref or deadlock. Some resources have not been initialized after calling add_disk in the process of creating dm device, which means users can get device and use some uninitialized resources. Using uninitialized resources may trigger null-ptr-deref directly. What's more, IO can't be finished by waiting specific resources to be initialized and the resources can't be initialized since IO is inflight, which triggers deadlock. Don't get gendisk if queue has not been registered to fix this. Signed-off-by: Li Lingfeng <lilingfeng3(a)huawei.com> --- block/blk-sysfs.c | 4 ++-- block/blk-throttle.c | 2 +- fs/block_dev.c | 6 ++++++ include/linux/blkdev.h | 2 +- 4 files changed, 10 insertions(+), 4 deletions(-) diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c index db7c0503bead..c95be9626a09 100644 --- a/block/blk-sysfs.c +++ b/block/blk-sysfs.c @@ -908,7 +908,7 @@ int blk_register_queue(struct gendisk *disk) wbt_enable_default(q); blk_throtl_register_queue(q); spin_lock_irq(&q->queue_lock); - blk_queue_flag_set(QUEUE_FLAG_THROTL_INIT_DONE, q); + blk_queue_flag_set(QUEUE_FLAG_REGISTER_DONE, q); spin_unlock_irq(&q->queue_lock); /* Now everything is ready and send out KOBJ_ADD uevent */ @@ -944,7 +944,7 @@ void blk_unregister_queue(struct gendisk *disk) return; spin_lock_irq(&q->queue_lock); - blk_queue_flag_clear(QUEUE_FLAG_THROTL_INIT_DONE, q); + blk_queue_flag_clear(QUEUE_FLAG_REGISTER_DONE, q); spin_unlock_irq(&q->queue_lock); /* diff --git a/block/blk-throttle.c b/block/blk-throttle.c index 2686634d2cc4..42ce18f991c4 100644 --- a/block/blk-throttle.c +++ b/block/blk-throttle.c @@ -1518,7 +1518,7 @@ static void tg_conf_updated(struct throtl_grp *tg, u64 *old_limits, bool global) static inline int throtl_check_init_done(struct request_queue *q) { - if (test_bit(QUEUE_FLAG_THROTL_INIT_DONE, &q->queue_flags)) + if (test_bit(QUEUE_FLAG_REGISTER_DONE, &q->queue_flags)) return 0; return blk_queue_dying(q) ? -ENODEV : -EBUSY; diff --git a/fs/block_dev.c b/fs/block_dev.c index 1c56cddcaa3d..a0e4d3ec300e 100644 --- a/fs/block_dev.c +++ b/fs/block_dev.c @@ -1136,6 +1136,12 @@ static struct gendisk *bdev_get_gendisk(struct block_device *bdev, int *partno) if (!disk) return NULL; + + if (!test_bit(QUEUE_FLAG_REGISTER_DONE, &disk->queue->queue_flags)) { + put_disk_and_module(disk); + return NULL; + } + /* * Now that we hold gendisk reference we make sure bdev we looked up is * not stale. If it is, it means device got removed and created before diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 9ede0a81e466..50b4fd0a0687 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -612,7 +612,7 @@ struct request_queue { /* Keep blk_queue_flag_name[] in sync with the definitions below */ #define QUEUE_FLAG_STOPPED 0 /* queue is stopped */ #define QUEUE_FLAG_DYING 1 /* queue being torn down */ -#define QUEUE_FLAG_THROTL_INIT_DONE 2 /* io throttle can be online */ +#define QUEUE_FLAG_REGISTER_DONE 2 /* blkdev can be opened */ #define QUEUE_FLAG_NOMERGES 3 /* disable merge attempts */ #define QUEUE_FLAG_SAME_COMP 4 /* complete on same CPU-group */ #define QUEUE_FLAG_FAIL_IO 5 /* fake timeout */ -- 2.31.1

2 1

[PATCH openEuler-23.09 v3 0/9] enable memcg qos and memcg async reclaim
by Lu Jialin 05 Sep '23

05 Sep '23

v3->v2: correct input param of try_to_free_mem_cgroup_pages v2->v1: introduce CONFIG_CGROUP_V1_KILL to isolated cgroup.kill in cgroupv1 Lu Jialin (9): memcg: Export memcg.{min/low} from cgroupv2 to cgroupv1 memcg: Export memcg.high from cgroupv2 to cgroupv1 memcg: Export memory.events and memory.events.local from cgroupv2 to cgroupv1 memcg: Fix inconsistent oom event behavior for OOM_MEMCG_KILL cgroup: Export cgroup.kill from cgroupv2 to cgroupv1 memcg: Introduce CONFIG_MEMCG_V1_THRESHOLD_QOS memcg: enable memcg async reclaim memcg: export high_async_ratio to userland cgroup: add config isolation for cgroup_kill in cgroupv1 arch/arm64/configs/openeuler_defconfig | 2 + arch/x86/configs/openeuler_defconfig | 2 + include/linux/memcontrol.h | 23 +- init/Kconfig | 10 + kernel/cgroup/cgroup-internal.h | 3 + kernel/cgroup/cgroup-v1.c | 7 + kernel/cgroup/cgroup.c | 4 +- mm/memcontrol.c | 382 ++++++++++++++++++------- 8 files changed, 321 insertions(+), 112 deletions(-) -- 2.34.1

2 10