- Kernel - mailweb.openeuler.org

[PATCH OLK-6.6] ext4: Track data blocks freeing operation in journal
by Zhihao Cheng 05 Sep '24

05 Sep '24

Offering: HULK hulk inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I9DN5Z -------------------------------- Since commit 7f6416dcd4a3f ("ext4: implement writeback iomap path"), the order mode is removed in iomap framework, which lets ext4_mb_clear_bb() free data blocks immediately. It may cause stale data read from truncated file in power-cut case. Following is details: P1 P2 vfs_truncate(file A) ext4_setattr EXT4_I(inode)->i_disksize = attr->ia_size // record in journal ext4_truncate ext4_mb_clear_bb mb_free_blocks // free block i vfs_write(file B) // get block i and writeback >> powercut << In the next mount, inode size and extent tree is stale(before truncated), the content in block i is file B. Fix the problem by tracking free data blocks in journal for iomap/non-writeback case. Fixes: 7f6416dcd4a3 ("ext4: implement writeback iomap path") Signed-off-by: Zhihao Cheng <chengzhihao1(a)huawei.com> --- fs/ext4/ext4_jbd2.c | 9 ++------- fs/ext4/ext4_jbd2.h | 7 +++++++ 2 files changed, 9 insertions(+), 7 deletions(-) diff --git a/fs/ext4/ext4_jbd2.c b/fs/ext4/ext4_jbd2.c index 94c8073b49e7..cac0eb5068d3 100644 --- a/fs/ext4/ext4_jbd2.c +++ b/fs/ext4/ext4_jbd2.c @@ -11,18 +11,13 @@ int ext4_inode_journal_mode(struct inode *inode) { if (EXT4_JOURNAL(inode) == NULL) return EXT4_INODE_WRITEBACK_DATA_MODE; /* writeback */ - /* - * Ordered mode is no longer needed for the inode that use the - * iomap path, always use writeback mode. - */ - if (ext4_test_inode_state(inode, EXT4_STATE_BUFFERED_IOMAP)) - return EXT4_INODE_WRITEBACK_DATA_MODE; /* writeback */ /* We do not support data journalling with delayed allocation */ if (!S_ISREG(inode->i_mode) || ext4_test_inode_flag(inode, EXT4_INODE_EA_INODE) || test_opt(inode->i_sb, DATA_FLAGS) == EXT4_MOUNT_JOURNAL_DATA || (ext4_test_inode_flag(inode, EXT4_INODE_JOURNAL_DATA) && - !test_opt(inode->i_sb, DELALLOC))) { + !test_opt(inode->i_sb, DELALLOC) && + !ext4_test_inode_state(inode, EXT4_STATE_BUFFERED_IOMAP))) { /* We do not support data journalling for encrypted data */ if (S_ISREG(inode->i_mode) && IS_ENCRYPTED(inode)) return EXT4_INODE_ORDERED_DATA_MODE; /* ordered */ diff --git a/fs/ext4/ext4_jbd2.h b/fs/ext4/ext4_jbd2.h index 0c77697d5e90..c52d1caf6622 100644 --- a/fs/ext4/ext4_jbd2.h +++ b/fs/ext4/ext4_jbd2.h @@ -467,6 +467,13 @@ static inline int ext4_should_journal_data(struct inode *inode) static inline int ext4_should_order_data(struct inode *inode) { + /* + * Ordered mode is no longer needed for the inode that use the + * iomap path, always use writeback mode. + */ + if (ext4_test_inode_state(inode, EXT4_STATE_BUFFERED_IOMAP)) + return 0; /* writeback */ + return ext4_inode_journal_mode(inode) & EXT4_INODE_ORDERED_DATA_MODE; } -- 2.31.1

2 1

[openeuler:OLK-6.6 13795/13866] mm/shmem.c:1641:57: error: call to '__compiletime_assert_745' declared with 'error' attribute: BUILD_BUG failed
by kernel test robot 05 Sep '24

05 Sep '24

tree: https://gitee.com/openeuler/kernel.git OLK-6.6 head: 552667f6edabe9e4bc2c86f990f53749bebfcbed commit: c7fcbe1041758d0dedc32502609a73a22884d7b8 [13795/13866] mm: shmem: Merge shmem_alloc_hugefolio() with shmem_alloc_folio() config: x86_64-randconfig-014-20240905 (https://download.01.org/0day-ci/archive/20240905/202409051939.TTEs2Xg7-lkp@…) compiler: clang version 18.1.5 (https://github.com/llvm/llvm-project 617a15a9eac96088ae5e9134248d8236e34b91b1) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240905/202409051939.TTEs2Xg7-lkp@…) If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot <lkp(a)intel.com> | Closes: https://lore.kernel.org/oe-kbuild-all/202409051939.TTEs2Xg7-lkp@intel.com/ All errors (new ones prefixed by >>): mm/shmem.c:1660:6: warning: variable 'folio' is used uninitialized whenever 'if' condition is true [-Wsometimes-uninitialized] 1660 | if (!shmem_prepare_alloc(&gfp)) | ^~~~~~~~~~~~~~~~~~~~~~~~~~ mm/shmem.c:1688:7: note: uninitialized use occurs here 1688 | if (!folio) | ^~~~~ mm/shmem.c:1660:2: note: remove the 'if' if its condition is always false 1660 | if (!shmem_prepare_alloc(&gfp)) | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1661 | goto no_mem; | ~~~~~~~~~~~ mm/shmem.c:1653:21: note: initialize the variable 'folio' to silence this warning 1653 | struct folio *folio; | ^ | = NULL >> mm/shmem.c:1641:57: error: call to '__compiletime_assert_745' declared with 'error' attribute: BUILD_BUG failed 1641 | folio = vma_alloc_folio(gfp, order, &pvma, 0, order == HPAGE_PMD_ORDER); | ^ include/linux/huge_mm.h:110:26: note: expanded from macro 'HPAGE_PMD_ORDER' 110 | #define HPAGE_PMD_ORDER (HPAGE_PMD_SHIFT-PAGE_SHIFT) | ^ include/linux/huge_mm.h:106:28: note: expanded from macro 'HPAGE_PMD_SHIFT' 106 | #define HPAGE_PMD_SHIFT ({ BUILD_BUG(); 0; }) | ^ include/linux/build_bug.h:59:21: note: expanded from macro 'BUILD_BUG' 59 | #define BUILD_BUG() BUILD_BUG_ON_MSG(1, "BUILD_BUG failed") | ^ note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all) include/linux/compiler_types.h:439:2: note: expanded from macro '_compiletime_assert' 439 | __compiletime_assert(condition, msg, prefix, suffix) | ^ include/linux/compiler_types.h:432:4: note: expanded from macro '__compiletime_assert' 432 | prefix ## suffix(); \ | ^ <scratch space>:61:1: note: expanded from here 61 | __compiletime_assert_745 | ^ >> mm/shmem.c:1641:57: error: call to '__compiletime_assert_745' declared with 'error' attribute: BUILD_BUG failed include/linux/huge_mm.h:110:26: note: expanded from macro 'HPAGE_PMD_ORDER' 110 | #define HPAGE_PMD_ORDER (HPAGE_PMD_SHIFT-PAGE_SHIFT) | ^ include/linux/huge_mm.h:106:28: note: expanded from macro 'HPAGE_PMD_SHIFT' 106 | #define HPAGE_PMD_SHIFT ({ BUILD_BUG(); 0; }) | ^ include/linux/build_bug.h:59:21: note: expanded from macro 'BUILD_BUG' 59 | #define BUILD_BUG() BUILD_BUG_ON_MSG(1, "BUILD_BUG failed") | ^ note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all) include/linux/compiler_types.h:439:2: note: expanded from macro '_compiletime_assert' 439 | __compiletime_assert(condition, msg, prefix, suffix) | ^ include/linux/compiler_types.h:432:4: note: expanded from macro '__compiletime_assert' 432 | prefix ## suffix(); \ | ^ <scratch space>:61:1: note: expanded from here 61 | __compiletime_assert_745 | ^ 1 warning and 2 errors generated. vim +1641 mm/shmem.c 1633 1634 static struct folio *shmem_alloc_folio(gfp_t gfp, int order, 1635 struct shmem_inode_info *info, pgoff_t index) 1636 { 1637 struct vm_area_struct pvma; 1638 struct folio *folio; 1639 1640 shmem_pseudo_vma_init(&pvma, info, index); > 1641 folio = vma_alloc_folio(gfp, order, &pvma, 0, order == HPAGE_PMD_ORDER); 1642 shmem_pseudo_vma_destroy(&pvma); 1643 1644 return folio; 1645 } 1646 -- 0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki

1 0

[PATCH openEuler-23.09] tools: Add dynamic process-level cgroup memory monitoring tool
by Taoxy2004 05 Sep '24

05 Sep '24

community inclusion category: feature bugzilla: https://gitee.com/openeuler/open-source-summer/issues/I9JQ4D ---------------------------------------------------------------------- This patch introduces a new tool called "probeCgroup" that enables dynamic monitoring of memory usage at the process level within cgroups. By using kprobes at relevant cgroup functions, this tool can track memory allocations and deallocations for individual processes within a cgroup, providing detailed statistics on memory usage. The key features of the tool include: 1. Dynamic insertion of kprobes at critical points in the cgroup subsystem. 2. Tracking memory allocation and deallocation events for each process by recording page addresses in a hash table. 3. Providing real-time statistics on memory usage at the process level. 4. Providing statistics on memory usage for processes that are OOM. Signed-off-by: Taoxy2004 <221870066(a)smail.nju.edu.cn> --- tools/probeCgroup/Makefile | 7 + tools/probeCgroup/README.md | 29 + tools/probeCgroup/probeCgroup.c | 612 ++++++++++++++++++ tools/probeCgroup/probeCgroup.h | 415 ++++++++++++ tools/probeCgroup/run.sh | 8 + tools/probeCgroup/scripts/script1.sh | 10 + tools/probeCgroup/scripts/script2.sh | 14 + tools/probeCgroup/scripts/script3.sh | 11 + .../testcases/1_load_unload_test.py | 24 + .../testcases/2_multiple_process_test.py | 48 ++ .../testcases/3_multiple_cgroup_test.py | 55 ++ tools/probeCgroup/testcases/4_oom_test.py | 52 ++ .../testcases/5_multiple_threads_test.py | 45 ++ tools/probeCgroup/testcases/cgroup_utils.py | 114 ++++ tools/probeCgroup/testcases/mem-allocate.c | 35 + .../testcases/multiple-thread-mem-allocate.c | 60 ++ tools/probeCgroup/testcases/run.py | 32 + .../testcases/simple-mem-allocate.c | 27 + 18 files changed, 1598 insertions(+) create mode 100644 tools/probeCgroup/Makefile create mode 100644 tools/probeCgroup/README.md create mode 100644 tools/probeCgroup/probeCgroup.c create mode 100644 tools/probeCgroup/probeCgroup.h create mode 100755 tools/probeCgroup/run.sh create mode 100755 tools/probeCgroup/scripts/script1.sh create mode 100755 tools/probeCgroup/scripts/script2.sh create mode 100755 tools/probeCgroup/scripts/script3.sh create mode 100755 tools/probeCgroup/testcases/1_load_unload_test.py create mode 100755 tools/probeCgroup/testcases/2_multiple_process_test.py create mode 100755 tools/probeCgroup/testcases/3_multiple_cgroup_test.py create mode 100755 tools/probeCgroup/testcases/4_oom_test.py create mode 100755 tools/probeCgroup/testcases/5_multiple_threads_test.py create mode 100644 tools/probeCgroup/testcases/cgroup_utils.py create mode 100644 tools/probeCgroup/testcases/mem-allocate.c create mode 100644 tools/probeCgroup/testcases/multiple-thread-mem-allocate.c create mode 100755 tools/probeCgroup/testcases/run.py create mode 100644 tools/probeCgroup/testcases/simple-mem-allocate.c diff --git a/tools/probeCgroup/Makefile b/tools/probeCgroup/Makefile new file mode 100644 index 000000000000..606c951e5487 --- /dev/null +++ b/tools/probeCgroup/Makefile @@ -0,0 +1,7 @@ +obj-m := probeCgroup.o +CROSS_COMPILE = '' +KDIR := /lib/modules/$(shell uname -r)/build +all: + make -C $(KDIR) M=$(PWD) modules +clean: + rm -f *.ko *.o *.mod *.mod.o *.mod.c .*.cmd *.symvers module* diff --git a/tools/probeCgroup/README.md b/tools/probeCgroup/README.md new file mode 100644 index 000000000000..ff0b6fc21228 --- /dev/null +++ b/tools/probeCgroup/README.md @@ -0,0 +1,29 @@ +# probeCgroup + +#### Description +probeCgroup is a process-level cgroup memory monitoring tool based on dynamic tracing (kprobe/kretprobe) technology. By inserting kprobes and kretprobes at the entry and exit points of relevant cgroup functions, this tool can track the memory usage of individual processes within each cgroup in real time. + +#### Software Architecture +1. Dynamic Tracing : Insert kprobes and kretprobes at critical points in cgroup functions to capture memory allocation and release events. +2. Hash Table Recording : Record the addresses of pages currently used by each process in a hash table, so that when a page is released, the process it belongs to can be identified. +3. Real-Time Statistics : Provide real-time statistics showing the memory usage of individual processes within each cgroup. + +#### Instruction +1. Compile and Load the Module + a. In the 'probeCgroup' directory, run the 'make' command to compile the module. + b. Load the module: 'insmod probeCgroup.ko'. + c. View memory statistics: 'cat /proc/cgroup_memory_usage_per_process'. + If an OOM (Out of Memory) event occurs in a cgroup, you can see "oom:" followed by the process that experienced the OOM and its memory usage at the time. + +2. Automate OOM Scenario + In the 'probeCgroup' directory, run './run.sh'. This script will automatically set up an OOM scenario and output the content of '/proc/cgroup_memory_usage_per_process' after execution. + +3. Perform More Tests + a. After compiling the module, in the 'testcases' directory, run './run.py'. + b. This script will perform various tests, including: + - Loading and unloading the module + - Each cgroup containing multiple processes + - Creating multiple cgroups + - OOM scenarios + - Multithreading + c. The tests will take approximately one minute to complete. diff --git a/tools/probeCgroup/probeCgroup.c b/tools/probeCgroup/probeCgroup.c new file mode 100644 index 000000000000..9883cb1e082d --- /dev/null +++ b/tools/probeCgroup/probeCgroup.c @@ -0,0 +1,612 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * probeCgroup.c - A tool used to get memory usage for each process in a cgroup + * + * Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + */ + +#include "probeCgroup.h" + +// kretprobe at mem_cgroup_charge +struct charge_data { + struct cgroup *cgrp; + struct mem_cgroup *memcg; + struct task_struct *task; + unsigned long addr; +}; + +static int mem_cgroup_charge_entry_handler(struct kretprobe_instance *ri, + struct pt_regs *regs) +{ + struct charge_data *data; + struct folio *page; + struct mm_struct *mm; + struct mem_cgroup *memcg; + struct cgroup_subsys_state css; + struct cgroup *cgrp; + + if (!current->mm) + return 1; + page = (struct folio *)regs->di; + mm = (struct mm_struct *)regs->si; + if (mm == NULL || page == NULL) + return -1; + memcg = get_mem_cgroup_from_mm(mm); + if (memcg != NULL) { + css = memcg->css; + cgrp = css.cgroup; + + data = (struct charge_data *)ri->data; + data->memcg = memcg; + data->addr = (unsigned long)page; + data->task = current; + data->cgrp = cgrp; + } + return 0; +} + +NOKPROBE_SYMBOL(mem_cgroup_charge_entry_handler); + +static int mem_cgroup_charge_ret_handler(struct kretprobe_instance *ri, + struct pt_regs *regs) +{ + unsigned long retval = regs_return_value(regs); + struct charge_data *data = (struct charge_data *)ri->data; + int id; + struct cgroup_info *cgrp_info; + struct task_info *tsk_info; + + if (data->memcg != NULL && retval == 0) { + id = ((data->memcg)->css).id; + + spin_lock(&lock); + cgrp_info = find_cgroup_info(id); + if (cgrp_info == NULL) { + cgrp_info = create_cgroup_info(data->cgrp, data->memcg); + if (cgrp_info == NULL) { + spin_unlock(&lock); + return -1; + } + add_cgroup_info(cgrp_info); + } + spin_unlock(&lock); + + read_lock(&cgrp_info->cgrp_lock); + tsk_info = find_task_info(cgrp_info, data->task->tgid); + read_unlock(&cgrp_info->cgrp_lock); + + // for some cases, task->comm changes over time + if (tsk_info != NULL + && strcmp(data->task->comm, tsk_info->comm) != 0) { + strscpy(tsk_info->comm, data->task->comm, + sizeof(tsk_info->comm)); + } + + if (tsk_info == NULL) { + tsk_info = create_task_info(data->task); + if (tsk_info == NULL) + return -1; + add_task_to_cgroup_info(cgrp_info, tsk_info); + } + + if (HashMap_insert(tsk_info->pages, data->addr)) { + //update counter + spin_lock(&(tsk_info->cnt_lock)); + tsk_info->count += + folio_nr_pages((struct folio *)data->addr); + spin_unlock(&(tsk_info->cnt_lock)); + } + } + + return 0; + +} + +NOKPROBE_SYMBOL(mem_cgroup_charge_ret_handler); + +static struct kretprobe mem_cgroup_charge_kretprobe = { + .handler = mem_cgroup_charge_ret_handler, + .entry_handler = mem_cgroup_charge_entry_handler, + .data_size = sizeof(struct charge_data), + .maxactive = 20, +}; + +static int mem_cgroup_charge_kretprobe_init(void) +{ + int ret; + + mem_cgroup_charge_kretprobe.kp.symbol_name = "__mem_cgroup_charge"; + ret = register_kretprobe(&mem_cgroup_charge_kretprobe); + if (ret < 0) { + pr_err("register_kretprobe failed, returned %d\n", ret); + return ret; + } + pr_info("Planted return probe at %s: %p\n", + mem_cgroup_charge_kretprobe.kp.symbol_name, + mem_cgroup_charge_kretprobe.kp.addr); + return 0; +} + +static void mem_cgroup_charge_kretprobe_exit(void) +{ + unregister_kretprobe(&mem_cgroup_charge_kretprobe); + pr_info("kretprobe at %p unregistered\n", + mem_cgroup_charge_kretprobe.kp.addr); + + /* nmissed > 0 suggests that maxactive was set too low. */ + pr_info("Missed probing %d instances of %s\n", + mem_cgroup_charge_kretprobe.nmissed, + mem_cgroup_charge_kretprobe.kp.symbol_name); +} + +// kretprobe at uncharge_folio + +struct uncharge_data { + struct cgroup *cgrp; + struct mem_cgroup *memcg; + unsigned long addr; + bool isKmem; + int nr_pages; +}; + +static int uncharge_folio_entry_handler(struct kretprobe_instance *ri, + struct pt_regs *regs) +{ + struct uncharge_data *data; + struct folio *page; + struct mem_cgroup *memcg = NULL; + struct cgroup_subsys_state css; + struct cgroup *cgrp; + struct obj_cgroup *objcg; + int nr_pages = 0; + + data = (struct uncharge_data *)ri->data; + page = (struct folio *)regs->di; + if (page == NULL) { + data->memcg = NULL; + return -1; + } + if (page->memcg_data & MEMCG_DATA_KMEM) { // if the page belongs to kmem + if (!folio_test_large(page)) + nr_pages = 1; + else + nr_pages = page->_folio_nr_pages; + // nr_pages = thp_nr_pages(page); + objcg = __folio_objcg(page); + if (objcg != NULL) + memcg = objcg->memcg; + data->isKmem = true; + data->nr_pages = nr_pages; + } else { + memcg = __folio_memcg(page); + data->isKmem = false; + } + + if (memcg != NULL) { + css = memcg->css; + cgrp = css.cgroup; + + data->memcg = memcg; + data->addr = (unsigned long)page; + data->cgrp = cgrp; + } + return 0; +} + +NOKPROBE_SYMBOL(uncharge_folio_entry_handler); + +static int uncharge_folio_ret_handler(struct kretprobe_instance *ri, + struct pt_regs *regs) +{ + struct uncharge_data *data = (struct uncharge_data *)ri->data; + int id; + struct cgroup_info *cgrp_info; + int ret = -1; + + if (data->memcg != NULL) { + id = ((data->memcg)->css).id; + cgrp_info = find_cgroup_info(id); + if (cgrp_info == NULL) + return -1; + if (data->isKmem) + ret = -1; + else + ret = remove_page_from_cgroup_info(data->addr, cgrp_info); + } + + return ret; +} + +NOKPROBE_SYMBOL(uncharge_folio_ret_handler); + +static struct kretprobe uncharge_folio_kretprobe = { + .handler = uncharge_folio_ret_handler, + .entry_handler = uncharge_folio_entry_handler, + .data_size = sizeof(struct uncharge_data), + .maxactive = 20, +}; + +static int uncharge_folio_kretprobe_init(void) +{ + int ret; + + uncharge_folio_kretprobe.kp.symbol_name = "uncharge_folio"; + ret = register_kretprobe(&uncharge_folio_kretprobe); + if (ret < 0) { + pr_err("register_kretprobe failed, returned %d\n", ret); + return ret; + } + pr_info("Planted return probe at %s: %p\n", + uncharge_folio_kretprobe.kp.symbol_name, + uncharge_folio_kretprobe.kp.addr); + return 0; +} + +static void uncharge_folio_kretprobe_exit(void) +{ + unregister_kretprobe(&uncharge_folio_kretprobe); + pr_info("kretprobe at %p unregistered\n", + uncharge_folio_kretprobe.kp.addr); + + /* nmissed > 0 suggests that maxactive was set too low. */ + pr_info("Missed probing %d instances of %s\n", + uncharge_folio_kretprobe.nmissed, + uncharge_folio_kretprobe.kp.symbol_name); +} + +//kprobe at do_exit +static struct kprobe do_exit_kprobe; +static int do_exit_kprobe_pre_handler(struct kprobe *p, struct pt_regs *regs) +{ + struct task_struct *cur = current; + int tgid = cur->tgid; + struct mm_struct *mm = cur->mm; + struct mem_cgroup *memcg = get_mem_cgroup_from_mm(mm); + struct cgroup_subsys_state css; + struct cgroup *cgrp; + struct cgroup_info *cgrp_info; + struct task_info *tsk_info; + int id; + + if (memcg != NULL) { + css = memcg->css; + cgrp = css.cgroup; + id = (memcg->css).id; + cgrp_info = find_cgroup_info(id); + if (cgrp_info != NULL) { + write_lock(&cgrp_info->cgrp_lock); + tsk_info = find_task_info(cgrp_info, tgid); + if (tsk_info != NULL) { + list_del(&tsk_info->list); + write_unlock(&cgrp_info->cgrp_lock); + remove_task_from_cgroup_info(cgrp_info, + tsk_info); + } else { + write_unlock(&cgrp_info->cgrp_lock); + } + return 0; + } + } + return 0; +} + +static void do_exit_kprobe_post_handler(struct kprobe *p, + struct pt_regs *regs, + unsigned long flags) +{ + +} + +static int do_exit_kprobe_init(void) +{ + do_exit_kprobe.pre_handler = do_exit_kprobe_pre_handler; + do_exit_kprobe.post_handler = do_exit_kprobe_post_handler; + do_exit_kprobe.symbol_name = "do_exit"; + if (register_kprobe(&do_exit_kprobe)) { + pr_alert("register_kprobe on do_exit failed!\n"); + return -EINVAL; + } + return 0; +} + +static void do_exit_kprobe_exit(void) +{ + unregister_kprobe(&do_exit_kprobe); +} + +//kprobe at mark_oom_victim +static struct kprobe mark_oom_victim_kprobe; + +static int mark_oom_victim_kprobe_pre_handler(struct kprobe *p, + struct pt_regs *regs) +{ + struct task_struct *victim; + int tgid; + struct mm_struct *mm; + struct mem_cgroup *memcg; + struct cgroup_subsys_state css; + struct cgroup *cgrp; + struct cgroup_info *cgrp_info; + struct task_info *tsk_info; + int id; + struct task_info *oom_info; + + victim = (struct task_struct *)regs->di; + tgid = victim->tgid; + mm = victim->mm; + memcg = get_mem_cgroup_from_mm(mm); + if (memcg != NULL) { + css = memcg->css; + cgrp = css.cgroup; + id = (memcg->css).id; + cgrp_info = find_cgroup_info(id); + if (cgrp_info != NULL) { + read_lock(&cgrp_info->cgrp_lock); + tsk_info = find_task_info(cgrp_info, tgid); + read_unlock(&cgrp_info->cgrp_lock); + if (tsk_info != NULL) { + oom_info = create_oom_task_info(tsk_info); + if (oom_info != NULL) { + add_oom_task_to_cgroup_info(cgrp_info, + oom_info); + } + return 0; + } + } + } + return 0; +} + +static void mark_oom_victim_kprobe_post_handler(struct kprobe *p, + struct pt_regs *regs, + unsigned long flags) +{ + +} + +static int mark_oom_victim_kprobe_init(void) +{ + mark_oom_victim_kprobe.pre_handler = mark_oom_victim_kprobe_pre_handler; + mark_oom_victim_kprobe.post_handler = + mark_oom_victim_kprobe_post_handler; + mark_oom_victim_kprobe.symbol_name = "mark_oom_victim"; + if (register_kprobe(&mark_oom_victim_kprobe)) { + pr_alert("register_kprobe on mark_oom_victim failed!\n"); + return -EINVAL; + } + return 0; +} + +static void mark_oom_victim_kprobe_exit(void) +{ + unregister_kprobe(&mark_oom_victim_kprobe); +} + +//kretporbe at cgroup_destroy_locked +struct destroy_data { + struct cgroup *cgrp; +}; + +static int cgroup_destroy_locked_entry_handler(struct kretprobe_instance + *ri, struct pt_regs *regs) +{ + struct destroy_data *data; + + data = (struct destroy_data *)ri->data; + data->cgrp = (struct cgroup *)regs->di; + return 0; +} + +NOKPROBE_SYMBOL(cgroup_destroy_locked_entry_handler); + +static int cgroup_destroy_locked_ret_handler(struct kretprobe_instance *ri, + struct pt_regs *regs) +{ + struct destroy_data *data = (struct destroy_data *)ri->data; + struct cgroup *cgrp = data->cgrp; + struct cgroup_info *cgrp_info = NULL; + unsigned long retval = regs_return_value(regs); + + if (!cgrp) + return -1; + if (retval != 0) + return -1; + list_for_each_entry(cgrp_info, &all_cgroup_info, list) { + if (cgrp_info->cgrp == cgrp) { + spin_lock(&lock); + list_del(&cgrp_info->list); + spin_unlock(&lock); + destroy_cgroup_info(cgrp_info); + return 0; + } + } + return -1; +} + +NOKPROBE_SYMBOL(cgroup_destroy_locked_ret_handler); + +static struct kretprobe cgroup_destroy_locked_kretprobe = { + .handler = cgroup_destroy_locked_ret_handler, + .entry_handler = cgroup_destroy_locked_entry_handler, + .data_size = sizeof(struct destroy_data), + .maxactive = 20, +}; + +static int cgroup_destroy_locked_kretprobe_init(void) +{ + int ret; + + cgroup_destroy_locked_kretprobe.kp.symbol_name = + "cgroup_destroy_locked"; + ret = register_kretprobe(&cgroup_destroy_locked_kretprobe); + if (ret < 0) { + pr_err("register_kretprobe failed, returned %d\n", ret); + return ret; + } + pr_info("Planted return probe at %s: %p\n", + cgroup_destroy_locked_kretprobe.kp.symbol_name, + cgroup_destroy_locked_kretprobe.kp.addr); + return 0; +} + +static void cgroup_destroy_locked_kretprobe_exit(void) +{ + unregister_kretprobe(&cgroup_destroy_locked_kretprobe); + pr_info("kretprobe at %p unregistered\n", + cgroup_destroy_locked_kretprobe.kp.addr); + + /* nmissed > 0 suggests that maxactive was set too low. */ + pr_info("Missed probing %d instances of %s\n", + cgroup_destroy_locked_kretprobe.nmissed, + cgroup_destroy_locked_kretprobe.kp.symbol_name); +} + +// print the tasks in order of their memory usage +static void print_sorted_tasks_list(struct cgroup_info *cgrp_info, + int type, struct seq_file *m) +{ + struct list_head *cur, *insert_pos; + struct task_info *task, *insert_task; + struct list_head new_list = LIST_HEAD_INIT(new_list); + struct list_head *old_list; + struct task_info *new_task, *next_task; + + if (type == 0) { + if (cgrp_info == NULL) + return; + read_lock(&cgrp_info->cgrp_lock); + old_list = &cgrp_info->tasks_list; + } else { + if (cgrp_info == NULL) + return; + old_list = &cgrp_info->oom_list; + } + + list_for_each_entry_safe(task, insert_task, old_list, list) { + new_task = kmalloc(sizeof(struct task_info), GFP_ATOMIC); + if (!new_task) + return; + new_task->tgid = task->tgid; + strscpy(new_task->comm, task->comm, sizeof(new_task->comm)); + new_task->count = task->count; + new_task->pages = NULL; + INIT_LIST_HEAD(&new_task->list); + + //insertion sort + cur = &new_list; + insert_pos = cur->next; + while (insert_pos != &new_list) { + next_task = + list_entry(insert_pos, struct task_info, list); + if (new_task->count >= next_task->count) + break; + cur = insert_pos; + insert_pos = insert_pos->next; + } + + (&new_task->list)->prev = insert_pos->prev; + (insert_pos->prev)->next = (&new_task->list); + (&new_task->list)->next = insert_pos; + insert_pos->prev = (&new_task->list); + } + if (type == 0) + read_unlock(&cgrp_info->cgrp_lock); + + //print + if (type == 1 && (&new_list) != new_list.next) { + seq_puts(m, "oom:\n"); + seq_printf(m, "%10s %20s %20s\n", "pid", "command", + "memory usage (KB)"); + } + if (type == 0) + seq_printf(m, "%10s %20s %20s\n", "pid", "command", + "memory usage (KB)"); + list_for_each_entry_safe(task, insert_task, &new_list, list) { + seq_printf(m, "%10d %20s %20d\n", task->tgid, task->comm, + (task->count) * 4); + } + + list_for_each_entry_safe(task, insert_task, &new_list, list) { + list_del(&task->list); + kfree(task); + } +} + +static struct proc_dir_entry *cgroup_info_read; +#define procfs_file_read "cgroup_memory_usage_per_process" + +void seq_print_tasks(struct cgroup_info *cgroup_info, struct seq_file *m) +{ + if (!cgroup_info) + return; + + print_sorted_tasks_list(cgroup_info, 0, m); +} + +void seq_print_oom_tasks(struct cgroup_info *cgroup_info, struct seq_file *m) +{ + if (!cgroup_info) + return; + + print_sorted_tasks_list(cgroup_info, 1, m); +} + +void seq_print_cgroups(struct seq_file *m) +{ + struct cgroup_info *cgrp, *pos; + + spin_lock(&lock); + list_for_each_entry_safe(cgrp, pos, &all_cgroup_info, list) { + seq_printf(m, "cgroup name : %s\n", cgrp->name); + seq_print_tasks(cgrp, m); + seq_print_oom_tasks(cgrp, m); + seq_puts(m, "\n"); + } + spin_unlock(&lock); +} + +static int memory_usage_show(struct seq_file *m, void *v) +{ + seq_print_cgroups(m); + return 0; +} + +static int __init global_init(void) +{ + int ret = 0; + + cgroup_info_read = + proc_create_single(procfs_file_read, 0, NULL, memory_usage_show); + if (!cgroup_info_read) + return -ENOMEM; + ret = mem_cgroup_charge_kretprobe_init(); + uncharge_folio_kretprobe_init(); + do_exit_kprobe_init(); + mark_oom_victim_kprobe_init(); + cgroup_destroy_locked_kretprobe_init(); + + return ret; +} + +static void __exit global_exit(void) +{ + struct cgroup_info *cgrp_info, *pos; + + mem_cgroup_charge_kretprobe_exit(); + uncharge_folio_kretprobe_exit(); + do_exit_kprobe_exit(); + mark_oom_victim_kprobe_exit(); + cgroup_destroy_locked_kretprobe_exit(); + + remove_proc_entry(procfs_file_read, NULL); + + //release all memory use + list_for_each_entry_safe(cgrp_info, pos, &all_cgroup_info, list) { + list_del(&cgrp_info->list); + destroy_cgroup_info(cgrp_info); + } +} + +module_init(global_init) +module_exit(global_exit) +MODULE_LICENSE("GPL"); diff --git a/tools/probeCgroup/probeCgroup.h b/tools/probeCgroup/probeCgroup.h new file mode 100644 index 000000000000..953a6e0aca31 --- /dev/null +++ b/tools/probeCgroup/probeCgroup.h @@ -0,0 +1,415 @@ +/* SPDX-License-Identifier: GPL-2.0*/ +/* + * probeCgroup.h + * + * Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + */ + +#include <linux/kernel.h> +#include <linux/module.h> +#include <linux/kprobes.h> +#include <linux/ktime.h> +#include <linux/limits.h> +#include <linux/sched.h> +#include <linux/mm_types.h> +#include <linux/memcontrol.h> +#include <linux/cgroup-defs.h> +#include <linux/kernfs.h> +#include <linux/string.h> +#include <linux/list.h> +#include <linux/oom.h> +#include <linux/fs.h> +#include <linux/proc_fs.h> +#include <linux/huge_mm.h> +#include <linux/page-flags.h> +#include <linux/spinlock.h> +#include <linux/rwlock.h> + +static spinlock_t lock; // global lock for the list of cgroup_info + +struct HashNode { + unsigned long addr; + struct HashNode *next; +}; + +struct HashNode *HashNode_create(unsigned long addr) +{ + struct HashNode *node = NULL; + + node = kzalloc(sizeof(struct HashNode), GFP_ATOMIC); + if (node == NULL) + return NULL; + node->addr = addr; + node->next = NULL; + return node; +} + +struct HashBucket { + struct HashNode *head; + spinlock_t bkt_lock; +}; + +void HashBucket_init(struct HashBucket *bkt) +{ + bkt->head = NULL; + spin_lock_init(&bkt->bkt_lock); +} + +bool HashBucket_insert(struct HashBucket *bkt, unsigned long addr) +{ + struct HashNode *new_node; + struct HashNode *node; + struct HashNode *prev; + bool ret = true; + + if (bkt == NULL) + return false; + + prev = NULL; + new_node = NULL; + new_node = HashNode_create(addr); + + spin_lock(&bkt->bkt_lock); + node = bkt->head; + while (node != NULL && node->addr != addr) { + prev = node; + node = node->next; + } + if (node == NULL) { + if (new_node == NULL) { + pr_info("not enough memory for HashNode\n"); + spin_unlock(&bkt->bkt_lock); + return false; + } + if (bkt->head == NULL) + bkt->head = new_node; + else + prev->next = new_node; + spin_unlock(&bkt->bkt_lock); + ret = true; + } else { + spin_unlock(&bkt->bkt_lock); + kfree(new_node); + ret = false; + } + + return ret; +} + +bool HashBucket_erase(struct HashBucket *bkt, unsigned long addr) +{ + struct HashNode *node; + struct HashNode *prev; + bool ret = true; + + if (bkt == NULL) + return false; + + spin_lock(&bkt->bkt_lock); + node = bkt->head; + prev = NULL; + while (node != NULL && node->addr != addr) { + prev = node; + node = node->next; + } + if (node == NULL) { + spin_unlock(&bkt->bkt_lock); + ret = false; + } else { + if (bkt->head == node) + bkt->head = node->next; + else + prev->next = node->next; + kfree(node); + spin_unlock(&bkt->bkt_lock); + ret = true; + } + + return ret; +} + +void HashBucket_clear(struct HashBucket *bkt) +{ + struct HashNode *node; + struct HashNode *prev; + + if (bkt == NULL) + return; + + spin_lock(&bkt->bkt_lock); + node = bkt->head; + prev = NULL; + bkt->head = NULL; + while (node != NULL) { + prev = node; + node = node->next; + kfree(prev); + } + spin_unlock(&bkt->bkt_lock); +} + +struct HashMap { + unsigned long size; + struct HashBucket *HashTable; +}; + +unsigned long hash_func(unsigned long addr, unsigned long size) +{ + return addr % size; +} + +struct HashMap *HashMap_create(unsigned long size) +{ + struct HashMap *hm = NULL; + struct HashBucket *ht = NULL; + int i = 0; + + hm = kmalloc(sizeof(struct HashMap), GFP_ATOMIC); + if (hm == NULL) + return NULL; + ht = kmalloc((size * sizeof(struct HashBucket)), GFP_ATOMIC); + if (ht == NULL) { + kfree(hm); + return NULL; + } + for (i = 0; i < size; i++) + HashBucket_init(&(ht[i])); + + hm->size = size; + hm->HashTable = ht; + return hm; +} + +bool HashMap_insert(struct HashMap *hm, unsigned long addr) +{ + unsigned long index; + + if (hm == NULL) + return false; + index = hash_func(addr, hm->size); + if (hm->HashTable == NULL) + return false; + return HashBucket_insert(&(hm->HashTable[index]), addr); +} + +bool HashMap_erase(struct HashMap *hm, unsigned long addr) +{ + unsigned long index; + + if (hm == NULL) + return false; + index = hash_func(addr, hm->size); + if (hm->HashTable == NULL) + return false; + return HashBucket_erase(&(hm->HashTable[index]), addr); +} + +void HashMap_clear(struct HashMap *hm) +{ + unsigned long size; + struct HashBucket *ht; + int i; + + if (hm == NULL) + return; + size = hm->size; + ht = hm->HashTable; + if (ht == NULL) + return; + hm->HashTable = NULL; + for (i = 0; i < size; i++) + HashBucket_clear(&(ht[i])); + + kfree(ht); + kfree(hm); +} + +//struct that save the information for each task +struct task_info { + int tgid; + char comm[TASK_COMM_LEN]; + int count; // number of pages + struct HashMap *pages; + struct list_head list; + spinlock_t cnt_lock; +}; + +// struct that save the information for each cgroup +struct cgroup_info { + struct cgroup *cgrp; + struct mem_cgroup *memcg; + int id; + char name[64]; + struct list_head list; + struct list_head tasks_list; + struct list_head oom_list; + rwlock_t cgrp_lock; + unsigned int cached_bytes; +}; + +static LIST_HEAD(all_cgroup_info); // a list that linked all the cgroup_info struct + +static struct task_info *create_task_info(struct task_struct *cur_task) +{ + struct task_info *tsk_info = + kmalloc(sizeof(struct task_info), GFP_ATOMIC); + if (!tsk_info) + return NULL; + + // initialization + tsk_info->tgid = cur_task->tgid; + strscpy(tsk_info->comm, cur_task->comm, sizeof(tsk_info->comm)); + tsk_info->count = 0; + tsk_info->pages = NULL; + tsk_info->pages = HashMap_create(1023); + INIT_LIST_HEAD(&tsk_info->list); + spin_lock_init(&(tsk_info->cnt_lock)); + + return tsk_info; +} + +static int +add_task_to_cgroup_info(struct cgroup_info *cgrp, struct task_info *task) +{ + if (!cgrp || !task) + return -EINVAL; + + write_lock(&cgrp->cgrp_lock); + list_add_tail(&task->list, &cgrp->tasks_list); + write_unlock(&cgrp->cgrp_lock); + return 0; +} + +static int +remove_task_from_cgroup_info(struct cgroup_info *cgrp, struct task_info *task) +{ + if (cgrp == NULL || task == NULL) + return -EINVAL; + + HashMap_clear(task->pages); + // kfree(task->pages); + kfree(task); + return 0; +} + +static struct task_info *find_task_info(struct cgroup_info *cgrp, int tgid) +{ + struct task_info *tsk_info, *pos; + + list_for_each_entry_safe(tsk_info, pos, &cgrp->tasks_list, list) { + if (tsk_info->tgid == tgid) + return tsk_info; + } + return NULL; +} + +static int +remove_page_from_cgroup_info(unsigned long addr, struct cgroup_info *cgrp) +{ + struct task_info *tsk_info, *pos; + + read_lock(&cgrp->cgrp_lock); + list_for_each_entry_safe(tsk_info, pos, &cgrp->tasks_list, list) { + if (HashMap_erase(tsk_info->pages, addr)) { + spin_lock(&(tsk_info->cnt_lock)); + tsk_info->count -= folio_nr_pages((struct folio *)addr); + spin_unlock(&(tsk_info->cnt_lock)); + read_unlock(&cgrp->cgrp_lock); + return 0; + } + } + read_unlock(&cgrp->cgrp_lock); + return -1; +} + +static struct cgroup_info *create_cgroup_info(struct cgroup *cgrp, + struct mem_cgroup *memcg) +{ + struct cgroup_info *cgrp_info = + kmalloc(sizeof(struct cgroup_info), GFP_ATOMIC); + struct kernfs_node *kn; + + if (!cgrp_info) + return NULL; + + cgrp_info->cgrp = cgrp; + cgrp_info->memcg = memcg; + cgrp_info->id = (memcg->css).id; + kn = cgrp->kn; + strscpy(cgrp_info->name, kn->name, sizeof(cgrp_info->name)); + INIT_LIST_HEAD(&cgrp_info->list); + INIT_LIST_HEAD(&cgrp_info->tasks_list); + INIT_LIST_HEAD(&cgrp_info->oom_list); + rwlock_init(&(cgrp_info->cgrp_lock)); + cgrp_info->cached_bytes = 0; + + return cgrp_info; +} + +static void destroy_cgroup_info(struct cgroup_info *cgrp_info) +{ + struct task_info *task, *tmp; + + if (!cgrp_info) + return; + + write_lock(&cgrp_info->cgrp_lock); + list_for_each_entry_safe(task, tmp, &cgrp_info->tasks_list, list) { + list_del(&task->list); + remove_task_from_cgroup_info(cgrp_info, task); + } + write_unlock(&cgrp_info->cgrp_lock); + list_for_each_entry_safe(task, tmp, &cgrp_info->oom_list, list) { + list_del(&task->list); + remove_task_from_cgroup_info(cgrp_info, task); + } + + kfree(cgrp_info); +} + +static int add_cgroup_info(struct cgroup_info *cgrp_info) +{ + if (!cgrp_info) + return -EINVAL; + + list_add_tail(&cgrp_info->list, &all_cgroup_info); + return 0; +} + +static struct cgroup_info *find_cgroup_info(int id) +{ + struct cgroup_info *cgrp_info = NULL; + + list_for_each_entry(cgrp_info, &all_cgroup_info, list) { + if (cgrp_info->id == id) + return cgrp_info; + } + + return NULL; +} + +static struct task_info *create_oom_task_info(struct task_info *tsk_info) +{ + struct task_info *oom_tsk_info = + kmalloc(sizeof(struct task_info), GFP_ATOMIC); + if (!oom_tsk_info) + return NULL; + + oom_tsk_info->tgid = tsk_info->tgid; + strscpy(oom_tsk_info->comm, tsk_info->comm, sizeof(oom_tsk_info->comm)); + oom_tsk_info->count = tsk_info->count; + oom_tsk_info->pages = NULL; + INIT_LIST_HEAD(&oom_tsk_info->list); + + return oom_tsk_info; +} + +static int +add_oom_task_to_cgroup_info(struct cgroup_info *cgrp, + struct task_info *oom_task) +{ + if (!cgrp || !oom_task) + return -EINVAL; + list_add_tail(&oom_task->list, &cgrp->oom_list); + return 0; +} diff --git a/tools/probeCgroup/run.sh b/tools/probeCgroup/run.sh new file mode 100755 index 000000000000..7e1ffefe66d1 --- /dev/null +++ b/tools/probeCgroup/run.sh @@ -0,0 +1,8 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +cd scripts +./script1.sh +./script2.sh +./script3.sh diff --git a/tools/probeCgroup/scripts/script1.sh b/tools/probeCgroup/scripts/script1.sh new file mode 100755 index 000000000000..539e8258afb9 --- /dev/null +++ b/tools/probeCgroup/scripts/script1.sh @@ -0,0 +1,10 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +cd .. +make +insmod probeCgroup.ko + +cd testcases +gcc simple-mem-allocate.c -o simple-mem-allocate diff --git a/tools/probeCgroup/scripts/script2.sh b/tools/probeCgroup/scripts/script2.sh new file mode 100755 index 000000000000..2ad515cfb912 --- /dev/null +++ b/tools/probeCgroup/scripts/script2.sh @@ -0,0 +1,14 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +current_dir=$(pwd) +cd /sys/fs/cgroup/memory +mkdir test +cd test +sh -c "echo $$ >> cgroup.procs" +sh -c "echo 5M > memory.limit_in_bytes" +sh -c "echo 0 > memory.swappiness" +cd "$current_dir" +cd ../testcases +./simple-mem-allocate diff --git a/tools/probeCgroup/scripts/script3.sh b/tools/probeCgroup/scripts/script3.sh new file mode 100755 index 000000000000..127eb45de5c9 --- /dev/null +++ b/tools/probeCgroup/scripts/script3.sh @@ -0,0 +1,11 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +cd /proc +cat cgroup_memory_usage_per_process + +cat /sys/fs/cgroup/memory/test/cgroup.procs > /sys/fs/cgroup/memory/cgroup.procs +rmdir /sys/fs/cgroup/memory/test +# cat cgroup_memory_usage_per_process +rmmod probeCgroup diff --git a/tools/probeCgroup/testcases/1_load_unload_test.py b/tools/probeCgroup/testcases/1_load_unload_test.py new file mode 100755 index 000000000000..5389a14a1dac --- /dev/null +++ b/tools/probeCgroup/testcases/1_load_unload_test.py @@ -0,0 +1,24 @@ +#!/usr/bin/env python +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +import os +import subprocess +import time + +def test_module_load_unload(): + try: + subprocess.check_call(['insmod', '../probeCgroup.ko']) + time.sleep(1) + print('loading module successfully!') + subprocess.check_call(['rmmod', 'probeCgroup']) + output = subprocess.check_output(['lsmod']) + assert b'probeCgroup' not in output + print('unloading module successfully!') + except subprocess.CalledProcessError as e: + print('Load unload test failed. Insmod failed.') + except AssertionError as e: + print('Load unload test failed. Cannot remove module.') + +if __name__ == '__main__': + test_module_load_unload() \ No newline at end of file diff --git a/tools/probeCgroup/testcases/2_multiple_process_test.py b/tools/probeCgroup/testcases/2_multiple_process_test.py new file mode 100755 index 000000000000..d88c7f7f2952 --- /dev/null +++ b/tools/probeCgroup/testcases/2_multiple_process_test.py @@ -0,0 +1,48 @@ +#!/usr/bin/env python +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +from cgroup_utils import create_cgroup, add_process_to_cgroup, get_process_memory_usage, remove_cgroup, check_memory_usage, cleanup, check_kmem_usage +import os +import subprocess +import time + +def test_multiple_process(num_procs): + subprocess.check_call(['insmod', '../probeCgroup.ko']) + time.sleep(1) + + cgroup_name = 'test' + cgroup_path = create_cgroup(cgroup_name) + + processes = [] + pids = [] + + for i in range(num_procs): + process = subprocess.Popen(['./mem-allocate']) + pid = process.pid + add_process_to_cgroup(cgroup_path, pid) + processes.append(process) + pids.append(pid) + + time.sleep(0.1) + try: + count = 0 + for i in range (2000): + count += check_memory_usage(cgroup_name, pids, False) + time.sleep(0.01) + assert count <= 50, f"Memory read by probeCgroup is not accurate" + + remove_cgroup(cgroup_path, pids) + check_memory_usage(cgroup_name, pids, True) + cleanup(processes) + subprocess.check_call(['rmmod', 'probeCgroup']) + + print('pass multiple process test!') + except AssertionError as e: + print(f"Assertion failed: {e}") + remove_cgroup(cgroup_path, pids) + cleanup(processes) + subprocess.check_call(['rmmod', 'probeCgroup']) + +if __name__ == '__main__': + test_multiple_process(3) \ No newline at end of file diff --git a/tools/probeCgroup/testcases/3_multiple_cgroup_test.py b/tools/probeCgroup/testcases/3_multiple_cgroup_test.py new file mode 100755 index 000000000000..592a716df877 --- /dev/null +++ b/tools/probeCgroup/testcases/3_multiple_cgroup_test.py @@ -0,0 +1,55 @@ +#!/usr/bin/env python +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +from cgroup_utils import create_cgroup, add_process_to_cgroup, get_process_memory_usage, remove_cgroup, check_memory_usage, cleanup +import os +import subprocess +import time + +def test_multiple_cgroup(num_procs, num_cgroups): + subprocess.check_call(['insmod', '../probeCgroup.ko']) + time.sleep(1) + + cgroups = [] + processes = {} + pids = {} + for i in range(num_cgroups): + cgroup_name = f'test_{i}' + cgroup_path = create_cgroup(cgroup_name) + cgroups.append((cgroup_name, cgroup_path)) + + for j in range(num_procs): + process = subprocess.Popen(['./mem-allocate']) + pid = process.pid + add_process_to_cgroup(cgroup_path, pid) + if cgroup_path not in processes: + processes[cgroup_path] = [] + processes[cgroup_path].append(process) + if cgroup_path not in pids: + pids[cgroup_path] = [] + pids[cgroup_path].append(pid) + + time.sleep(0.1) + try: + for i in range (100): + for cgroup_name, cgroup_path in cgroups: + check_memory_usage(cgroup_name, pids[cgroup_path], False) + time.sleep(0.01) + + for cgroup_name, cgroup_path in cgroups: + remove_cgroup(cgroup_path, pids[cgroup_path]) + check_memory_usage(cgroup_name, pids[cgroup_path], True) + cleanup(processes[cgroup_path]) + subprocess.check_call(['rmmod', 'probeCgroup']) + + print('pass multiple cgroup test!') + except AssertionError as e: + print(f"Assertion failed: {e}") + for cgroup_name, cgroup_path in cgroups: + remove_cgroup(cgroup_path, pids[cgroup_path]) + cleanup(processes[cgroup_path]) + subprocess.check_call(['rmmod', 'probeCgroup']) + +if __name__ == '__main__': + test_multiple_cgroup(2,2) \ No newline at end of file diff --git a/tools/probeCgroup/testcases/4_oom_test.py b/tools/probeCgroup/testcases/4_oom_test.py new file mode 100755 index 000000000000..128a258c56f5 --- /dev/null +++ b/tools/probeCgroup/testcases/4_oom_test.py @@ -0,0 +1,52 @@ +#!/usr/bin/env python +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +from cgroup_utils import create_cgroup, add_process_to_cgroup, get_process_memory_usage, remove_cgroup, check_memory_usage, cleanup, get_oom_process_memory_usage +import os +import subprocess +import time + +def test_oom(num_procs): + subprocess.check_call(['insmod', '../probeCgroup.ko']) + time.sleep(1) + + cgroup_name = 'test' + cgroup_path = create_cgroup(cgroup_name) + + with open(f"/sys/fs/cgroup/memory/{cgroup_name}/memory.limit_in_bytes", 'w') as limit_file: + limit_file.write("5M") + with open(f"/sys/fs/cgroup/memory/{cgroup_name}/memory.swappiness", 'w') as swap_file: + swap_file.write("0") + + processes = [] + pids = [] + for i in range(num_procs): + process = subprocess.Popen(['./simple-mem-allocate']) + pid = process.pid + add_process_to_cgroup(cgroup_path, pid) + processes.append(process) + pids.append(pid) + + time.sleep(6) + + try: + for pid in pids: + memory_usage = get_oom_process_memory_usage(pid, cgroup_name) + assert memory_usage is not None, f"Memory usage(oom) not found for PID {pid}" + assert memory_usage > 0, f"Memory usage should be greater than zero for PID {pid}" + + remove_cgroup(cgroup_path, pids) + check_memory_usage(cgroup_name, pids, True) + cleanup(processes) + subprocess.check_call(['rmmod', 'probeCgroup']) + + print('pass oom test!') + except AssertionError as e: + print(f"Assertion failed: {e}") + remove_cgroup(cgroup_path, pids) + cleanup(processes) + subprocess.check_call(['rmmod', 'probeCgroup']) + +if __name__ == '__main__': + test_oom(1) \ No newline at end of file diff --git a/tools/probeCgroup/testcases/5_multiple_threads_test.py b/tools/probeCgroup/testcases/5_multiple_threads_test.py new file mode 100755 index 000000000000..7e1b86dabe48 --- /dev/null +++ b/tools/probeCgroup/testcases/5_multiple_threads_test.py @@ -0,0 +1,45 @@ +#!/usr/bin/env python +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +from cgroup_utils import create_cgroup, add_process_to_cgroup, get_process_memory_usage, remove_cgroup, check_memory_usage, cleanup, check_kmem_usage +import os +import subprocess +import time + +def test_multiple_thread(num_procs): + subprocess.check_call(['insmod', '../probeCgroup.ko']) + time.sleep(1) + + cgroup_name = 'test' + cgroup_path = create_cgroup(cgroup_name) + + processes = [] + pids = [] + for i in range(num_procs): + process = subprocess.Popen(['./multiple-thread-mem-allocate']) + pid = process.pid + add_process_to_cgroup(cgroup_path, pid) + processes.append(process) + pids.append(pid) + + time.sleep(1) + try: + for i in range (200): + check_memory_usage(cgroup_name, pids, False) + time.sleep(0.01) + + remove_cgroup(cgroup_path, pids) + check_memory_usage(cgroup_name, pids, True) + cleanup(processes) + subprocess.check_call(['rmmod', 'probeCgroup']) + + print('pass multiple threads test!') + except AssertionError as e: + print(f"Assertion failed: {e}") + remove_cgroup(cgroup_path, pids) + cleanup(processes) + subprocess.check_call(['rmmod', 'probeCgroup']) + +if __name__ == '__main__': + test_multiple_thread(5) \ No newline at end of file diff --git a/tools/probeCgroup/testcases/cgroup_utils.py b/tools/probeCgroup/testcases/cgroup_utils.py new file mode 100644 index 000000000000..e00c580e724e --- /dev/null +++ b/tools/probeCgroup/testcases/cgroup_utils.py @@ -0,0 +1,114 @@ +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +import os +import subprocess +import time + +def create_cgroup(cgroup_name): + cgroup_path = f'/sys/fs/cgroup/memory/{cgroup_name}' + + try: + os.makedirs(cgroup_path) + except FileExistsError: + pass + + return cgroup_path + +def add_process_to_cgroup(cgroup_path, pid): + with open(os.path.join(cgroup_path, 'cgroup.procs'), 'w') as procs_file: + procs_file.write(str(pid)) + +def get_process_memory_usage(pid, cgroup_name): + cur_name = '' + with open('/proc/cgroup_memory_usage_per_process', 'r') as file: + for line in file: + parts = line.strip().split() + if len(parts) >= 4 and parts[0] == 'cgroup': + cur_name = parts[3] + if len(parts) >= 3 and parts[0] != 'cgroup' and cur_name == cgroup_name and parts[0] != 'pid' and int(parts[0]) == pid: + return int(parts[2]) + return None + +def get_process_kmem_usage(pid, cgroup_name): + cur_name = '' + with open('/proc/cgroup_memory_usage_per_process', 'r') as file: + for line in file: + parts = line.strip().split() + if len(parts) >= 4 and parts[0] == 'cgroup': + cur_name = parts[3] + if len(parts) >= 4 and parts[0] != 'cgroup' and cur_name == cgroup_name and parts[0] != 'pid' and int(parts[0]) == pid: + return int(parts[3]) + return None + +def remove_cgroup(cgroup_path, pids): + for pid in pids: + with open('/sys/fs/cgroup/memory/cgroup.procs', 'w') as backup_file: + backup_file.write(str(pid)) + os.rmdir(cgroup_path) + return + +def check_memory_usage(cgroup_name, pids, delete): + memory_sum = 0 + for pid in pids: + memory_usage = get_process_memory_usage(pid, cgroup_name) + if delete == False: + assert memory_usage is not None, f"Memory usage not found for PID {pid}" + assert memory_usage >= 0, f"Memory usage should be greater than zero for PID {pid}" + memory_sum += memory_usage + else: + assert memory_usage is None, f"Error: Memory usage should not be available for PID {pid} after deleting the cgroup." + if delete == False: + with open(f"/sys/fs/cgroup/memory/{cgroup_name}/memory.usage_in_bytes", 'r') as file: + content = file.readline().strip() + memory_read = int(content) + memory_sum *= 1024 + delta = abs(memory_read - memory_sum) + # print(f"read: {memory_read}") + # print(f"sum : {memory_sum}") + if (delta > max(memory_read, memory_sum) * 0.1): + return 1 + else: + return 0 + else: + return 0 + +def check_kmem_usage(cgroup_name, pids, delete): + kmem_sum = 0 + for pid in pids: + kmem_usage = get_process_kmem_usage(pid, cgroup_name) + if delete == False: + assert kmem_usage is not None, f"Kmem usage not found for PID {pid}" + assert kmem_usage >= 0, f"Kmem usage should be greater than zero for PID {pid}" + kmem_sum += kmem_usage + else: + assert kmem_usage is None, f"Error: Kmem usage should not be available for PID {pid} after deleting the cgroup." + if delete == False: + with open(f"/sys/fs/cgroup/memory/{cgroup_name}/memory.kmem.usage_in_bytes", 'r') as file: + content = file.readline().strip() + kmem_read = int(content) + kmem_sum *= 1024 + delta = abs(kmem_read - kmem_sum) + # print(f"kmem read: {kmem_read}") + # print(f"kmem sum : {kmem_sum}") + # assert delta <= max(kmem_read, kmem_sum) * 0.2, f"Kmem read by probeCgroup is not accurate, {kmem_read}, {kmem_sum}" + +def cleanup(processes): + for process in processes: + process.terminate() + process.wait() + +def get_oom_process_memory_usage(pid, cgroup_name): + cur_name = '' + oom = False + with open('/proc/cgroup_memory_usage_per_process', 'r') as file: + for line in file: + parts = line.strip().split() + if len(parts) >= 4 and parts[0] == 'cgroup': + cur_name = parts[3] + oom = False + if len(parts) >= 1 and parts[0] == 'oom:': + oom = True + if len(parts) >= 3 and parts[0] != 'cgroup' and cur_name == cgroup_name and parts[0] != 'pid' and int(parts[0]) == pid and oom == True: + return int(parts[2]) + return None \ No newline at end of file diff --git a/tools/probeCgroup/testcases/mem-allocate.c b/tools/probeCgroup/testcases/mem-allocate.c new file mode 100644 index 000000000000..e78e37cae61b --- /dev/null +++ b/tools/probeCgroup/testcases/mem-allocate.c @@ -0,0 +1,35 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * mem-allocate.c - The program to test probeCgroup + * + * Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + */ + +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <unistd.h> + +#define MB (1024 * 1024) + +char *arr[40]; + +int main(int argc, char *argv[]) +{ + char *p; + int i = 0; + + while (1) { + for (i = 0; i < 40; i++) { + p = (char *)malloc(MB); + memset(p, 0, MB); + arr[i] = p; + usleep(100000); + } + for (int i = 0; i < 40; i++) { + free(arr[i]); + usleep(100000); + } + } + return 0; +} diff --git a/tools/probeCgroup/testcases/multiple-thread-mem-allocate.c b/tools/probeCgroup/testcases/multiple-thread-mem-allocate.c new file mode 100644 index 000000000000..55f4c068f55e --- /dev/null +++ b/tools/probeCgroup/testcases/multiple-thread-mem-allocate.c @@ -0,0 +1,60 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * multiple-thread-mem-allocate.c - The program to test probeCgroup + * + * Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + */ + +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <unistd.h> +#include <pthread.h> + +#define MB (1024 * 1024) + +void *memory_test(void *) +{ + char *arr[25]; + char *p; + int i = 0; + int cnt = 0; + + while (1) { + for (i = 0; i < 20; i++) { + p = (char *)malloc(MB); + memset(p, 0, MB); + arr[i] = p; + usleep(10000); + } + for (int i = 0; i < 20; i++) { + free(arr[i]); + usleep(10000); + } + } +} + +int main(int argc, char *argv[]) +{ + pthread_t threads[4]; + int rc; + + // create threads + for (int i = 0; i < 4; i++) { + rc = pthread_create(&threads[i], NULL, memory_test, NULL); + if (rc != 0) { + fprintf(stderr, "Error creating thread: %d\n", rc); + return 1; + } + } + + for (int i = 0; i < 4; i++) { + rc = pthread_join(threads[i], NULL); + if (rc != 0) { + fprintf(stderr, "Error joining thread: %d\n", rc); + return 1; + } + } + + return 0; +} diff --git a/tools/probeCgroup/testcases/run.py b/tools/probeCgroup/testcases/run.py new file mode 100755 index 000000000000..8ffd0ca720d8 --- /dev/null +++ b/tools/probeCgroup/testcases/run.py @@ -0,0 +1,32 @@ +#!/usr/bin/env python +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +import os +import subprocess +import sys + +def run_tests(directory): + """Run all Python scripts in the given directory.""" + python_files = [f for f in os.listdir(directory) if f.endswith('test.py')] + python_files.sort() + + for filename in python_files: + try: + filepath = os.path.join(directory, filename) + + subprocess.check_call([sys.executable, filepath]) + except subprocess.CalledProcessError as e: + print(f"Error executing {filename}:") + return + except Exception as e: + print(f"Error executing {filename}:") + print(e) + return + +if __name__ == '__main__': + tests_directory = '.' + subprocess.check_call(['gcc', 'mem-allocate.c', '-o', 'mem-allocate']) + subprocess.check_call(['gcc', 'simple-mem-allocate.c', '-o', 'simple-mem-allocate']) + subprocess.check_call(['gcc', 'multiple-thread-mem-allocate.c', '-o', 'multiple-thread-mem-allocate']) + run_tests(tests_directory) \ No newline at end of file diff --git a/tools/probeCgroup/testcases/simple-mem-allocate.c b/tools/probeCgroup/testcases/simple-mem-allocate.c new file mode 100644 index 000000000000..16328b10ba48 --- /dev/null +++ b/tools/probeCgroup/testcases/simple-mem-allocate.c @@ -0,0 +1,27 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * mem-allocate.c - The program to test probeCgroup + * + * Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + */ + +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <unistd.h> + +#define MB (1024 * 1024) + +int main(int argc, char *argv[]) +{ + char *p; + int i = 0; + + while (1) { + p = (char *)malloc(MB); + memset(p, 0, MB); + sleep(1); + } + + return 0; +} -- 2.43.0

2 1

openEuler Kernel SIG双周例会
by openEuler conference 05 Sep '24

05 Sep '24

您好！ Kernel 邀请您参加 2024-09-06 14:00 召开的WeLink会议(自动录制) 会议主题：openEuler Kernel SIG双周例会会议内容： 1. 进展update 2. 议题征集中（新增议题可回复本邮件申请，也可直接填写至会议纪要看板）会议链接：https://meeting.huaweicloud.com:36443/#/j/989033877 会议纪要：https://etherpad.openeuler.org/p/Kernel-meetings 更多资讯尽在：https://www.openeuler.org/zh/ Hello! Kernel invites you to attend the WeLink conference(auto recording) will be held at 2024-09-06 14:00, The subject of the conference is openEuler Kernel SIG双周例会, Summary: 1. 进展update 2. 议题征集中（新增议题可回复本邮件申请，也可直接填写至会议纪要看板） You can join the meeting at https://meeting.huaweicloud.com:36443/#/j/989033877. Add topics at https://etherpad.openeuler.org/p/Kernel-meetings. More information: https://www.openeuler.org/en/

1 0

[PATCH OLK-5.10] Revert "mm/writeback: fix possible divide-by-zero in wb_dirty_limits(), again"
by Kaixiong Yu 05 Sep '24

05 Sep '24

From: Jan Kara <jack(a)suse.cz> stable inclusion from stable-v5.10.222 commit 145faa3d03688cbb7bbaaecbd84c01539852942c category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/IAD6H2 CVE: CVE-2024-42102 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id… -------------------------------- commit 30139c702048f1097342a31302cbd3d478f50c63 upstream. Patch series "mm: Avoid possible overflows in dirty throttling". Dirty throttling logic assumes dirty limits in page units fit into 32-bits. This patch series makes sure this is true (see patch 2/2 for more details). This patch (of 2): This reverts commit 9319b647902cbd5cc884ac08a8a6d54ce111fc78. The commit is broken in several ways. Firstly, the removed (u64) cast from the multiplication will introduce a multiplication overflow on 32-bit archs if wb_thresh * bg_thresh >= 1<<32 (which is actually common - the default settings with 4GB of RAM will trigger this). Secondly, the div64_u64() is unnecessarily expensive on 32-bit archs. We have div64_ul() in case we want to be safe & cheap. Thirdly, if dirty thresholds are larger than 1<<32 pages, then dirty balancing is going to blow up in many other spectacular ways anyway so trying to fix one possible overflow is just moot. Link: https://lkml.kernel.org/r/20240621144017.30993-1-jack@suse.cz Link: https://lkml.kernel.org/r/20240621144246.11148-1-jack@suse.cz Fixes: 9319b647902c ("mm/writeback: fix possible divide-by-zero in wb_dirty_limits(), again") Signed-off-by: Jan Kara <jack(a)suse.cz> Reviewed-By: Zach O'Keefe <zokeefe(a)google.com> Cc: <stable(a)vger.kernel.org> Signed-off-by: Andrew Morton <akpm(a)linux-foundation.org> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> Signed-off-by: Kaixiong Yu <yukaixiong(a)huawei.com> --- mm/page-writeback.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/page-writeback.c b/mm/page-writeback.c index 0d7cc65c6367..31bc5904bbf8 100644 --- a/mm/page-writeback.c +++ b/mm/page-writeback.c @@ -1541,7 +1541,7 @@ static inline void wb_dirty_limits(struct dirty_throttle_control *dtc) */ dtc->wb_thresh = __wb_calc_thresh(dtc); dtc->wb_bg_thresh = dtc->thresh ? - div64_u64(dtc->wb_thresh * dtc->bg_thresh, dtc->thresh) : 0; + div_u64((u64)dtc->wb_thresh * dtc->bg_thresh, dtc->thresh) : 0; /* * In order to avoid the stacked BDI deadlock we need -- 2.25.1

2 1

[openeuler:OLK-6.6 2700/13861] drivers/platform/mpam/mpam_devices.c:247:32: error: implicit declaration of function '__acpi_get_mem_attribute'
by kernel test robot 05 Sep '24

05 Sep '24

Hi James, FYI, the error/warning still remains. tree: https://gitee.com/openeuler/kernel.git OLK-6.6 head: 392f26f46bac569be21dbe2b3ca67641e02cb148 commit: 3e9e723f3bf92a19e5e15dda89bbb136ce463294 [2700/13861] arm_mpam: Add probe/remove for mpam msc driver and kbuild boiler plate config: arm64-randconfig-004-20240905 (https://download.01.org/0day-ci/archive/20240905/202409051300.VBHKGPkR-lkp@…) compiler: aarch64-linux-gcc (GCC) 14.1.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240905/202409051300.VBHKGPkR-lkp@…) If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot <lkp(a)intel.com> | Closes: https://lore.kernel.org/oe-kbuild-all/202409051300.VBHKGPkR-lkp@intel.com/ All errors (new ones prefixed by >>): In file included from include/linux/mmzone.h:8, from include/linux/gfp.h:7, from include/linux/slab.h:16, from include/linux/resource_ext.h:11, from include/linux/acpi.h:13, from drivers/platform/mpam/mpam_devices.c:6: drivers/platform/mpam/mpam_devices.c: In function 'mpam_msc_drv_probe': drivers/platform/mpam/mpam_devices.c:212:38: error: 'struct mpam_msc' has no member named 'mon_sel_lock'; did you mean 'part_sel_lock'? 212 | spin_lock_init(&msc->mon_sel_lock); | ^~~~~~~~~~~~ include/linux/spinlock.h:335:45: note: in definition of macro 'spin_lock_init' 335 | __raw_spin_lock_init(spinlock_check(lock), \ | ^~~~ >> drivers/platform/mpam/mpam_devices.c:247:32: error: implicit declaration of function '__acpi_get_mem_attribute' [-Wimplicit-function-declaration] 247 | prot = __acpi_get_mem_attribute(msc->pcc_chan->shmem_base_addr); | ^~~~~~~~~~~~~~~~~~~~~~~~ >> drivers/platform/mpam/mpam_devices.c:247:32: error: incompatible types when assigning to type 'pgprot_t' from type 'int' vim +/__acpi_get_mem_attribute +247 drivers/platform/mpam/mpam_devices.c 170 171 static int mpam_msc_drv_probe(struct platform_device *pdev) 172 { 173 int err; 174 pgprot_t prot; 175 void * __iomem io; 176 struct mpam_msc *msc; 177 struct resource *msc_res; 178 void *plat_data = pdev->dev.platform_data; 179 180 mutex_lock(&mpam_list_lock); 181 do { 182 msc = devm_kzalloc(&pdev->dev, sizeof(*msc), GFP_KERNEL); 183 if (!msc) { 184 err = -ENOMEM; 185 break; 186 } 187 188 INIT_LIST_HEAD_RCU(&msc->glbl_list); 189 msc->pdev = pdev; 190 191 err = device_property_read_u32(&pdev->dev, "arm,not-ready-us", 192 &msc->nrdy_usec); 193 if (err) { 194 /* This will prevent CSU monitors being usable */ 195 msc->nrdy_usec = 0; 196 } 197 198 err = get_msc_affinity(msc); 199 if (err) 200 break; 201 if (cpumask_empty(&msc->accessibility)) { 202 pr_err_once("msc:%u is not accessible from any CPU!", 203 msc->id); 204 err = -EINVAL; 205 break; 206 } 207 208 mutex_init(&msc->lock); 209 msc->id = mpam_num_msc++; 210 INIT_LIST_HEAD_RCU(&msc->ris); 211 spin_lock_init(&msc->part_sel_lock); > 212 spin_lock_init(&msc->mon_sel_lock); 213 214 if (device_property_read_u32(&pdev->dev, "pcc-channel", 215 &msc->pcc_subspace_id)) 216 msc->iface = MPAM_IFACE_MMIO; 217 else 218 msc->iface = MPAM_IFACE_PCC; 219 220 if (msc->iface == MPAM_IFACE_MMIO) { 221 io = devm_platform_get_and_ioremap_resource(pdev, 0, 222 &msc_res); 223 if (IS_ERR(io)) { 224 pr_err("Failed to map MSC base address\n"); 225 devm_kfree(&pdev->dev, msc); 226 err = PTR_ERR(io); 227 break; 228 } 229 msc->mapped_hwpage_sz = msc_res->end - msc_res->start; 230 msc->mapped_hwpage = io; 231 } else if (msc->iface == MPAM_IFACE_PCC) { 232 msc->pcc_cl.dev = &pdev->dev; 233 msc->pcc_cl.rx_callback = mpam_pcc_rx_callback; 234 msc->pcc_cl.tx_block = false; 235 msc->pcc_cl.tx_tout = 1000; /* 1s */ 236 msc->pcc_cl.knows_txdone = false; 237 238 msc->pcc_chan = pcc_mbox_request_channel(&msc->pcc_cl, 239 msc->pcc_subspace_id); 240 if (IS_ERR(msc->pcc_chan)) { 241 pr_err("Failed to request MSC PCC channel\n"); 242 devm_kfree(&pdev->dev, msc); 243 err = PTR_ERR(msc->pcc_chan); 244 break; 245 } 246 > 247 prot = __acpi_get_mem_attribute(msc->pcc_chan->shmem_base_addr); 248 io = ioremap_prot(msc->pcc_chan->shmem_base_addr, 249 msc->pcc_chan->shmem_size, pgprot_val(prot)); 250 if (IS_ERR(io)) { 251 pr_err("Failed to map MSC base address\n"); 252 pcc_mbox_free_channel(msc->pcc_chan); 253 devm_kfree(&pdev->dev, msc); 254 err = PTR_ERR(io); 255 break; 256 } 257 258 /* TODO: issue a read to update the registers */ 259 260 msc->mapped_hwpage_sz = msc->pcc_chan->shmem_size; 261 msc->mapped_hwpage = io + sizeof(struct acpi_pcct_shared_memory); 262 } 263 264 list_add_rcu(&msc->glbl_list, &mpam_all_msc); 265 platform_set_drvdata(pdev, msc); 266 } while (0); 267 mutex_unlock(&mpam_list_lock); 268 269 if (!err) { 270 /* Create RIS entries described by firmware */ 271 if (!acpi_disabled) 272 err = acpi_mpam_parse_resources(msc, plat_data); 273 else 274 err = mpam_dt_parse_resources(msc, plat_data); 275 } 276 277 if (!err && fw_num_msc == mpam_num_msc) 278 mpam_discovery_complete(); 279 280 return err; 281 } 282 -- 0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki

1 0

[PATCH openEuler-22.03-LTS-SP1 0/2] nfc: pn533: Wait for out_urb's completion in pn533_usb_send_frame()
by Kaixiong Yu 05 Sep '24

05 Sep '24

fix CVE-2023-52907 Fedor Pchelkin (1): nfc: pn533: initialize struct pn533_out_arg properly Minsuk Kang (1): nfc: pn533: Wait for out_urb's completion in pn533_usb_send_frame() drivers/nfc/pn533/usb.c | 45 ++++++++++++++++++++++++++++++++++++++--- 1 file changed, 42 insertions(+), 3 deletions(-) -- 2.25.1

2 3

[PATCH openEuler-22.03-LTS-SP1 0/1] zap_pid_ns_processes: clear TIF_NOTIFY_SIGNAL along with TIF_SIGPENDING
by Chen Ridong 05 Sep '24

05 Sep '24

zap_pid_ns_processes: clear TIF_NOTIFY_SIGNAL along with TIF_SIGPENDING Oleg Nesterov (1): zap_pid_ns_processes: clear TIF_NOTIFY_SIGNAL along with TIF_SIGPENDING kernel/pid_namespace.c | 1 + 1 file changed, 1 insertion(+) -- 2.34.1

2 2

[PATCH OLK-5.10 0/1] zap_pid_ns_processes: clear TIF_NOTIFY_SIGNAL along with TIF_SIGPENDING
by Chen Ridong 05 Sep '24

05 Sep '24

zap_pid_ns_processes: clear TIF_NOTIFY_SIGNAL along with TIF_SIGPENDING Oleg Nesterov (1): zap_pid_ns_processes: clear TIF_NOTIFY_SIGNAL along with TIF_SIGPENDING kernel/pid_namespace.c | 1 + 1 file changed, 1 insertion(+) -- 2.34.1

2 2

[PATCH openEuler-1.0-LTS 0/2] nfc: pn533: Wait for out_urb's completion in pn533_usb_send_frame()
by Kaixiong Yu 05 Sep '24

05 Sep '24

fix CVE-2023-52907 Fedor Pchelkin (1): nfc: pn533: initialize struct pn533_out_arg properly Minsuk Kang (1): nfc: pn533: Wait for out_urb's completion in pn533_usb_send_frame() drivers/nfc/pn533/usb.c | 45 ++++++++++++++++++++++++++++++++++++++--- 1 file changed, 42 insertions(+), 3 deletions(-) -- 2.25.1

2 3

[openeuler:openEuler-1.0-LTS 23675/23694] block/blk-io-hierarchy/iodump.c:571:7: warning: no previous prototype for function '__bio_stage_hierarchy_start'
by kernel test robot 05 Sep '24

05 Sep '24

tree: https://gitee.com/openeuler/kernel.git openEuler-1.0-LTS head: bd317843995067c2f78bd884340fcd0a27acab8d commit: ac91631e9ef7b530c02e40d10085072c08fb4a0a [23675/23694] blk-io-hierarchy: support new stage for bio lifetime config: x86_64-allyesconfig (https://download.01.org/0day-ci/archive/20240905/202409051155.tjBfzaXC-lkp@…) compiler: clang version 18.1.5 (https://github.com/llvm/llvm-project 617a15a9eac96088ae5e9134248d8236e34b91b1) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240905/202409051155.tjBfzaXC-lkp@…) If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot <lkp(a)intel.com> | Closes: https://lore.kernel.org/oe-kbuild-all/202409051155.tjBfzaXC-lkp@intel.com/ All warnings (new ones prefixed by >>): In file included from block/blk-io-hierarchy/iodump.c:15: In file included from include/linux/blktrace_api.h:5: In file included from include/linux/blkdev.h:16: include/linux/pagemap.h:425:21: warning: cast from 'int (*)(struct file *, struct page *)' to 'filler_t *' (aka 'int (*)(void *, struct page *)') converts to incompatible function type [-Wcast-function-type-strict] 425 | filler_t *filler = (filler_t *)mapping->a_ops->readpage; | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> block/blk-io-hierarchy/iodump.c:571:7: warning: no previous prototype for function '__bio_stage_hierarchy_start' [-Wmissing-prototypes] 571 | void *__bio_stage_hierarchy_start(struct bio_stage_dump_data *bstage_ddata, | ^ block/blk-io-hierarchy/iodump.c:571:1: note: declare 'static' if the function is not intended to be used outside of this translation unit 571 | void *__bio_stage_hierarchy_start(struct bio_stage_dump_data *bstage_ddata, | ^ | static 2 warnings generated. vim +/__bio_stage_hierarchy_start +571 block/blk-io-hierarchy/iodump.c 570 > 571 void *__bio_stage_hierarchy_start(struct bio_stage_dump_data *bstage_ddata, 572 loff_t *pos) 573 { 574 struct pos_data *pdata = &bstage_ddata->pdata; 575 struct rq_dump_data *rq_ddata = &bstage_ddata->rq_ddata; 576 577 retry: 578 if (stage_is_bio(pdata->stage)) { 579 struct list_head *list; 580 struct bio_dump_data *bio_ddata = 581 get_bio_stage_ddata(rq_ddata->q, pdata->stage); 582 583 if (!bio_ddata) { 584 bio_stage_start_next_stage(bstage_ddata, pos); 585 goto retry; 586 } 587 588 spin_lock_irq(&bio_ddata->lock); 589 list = seq_list_start(&bio_ddata->head, pdata->count); 590 if (list) 591 return list; 592 593 spin_unlock_irq(&bio_ddata->lock); 594 bio_stage_start_next_stage(bstage_ddata, pos); 595 goto retry; 596 } 597 598 if (pdata->stage == STAGE_BIO && 599 __rq_hierarchy_start(rq_ddata, pdata->count)) 600 return bstage_ddata; 601 602 return NULL; 603 } 604 -- 0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki

1 0

[PATCH openEuler-1.0-LTS 0/2] nfc: pn533: Wait for out_urb's completion in pn533_usb_send_frame()
by Kaixiong Yu 05 Sep '24

05 Sep '24

fix CVE-2023-52907 Fedor Pchelkin (1): nfc: pn533: initialize struct pn533_out_arg properly Minsuk Kang (1): nfc: pn533: Wait for out_urb's completion in pn533_usb_send_frame() drivers/nfc/pn533/usb.c | 45 ++++++++++++++++++++++++++++++++++++++--- 1 file changed, 42 insertions(+), 3 deletions(-) -- 2.25.1

2 3

[PATCH OLK-5.10] Revert "mm/writeback: fix possible divide-by-zero in wb_dirty_limits(), again"
by Kaixiong Yu 05 Sep '24

05 Sep '24

From: Jan Kara <jack(a)suse.cz> stable inclusion from stable-v5.10.222 commit 145faa3d03688cbb7bbaaecbd84c01539852942c category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/IAD6H2 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id… -------------------------------- commit 30139c702048f1097342a31302cbd3d478f50c63 upstream. Patch series "mm: Avoid possible overflows in dirty throttling". Dirty throttling logic assumes dirty limits in page units fit into 32-bits. This patch series makes sure this is true (see patch 2/2 for more details). This patch (of 2): This reverts commit 9319b647902cbd5cc884ac08a8a6d54ce111fc78. The commit is broken in several ways. Firstly, the removed (u64) cast from the multiplication will introduce a multiplication overflow on 32-bit archs if wb_thresh * bg_thresh >= 1<<32 (which is actually common - the default settings with 4GB of RAM will trigger this). Secondly, the div64_u64() is unnecessarily expensive on 32-bit archs. We have div64_ul() in case we want to be safe & cheap. Thirdly, if dirty thresholds are larger than 1<<32 pages, then dirty balancing is going to blow up in many other spectacular ways anyway so trying to fix one possible overflow is just moot. Link: https://lkml.kernel.org/r/20240621144017.30993-1-jack@suse.cz Link: https://lkml.kernel.org/r/20240621144246.11148-1-jack@suse.cz Fixes: 9319b647902c ("mm/writeback: fix possible divide-by-zero in wb_dirty_limits(), again") Signed-off-by: Jan Kara <jack(a)suse.cz> Reviewed-By: Zach O'Keefe <zokeefe(a)google.com> Cc: <stable(a)vger.kernel.org> Signed-off-by: Andrew Morton <akpm(a)linux-foundation.org> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> Signed-off-by: Kaixiong Yu <yukaixiong(a)huawei.com> --- mm/page-writeback.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/page-writeback.c b/mm/page-writeback.c index 0d7cc65c6367..31bc5904bbf8 100644 --- a/mm/page-writeback.c +++ b/mm/page-writeback.c @@ -1541,7 +1541,7 @@ static inline void wb_dirty_limits(struct dirty_throttle_control *dtc) */ dtc->wb_thresh = __wb_calc_thresh(dtc); dtc->wb_bg_thresh = dtc->thresh ? - div64_u64(dtc->wb_thresh * dtc->bg_thresh, dtc->thresh) : 0; + div_u64((u64)dtc->wb_thresh * dtc->bg_thresh, dtc->thresh) : 0; /* * In order to avoid the stacked BDI deadlock we need -- 2.25.1

2 1

[PATCH openEuler-23.09] Add dynamic kprobe-based process-level cgroup memory monitoring tool
by Taoxy2004 05 Sep '24

05 Sep '24

--- tools/probeCgroup: This patch introduces a new tool called "probeCgroup" that enables dynamic monitoring of memory usage at the process level within cgroups. By using kprobes at relevant cgroup functions, this tool can track memory allocations and deallocations for individual processes within a cgroup, providing detailed statistics on memory usage. The key features of the tool include: 1. Dynamic insertion of kprobes at critical points in the cgroup subsystem. 2. Tracking memory allocation and deallocation events for each process by recording page addresses in a hash table. 3. Providing real-time statistics on memory usage at the process level. 4. Providing statistics on memory usage for processes that are OOM. Signed-off-by: Taoxy2004 <221870066(a)smail.nju.edu.cn> --- tools/probeCgroup/Makefile | 7 + tools/probeCgroup/README.md | 29 + tools/probeCgroup/probeCgroup.c | 612 ++++++++++++++++++ tools/probeCgroup/probeCgroup.h | 415 ++++++++++++ tools/probeCgroup/run.sh | 8 + tools/probeCgroup/scripts/script1.sh | 10 + tools/probeCgroup/scripts/script2.sh | 14 + tools/probeCgroup/scripts/script3.sh | 11 + .../testcases/1_load_unload_test.py | 24 + .../testcases/2_multiple_process_test.py | 48 ++ .../testcases/3_multiple_cgroup_test.py | 55 ++ tools/probeCgroup/testcases/4_oom_test.py | 52 ++ .../testcases/5_multiple_threads_test.py | 45 ++ tools/probeCgroup/testcases/cgroup_utils.py | 115 ++++ tools/probeCgroup/testcases/mem-allocate.c | 35 + .../testcases/multiple-thread-mem-allocate.c | 60 ++ tools/probeCgroup/testcases/run.py | 32 + .../testcases/simple-mem-allocate.c | 27 + 18 files changed, 1599 insertions(+) create mode 100644 tools/probeCgroup/Makefile create mode 100644 tools/probeCgroup/README.md create mode 100644 tools/probeCgroup/probeCgroup.c create mode 100644 tools/probeCgroup/probeCgroup.h create mode 100755 tools/probeCgroup/run.sh create mode 100755 tools/probeCgroup/scripts/script1.sh create mode 100755 tools/probeCgroup/scripts/script2.sh create mode 100755 tools/probeCgroup/scripts/script3.sh create mode 100755 tools/probeCgroup/testcases/1_load_unload_test.py create mode 100755 tools/probeCgroup/testcases/2_multiple_process_test.py create mode 100755 tools/probeCgroup/testcases/3_multiple_cgroup_test.py create mode 100755 tools/probeCgroup/testcases/4_oom_test.py create mode 100755 tools/probeCgroup/testcases/5_multiple_threads_test.py create mode 100644 tools/probeCgroup/testcases/cgroup_utils.py create mode 100644 tools/probeCgroup/testcases/mem-allocate.c create mode 100644 tools/probeCgroup/testcases/multiple-thread-mem-allocate.c create mode 100755 tools/probeCgroup/testcases/run.py create mode 100644 tools/probeCgroup/testcases/simple-mem-allocate.c diff --git a/tools/probeCgroup/Makefile b/tools/probeCgroup/Makefile new file mode 100644 index 000000000000..606c951e5487 --- /dev/null +++ b/tools/probeCgroup/Makefile @@ -0,0 +1,7 @@ +obj-m := probeCgroup.o +CROSS_COMPILE = '' +KDIR := /lib/modules/$(shell uname -r)/build +all: + make -C $(KDIR) M=$(PWD) modules +clean: + rm -f *.ko *.o *.mod *.mod.o *.mod.c .*.cmd *.symvers module* diff --git a/tools/probeCgroup/README.md b/tools/probeCgroup/README.md new file mode 100644 index 000000000000..ff0b6fc21228 --- /dev/null +++ b/tools/probeCgroup/README.md @@ -0,0 +1,29 @@ +# probeCgroup + +#### Description +probeCgroup is a process-level cgroup memory monitoring tool based on dynamic tracing (kprobe/kretprobe) technology. By inserting kprobes and kretprobes at the entry and exit points of relevant cgroup functions, this tool can track the memory usage of individual processes within each cgroup in real time. + +#### Software Architecture +1. Dynamic Tracing : Insert kprobes and kretprobes at critical points in cgroup functions to capture memory allocation and release events. +2. Hash Table Recording : Record the addresses of pages currently used by each process in a hash table, so that when a page is released, the process it belongs to can be identified. +3. Real-Time Statistics : Provide real-time statistics showing the memory usage of individual processes within each cgroup. + +#### Instruction +1. Compile and Load the Module + a. In the 'probeCgroup' directory, run the 'make' command to compile the module. + b. Load the module: 'insmod probeCgroup.ko'. + c. View memory statistics: 'cat /proc/cgroup_memory_usage_per_process'. + If an OOM (Out of Memory) event occurs in a cgroup, you can see "oom:" followed by the process that experienced the OOM and its memory usage at the time. + +2. Automate OOM Scenario + In the 'probeCgroup' directory, run './run.sh'. This script will automatically set up an OOM scenario and output the content of '/proc/cgroup_memory_usage_per_process' after execution. + +3. Perform More Tests + a. After compiling the module, in the 'testcases' directory, run './run.py'. + b. This script will perform various tests, including: + - Loading and unloading the module + - Each cgroup containing multiple processes + - Creating multiple cgroups + - OOM scenarios + - Multithreading + c. The tests will take approximately one minute to complete. diff --git a/tools/probeCgroup/probeCgroup.c b/tools/probeCgroup/probeCgroup.c new file mode 100644 index 000000000000..9883cb1e082d --- /dev/null +++ b/tools/probeCgroup/probeCgroup.c @@ -0,0 +1,612 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * probeCgroup.c - A tool used to get memory usage for each process in a cgroup + * + * Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + */ + +#include "probeCgroup.h" + +// kretprobe at mem_cgroup_charge +struct charge_data { + struct cgroup *cgrp; + struct mem_cgroup *memcg; + struct task_struct *task; + unsigned long addr; +}; + +static int mem_cgroup_charge_entry_handler(struct kretprobe_instance *ri, + struct pt_regs *regs) +{ + struct charge_data *data; + struct folio *page; + struct mm_struct *mm; + struct mem_cgroup *memcg; + struct cgroup_subsys_state css; + struct cgroup *cgrp; + + if (!current->mm) + return 1; + page = (struct folio *)regs->di; + mm = (struct mm_struct *)regs->si; + if (mm == NULL || page == NULL) + return -1; + memcg = get_mem_cgroup_from_mm(mm); + if (memcg != NULL) { + css = memcg->css; + cgrp = css.cgroup; + + data = (struct charge_data *)ri->data; + data->memcg = memcg; + data->addr = (unsigned long)page; + data->task = current; + data->cgrp = cgrp; + } + return 0; +} + +NOKPROBE_SYMBOL(mem_cgroup_charge_entry_handler); + +static int mem_cgroup_charge_ret_handler(struct kretprobe_instance *ri, + struct pt_regs *regs) +{ + unsigned long retval = regs_return_value(regs); + struct charge_data *data = (struct charge_data *)ri->data; + int id; + struct cgroup_info *cgrp_info; + struct task_info *tsk_info; + + if (data->memcg != NULL && retval == 0) { + id = ((data->memcg)->css).id; + + spin_lock(&lock); + cgrp_info = find_cgroup_info(id); + if (cgrp_info == NULL) { + cgrp_info = create_cgroup_info(data->cgrp, data->memcg); + if (cgrp_info == NULL) { + spin_unlock(&lock); + return -1; + } + add_cgroup_info(cgrp_info); + } + spin_unlock(&lock); + + read_lock(&cgrp_info->cgrp_lock); + tsk_info = find_task_info(cgrp_info, data->task->tgid); + read_unlock(&cgrp_info->cgrp_lock); + + // for some cases, task->comm changes over time + if (tsk_info != NULL + && strcmp(data->task->comm, tsk_info->comm) != 0) { + strscpy(tsk_info->comm, data->task->comm, + sizeof(tsk_info->comm)); + } + + if (tsk_info == NULL) { + tsk_info = create_task_info(data->task); + if (tsk_info == NULL) + return -1; + add_task_to_cgroup_info(cgrp_info, tsk_info); + } + + if (HashMap_insert(tsk_info->pages, data->addr)) { + //update counter + spin_lock(&(tsk_info->cnt_lock)); + tsk_info->count += + folio_nr_pages((struct folio *)data->addr); + spin_unlock(&(tsk_info->cnt_lock)); + } + } + + return 0; + +} + +NOKPROBE_SYMBOL(mem_cgroup_charge_ret_handler); + +static struct kretprobe mem_cgroup_charge_kretprobe = { + .handler = mem_cgroup_charge_ret_handler, + .entry_handler = mem_cgroup_charge_entry_handler, + .data_size = sizeof(struct charge_data), + .maxactive = 20, +}; + +static int mem_cgroup_charge_kretprobe_init(void) +{ + int ret; + + mem_cgroup_charge_kretprobe.kp.symbol_name = "__mem_cgroup_charge"; + ret = register_kretprobe(&mem_cgroup_charge_kretprobe); + if (ret < 0) { + pr_err("register_kretprobe failed, returned %d\n", ret); + return ret; + } + pr_info("Planted return probe at %s: %p\n", + mem_cgroup_charge_kretprobe.kp.symbol_name, + mem_cgroup_charge_kretprobe.kp.addr); + return 0; +} + +static void mem_cgroup_charge_kretprobe_exit(void) +{ + unregister_kretprobe(&mem_cgroup_charge_kretprobe); + pr_info("kretprobe at %p unregistered\n", + mem_cgroup_charge_kretprobe.kp.addr); + + /* nmissed > 0 suggests that maxactive was set too low. */ + pr_info("Missed probing %d instances of %s\n", + mem_cgroup_charge_kretprobe.nmissed, + mem_cgroup_charge_kretprobe.kp.symbol_name); +} + +// kretprobe at uncharge_folio + +struct uncharge_data { + struct cgroup *cgrp; + struct mem_cgroup *memcg; + unsigned long addr; + bool isKmem; + int nr_pages; +}; + +static int uncharge_folio_entry_handler(struct kretprobe_instance *ri, + struct pt_regs *regs) +{ + struct uncharge_data *data; + struct folio *page; + struct mem_cgroup *memcg = NULL; + struct cgroup_subsys_state css; + struct cgroup *cgrp; + struct obj_cgroup *objcg; + int nr_pages = 0; + + data = (struct uncharge_data *)ri->data; + page = (struct folio *)regs->di; + if (page == NULL) { + data->memcg = NULL; + return -1; + } + if (page->memcg_data & MEMCG_DATA_KMEM) { // if the page belongs to kmem + if (!folio_test_large(page)) + nr_pages = 1; + else + nr_pages = page->_folio_nr_pages; + // nr_pages = thp_nr_pages(page); + objcg = __folio_objcg(page); + if (objcg != NULL) + memcg = objcg->memcg; + data->isKmem = true; + data->nr_pages = nr_pages; + } else { + memcg = __folio_memcg(page); + data->isKmem = false; + } + + if (memcg != NULL) { + css = memcg->css; + cgrp = css.cgroup; + + data->memcg = memcg; + data->addr = (unsigned long)page; + data->cgrp = cgrp; + } + return 0; +} + +NOKPROBE_SYMBOL(uncharge_folio_entry_handler); + +static int uncharge_folio_ret_handler(struct kretprobe_instance *ri, + struct pt_regs *regs) +{ + struct uncharge_data *data = (struct uncharge_data *)ri->data; + int id; + struct cgroup_info *cgrp_info; + int ret = -1; + + if (data->memcg != NULL) { + id = ((data->memcg)->css).id; + cgrp_info = find_cgroup_info(id); + if (cgrp_info == NULL) + return -1; + if (data->isKmem) + ret = -1; + else + ret = remove_page_from_cgroup_info(data->addr, cgrp_info); + } + + return ret; +} + +NOKPROBE_SYMBOL(uncharge_folio_ret_handler); + +static struct kretprobe uncharge_folio_kretprobe = { + .handler = uncharge_folio_ret_handler, + .entry_handler = uncharge_folio_entry_handler, + .data_size = sizeof(struct uncharge_data), + .maxactive = 20, +}; + +static int uncharge_folio_kretprobe_init(void) +{ + int ret; + + uncharge_folio_kretprobe.kp.symbol_name = "uncharge_folio"; + ret = register_kretprobe(&uncharge_folio_kretprobe); + if (ret < 0) { + pr_err("register_kretprobe failed, returned %d\n", ret); + return ret; + } + pr_info("Planted return probe at %s: %p\n", + uncharge_folio_kretprobe.kp.symbol_name, + uncharge_folio_kretprobe.kp.addr); + return 0; +} + +static void uncharge_folio_kretprobe_exit(void) +{ + unregister_kretprobe(&uncharge_folio_kretprobe); + pr_info("kretprobe at %p unregistered\n", + uncharge_folio_kretprobe.kp.addr); + + /* nmissed > 0 suggests that maxactive was set too low. */ + pr_info("Missed probing %d instances of %s\n", + uncharge_folio_kretprobe.nmissed, + uncharge_folio_kretprobe.kp.symbol_name); +} + +//kprobe at do_exit +static struct kprobe do_exit_kprobe; +static int do_exit_kprobe_pre_handler(struct kprobe *p, struct pt_regs *regs) +{ + struct task_struct *cur = current; + int tgid = cur->tgid; + struct mm_struct *mm = cur->mm; + struct mem_cgroup *memcg = get_mem_cgroup_from_mm(mm); + struct cgroup_subsys_state css; + struct cgroup *cgrp; + struct cgroup_info *cgrp_info; + struct task_info *tsk_info; + int id; + + if (memcg != NULL) { + css = memcg->css; + cgrp = css.cgroup; + id = (memcg->css).id; + cgrp_info = find_cgroup_info(id); + if (cgrp_info != NULL) { + write_lock(&cgrp_info->cgrp_lock); + tsk_info = find_task_info(cgrp_info, tgid); + if (tsk_info != NULL) { + list_del(&tsk_info->list); + write_unlock(&cgrp_info->cgrp_lock); + remove_task_from_cgroup_info(cgrp_info, + tsk_info); + } else { + write_unlock(&cgrp_info->cgrp_lock); + } + return 0; + } + } + return 0; +} + +static void do_exit_kprobe_post_handler(struct kprobe *p, + struct pt_regs *regs, + unsigned long flags) +{ + +} + +static int do_exit_kprobe_init(void) +{ + do_exit_kprobe.pre_handler = do_exit_kprobe_pre_handler; + do_exit_kprobe.post_handler = do_exit_kprobe_post_handler; + do_exit_kprobe.symbol_name = "do_exit"; + if (register_kprobe(&do_exit_kprobe)) { + pr_alert("register_kprobe on do_exit failed!\n"); + return -EINVAL; + } + return 0; +} + +static void do_exit_kprobe_exit(void) +{ + unregister_kprobe(&do_exit_kprobe); +} + +//kprobe at mark_oom_victim +static struct kprobe mark_oom_victim_kprobe; + +static int mark_oom_victim_kprobe_pre_handler(struct kprobe *p, + struct pt_regs *regs) +{ + struct task_struct *victim; + int tgid; + struct mm_struct *mm; + struct mem_cgroup *memcg; + struct cgroup_subsys_state css; + struct cgroup *cgrp; + struct cgroup_info *cgrp_info; + struct task_info *tsk_info; + int id; + struct task_info *oom_info; + + victim = (struct task_struct *)regs->di; + tgid = victim->tgid; + mm = victim->mm; + memcg = get_mem_cgroup_from_mm(mm); + if (memcg != NULL) { + css = memcg->css; + cgrp = css.cgroup; + id = (memcg->css).id; + cgrp_info = find_cgroup_info(id); + if (cgrp_info != NULL) { + read_lock(&cgrp_info->cgrp_lock); + tsk_info = find_task_info(cgrp_info, tgid); + read_unlock(&cgrp_info->cgrp_lock); + if (tsk_info != NULL) { + oom_info = create_oom_task_info(tsk_info); + if (oom_info != NULL) { + add_oom_task_to_cgroup_info(cgrp_info, + oom_info); + } + return 0; + } + } + } + return 0; +} + +static void mark_oom_victim_kprobe_post_handler(struct kprobe *p, + struct pt_regs *regs, + unsigned long flags) +{ + +} + +static int mark_oom_victim_kprobe_init(void) +{ + mark_oom_victim_kprobe.pre_handler = mark_oom_victim_kprobe_pre_handler; + mark_oom_victim_kprobe.post_handler = + mark_oom_victim_kprobe_post_handler; + mark_oom_victim_kprobe.symbol_name = "mark_oom_victim"; + if (register_kprobe(&mark_oom_victim_kprobe)) { + pr_alert("register_kprobe on mark_oom_victim failed!\n"); + return -EINVAL; + } + return 0; +} + +static void mark_oom_victim_kprobe_exit(void) +{ + unregister_kprobe(&mark_oom_victim_kprobe); +} + +//kretporbe at cgroup_destroy_locked +struct destroy_data { + struct cgroup *cgrp; +}; + +static int cgroup_destroy_locked_entry_handler(struct kretprobe_instance + *ri, struct pt_regs *regs) +{ + struct destroy_data *data; + + data = (struct destroy_data *)ri->data; + data->cgrp = (struct cgroup *)regs->di; + return 0; +} + +NOKPROBE_SYMBOL(cgroup_destroy_locked_entry_handler); + +static int cgroup_destroy_locked_ret_handler(struct kretprobe_instance *ri, + struct pt_regs *regs) +{ + struct destroy_data *data = (struct destroy_data *)ri->data; + struct cgroup *cgrp = data->cgrp; + struct cgroup_info *cgrp_info = NULL; + unsigned long retval = regs_return_value(regs); + + if (!cgrp) + return -1; + if (retval != 0) + return -1; + list_for_each_entry(cgrp_info, &all_cgroup_info, list) { + if (cgrp_info->cgrp == cgrp) { + spin_lock(&lock); + list_del(&cgrp_info->list); + spin_unlock(&lock); + destroy_cgroup_info(cgrp_info); + return 0; + } + } + return -1; +} + +NOKPROBE_SYMBOL(cgroup_destroy_locked_ret_handler); + +static struct kretprobe cgroup_destroy_locked_kretprobe = { + .handler = cgroup_destroy_locked_ret_handler, + .entry_handler = cgroup_destroy_locked_entry_handler, + .data_size = sizeof(struct destroy_data), + .maxactive = 20, +}; + +static int cgroup_destroy_locked_kretprobe_init(void) +{ + int ret; + + cgroup_destroy_locked_kretprobe.kp.symbol_name = + "cgroup_destroy_locked"; + ret = register_kretprobe(&cgroup_destroy_locked_kretprobe); + if (ret < 0) { + pr_err("register_kretprobe failed, returned %d\n", ret); + return ret; + } + pr_info("Planted return probe at %s: %p\n", + cgroup_destroy_locked_kretprobe.kp.symbol_name, + cgroup_destroy_locked_kretprobe.kp.addr); + return 0; +} + +static void cgroup_destroy_locked_kretprobe_exit(void) +{ + unregister_kretprobe(&cgroup_destroy_locked_kretprobe); + pr_info("kretprobe at %p unregistered\n", + cgroup_destroy_locked_kretprobe.kp.addr); + + /* nmissed > 0 suggests that maxactive was set too low. */ + pr_info("Missed probing %d instances of %s\n", + cgroup_destroy_locked_kretprobe.nmissed, + cgroup_destroy_locked_kretprobe.kp.symbol_name); +} + +// print the tasks in order of their memory usage +static void print_sorted_tasks_list(struct cgroup_info *cgrp_info, + int type, struct seq_file *m) +{ + struct list_head *cur, *insert_pos; + struct task_info *task, *insert_task; + struct list_head new_list = LIST_HEAD_INIT(new_list); + struct list_head *old_list; + struct task_info *new_task, *next_task; + + if (type == 0) { + if (cgrp_info == NULL) + return; + read_lock(&cgrp_info->cgrp_lock); + old_list = &cgrp_info->tasks_list; + } else { + if (cgrp_info == NULL) + return; + old_list = &cgrp_info->oom_list; + } + + list_for_each_entry_safe(task, insert_task, old_list, list) { + new_task = kmalloc(sizeof(struct task_info), GFP_ATOMIC); + if (!new_task) + return; + new_task->tgid = task->tgid; + strscpy(new_task->comm, task->comm, sizeof(new_task->comm)); + new_task->count = task->count; + new_task->pages = NULL; + INIT_LIST_HEAD(&new_task->list); + + //insertion sort + cur = &new_list; + insert_pos = cur->next; + while (insert_pos != &new_list) { + next_task = + list_entry(insert_pos, struct task_info, list); + if (new_task->count >= next_task->count) + break; + cur = insert_pos; + insert_pos = insert_pos->next; + } + + (&new_task->list)->prev = insert_pos->prev; + (insert_pos->prev)->next = (&new_task->list); + (&new_task->list)->next = insert_pos; + insert_pos->prev = (&new_task->list); + } + if (type == 0) + read_unlock(&cgrp_info->cgrp_lock); + + //print + if (type == 1 && (&new_list) != new_list.next) { + seq_puts(m, "oom:\n"); + seq_printf(m, "%10s %20s %20s\n", "pid", "command", + "memory usage (KB)"); + } + if (type == 0) + seq_printf(m, "%10s %20s %20s\n", "pid", "command", + "memory usage (KB)"); + list_for_each_entry_safe(task, insert_task, &new_list, list) { + seq_printf(m, "%10d %20s %20d\n", task->tgid, task->comm, + (task->count) * 4); + } + + list_for_each_entry_safe(task, insert_task, &new_list, list) { + list_del(&task->list); + kfree(task); + } +} + +static struct proc_dir_entry *cgroup_info_read; +#define procfs_file_read "cgroup_memory_usage_per_process" + +void seq_print_tasks(struct cgroup_info *cgroup_info, struct seq_file *m) +{ + if (!cgroup_info) + return; + + print_sorted_tasks_list(cgroup_info, 0, m); +} + +void seq_print_oom_tasks(struct cgroup_info *cgroup_info, struct seq_file *m) +{ + if (!cgroup_info) + return; + + print_sorted_tasks_list(cgroup_info, 1, m); +} + +void seq_print_cgroups(struct seq_file *m) +{ + struct cgroup_info *cgrp, *pos; + + spin_lock(&lock); + list_for_each_entry_safe(cgrp, pos, &all_cgroup_info, list) { + seq_printf(m, "cgroup name : %s\n", cgrp->name); + seq_print_tasks(cgrp, m); + seq_print_oom_tasks(cgrp, m); + seq_puts(m, "\n"); + } + spin_unlock(&lock); +} + +static int memory_usage_show(struct seq_file *m, void *v) +{ + seq_print_cgroups(m); + return 0; +} + +static int __init global_init(void) +{ + int ret = 0; + + cgroup_info_read = + proc_create_single(procfs_file_read, 0, NULL, memory_usage_show); + if (!cgroup_info_read) + return -ENOMEM; + ret = mem_cgroup_charge_kretprobe_init(); + uncharge_folio_kretprobe_init(); + do_exit_kprobe_init(); + mark_oom_victim_kprobe_init(); + cgroup_destroy_locked_kretprobe_init(); + + return ret; +} + +static void __exit global_exit(void) +{ + struct cgroup_info *cgrp_info, *pos; + + mem_cgroup_charge_kretprobe_exit(); + uncharge_folio_kretprobe_exit(); + do_exit_kprobe_exit(); + mark_oom_victim_kprobe_exit(); + cgroup_destroy_locked_kretprobe_exit(); + + remove_proc_entry(procfs_file_read, NULL); + + //release all memory use + list_for_each_entry_safe(cgrp_info, pos, &all_cgroup_info, list) { + list_del(&cgrp_info->list); + destroy_cgroup_info(cgrp_info); + } +} + +module_init(global_init) +module_exit(global_exit) +MODULE_LICENSE("GPL"); diff --git a/tools/probeCgroup/probeCgroup.h b/tools/probeCgroup/probeCgroup.h new file mode 100644 index 000000000000..953a6e0aca31 --- /dev/null +++ b/tools/probeCgroup/probeCgroup.h @@ -0,0 +1,415 @@ +/* SPDX-License-Identifier: GPL-2.0*/ +/* + * probeCgroup.h + * + * Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + */ + +#include <linux/kernel.h> +#include <linux/module.h> +#include <linux/kprobes.h> +#include <linux/ktime.h> +#include <linux/limits.h> +#include <linux/sched.h> +#include <linux/mm_types.h> +#include <linux/memcontrol.h> +#include <linux/cgroup-defs.h> +#include <linux/kernfs.h> +#include <linux/string.h> +#include <linux/list.h> +#include <linux/oom.h> +#include <linux/fs.h> +#include <linux/proc_fs.h> +#include <linux/huge_mm.h> +#include <linux/page-flags.h> +#include <linux/spinlock.h> +#include <linux/rwlock.h> + +static spinlock_t lock; // global lock for the list of cgroup_info + +struct HashNode { + unsigned long addr; + struct HashNode *next; +}; + +struct HashNode *HashNode_create(unsigned long addr) +{ + struct HashNode *node = NULL; + + node = kzalloc(sizeof(struct HashNode), GFP_ATOMIC); + if (node == NULL) + return NULL; + node->addr = addr; + node->next = NULL; + return node; +} + +struct HashBucket { + struct HashNode *head; + spinlock_t bkt_lock; +}; + +void HashBucket_init(struct HashBucket *bkt) +{ + bkt->head = NULL; + spin_lock_init(&bkt->bkt_lock); +} + +bool HashBucket_insert(struct HashBucket *bkt, unsigned long addr) +{ + struct HashNode *new_node; + struct HashNode *node; + struct HashNode *prev; + bool ret = true; + + if (bkt == NULL) + return false; + + prev = NULL; + new_node = NULL; + new_node = HashNode_create(addr); + + spin_lock(&bkt->bkt_lock); + node = bkt->head; + while (node != NULL && node->addr != addr) { + prev = node; + node = node->next; + } + if (node == NULL) { + if (new_node == NULL) { + pr_info("not enough memory for HashNode\n"); + spin_unlock(&bkt->bkt_lock); + return false; + } + if (bkt->head == NULL) + bkt->head = new_node; + else + prev->next = new_node; + spin_unlock(&bkt->bkt_lock); + ret = true; + } else { + spin_unlock(&bkt->bkt_lock); + kfree(new_node); + ret = false; + } + + return ret; +} + +bool HashBucket_erase(struct HashBucket *bkt, unsigned long addr) +{ + struct HashNode *node; + struct HashNode *prev; + bool ret = true; + + if (bkt == NULL) + return false; + + spin_lock(&bkt->bkt_lock); + node = bkt->head; + prev = NULL; + while (node != NULL && node->addr != addr) { + prev = node; + node = node->next; + } + if (node == NULL) { + spin_unlock(&bkt->bkt_lock); + ret = false; + } else { + if (bkt->head == node) + bkt->head = node->next; + else + prev->next = node->next; + kfree(node); + spin_unlock(&bkt->bkt_lock); + ret = true; + } + + return ret; +} + +void HashBucket_clear(struct HashBucket *bkt) +{ + struct HashNode *node; + struct HashNode *prev; + + if (bkt == NULL) + return; + + spin_lock(&bkt->bkt_lock); + node = bkt->head; + prev = NULL; + bkt->head = NULL; + while (node != NULL) { + prev = node; + node = node->next; + kfree(prev); + } + spin_unlock(&bkt->bkt_lock); +} + +struct HashMap { + unsigned long size; + struct HashBucket *HashTable; +}; + +unsigned long hash_func(unsigned long addr, unsigned long size) +{ + return addr % size; +} + +struct HashMap *HashMap_create(unsigned long size) +{ + struct HashMap *hm = NULL; + struct HashBucket *ht = NULL; + int i = 0; + + hm = kmalloc(sizeof(struct HashMap), GFP_ATOMIC); + if (hm == NULL) + return NULL; + ht = kmalloc((size * sizeof(struct HashBucket)), GFP_ATOMIC); + if (ht == NULL) { + kfree(hm); + return NULL; + } + for (i = 0; i < size; i++) + HashBucket_init(&(ht[i])); + + hm->size = size; + hm->HashTable = ht; + return hm; +} + +bool HashMap_insert(struct HashMap *hm, unsigned long addr) +{ + unsigned long index; + + if (hm == NULL) + return false; + index = hash_func(addr, hm->size); + if (hm->HashTable == NULL) + return false; + return HashBucket_insert(&(hm->HashTable[index]), addr); +} + +bool HashMap_erase(struct HashMap *hm, unsigned long addr) +{ + unsigned long index; + + if (hm == NULL) + return false; + index = hash_func(addr, hm->size); + if (hm->HashTable == NULL) + return false; + return HashBucket_erase(&(hm->HashTable[index]), addr); +} + +void HashMap_clear(struct HashMap *hm) +{ + unsigned long size; + struct HashBucket *ht; + int i; + + if (hm == NULL) + return; + size = hm->size; + ht = hm->HashTable; + if (ht == NULL) + return; + hm->HashTable = NULL; + for (i = 0; i < size; i++) + HashBucket_clear(&(ht[i])); + + kfree(ht); + kfree(hm); +} + +//struct that save the information for each task +struct task_info { + int tgid; + char comm[TASK_COMM_LEN]; + int count; // number of pages + struct HashMap *pages; + struct list_head list; + spinlock_t cnt_lock; +}; + +// struct that save the information for each cgroup +struct cgroup_info { + struct cgroup *cgrp; + struct mem_cgroup *memcg; + int id; + char name[64]; + struct list_head list; + struct list_head tasks_list; + struct list_head oom_list; + rwlock_t cgrp_lock; + unsigned int cached_bytes; +}; + +static LIST_HEAD(all_cgroup_info); // a list that linked all the cgroup_info struct + +static struct task_info *create_task_info(struct task_struct *cur_task) +{ + struct task_info *tsk_info = + kmalloc(sizeof(struct task_info), GFP_ATOMIC); + if (!tsk_info) + return NULL; + + // initialization + tsk_info->tgid = cur_task->tgid; + strscpy(tsk_info->comm, cur_task->comm, sizeof(tsk_info->comm)); + tsk_info->count = 0; + tsk_info->pages = NULL; + tsk_info->pages = HashMap_create(1023); + INIT_LIST_HEAD(&tsk_info->list); + spin_lock_init(&(tsk_info->cnt_lock)); + + return tsk_info; +} + +static int +add_task_to_cgroup_info(struct cgroup_info *cgrp, struct task_info *task) +{ + if (!cgrp || !task) + return -EINVAL; + + write_lock(&cgrp->cgrp_lock); + list_add_tail(&task->list, &cgrp->tasks_list); + write_unlock(&cgrp->cgrp_lock); + return 0; +} + +static int +remove_task_from_cgroup_info(struct cgroup_info *cgrp, struct task_info *task) +{ + if (cgrp == NULL || task == NULL) + return -EINVAL; + + HashMap_clear(task->pages); + // kfree(task->pages); + kfree(task); + return 0; +} + +static struct task_info *find_task_info(struct cgroup_info *cgrp, int tgid) +{ + struct task_info *tsk_info, *pos; + + list_for_each_entry_safe(tsk_info, pos, &cgrp->tasks_list, list) { + if (tsk_info->tgid == tgid) + return tsk_info; + } + return NULL; +} + +static int +remove_page_from_cgroup_info(unsigned long addr, struct cgroup_info *cgrp) +{ + struct task_info *tsk_info, *pos; + + read_lock(&cgrp->cgrp_lock); + list_for_each_entry_safe(tsk_info, pos, &cgrp->tasks_list, list) { + if (HashMap_erase(tsk_info->pages, addr)) { + spin_lock(&(tsk_info->cnt_lock)); + tsk_info->count -= folio_nr_pages((struct folio *)addr); + spin_unlock(&(tsk_info->cnt_lock)); + read_unlock(&cgrp->cgrp_lock); + return 0; + } + } + read_unlock(&cgrp->cgrp_lock); + return -1; +} + +static struct cgroup_info *create_cgroup_info(struct cgroup *cgrp, + struct mem_cgroup *memcg) +{ + struct cgroup_info *cgrp_info = + kmalloc(sizeof(struct cgroup_info), GFP_ATOMIC); + struct kernfs_node *kn; + + if (!cgrp_info) + return NULL; + + cgrp_info->cgrp = cgrp; + cgrp_info->memcg = memcg; + cgrp_info->id = (memcg->css).id; + kn = cgrp->kn; + strscpy(cgrp_info->name, kn->name, sizeof(cgrp_info->name)); + INIT_LIST_HEAD(&cgrp_info->list); + INIT_LIST_HEAD(&cgrp_info->tasks_list); + INIT_LIST_HEAD(&cgrp_info->oom_list); + rwlock_init(&(cgrp_info->cgrp_lock)); + cgrp_info->cached_bytes = 0; + + return cgrp_info; +} + +static void destroy_cgroup_info(struct cgroup_info *cgrp_info) +{ + struct task_info *task, *tmp; + + if (!cgrp_info) + return; + + write_lock(&cgrp_info->cgrp_lock); + list_for_each_entry_safe(task, tmp, &cgrp_info->tasks_list, list) { + list_del(&task->list); + remove_task_from_cgroup_info(cgrp_info, task); + } + write_unlock(&cgrp_info->cgrp_lock); + list_for_each_entry_safe(task, tmp, &cgrp_info->oom_list, list) { + list_del(&task->list); + remove_task_from_cgroup_info(cgrp_info, task); + } + + kfree(cgrp_info); +} + +static int add_cgroup_info(struct cgroup_info *cgrp_info) +{ + if (!cgrp_info) + return -EINVAL; + + list_add_tail(&cgrp_info->list, &all_cgroup_info); + return 0; +} + +static struct cgroup_info *find_cgroup_info(int id) +{ + struct cgroup_info *cgrp_info = NULL; + + list_for_each_entry(cgrp_info, &all_cgroup_info, list) { + if (cgrp_info->id == id) + return cgrp_info; + } + + return NULL; +} + +static struct task_info *create_oom_task_info(struct task_info *tsk_info) +{ + struct task_info *oom_tsk_info = + kmalloc(sizeof(struct task_info), GFP_ATOMIC); + if (!oom_tsk_info) + return NULL; + + oom_tsk_info->tgid = tsk_info->tgid; + strscpy(oom_tsk_info->comm, tsk_info->comm, sizeof(oom_tsk_info->comm)); + oom_tsk_info->count = tsk_info->count; + oom_tsk_info->pages = NULL; + INIT_LIST_HEAD(&oom_tsk_info->list); + + return oom_tsk_info; +} + +static int +add_oom_task_to_cgroup_info(struct cgroup_info *cgrp, + struct task_info *oom_task) +{ + if (!cgrp || !oom_task) + return -EINVAL; + list_add_tail(&oom_task->list, &cgrp->oom_list); + return 0; +} diff --git a/tools/probeCgroup/run.sh b/tools/probeCgroup/run.sh new file mode 100755 index 000000000000..7e1ffefe66d1 --- /dev/null +++ b/tools/probeCgroup/run.sh @@ -0,0 +1,8 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +cd scripts +./script1.sh +./script2.sh +./script3.sh diff --git a/tools/probeCgroup/scripts/script1.sh b/tools/probeCgroup/scripts/script1.sh new file mode 100755 index 000000000000..539e8258afb9 --- /dev/null +++ b/tools/probeCgroup/scripts/script1.sh @@ -0,0 +1,10 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +cd .. +make +insmod probeCgroup.ko + +cd testcases +gcc simple-mem-allocate.c -o simple-mem-allocate diff --git a/tools/probeCgroup/scripts/script2.sh b/tools/probeCgroup/scripts/script2.sh new file mode 100755 index 000000000000..2ad515cfb912 --- /dev/null +++ b/tools/probeCgroup/scripts/script2.sh @@ -0,0 +1,14 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +current_dir=$(pwd) +cd /sys/fs/cgroup/memory +mkdir test +cd test +sh -c "echo $$ >> cgroup.procs" +sh -c "echo 5M > memory.limit_in_bytes" +sh -c "echo 0 > memory.swappiness" +cd "$current_dir" +cd ../testcases +./simple-mem-allocate diff --git a/tools/probeCgroup/scripts/script3.sh b/tools/probeCgroup/scripts/script3.sh new file mode 100755 index 000000000000..127eb45de5c9 --- /dev/null +++ b/tools/probeCgroup/scripts/script3.sh @@ -0,0 +1,11 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +cd /proc +cat cgroup_memory_usage_per_process + +cat /sys/fs/cgroup/memory/test/cgroup.procs > /sys/fs/cgroup/memory/cgroup.procs +rmdir /sys/fs/cgroup/memory/test +# cat cgroup_memory_usage_per_process +rmmod probeCgroup diff --git a/tools/probeCgroup/testcases/1_load_unload_test.py b/tools/probeCgroup/testcases/1_load_unload_test.py new file mode 100755 index 000000000000..5389a14a1dac --- /dev/null +++ b/tools/probeCgroup/testcases/1_load_unload_test.py @@ -0,0 +1,24 @@ +#!/usr/bin/env python +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +import os +import subprocess +import time + +def test_module_load_unload(): + try: + subprocess.check_call(['insmod', '../probeCgroup.ko']) + time.sleep(1) + print('loading module successfully!') + subprocess.check_call(['rmmod', 'probeCgroup']) + output = subprocess.check_output(['lsmod']) + assert b'probeCgroup' not in output + print('unloading module successfully!') + except subprocess.CalledProcessError as e: + print('Load unload test failed. Insmod failed.') + except AssertionError as e: + print('Load unload test failed. Cannot remove module.') + +if __name__ == '__main__': + test_module_load_unload() \ No newline at end of file diff --git a/tools/probeCgroup/testcases/2_multiple_process_test.py b/tools/probeCgroup/testcases/2_multiple_process_test.py new file mode 100755 index 000000000000..d88c7f7f2952 --- /dev/null +++ b/tools/probeCgroup/testcases/2_multiple_process_test.py @@ -0,0 +1,48 @@ +#!/usr/bin/env python +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +from cgroup_utils import create_cgroup, add_process_to_cgroup, get_process_memory_usage, remove_cgroup, check_memory_usage, cleanup, check_kmem_usage +import os +import subprocess +import time + +def test_multiple_process(num_procs): + subprocess.check_call(['insmod', '../probeCgroup.ko']) + time.sleep(1) + + cgroup_name = 'test' + cgroup_path = create_cgroup(cgroup_name) + + processes = [] + pids = [] + + for i in range(num_procs): + process = subprocess.Popen(['./mem-allocate']) + pid = process.pid + add_process_to_cgroup(cgroup_path, pid) + processes.append(process) + pids.append(pid) + + time.sleep(0.1) + try: + count = 0 + for i in range (2000): + count += check_memory_usage(cgroup_name, pids, False) + time.sleep(0.01) + assert count <= 50, f"Memory read by probeCgroup is not accurate" + + remove_cgroup(cgroup_path, pids) + check_memory_usage(cgroup_name, pids, True) + cleanup(processes) + subprocess.check_call(['rmmod', 'probeCgroup']) + + print('pass multiple process test!') + except AssertionError as e: + print(f"Assertion failed: {e}") + remove_cgroup(cgroup_path, pids) + cleanup(processes) + subprocess.check_call(['rmmod', 'probeCgroup']) + +if __name__ == '__main__': + test_multiple_process(3) \ No newline at end of file diff --git a/tools/probeCgroup/testcases/3_multiple_cgroup_test.py b/tools/probeCgroup/testcases/3_multiple_cgroup_test.py new file mode 100755 index 000000000000..592a716df877 --- /dev/null +++ b/tools/probeCgroup/testcases/3_multiple_cgroup_test.py @@ -0,0 +1,55 @@ +#!/usr/bin/env python +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +from cgroup_utils import create_cgroup, add_process_to_cgroup, get_process_memory_usage, remove_cgroup, check_memory_usage, cleanup +import os +import subprocess +import time + +def test_multiple_cgroup(num_procs, num_cgroups): + subprocess.check_call(['insmod', '../probeCgroup.ko']) + time.sleep(1) + + cgroups = [] + processes = {} + pids = {} + for i in range(num_cgroups): + cgroup_name = f'test_{i}' + cgroup_path = create_cgroup(cgroup_name) + cgroups.append((cgroup_name, cgroup_path)) + + for j in range(num_procs): + process = subprocess.Popen(['./mem-allocate']) + pid = process.pid + add_process_to_cgroup(cgroup_path, pid) + if cgroup_path not in processes: + processes[cgroup_path] = [] + processes[cgroup_path].append(process) + if cgroup_path not in pids: + pids[cgroup_path] = [] + pids[cgroup_path].append(pid) + + time.sleep(0.1) + try: + for i in range (100): + for cgroup_name, cgroup_path in cgroups: + check_memory_usage(cgroup_name, pids[cgroup_path], False) + time.sleep(0.01) + + for cgroup_name, cgroup_path in cgroups: + remove_cgroup(cgroup_path, pids[cgroup_path]) + check_memory_usage(cgroup_name, pids[cgroup_path], True) + cleanup(processes[cgroup_path]) + subprocess.check_call(['rmmod', 'probeCgroup']) + + print('pass multiple cgroup test!') + except AssertionError as e: + print(f"Assertion failed: {e}") + for cgroup_name, cgroup_path in cgroups: + remove_cgroup(cgroup_path, pids[cgroup_path]) + cleanup(processes[cgroup_path]) + subprocess.check_call(['rmmod', 'probeCgroup']) + +if __name__ == '__main__': + test_multiple_cgroup(2,2) \ No newline at end of file diff --git a/tools/probeCgroup/testcases/4_oom_test.py b/tools/probeCgroup/testcases/4_oom_test.py new file mode 100755 index 000000000000..128a258c56f5 --- /dev/null +++ b/tools/probeCgroup/testcases/4_oom_test.py @@ -0,0 +1,52 @@ +#!/usr/bin/env python +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +from cgroup_utils import create_cgroup, add_process_to_cgroup, get_process_memory_usage, remove_cgroup, check_memory_usage, cleanup, get_oom_process_memory_usage +import os +import subprocess +import time + +def test_oom(num_procs): + subprocess.check_call(['insmod', '../probeCgroup.ko']) + time.sleep(1) + + cgroup_name = 'test' + cgroup_path = create_cgroup(cgroup_name) + + with open(f"/sys/fs/cgroup/memory/{cgroup_name}/memory.limit_in_bytes", 'w') as limit_file: + limit_file.write("5M") + with open(f"/sys/fs/cgroup/memory/{cgroup_name}/memory.swappiness", 'w') as swap_file: + swap_file.write("0") + + processes = [] + pids = [] + for i in range(num_procs): + process = subprocess.Popen(['./simple-mem-allocate']) + pid = process.pid + add_process_to_cgroup(cgroup_path, pid) + processes.append(process) + pids.append(pid) + + time.sleep(6) + + try: + for pid in pids: + memory_usage = get_oom_process_memory_usage(pid, cgroup_name) + assert memory_usage is not None, f"Memory usage(oom) not found for PID {pid}" + assert memory_usage > 0, f"Memory usage should be greater than zero for PID {pid}" + + remove_cgroup(cgroup_path, pids) + check_memory_usage(cgroup_name, pids, True) + cleanup(processes) + subprocess.check_call(['rmmod', 'probeCgroup']) + + print('pass oom test!') + except AssertionError as e: + print(f"Assertion failed: {e}") + remove_cgroup(cgroup_path, pids) + cleanup(processes) + subprocess.check_call(['rmmod', 'probeCgroup']) + +if __name__ == '__main__': + test_oom(1) \ No newline at end of file diff --git a/tools/probeCgroup/testcases/5_multiple_threads_test.py b/tools/probeCgroup/testcases/5_multiple_threads_test.py new file mode 100755 index 000000000000..7e1b86dabe48 --- /dev/null +++ b/tools/probeCgroup/testcases/5_multiple_threads_test.py @@ -0,0 +1,45 @@ +#!/usr/bin/env python +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +from cgroup_utils import create_cgroup, add_process_to_cgroup, get_process_memory_usage, remove_cgroup, check_memory_usage, cleanup, check_kmem_usage +import os +import subprocess +import time + +def test_multiple_thread(num_procs): + subprocess.check_call(['insmod', '../probeCgroup.ko']) + time.sleep(1) + + cgroup_name = 'test' + cgroup_path = create_cgroup(cgroup_name) + + processes = [] + pids = [] + for i in range(num_procs): + process = subprocess.Popen(['./multiple-thread-mem-allocate']) + pid = process.pid + add_process_to_cgroup(cgroup_path, pid) + processes.append(process) + pids.append(pid) + + time.sleep(1) + try: + for i in range (200): + check_memory_usage(cgroup_name, pids, False) + time.sleep(0.01) + + remove_cgroup(cgroup_path, pids) + check_memory_usage(cgroup_name, pids, True) + cleanup(processes) + subprocess.check_call(['rmmod', 'probeCgroup']) + + print('pass multiple threads test!') + except AssertionError as e: + print(f"Assertion failed: {e}") + remove_cgroup(cgroup_path, pids) + cleanup(processes) + subprocess.check_call(['rmmod', 'probeCgroup']) + +if __name__ == '__main__': + test_multiple_thread(5) \ No newline at end of file diff --git a/tools/probeCgroup/testcases/cgroup_utils.py b/tools/probeCgroup/testcases/cgroup_utils.py new file mode 100644 index 000000000000..f70c68c1f188 --- /dev/null +++ b/tools/probeCgroup/testcases/cgroup_utils.py @@ -0,0 +1,115 @@ +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +import os +import subprocess +import time + +def create_cgroup(cgroup_name): + cgroup_path = f'/sys/fs/cgroup/memory/{cgroup_name}' + + try: + os.makedirs(cgroup_path) + except FileExistsError: + pass + + return cgroup_path + +def add_process_to_cgroup(cgroup_path, pid): + with open(os.path.join(cgroup_path, 'cgroup.procs'), 'w') as procs_file: + procs_file.write(str(pid)) + +def get_process_memory_usage(pid, cgroup_name): + cur_name = '' + with open('/proc/cgroup_memory_usage_per_process', 'r') as file: + for line in file: + parts = line.strip().split() + if len(parts) >= 4 and parts[0] == 'cgroup': + cur_name = parts[3] + if len(parts) >= 3 and parts[0] != 'cgroup' and cur_name == cgroup_name and parts[0] != 'pid' and int(parts[0]) == pid: + return int(parts[2]) + return None + +def get_process_kmem_usage(pid, cgroup_name): + cur_name = '' + with open('/proc/cgroup_memory_usage_per_process', 'r') as file: + for line in file: + parts = line.strip().split() + if len(parts) >= 4 and parts[0] == 'cgroup': + cur_name = parts[3] + if len(parts) >= 4 and parts[0] != 'cgroup' and cur_name == cgroup_name and parts[0] != 'pid' and int(parts[0]) == pid: + return int(parts[3]) + return None + +def remove_cgroup(cgroup_path, pids): + for pid in pids: + with open('/sys/fs/cgroup/memory/cgroup.procs', 'w') as backup_file: + backup_file.write(str(pid)) + os.rmdir(cgroup_path) + return + +def check_memory_usage(cgroup_name, pids, delete): + memory_sum = 0 + for pid in pids: + memory_usage = get_process_memory_usage(pid, cgroup_name) + if delete == False: + assert memory_usage is not None, f"Memory usage not found for PID {pid}" + assert memory_usage >= 0, f"Memory usage should be greater than zero for PID {pid}" + memory_sum += memory_usage + else: + assert memory_usage is None, f"Error: Memory usage should not be available for PID {pid} after deleting the cgroup." + if delete == False: + with open(f"/sys/fs/cgroup/memory/{cgroup_name}/memory.usage_in_bytes", 'r') as file: + content = file.readline().strip() + memory_read = int(content) + memory_sum *= 1024 + delta = abs(memory_read - memory_sum) + # print(f"read: {memory_read}") + # print(f"sum : {memory_sum}") + if (delta > max(memory_read, memory_sum) * 0.1): + return 1 + else: + return 0 + else: + return 0 + +def check_kmem_usage(cgroup_name, pids, delete): + kmem_sum = 0 + for pid in pids: + kmem_usage = get_process_kmem_usage(pid, cgroup_name) + if delete == False: + assert kmem_usage is not None, f"Kmem usage not found for PID {pid}" + assert kmem_usage >= 0, f"Kmem usage should be greater than zero for PID {pid}" + kmem_sum += kmem_usage + else: + assert kmem_usage is None, f"Error: Kmem usage should not be available for PID {pid} after deleting the cgroup." + if delete == False: + with open(f"/sys/fs/cgroup/memory/{cgroup_name}/memory.kmem.usage_in_bytes", 'r') as file: + content = file.readline().strip() + kmem_read = int(content) + kmem_sum *= 1024 + delta = abs(kmem_read - kmem_sum) + # print(f"kmem read: {kmem_read}") + # print(f"kmem sum : {kmem_sum}") + # assert delta <= max(kmem_read, kmem_sum) * 0.2, f"Kmem read by probeCgroup is not accurate, {kmem_read}, {kmem_sum}" + +def cleanup(processes): + for process in processes: + process.terminate() + process.wait() + +def get_oom_process_memory_usage(pid, cgroup_name): + # �� + cur_name = '' + oom = False + with open('/proc/cgroup_memory_usage_per_process', 'r') as file: + for line in file: + parts = line.strip().split() + if len(parts) >= 4 and parts[0] == 'cgroup': + cur_name = parts[3] + oom = False + if len(parts) >= 1 and parts[0] == 'oom:': + oom = True + if len(parts) >= 3 and parts[0] != 'cgroup' and cur_name == cgroup_name and parts[0] != 'pid' and int(parts[0]) == pid and oom == True: + return int(parts[2]) + return None \ No newline at end of file diff --git a/tools/probeCgroup/testcases/mem-allocate.c b/tools/probeCgroup/testcases/mem-allocate.c new file mode 100644 index 000000000000..e78e37cae61b --- /dev/null +++ b/tools/probeCgroup/testcases/mem-allocate.c @@ -0,0 +1,35 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * mem-allocate.c - The program to test probeCgroup + * + * Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + */ + +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <unistd.h> + +#define MB (1024 * 1024) + +char *arr[40]; + +int main(int argc, char *argv[]) +{ + char *p; + int i = 0; + + while (1) { + for (i = 0; i < 40; i++) { + p = (char *)malloc(MB); + memset(p, 0, MB); + arr[i] = p; + usleep(100000); + } + for (int i = 0; i < 40; i++) { + free(arr[i]); + usleep(100000); + } + } + return 0; +} diff --git a/tools/probeCgroup/testcases/multiple-thread-mem-allocate.c b/tools/probeCgroup/testcases/multiple-thread-mem-allocate.c new file mode 100644 index 000000000000..55f4c068f55e --- /dev/null +++ b/tools/probeCgroup/testcases/multiple-thread-mem-allocate.c @@ -0,0 +1,60 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * multiple-thread-mem-allocate.c - The program to test probeCgroup + * + * Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + */ + +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <unistd.h> +#include <pthread.h> + +#define MB (1024 * 1024) + +void *memory_test(void *) +{ + char *arr[25]; + char *p; + int i = 0; + int cnt = 0; + + while (1) { + for (i = 0; i < 20; i++) { + p = (char *)malloc(MB); + memset(p, 0, MB); + arr[i] = p; + usleep(10000); + } + for (int i = 0; i < 20; i++) { + free(arr[i]); + usleep(10000); + } + } +} + +int main(int argc, char *argv[]) +{ + pthread_t threads[4]; + int rc; + + // create threads + for (int i = 0; i < 4; i++) { + rc = pthread_create(&threads[i], NULL, memory_test, NULL); + if (rc != 0) { + fprintf(stderr, "Error creating thread: %d\n", rc); + return 1; + } + } + + for (int i = 0; i < 4; i++) { + rc = pthread_join(threads[i], NULL); + if (rc != 0) { + fprintf(stderr, "Error joining thread: %d\n", rc); + return 1; + } + } + + return 0; +} diff --git a/tools/probeCgroup/testcases/run.py b/tools/probeCgroup/testcases/run.py new file mode 100755 index 000000000000..8ffd0ca720d8 --- /dev/null +++ b/tools/probeCgroup/testcases/run.py @@ -0,0 +1,32 @@ +#!/usr/bin/env python +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +import os +import subprocess +import sys + +def run_tests(directory): + """Run all Python scripts in the given directory.""" + python_files = [f for f in os.listdir(directory) if f.endswith('test.py')] + python_files.sort() + + for filename in python_files: + try: + filepath = os.path.join(directory, filename) + + subprocess.check_call([sys.executable, filepath]) + except subprocess.CalledProcessError as e: + print(f"Error executing {filename}:") + return + except Exception as e: + print(f"Error executing {filename}:") + print(e) + return + +if __name__ == '__main__': + tests_directory = '.' + subprocess.check_call(['gcc', 'mem-allocate.c', '-o', 'mem-allocate']) + subprocess.check_call(['gcc', 'simple-mem-allocate.c', '-o', 'simple-mem-allocate']) + subprocess.check_call(['gcc', 'multiple-thread-mem-allocate.c', '-o', 'multiple-thread-mem-allocate']) + run_tests(tests_directory) \ No newline at end of file diff --git a/tools/probeCgroup/testcases/simple-mem-allocate.c b/tools/probeCgroup/testcases/simple-mem-allocate.c new file mode 100644 index 000000000000..16328b10ba48 --- /dev/null +++ b/tools/probeCgroup/testcases/simple-mem-allocate.c @@ -0,0 +1,27 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * mem-allocate.c - The program to test probeCgroup + * + * Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + */ + +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <unistd.h> + +#define MB (1024 * 1024) + +int main(int argc, char *argv[]) +{ + char *p; + int i = 0; + + while (1) { + p = (char *)malloc(MB); + memset(p, 0, MB); + sleep(1); + } + + return 0; +} -- 2.43.0

2 1

[PATCH OLK-6.6 0/4] Some fixes About cpuset partition
by Chen Ridong 05 Sep '24

05 Sep '24

Chen Ridong (1): cgroup/cpuset: fix panic caused by partcmd_update Waiman Long (3): cgroup/cpuset: Optimize isolated partition only generate_sched_domains() calls cgroup/cpuset: Fix remote root partition creation problem cgroup/cpuset: Clear effective_xcpus on cpus_allowed clearing only if cpus.exclusive not set kernel/cgroup/cpuset.c | 68 +++++++++++++++++++++++++++++++++--------- 1 file changed, 54 insertions(+), 14 deletions(-) -- 2.34.1

2 5

[PATCH penEuler-23.09] Add dynamic kprobe-based process-level cgroup memory monitoring tool
by Taoxy2004 05 Sep '24

05 Sep '24

--- tools/probeCgroup: This patch introduces a new tool called "probeCgroup" that enables dynamic monitoring of memory usage at the process level within cgroups. By using kprobes at relevant cgroup functions, this tool can track memory allocations and deallocations for individual processes within a cgroup, providing detailed statistics on memory usage. The key features of the tool include: 1. Dynamic insertion of kprobes at critical points in the cgroup subsystem. 2. Tracking memory allocation and deallocation events for each process by recording page addresses in a hash table. 3. Providing real-time statistics on memory usage at the process level. 4. Providing statistics on memory usage for processes that are OOM. Signed-off-by: Taoxy2004 <221870066(a)smail.nju.edu.cn> --- tools/probeCgroup/Makefile | 7 + tools/probeCgroup/README.md | 29 + tools/probeCgroup/probeCgroup.c | 612 ++++++++++++++++++ tools/probeCgroup/probeCgroup.h | 415 ++++++++++++ tools/probeCgroup/run.sh | 8 + tools/probeCgroup/scripts/script1.sh | 10 + tools/probeCgroup/scripts/script2.sh | 14 + tools/probeCgroup/scripts/script3.sh | 11 + .../testcases/1_load_unload_test.py | 24 + .../testcases/2_multiple_process_test.py | 48 ++ .../testcases/3_multiple_cgroup_test.py | 55 ++ tools/probeCgroup/testcases/4_oom_test.py | 52 ++ .../testcases/5_multiple_threads_test.py | 45 ++ tools/probeCgroup/testcases/cgroup_utils.py | 115 ++++ tools/probeCgroup/testcases/mem-allocate.c | 35 + .../testcases/multiple-thread-mem-allocate.c | 60 ++ tools/probeCgroup/testcases/run.py | 32 + .../testcases/simple-mem-allocate.c | 27 + 18 files changed, 1599 insertions(+) create mode 100644 tools/probeCgroup/Makefile create mode 100644 tools/probeCgroup/README.md create mode 100644 tools/probeCgroup/probeCgroup.c create mode 100644 tools/probeCgroup/probeCgroup.h create mode 100755 tools/probeCgroup/run.sh create mode 100755 tools/probeCgroup/scripts/script1.sh create mode 100755 tools/probeCgroup/scripts/script2.sh create mode 100755 tools/probeCgroup/scripts/script3.sh create mode 100755 tools/probeCgroup/testcases/1_load_unload_test.py create mode 100755 tools/probeCgroup/testcases/2_multiple_process_test.py create mode 100755 tools/probeCgroup/testcases/3_multiple_cgroup_test.py create mode 100755 tools/probeCgroup/testcases/4_oom_test.py create mode 100755 tools/probeCgroup/testcases/5_multiple_threads_test.py create mode 100644 tools/probeCgroup/testcases/cgroup_utils.py create mode 100644 tools/probeCgroup/testcases/mem-allocate.c create mode 100644 tools/probeCgroup/testcases/multiple-thread-mem-allocate.c create mode 100755 tools/probeCgroup/testcases/run.py create mode 100644 tools/probeCgroup/testcases/simple-mem-allocate.c diff --git a/tools/probeCgroup/Makefile b/tools/probeCgroup/Makefile new file mode 100644 index 000000000000..606c951e5487 --- /dev/null +++ b/tools/probeCgroup/Makefile @@ -0,0 +1,7 @@ +obj-m := probeCgroup.o +CROSS_COMPILE = '' +KDIR := /lib/modules/$(shell uname -r)/build +all: + make -C $(KDIR) M=$(PWD) modules +clean: + rm -f *.ko *.o *.mod *.mod.o *.mod.c .*.cmd *.symvers module* diff --git a/tools/probeCgroup/README.md b/tools/probeCgroup/README.md new file mode 100644 index 000000000000..ff0b6fc21228 --- /dev/null +++ b/tools/probeCgroup/README.md @@ -0,0 +1,29 @@ +# probeCgroup + +#### Description +probeCgroup is a process-level cgroup memory monitoring tool based on dynamic tracing (kprobe/kretprobe) technology. By inserting kprobes and kretprobes at the entry and exit points of relevant cgroup functions, this tool can track the memory usage of individual processes within each cgroup in real time. + +#### Software Architecture +1. Dynamic Tracing : Insert kprobes and kretprobes at critical points in cgroup functions to capture memory allocation and release events. +2. Hash Table Recording : Record the addresses of pages currently used by each process in a hash table, so that when a page is released, the process it belongs to can be identified. +3. Real-Time Statistics : Provide real-time statistics showing the memory usage of individual processes within each cgroup. + +#### Instruction +1. Compile and Load the Module + a. In the 'probeCgroup' directory, run the 'make' command to compile the module. + b. Load the module: 'insmod probeCgroup.ko'. + c. View memory statistics: 'cat /proc/cgroup_memory_usage_per_process'. + If an OOM (Out of Memory) event occurs in a cgroup, you can see "oom:" followed by the process that experienced the OOM and its memory usage at the time. + +2. Automate OOM Scenario + In the 'probeCgroup' directory, run './run.sh'. This script will automatically set up an OOM scenario and output the content of '/proc/cgroup_memory_usage_per_process' after execution. + +3. Perform More Tests + a. After compiling the module, in the 'testcases' directory, run './run.py'. + b. This script will perform various tests, including: + - Loading and unloading the module + - Each cgroup containing multiple processes + - Creating multiple cgroups + - OOM scenarios + - Multithreading + c. The tests will take approximately one minute to complete. diff --git a/tools/probeCgroup/probeCgroup.c b/tools/probeCgroup/probeCgroup.c new file mode 100644 index 000000000000..9883cb1e082d --- /dev/null +++ b/tools/probeCgroup/probeCgroup.c @@ -0,0 +1,612 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * probeCgroup.c - A tool used to get memory usage for each process in a cgroup + * + * Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + */ + +#include "probeCgroup.h" + +// kretprobe at mem_cgroup_charge +struct charge_data { + struct cgroup *cgrp; + struct mem_cgroup *memcg; + struct task_struct *task; + unsigned long addr; +}; + +static int mem_cgroup_charge_entry_handler(struct kretprobe_instance *ri, + struct pt_regs *regs) +{ + struct charge_data *data; + struct folio *page; + struct mm_struct *mm; + struct mem_cgroup *memcg; + struct cgroup_subsys_state css; + struct cgroup *cgrp; + + if (!current->mm) + return 1; + page = (struct folio *)regs->di; + mm = (struct mm_struct *)regs->si; + if (mm == NULL || page == NULL) + return -1; + memcg = get_mem_cgroup_from_mm(mm); + if (memcg != NULL) { + css = memcg->css; + cgrp = css.cgroup; + + data = (struct charge_data *)ri->data; + data->memcg = memcg; + data->addr = (unsigned long)page; + data->task = current; + data->cgrp = cgrp; + } + return 0; +} + +NOKPROBE_SYMBOL(mem_cgroup_charge_entry_handler); + +static int mem_cgroup_charge_ret_handler(struct kretprobe_instance *ri, + struct pt_regs *regs) +{ + unsigned long retval = regs_return_value(regs); + struct charge_data *data = (struct charge_data *)ri->data; + int id; + struct cgroup_info *cgrp_info; + struct task_info *tsk_info; + + if (data->memcg != NULL && retval == 0) { + id = ((data->memcg)->css).id; + + spin_lock(&lock); + cgrp_info = find_cgroup_info(id); + if (cgrp_info == NULL) { + cgrp_info = create_cgroup_info(data->cgrp, data->memcg); + if (cgrp_info == NULL) { + spin_unlock(&lock); + return -1; + } + add_cgroup_info(cgrp_info); + } + spin_unlock(&lock); + + read_lock(&cgrp_info->cgrp_lock); + tsk_info = find_task_info(cgrp_info, data->task->tgid); + read_unlock(&cgrp_info->cgrp_lock); + + // for some cases, task->comm changes over time + if (tsk_info != NULL + && strcmp(data->task->comm, tsk_info->comm) != 0) { + strscpy(tsk_info->comm, data->task->comm, + sizeof(tsk_info->comm)); + } + + if (tsk_info == NULL) { + tsk_info = create_task_info(data->task); + if (tsk_info == NULL) + return -1; + add_task_to_cgroup_info(cgrp_info, tsk_info); + } + + if (HashMap_insert(tsk_info->pages, data->addr)) { + //update counter + spin_lock(&(tsk_info->cnt_lock)); + tsk_info->count += + folio_nr_pages((struct folio *)data->addr); + spin_unlock(&(tsk_info->cnt_lock)); + } + } + + return 0; + +} + +NOKPROBE_SYMBOL(mem_cgroup_charge_ret_handler); + +static struct kretprobe mem_cgroup_charge_kretprobe = { + .handler = mem_cgroup_charge_ret_handler, + .entry_handler = mem_cgroup_charge_entry_handler, + .data_size = sizeof(struct charge_data), + .maxactive = 20, +}; + +static int mem_cgroup_charge_kretprobe_init(void) +{ + int ret; + + mem_cgroup_charge_kretprobe.kp.symbol_name = "__mem_cgroup_charge"; + ret = register_kretprobe(&mem_cgroup_charge_kretprobe); + if (ret < 0) { + pr_err("register_kretprobe failed, returned %d\n", ret); + return ret; + } + pr_info("Planted return probe at %s: %p\n", + mem_cgroup_charge_kretprobe.kp.symbol_name, + mem_cgroup_charge_kretprobe.kp.addr); + return 0; +} + +static void mem_cgroup_charge_kretprobe_exit(void) +{ + unregister_kretprobe(&mem_cgroup_charge_kretprobe); + pr_info("kretprobe at %p unregistered\n", + mem_cgroup_charge_kretprobe.kp.addr); + + /* nmissed > 0 suggests that maxactive was set too low. */ + pr_info("Missed probing %d instances of %s\n", + mem_cgroup_charge_kretprobe.nmissed, + mem_cgroup_charge_kretprobe.kp.symbol_name); +} + +// kretprobe at uncharge_folio + +struct uncharge_data { + struct cgroup *cgrp; + struct mem_cgroup *memcg; + unsigned long addr; + bool isKmem; + int nr_pages; +}; + +static int uncharge_folio_entry_handler(struct kretprobe_instance *ri, + struct pt_regs *regs) +{ + struct uncharge_data *data; + struct folio *page; + struct mem_cgroup *memcg = NULL; + struct cgroup_subsys_state css; + struct cgroup *cgrp; + struct obj_cgroup *objcg; + int nr_pages = 0; + + data = (struct uncharge_data *)ri->data; + page = (struct folio *)regs->di; + if (page == NULL) { + data->memcg = NULL; + return -1; + } + if (page->memcg_data & MEMCG_DATA_KMEM) { // if the page belongs to kmem + if (!folio_test_large(page)) + nr_pages = 1; + else + nr_pages = page->_folio_nr_pages; + // nr_pages = thp_nr_pages(page); + objcg = __folio_objcg(page); + if (objcg != NULL) + memcg = objcg->memcg; + data->isKmem = true; + data->nr_pages = nr_pages; + } else { + memcg = __folio_memcg(page); + data->isKmem = false; + } + + if (memcg != NULL) { + css = memcg->css; + cgrp = css.cgroup; + + data->memcg = memcg; + data->addr = (unsigned long)page; + data->cgrp = cgrp; + } + return 0; +} + +NOKPROBE_SYMBOL(uncharge_folio_entry_handler); + +static int uncharge_folio_ret_handler(struct kretprobe_instance *ri, + struct pt_regs *regs) +{ + struct uncharge_data *data = (struct uncharge_data *)ri->data; + int id; + struct cgroup_info *cgrp_info; + int ret = -1; + + if (data->memcg != NULL) { + id = ((data->memcg)->css).id; + cgrp_info = find_cgroup_info(id); + if (cgrp_info == NULL) + return -1; + if (data->isKmem) + ret = -1; + else + ret = remove_page_from_cgroup_info(data->addr, cgrp_info); + } + + return ret; +} + +NOKPROBE_SYMBOL(uncharge_folio_ret_handler); + +static struct kretprobe uncharge_folio_kretprobe = { + .handler = uncharge_folio_ret_handler, + .entry_handler = uncharge_folio_entry_handler, + .data_size = sizeof(struct uncharge_data), + .maxactive = 20, +}; + +static int uncharge_folio_kretprobe_init(void) +{ + int ret; + + uncharge_folio_kretprobe.kp.symbol_name = "uncharge_folio"; + ret = register_kretprobe(&uncharge_folio_kretprobe); + if (ret < 0) { + pr_err("register_kretprobe failed, returned %d\n", ret); + return ret; + } + pr_info("Planted return probe at %s: %p\n", + uncharge_folio_kretprobe.kp.symbol_name, + uncharge_folio_kretprobe.kp.addr); + return 0; +} + +static void uncharge_folio_kretprobe_exit(void) +{ + unregister_kretprobe(&uncharge_folio_kretprobe); + pr_info("kretprobe at %p unregistered\n", + uncharge_folio_kretprobe.kp.addr); + + /* nmissed > 0 suggests that maxactive was set too low. */ + pr_info("Missed probing %d instances of %s\n", + uncharge_folio_kretprobe.nmissed, + uncharge_folio_kretprobe.kp.symbol_name); +} + +//kprobe at do_exit +static struct kprobe do_exit_kprobe; +static int do_exit_kprobe_pre_handler(struct kprobe *p, struct pt_regs *regs) +{ + struct task_struct *cur = current; + int tgid = cur->tgid; + struct mm_struct *mm = cur->mm; + struct mem_cgroup *memcg = get_mem_cgroup_from_mm(mm); + struct cgroup_subsys_state css; + struct cgroup *cgrp; + struct cgroup_info *cgrp_info; + struct task_info *tsk_info; + int id; + + if (memcg != NULL) { + css = memcg->css; + cgrp = css.cgroup; + id = (memcg->css).id; + cgrp_info = find_cgroup_info(id); + if (cgrp_info != NULL) { + write_lock(&cgrp_info->cgrp_lock); + tsk_info = find_task_info(cgrp_info, tgid); + if (tsk_info != NULL) { + list_del(&tsk_info->list); + write_unlock(&cgrp_info->cgrp_lock); + remove_task_from_cgroup_info(cgrp_info, + tsk_info); + } else { + write_unlock(&cgrp_info->cgrp_lock); + } + return 0; + } + } + return 0; +} + +static void do_exit_kprobe_post_handler(struct kprobe *p, + struct pt_regs *regs, + unsigned long flags) +{ + +} + +static int do_exit_kprobe_init(void) +{ + do_exit_kprobe.pre_handler = do_exit_kprobe_pre_handler; + do_exit_kprobe.post_handler = do_exit_kprobe_post_handler; + do_exit_kprobe.symbol_name = "do_exit"; + if (register_kprobe(&do_exit_kprobe)) { + pr_alert("register_kprobe on do_exit failed!\n"); + return -EINVAL; + } + return 0; +} + +static void do_exit_kprobe_exit(void) +{ + unregister_kprobe(&do_exit_kprobe); +} + +//kprobe at mark_oom_victim +static struct kprobe mark_oom_victim_kprobe; + +static int mark_oom_victim_kprobe_pre_handler(struct kprobe *p, + struct pt_regs *regs) +{ + struct task_struct *victim; + int tgid; + struct mm_struct *mm; + struct mem_cgroup *memcg; + struct cgroup_subsys_state css; + struct cgroup *cgrp; + struct cgroup_info *cgrp_info; + struct task_info *tsk_info; + int id; + struct task_info *oom_info; + + victim = (struct task_struct *)regs->di; + tgid = victim->tgid; + mm = victim->mm; + memcg = get_mem_cgroup_from_mm(mm); + if (memcg != NULL) { + css = memcg->css; + cgrp = css.cgroup; + id = (memcg->css).id; + cgrp_info = find_cgroup_info(id); + if (cgrp_info != NULL) { + read_lock(&cgrp_info->cgrp_lock); + tsk_info = find_task_info(cgrp_info, tgid); + read_unlock(&cgrp_info->cgrp_lock); + if (tsk_info != NULL) { + oom_info = create_oom_task_info(tsk_info); + if (oom_info != NULL) { + add_oom_task_to_cgroup_info(cgrp_info, + oom_info); + } + return 0; + } + } + } + return 0; +} + +static void mark_oom_victim_kprobe_post_handler(struct kprobe *p, + struct pt_regs *regs, + unsigned long flags) +{ + +} + +static int mark_oom_victim_kprobe_init(void) +{ + mark_oom_victim_kprobe.pre_handler = mark_oom_victim_kprobe_pre_handler; + mark_oom_victim_kprobe.post_handler = + mark_oom_victim_kprobe_post_handler; + mark_oom_victim_kprobe.symbol_name = "mark_oom_victim"; + if (register_kprobe(&mark_oom_victim_kprobe)) { + pr_alert("register_kprobe on mark_oom_victim failed!\n"); + return -EINVAL; + } + return 0; +} + +static void mark_oom_victim_kprobe_exit(void) +{ + unregister_kprobe(&mark_oom_victim_kprobe); +} + +//kretporbe at cgroup_destroy_locked +struct destroy_data { + struct cgroup *cgrp; +}; + +static int cgroup_destroy_locked_entry_handler(struct kretprobe_instance + *ri, struct pt_regs *regs) +{ + struct destroy_data *data; + + data = (struct destroy_data *)ri->data; + data->cgrp = (struct cgroup *)regs->di; + return 0; +} + +NOKPROBE_SYMBOL(cgroup_destroy_locked_entry_handler); + +static int cgroup_destroy_locked_ret_handler(struct kretprobe_instance *ri, + struct pt_regs *regs) +{ + struct destroy_data *data = (struct destroy_data *)ri->data; + struct cgroup *cgrp = data->cgrp; + struct cgroup_info *cgrp_info = NULL; + unsigned long retval = regs_return_value(regs); + + if (!cgrp) + return -1; + if (retval != 0) + return -1; + list_for_each_entry(cgrp_info, &all_cgroup_info, list) { + if (cgrp_info->cgrp == cgrp) { + spin_lock(&lock); + list_del(&cgrp_info->list); + spin_unlock(&lock); + destroy_cgroup_info(cgrp_info); + return 0; + } + } + return -1; +} + +NOKPROBE_SYMBOL(cgroup_destroy_locked_ret_handler); + +static struct kretprobe cgroup_destroy_locked_kretprobe = { + .handler = cgroup_destroy_locked_ret_handler, + .entry_handler = cgroup_destroy_locked_entry_handler, + .data_size = sizeof(struct destroy_data), + .maxactive = 20, +}; + +static int cgroup_destroy_locked_kretprobe_init(void) +{ + int ret; + + cgroup_destroy_locked_kretprobe.kp.symbol_name = + "cgroup_destroy_locked"; + ret = register_kretprobe(&cgroup_destroy_locked_kretprobe); + if (ret < 0) { + pr_err("register_kretprobe failed, returned %d\n", ret); + return ret; + } + pr_info("Planted return probe at %s: %p\n", + cgroup_destroy_locked_kretprobe.kp.symbol_name, + cgroup_destroy_locked_kretprobe.kp.addr); + return 0; +} + +static void cgroup_destroy_locked_kretprobe_exit(void) +{ + unregister_kretprobe(&cgroup_destroy_locked_kretprobe); + pr_info("kretprobe at %p unregistered\n", + cgroup_destroy_locked_kretprobe.kp.addr); + + /* nmissed > 0 suggests that maxactive was set too low. */ + pr_info("Missed probing %d instances of %s\n", + cgroup_destroy_locked_kretprobe.nmissed, + cgroup_destroy_locked_kretprobe.kp.symbol_name); +} + +// print the tasks in order of their memory usage +static void print_sorted_tasks_list(struct cgroup_info *cgrp_info, + int type, struct seq_file *m) +{ + struct list_head *cur, *insert_pos; + struct task_info *task, *insert_task; + struct list_head new_list = LIST_HEAD_INIT(new_list); + struct list_head *old_list; + struct task_info *new_task, *next_task; + + if (type == 0) { + if (cgrp_info == NULL) + return; + read_lock(&cgrp_info->cgrp_lock); + old_list = &cgrp_info->tasks_list; + } else { + if (cgrp_info == NULL) + return; + old_list = &cgrp_info->oom_list; + } + + list_for_each_entry_safe(task, insert_task, old_list, list) { + new_task = kmalloc(sizeof(struct task_info), GFP_ATOMIC); + if (!new_task) + return; + new_task->tgid = task->tgid; + strscpy(new_task->comm, task->comm, sizeof(new_task->comm)); + new_task->count = task->count; + new_task->pages = NULL; + INIT_LIST_HEAD(&new_task->list); + + //insertion sort + cur = &new_list; + insert_pos = cur->next; + while (insert_pos != &new_list) { + next_task = + list_entry(insert_pos, struct task_info, list); + if (new_task->count >= next_task->count) + break; + cur = insert_pos; + insert_pos = insert_pos->next; + } + + (&new_task->list)->prev = insert_pos->prev; + (insert_pos->prev)->next = (&new_task->list); + (&new_task->list)->next = insert_pos; + insert_pos->prev = (&new_task->list); + } + if (type == 0) + read_unlock(&cgrp_info->cgrp_lock); + + //print + if (type == 1 && (&new_list) != new_list.next) { + seq_puts(m, "oom:\n"); + seq_printf(m, "%10s %20s %20s\n", "pid", "command", + "memory usage (KB)"); + } + if (type == 0) + seq_printf(m, "%10s %20s %20s\n", "pid", "command", + "memory usage (KB)"); + list_for_each_entry_safe(task, insert_task, &new_list, list) { + seq_printf(m, "%10d %20s %20d\n", task->tgid, task->comm, + (task->count) * 4); + } + + list_for_each_entry_safe(task, insert_task, &new_list, list) { + list_del(&task->list); + kfree(task); + } +} + +static struct proc_dir_entry *cgroup_info_read; +#define procfs_file_read "cgroup_memory_usage_per_process" + +void seq_print_tasks(struct cgroup_info *cgroup_info, struct seq_file *m) +{ + if (!cgroup_info) + return; + + print_sorted_tasks_list(cgroup_info, 0, m); +} + +void seq_print_oom_tasks(struct cgroup_info *cgroup_info, struct seq_file *m) +{ + if (!cgroup_info) + return; + + print_sorted_tasks_list(cgroup_info, 1, m); +} + +void seq_print_cgroups(struct seq_file *m) +{ + struct cgroup_info *cgrp, *pos; + + spin_lock(&lock); + list_for_each_entry_safe(cgrp, pos, &all_cgroup_info, list) { + seq_printf(m, "cgroup name : %s\n", cgrp->name); + seq_print_tasks(cgrp, m); + seq_print_oom_tasks(cgrp, m); + seq_puts(m, "\n"); + } + spin_unlock(&lock); +} + +static int memory_usage_show(struct seq_file *m, void *v) +{ + seq_print_cgroups(m); + return 0; +} + +static int __init global_init(void) +{ + int ret = 0; + + cgroup_info_read = + proc_create_single(procfs_file_read, 0, NULL, memory_usage_show); + if (!cgroup_info_read) + return -ENOMEM; + ret = mem_cgroup_charge_kretprobe_init(); + uncharge_folio_kretprobe_init(); + do_exit_kprobe_init(); + mark_oom_victim_kprobe_init(); + cgroup_destroy_locked_kretprobe_init(); + + return ret; +} + +static void __exit global_exit(void) +{ + struct cgroup_info *cgrp_info, *pos; + + mem_cgroup_charge_kretprobe_exit(); + uncharge_folio_kretprobe_exit(); + do_exit_kprobe_exit(); + mark_oom_victim_kprobe_exit(); + cgroup_destroy_locked_kretprobe_exit(); + + remove_proc_entry(procfs_file_read, NULL); + + //release all memory use + list_for_each_entry_safe(cgrp_info, pos, &all_cgroup_info, list) { + list_del(&cgrp_info->list); + destroy_cgroup_info(cgrp_info); + } +} + +module_init(global_init) +module_exit(global_exit) +MODULE_LICENSE("GPL"); diff --git a/tools/probeCgroup/probeCgroup.h b/tools/probeCgroup/probeCgroup.h new file mode 100644 index 000000000000..953a6e0aca31 --- /dev/null +++ b/tools/probeCgroup/probeCgroup.h @@ -0,0 +1,415 @@ +/* SPDX-License-Identifier: GPL-2.0*/ +/* + * probeCgroup.h + * + * Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + */ + +#include <linux/kernel.h> +#include <linux/module.h> +#include <linux/kprobes.h> +#include <linux/ktime.h> +#include <linux/limits.h> +#include <linux/sched.h> +#include <linux/mm_types.h> +#include <linux/memcontrol.h> +#include <linux/cgroup-defs.h> +#include <linux/kernfs.h> +#include <linux/string.h> +#include <linux/list.h> +#include <linux/oom.h> +#include <linux/fs.h> +#include <linux/proc_fs.h> +#include <linux/huge_mm.h> +#include <linux/page-flags.h> +#include <linux/spinlock.h> +#include <linux/rwlock.h> + +static spinlock_t lock; // global lock for the list of cgroup_info + +struct HashNode { + unsigned long addr; + struct HashNode *next; +}; + +struct HashNode *HashNode_create(unsigned long addr) +{ + struct HashNode *node = NULL; + + node = kzalloc(sizeof(struct HashNode), GFP_ATOMIC); + if (node == NULL) + return NULL; + node->addr = addr; + node->next = NULL; + return node; +} + +struct HashBucket { + struct HashNode *head; + spinlock_t bkt_lock; +}; + +void HashBucket_init(struct HashBucket *bkt) +{ + bkt->head = NULL; + spin_lock_init(&bkt->bkt_lock); +} + +bool HashBucket_insert(struct HashBucket *bkt, unsigned long addr) +{ + struct HashNode *new_node; + struct HashNode *node; + struct HashNode *prev; + bool ret = true; + + if (bkt == NULL) + return false; + + prev = NULL; + new_node = NULL; + new_node = HashNode_create(addr); + + spin_lock(&bkt->bkt_lock); + node = bkt->head; + while (node != NULL && node->addr != addr) { + prev = node; + node = node->next; + } + if (node == NULL) { + if (new_node == NULL) { + pr_info("not enough memory for HashNode\n"); + spin_unlock(&bkt->bkt_lock); + return false; + } + if (bkt->head == NULL) + bkt->head = new_node; + else + prev->next = new_node; + spin_unlock(&bkt->bkt_lock); + ret = true; + } else { + spin_unlock(&bkt->bkt_lock); + kfree(new_node); + ret = false; + } + + return ret; +} + +bool HashBucket_erase(struct HashBucket *bkt, unsigned long addr) +{ + struct HashNode *node; + struct HashNode *prev; + bool ret = true; + + if (bkt == NULL) + return false; + + spin_lock(&bkt->bkt_lock); + node = bkt->head; + prev = NULL; + while (node != NULL && node->addr != addr) { + prev = node; + node = node->next; + } + if (node == NULL) { + spin_unlock(&bkt->bkt_lock); + ret = false; + } else { + if (bkt->head == node) + bkt->head = node->next; + else + prev->next = node->next; + kfree(node); + spin_unlock(&bkt->bkt_lock); + ret = true; + } + + return ret; +} + +void HashBucket_clear(struct HashBucket *bkt) +{ + struct HashNode *node; + struct HashNode *prev; + + if (bkt == NULL) + return; + + spin_lock(&bkt->bkt_lock); + node = bkt->head; + prev = NULL; + bkt->head = NULL; + while (node != NULL) { + prev = node; + node = node->next; + kfree(prev); + } + spin_unlock(&bkt->bkt_lock); +} + +struct HashMap { + unsigned long size; + struct HashBucket *HashTable; +}; + +unsigned long hash_func(unsigned long addr, unsigned long size) +{ + return addr % size; +} + +struct HashMap *HashMap_create(unsigned long size) +{ + struct HashMap *hm = NULL; + struct HashBucket *ht = NULL; + int i = 0; + + hm = kmalloc(sizeof(struct HashMap), GFP_ATOMIC); + if (hm == NULL) + return NULL; + ht = kmalloc((size * sizeof(struct HashBucket)), GFP_ATOMIC); + if (ht == NULL) { + kfree(hm); + return NULL; + } + for (i = 0; i < size; i++) + HashBucket_init(&(ht[i])); + + hm->size = size; + hm->HashTable = ht; + return hm; +} + +bool HashMap_insert(struct HashMap *hm, unsigned long addr) +{ + unsigned long index; + + if (hm == NULL) + return false; + index = hash_func(addr, hm->size); + if (hm->HashTable == NULL) + return false; + return HashBucket_insert(&(hm->HashTable[index]), addr); +} + +bool HashMap_erase(struct HashMap *hm, unsigned long addr) +{ + unsigned long index; + + if (hm == NULL) + return false; + index = hash_func(addr, hm->size); + if (hm->HashTable == NULL) + return false; + return HashBucket_erase(&(hm->HashTable[index]), addr); +} + +void HashMap_clear(struct HashMap *hm) +{ + unsigned long size; + struct HashBucket *ht; + int i; + + if (hm == NULL) + return; + size = hm->size; + ht = hm->HashTable; + if (ht == NULL) + return; + hm->HashTable = NULL; + for (i = 0; i < size; i++) + HashBucket_clear(&(ht[i])); + + kfree(ht); + kfree(hm); +} + +//struct that save the information for each task +struct task_info { + int tgid; + char comm[TASK_COMM_LEN]; + int count; // number of pages + struct HashMap *pages; + struct list_head list; + spinlock_t cnt_lock; +}; + +// struct that save the information for each cgroup +struct cgroup_info { + struct cgroup *cgrp; + struct mem_cgroup *memcg; + int id; + char name[64]; + struct list_head list; + struct list_head tasks_list; + struct list_head oom_list; + rwlock_t cgrp_lock; + unsigned int cached_bytes; +}; + +static LIST_HEAD(all_cgroup_info); // a list that linked all the cgroup_info struct + +static struct task_info *create_task_info(struct task_struct *cur_task) +{ + struct task_info *tsk_info = + kmalloc(sizeof(struct task_info), GFP_ATOMIC); + if (!tsk_info) + return NULL; + + // initialization + tsk_info->tgid = cur_task->tgid; + strscpy(tsk_info->comm, cur_task->comm, sizeof(tsk_info->comm)); + tsk_info->count = 0; + tsk_info->pages = NULL; + tsk_info->pages = HashMap_create(1023); + INIT_LIST_HEAD(&tsk_info->list); + spin_lock_init(&(tsk_info->cnt_lock)); + + return tsk_info; +} + +static int +add_task_to_cgroup_info(struct cgroup_info *cgrp, struct task_info *task) +{ + if (!cgrp || !task) + return -EINVAL; + + write_lock(&cgrp->cgrp_lock); + list_add_tail(&task->list, &cgrp->tasks_list); + write_unlock(&cgrp->cgrp_lock); + return 0; +} + +static int +remove_task_from_cgroup_info(struct cgroup_info *cgrp, struct task_info *task) +{ + if (cgrp == NULL || task == NULL) + return -EINVAL; + + HashMap_clear(task->pages); + // kfree(task->pages); + kfree(task); + return 0; +} + +static struct task_info *find_task_info(struct cgroup_info *cgrp, int tgid) +{ + struct task_info *tsk_info, *pos; + + list_for_each_entry_safe(tsk_info, pos, &cgrp->tasks_list, list) { + if (tsk_info->tgid == tgid) + return tsk_info; + } + return NULL; +} + +static int +remove_page_from_cgroup_info(unsigned long addr, struct cgroup_info *cgrp) +{ + struct task_info *tsk_info, *pos; + + read_lock(&cgrp->cgrp_lock); + list_for_each_entry_safe(tsk_info, pos, &cgrp->tasks_list, list) { + if (HashMap_erase(tsk_info->pages, addr)) { + spin_lock(&(tsk_info->cnt_lock)); + tsk_info->count -= folio_nr_pages((struct folio *)addr); + spin_unlock(&(tsk_info->cnt_lock)); + read_unlock(&cgrp->cgrp_lock); + return 0; + } + } + read_unlock(&cgrp->cgrp_lock); + return -1; +} + +static struct cgroup_info *create_cgroup_info(struct cgroup *cgrp, + struct mem_cgroup *memcg) +{ + struct cgroup_info *cgrp_info = + kmalloc(sizeof(struct cgroup_info), GFP_ATOMIC); + struct kernfs_node *kn; + + if (!cgrp_info) + return NULL; + + cgrp_info->cgrp = cgrp; + cgrp_info->memcg = memcg; + cgrp_info->id = (memcg->css).id; + kn = cgrp->kn; + strscpy(cgrp_info->name, kn->name, sizeof(cgrp_info->name)); + INIT_LIST_HEAD(&cgrp_info->list); + INIT_LIST_HEAD(&cgrp_info->tasks_list); + INIT_LIST_HEAD(&cgrp_info->oom_list); + rwlock_init(&(cgrp_info->cgrp_lock)); + cgrp_info->cached_bytes = 0; + + return cgrp_info; +} + +static void destroy_cgroup_info(struct cgroup_info *cgrp_info) +{ + struct task_info *task, *tmp; + + if (!cgrp_info) + return; + + write_lock(&cgrp_info->cgrp_lock); + list_for_each_entry_safe(task, tmp, &cgrp_info->tasks_list, list) { + list_del(&task->list); + remove_task_from_cgroup_info(cgrp_info, task); + } + write_unlock(&cgrp_info->cgrp_lock); + list_for_each_entry_safe(task, tmp, &cgrp_info->oom_list, list) { + list_del(&task->list); + remove_task_from_cgroup_info(cgrp_info, task); + } + + kfree(cgrp_info); +} + +static int add_cgroup_info(struct cgroup_info *cgrp_info) +{ + if (!cgrp_info) + return -EINVAL; + + list_add_tail(&cgrp_info->list, &all_cgroup_info); + return 0; +} + +static struct cgroup_info *find_cgroup_info(int id) +{ + struct cgroup_info *cgrp_info = NULL; + + list_for_each_entry(cgrp_info, &all_cgroup_info, list) { + if (cgrp_info->id == id) + return cgrp_info; + } + + return NULL; +} + +static struct task_info *create_oom_task_info(struct task_info *tsk_info) +{ + struct task_info *oom_tsk_info = + kmalloc(sizeof(struct task_info), GFP_ATOMIC); + if (!oom_tsk_info) + return NULL; + + oom_tsk_info->tgid = tsk_info->tgid; + strscpy(oom_tsk_info->comm, tsk_info->comm, sizeof(oom_tsk_info->comm)); + oom_tsk_info->count = tsk_info->count; + oom_tsk_info->pages = NULL; + INIT_LIST_HEAD(&oom_tsk_info->list); + + return oom_tsk_info; +} + +static int +add_oom_task_to_cgroup_info(struct cgroup_info *cgrp, + struct task_info *oom_task) +{ + if (!cgrp || !oom_task) + return -EINVAL; + list_add_tail(&oom_task->list, &cgrp->oom_list); + return 0; +} diff --git a/tools/probeCgroup/run.sh b/tools/probeCgroup/run.sh new file mode 100755 index 000000000000..7e1ffefe66d1 --- /dev/null +++ b/tools/probeCgroup/run.sh @@ -0,0 +1,8 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +cd scripts +./script1.sh +./script2.sh +./script3.sh diff --git a/tools/probeCgroup/scripts/script1.sh b/tools/probeCgroup/scripts/script1.sh new file mode 100755 index 000000000000..539e8258afb9 --- /dev/null +++ b/tools/probeCgroup/scripts/script1.sh @@ -0,0 +1,10 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +cd .. +make +insmod probeCgroup.ko + +cd testcases +gcc simple-mem-allocate.c -o simple-mem-allocate diff --git a/tools/probeCgroup/scripts/script2.sh b/tools/probeCgroup/scripts/script2.sh new file mode 100755 index 000000000000..2ad515cfb912 --- /dev/null +++ b/tools/probeCgroup/scripts/script2.sh @@ -0,0 +1,14 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +current_dir=$(pwd) +cd /sys/fs/cgroup/memory +mkdir test +cd test +sh -c "echo $$ >> cgroup.procs" +sh -c "echo 5M > memory.limit_in_bytes" +sh -c "echo 0 > memory.swappiness" +cd "$current_dir" +cd ../testcases +./simple-mem-allocate diff --git a/tools/probeCgroup/scripts/script3.sh b/tools/probeCgroup/scripts/script3.sh new file mode 100755 index 000000000000..127eb45de5c9 --- /dev/null +++ b/tools/probeCgroup/scripts/script3.sh @@ -0,0 +1,11 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +cd /proc +cat cgroup_memory_usage_per_process + +cat /sys/fs/cgroup/memory/test/cgroup.procs > /sys/fs/cgroup/memory/cgroup.procs +rmdir /sys/fs/cgroup/memory/test +# cat cgroup_memory_usage_per_process +rmmod probeCgroup diff --git a/tools/probeCgroup/testcases/1_load_unload_test.py b/tools/probeCgroup/testcases/1_load_unload_test.py new file mode 100755 index 000000000000..5389a14a1dac --- /dev/null +++ b/tools/probeCgroup/testcases/1_load_unload_test.py @@ -0,0 +1,24 @@ +#!/usr/bin/env python +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +import os +import subprocess +import time + +def test_module_load_unload(): + try: + subprocess.check_call(['insmod', '../probeCgroup.ko']) + time.sleep(1) + print('loading module successfully!') + subprocess.check_call(['rmmod', 'probeCgroup']) + output = subprocess.check_output(['lsmod']) + assert b'probeCgroup' not in output + print('unloading module successfully!') + except subprocess.CalledProcessError as e: + print('Load unload test failed. Insmod failed.') + except AssertionError as e: + print('Load unload test failed. Cannot remove module.') + +if __name__ == '__main__': + test_module_load_unload() \ No newline at end of file diff --git a/tools/probeCgroup/testcases/2_multiple_process_test.py b/tools/probeCgroup/testcases/2_multiple_process_test.py new file mode 100755 index 000000000000..d88c7f7f2952 --- /dev/null +++ b/tools/probeCgroup/testcases/2_multiple_process_test.py @@ -0,0 +1,48 @@ +#!/usr/bin/env python +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +from cgroup_utils import create_cgroup, add_process_to_cgroup, get_process_memory_usage, remove_cgroup, check_memory_usage, cleanup, check_kmem_usage +import os +import subprocess +import time + +def test_multiple_process(num_procs): + subprocess.check_call(['insmod', '../probeCgroup.ko']) + time.sleep(1) + + cgroup_name = 'test' + cgroup_path = create_cgroup(cgroup_name) + + processes = [] + pids = [] + + for i in range(num_procs): + process = subprocess.Popen(['./mem-allocate']) + pid = process.pid + add_process_to_cgroup(cgroup_path, pid) + processes.append(process) + pids.append(pid) + + time.sleep(0.1) + try: + count = 0 + for i in range (2000): + count += check_memory_usage(cgroup_name, pids, False) + time.sleep(0.01) + assert count <= 50, f"Memory read by probeCgroup is not accurate" + + remove_cgroup(cgroup_path, pids) + check_memory_usage(cgroup_name, pids, True) + cleanup(processes) + subprocess.check_call(['rmmod', 'probeCgroup']) + + print('pass multiple process test!') + except AssertionError as e: + print(f"Assertion failed: {e}") + remove_cgroup(cgroup_path, pids) + cleanup(processes) + subprocess.check_call(['rmmod', 'probeCgroup']) + +if __name__ == '__main__': + test_multiple_process(3) \ No newline at end of file diff --git a/tools/probeCgroup/testcases/3_multiple_cgroup_test.py b/tools/probeCgroup/testcases/3_multiple_cgroup_test.py new file mode 100755 index 000000000000..592a716df877 --- /dev/null +++ b/tools/probeCgroup/testcases/3_multiple_cgroup_test.py @@ -0,0 +1,55 @@ +#!/usr/bin/env python +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +from cgroup_utils import create_cgroup, add_process_to_cgroup, get_process_memory_usage, remove_cgroup, check_memory_usage, cleanup +import os +import subprocess +import time + +def test_multiple_cgroup(num_procs, num_cgroups): + subprocess.check_call(['insmod', '../probeCgroup.ko']) + time.sleep(1) + + cgroups = [] + processes = {} + pids = {} + for i in range(num_cgroups): + cgroup_name = f'test_{i}' + cgroup_path = create_cgroup(cgroup_name) + cgroups.append((cgroup_name, cgroup_path)) + + for j in range(num_procs): + process = subprocess.Popen(['./mem-allocate']) + pid = process.pid + add_process_to_cgroup(cgroup_path, pid) + if cgroup_path not in processes: + processes[cgroup_path] = [] + processes[cgroup_path].append(process) + if cgroup_path not in pids: + pids[cgroup_path] = [] + pids[cgroup_path].append(pid) + + time.sleep(0.1) + try: + for i in range (100): + for cgroup_name, cgroup_path in cgroups: + check_memory_usage(cgroup_name, pids[cgroup_path], False) + time.sleep(0.01) + + for cgroup_name, cgroup_path in cgroups: + remove_cgroup(cgroup_path, pids[cgroup_path]) + check_memory_usage(cgroup_name, pids[cgroup_path], True) + cleanup(processes[cgroup_path]) + subprocess.check_call(['rmmod', 'probeCgroup']) + + print('pass multiple cgroup test!') + except AssertionError as e: + print(f"Assertion failed: {e}") + for cgroup_name, cgroup_path in cgroups: + remove_cgroup(cgroup_path, pids[cgroup_path]) + cleanup(processes[cgroup_path]) + subprocess.check_call(['rmmod', 'probeCgroup']) + +if __name__ == '__main__': + test_multiple_cgroup(2,2) \ No newline at end of file diff --git a/tools/probeCgroup/testcases/4_oom_test.py b/tools/probeCgroup/testcases/4_oom_test.py new file mode 100755 index 000000000000..128a258c56f5 --- /dev/null +++ b/tools/probeCgroup/testcases/4_oom_test.py @@ -0,0 +1,52 @@ +#!/usr/bin/env python +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +from cgroup_utils import create_cgroup, add_process_to_cgroup, get_process_memory_usage, remove_cgroup, check_memory_usage, cleanup, get_oom_process_memory_usage +import os +import subprocess +import time + +def test_oom(num_procs): + subprocess.check_call(['insmod', '../probeCgroup.ko']) + time.sleep(1) + + cgroup_name = 'test' + cgroup_path = create_cgroup(cgroup_name) + + with open(f"/sys/fs/cgroup/memory/{cgroup_name}/memory.limit_in_bytes", 'w') as limit_file: + limit_file.write("5M") + with open(f"/sys/fs/cgroup/memory/{cgroup_name}/memory.swappiness", 'w') as swap_file: + swap_file.write("0") + + processes = [] + pids = [] + for i in range(num_procs): + process = subprocess.Popen(['./simple-mem-allocate']) + pid = process.pid + add_process_to_cgroup(cgroup_path, pid) + processes.append(process) + pids.append(pid) + + time.sleep(6) + + try: + for pid in pids: + memory_usage = get_oom_process_memory_usage(pid, cgroup_name) + assert memory_usage is not None, f"Memory usage(oom) not found for PID {pid}" + assert memory_usage > 0, f"Memory usage should be greater than zero for PID {pid}" + + remove_cgroup(cgroup_path, pids) + check_memory_usage(cgroup_name, pids, True) + cleanup(processes) + subprocess.check_call(['rmmod', 'probeCgroup']) + + print('pass oom test!') + except AssertionError as e: + print(f"Assertion failed: {e}") + remove_cgroup(cgroup_path, pids) + cleanup(processes) + subprocess.check_call(['rmmod', 'probeCgroup']) + +if __name__ == '__main__': + test_oom(1) \ No newline at end of file diff --git a/tools/probeCgroup/testcases/5_multiple_threads_test.py b/tools/probeCgroup/testcases/5_multiple_threads_test.py new file mode 100755 index 000000000000..7e1b86dabe48 --- /dev/null +++ b/tools/probeCgroup/testcases/5_multiple_threads_test.py @@ -0,0 +1,45 @@ +#!/usr/bin/env python +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +from cgroup_utils import create_cgroup, add_process_to_cgroup, get_process_memory_usage, remove_cgroup, check_memory_usage, cleanup, check_kmem_usage +import os +import subprocess +import time + +def test_multiple_thread(num_procs): + subprocess.check_call(['insmod', '../probeCgroup.ko']) + time.sleep(1) + + cgroup_name = 'test' + cgroup_path = create_cgroup(cgroup_name) + + processes = [] + pids = [] + for i in range(num_procs): + process = subprocess.Popen(['./multiple-thread-mem-allocate']) + pid = process.pid + add_process_to_cgroup(cgroup_path, pid) + processes.append(process) + pids.append(pid) + + time.sleep(1) + try: + for i in range (200): + check_memory_usage(cgroup_name, pids, False) + time.sleep(0.01) + + remove_cgroup(cgroup_path, pids) + check_memory_usage(cgroup_name, pids, True) + cleanup(processes) + subprocess.check_call(['rmmod', 'probeCgroup']) + + print('pass multiple threads test!') + except AssertionError as e: + print(f"Assertion failed: {e}") + remove_cgroup(cgroup_path, pids) + cleanup(processes) + subprocess.check_call(['rmmod', 'probeCgroup']) + +if __name__ == '__main__': + test_multiple_thread(5) \ No newline at end of file diff --git a/tools/probeCgroup/testcases/cgroup_utils.py b/tools/probeCgroup/testcases/cgroup_utils.py new file mode 100644 index 000000000000..f70c68c1f188 --- /dev/null +++ b/tools/probeCgroup/testcases/cgroup_utils.py @@ -0,0 +1,115 @@ +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +import os +import subprocess +import time + +def create_cgroup(cgroup_name): + cgroup_path = f'/sys/fs/cgroup/memory/{cgroup_name}' + + try: + os.makedirs(cgroup_path) + except FileExistsError: + pass + + return cgroup_path + +def add_process_to_cgroup(cgroup_path, pid): + with open(os.path.join(cgroup_path, 'cgroup.procs'), 'w') as procs_file: + procs_file.write(str(pid)) + +def get_process_memory_usage(pid, cgroup_name): + cur_name = '' + with open('/proc/cgroup_memory_usage_per_process', 'r') as file: + for line in file: + parts = line.strip().split() + if len(parts) >= 4 and parts[0] == 'cgroup': + cur_name = parts[3] + if len(parts) >= 3 and parts[0] != 'cgroup' and cur_name == cgroup_name and parts[0] != 'pid' and int(parts[0]) == pid: + return int(parts[2]) + return None + +def get_process_kmem_usage(pid, cgroup_name): + cur_name = '' + with open('/proc/cgroup_memory_usage_per_process', 'r') as file: + for line in file: + parts = line.strip().split() + if len(parts) >= 4 and parts[0] == 'cgroup': + cur_name = parts[3] + if len(parts) >= 4 and parts[0] != 'cgroup' and cur_name == cgroup_name and parts[0] != 'pid' and int(parts[0]) == pid: + return int(parts[3]) + return None + +def remove_cgroup(cgroup_path, pids): + for pid in pids: + with open('/sys/fs/cgroup/memory/cgroup.procs', 'w') as backup_file: + backup_file.write(str(pid)) + os.rmdir(cgroup_path) + return + +def check_memory_usage(cgroup_name, pids, delete): + memory_sum = 0 + for pid in pids: + memory_usage = get_process_memory_usage(pid, cgroup_name) + if delete == False: + assert memory_usage is not None, f"Memory usage not found for PID {pid}" + assert memory_usage >= 0, f"Memory usage should be greater than zero for PID {pid}" + memory_sum += memory_usage + else: + assert memory_usage is None, f"Error: Memory usage should not be available for PID {pid} after deleting the cgroup." + if delete == False: + with open(f"/sys/fs/cgroup/memory/{cgroup_name}/memory.usage_in_bytes", 'r') as file: + content = file.readline().strip() + memory_read = int(content) + memory_sum *= 1024 + delta = abs(memory_read - memory_sum) + # print(f"read: {memory_read}") + # print(f"sum : {memory_sum}") + if (delta > max(memory_read, memory_sum) * 0.1): + return 1 + else: + return 0 + else: + return 0 + +def check_kmem_usage(cgroup_name, pids, delete): + kmem_sum = 0 + for pid in pids: + kmem_usage = get_process_kmem_usage(pid, cgroup_name) + if delete == False: + assert kmem_usage is not None, f"Kmem usage not found for PID {pid}" + assert kmem_usage >= 0, f"Kmem usage should be greater than zero for PID {pid}" + kmem_sum += kmem_usage + else: + assert kmem_usage is None, f"Error: Kmem usage should not be available for PID {pid} after deleting the cgroup." + if delete == False: + with open(f"/sys/fs/cgroup/memory/{cgroup_name}/memory.kmem.usage_in_bytes", 'r') as file: + content = file.readline().strip() + kmem_read = int(content) + kmem_sum *= 1024 + delta = abs(kmem_read - kmem_sum) + # print(f"kmem read: {kmem_read}") + # print(f"kmem sum : {kmem_sum}") + # assert delta <= max(kmem_read, kmem_sum) * 0.2, f"Kmem read by probeCgroup is not accurate, {kmem_read}, {kmem_sum}" + +def cleanup(processes): + for process in processes: + process.terminate() + process.wait() + +def get_oom_process_memory_usage(pid, cgroup_name): + # �� + cur_name = '' + oom = False + with open('/proc/cgroup_memory_usage_per_process', 'r') as file: + for line in file: + parts = line.strip().split() + if len(parts) >= 4 and parts[0] == 'cgroup': + cur_name = parts[3] + oom = False + if len(parts) >= 1 and parts[0] == 'oom:': + oom = True + if len(parts) >= 3 and parts[0] != 'cgroup' and cur_name == cgroup_name and parts[0] != 'pid' and int(parts[0]) == pid and oom == True: + return int(parts[2]) + return None \ No newline at end of file diff --git a/tools/probeCgroup/testcases/mem-allocate.c b/tools/probeCgroup/testcases/mem-allocate.c new file mode 100644 index 000000000000..e78e37cae61b --- /dev/null +++ b/tools/probeCgroup/testcases/mem-allocate.c @@ -0,0 +1,35 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * mem-allocate.c - The program to test probeCgroup + * + * Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + */ + +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <unistd.h> + +#define MB (1024 * 1024) + +char *arr[40]; + +int main(int argc, char *argv[]) +{ + char *p; + int i = 0; + + while (1) { + for (i = 0; i < 40; i++) { + p = (char *)malloc(MB); + memset(p, 0, MB); + arr[i] = p; + usleep(100000); + } + for (int i = 0; i < 40; i++) { + free(arr[i]); + usleep(100000); + } + } + return 0; +} diff --git a/tools/probeCgroup/testcases/multiple-thread-mem-allocate.c b/tools/probeCgroup/testcases/multiple-thread-mem-allocate.c new file mode 100644 index 000000000000..55f4c068f55e --- /dev/null +++ b/tools/probeCgroup/testcases/multiple-thread-mem-allocate.c @@ -0,0 +1,60 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * multiple-thread-mem-allocate.c - The program to test probeCgroup + * + * Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + */ + +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <unistd.h> +#include <pthread.h> + +#define MB (1024 * 1024) + +void *memory_test(void *) +{ + char *arr[25]; + char *p; + int i = 0; + int cnt = 0; + + while (1) { + for (i = 0; i < 20; i++) { + p = (char *)malloc(MB); + memset(p, 0, MB); + arr[i] = p; + usleep(10000); + } + for (int i = 0; i < 20; i++) { + free(arr[i]); + usleep(10000); + } + } +} + +int main(int argc, char *argv[]) +{ + pthread_t threads[4]; + int rc; + + // create threads + for (int i = 0; i < 4; i++) { + rc = pthread_create(&threads[i], NULL, memory_test, NULL); + if (rc != 0) { + fprintf(stderr, "Error creating thread: %d\n", rc); + return 1; + } + } + + for (int i = 0; i < 4; i++) { + rc = pthread_join(threads[i], NULL); + if (rc != 0) { + fprintf(stderr, "Error joining thread: %d\n", rc); + return 1; + } + } + + return 0; +} diff --git a/tools/probeCgroup/testcases/run.py b/tools/probeCgroup/testcases/run.py new file mode 100755 index 000000000000..8ffd0ca720d8 --- /dev/null +++ b/tools/probeCgroup/testcases/run.py @@ -0,0 +1,32 @@ +#!/usr/bin/env python +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + +import os +import subprocess +import sys + +def run_tests(directory): + """Run all Python scripts in the given directory.""" + python_files = [f for f in os.listdir(directory) if f.endswith('test.py')] + python_files.sort() + + for filename in python_files: + try: + filepath = os.path.join(directory, filename) + + subprocess.check_call([sys.executable, filepath]) + except subprocess.CalledProcessError as e: + print(f"Error executing {filename}:") + return + except Exception as e: + print(f"Error executing {filename}:") + print(e) + return + +if __name__ == '__main__': + tests_directory = '.' + subprocess.check_call(['gcc', 'mem-allocate.c', '-o', 'mem-allocate']) + subprocess.check_call(['gcc', 'simple-mem-allocate.c', '-o', 'simple-mem-allocate']) + subprocess.check_call(['gcc', 'multiple-thread-mem-allocate.c', '-o', 'multiple-thread-mem-allocate']) + run_tests(tests_directory) \ No newline at end of file diff --git a/tools/probeCgroup/testcases/simple-mem-allocate.c b/tools/probeCgroup/testcases/simple-mem-allocate.c new file mode 100644 index 000000000000..16328b10ba48 --- /dev/null +++ b/tools/probeCgroup/testcases/simple-mem-allocate.c @@ -0,0 +1,27 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * mem-allocate.c - The program to test probeCgroup + * + * Copyright (C) Taoxy2004 <221870066(a)smail.nju.edu.cn> + */ + +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <unistd.h> + +#define MB (1024 * 1024) + +int main(int argc, char *argv[]) +{ + char *p; + int i = 0; + + while (1) { + p = (char *)malloc(MB); + memset(p, 0, MB); + sleep(1); + } + + return 0; +} -- 2.43.0

1 0

[PATCH openEuler-22.03-LTS-SP1 0/2] nfc: pn533: Wait for out_urb's completion in pn533_usb_send_frame()
by Kaixiong Yu 04 Sep '24

04 Sep '24

fix CVE-2023-52907 Fedor Pchelkin (1): nfc: pn533: initialize struct pn533_out_arg properly Minsuk Kang (1): nfc: pn533: Wait for out_urb's completion in pn533_usb_send_frame() drivers/nfc/pn533/usb.c | 45 ++++++++++++++++++++++++++++++++++++++--- 1 file changed, 42 insertions(+), 3 deletions(-) -- 2.25.1

2 3

[PATCH openEuler-1.0-LTS 0/2] nfc: pn533: Wait for out_urb's completion in pn533_usb_send_frame()
by Kaixiong Yu 04 Sep '24

04 Sep '24

fix CVE-2023-52907 Fedor Pchelkin (1): nfc: pn533: initialize struct pn533_out_arg properly Minsuk Kang (1): nfc: pn533: Wait for out_urb's completion in pn533_usb_send_frame() drivers/nfc/pn533/usb.c | 45 ++++++++++++++++++++++++++++++++++++++--- 1 file changed, 42 insertions(+), 3 deletions(-) -- 2.25.1

2 3

[PATCH openEuler-1.0-LTS] md/raid5: avoid BUG_ON() while continue reshape after reassembling
by Li Nan 04 Sep '24

04 Sep '24

From: Yu Kuai <yukuai3(a)huawei.com> stable inclusion from stable-v4.19.320 commit 2c92f8c1c456d556f15cbf51667b385026b2e6a0 category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/IAMNBN CVE: CVE-2024-43914 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id… -------------------------------- [ Upstream commit 305a5170dc5cf3d395bb4c4e9239bca6d0b54b49 ] Currently, mdadm support --revert-reshape to abort the reshape while reassembling, as the test 07revert-grow. However, following BUG_ON() can be triggerred by the test: kernel BUG at drivers/md/raid5.c:6278! invalid opcode: 0000 [#1] PREEMPT SMP PTI irq event stamp: 158985 CPU: 6 PID: 891 Comm: md0_reshape Not tainted 6.9.0-03335-g7592a0b0049a #94 RIP: 0010:reshape_request+0x3f1/0xe60 Call Trace: <TASK> raid5_sync_request+0x43d/0x550 md_do_sync+0xb7a/0x2110 md_thread+0x294/0x2b0 kthread+0x147/0x1c0 ret_from_fork+0x59/0x70 ret_from_fork_asm+0x1a/0x30 </TASK> Root cause is that --revert-reshape update the raid_disks from 5 to 4, while reshape position is still set, and after reassembling the array, reshape position will be read from super block, then during reshape the checking of 'writepos' that is caculated by old reshape position will fail. Fix this panic the easy way first, by converting the BUG_ON() to WARN_ON(), and stop the reshape if checkings fail. Noted that mdadm must fix --revert-shape as well, and probably md/raid should enhance metadata validation as well, however this means reassemble will fail and there must be user tools to fix the wrong metadata. Signed-off-by: Yu Kuai <yukuai3(a)huawei.com> Signed-off-by: Song Liu <song(a)kernel.org> Link: https://lore.kernel.org/r/20240611132251.1967786-13-yukuai1@huaweicloud.com Signed-off-by: Sasha Levin <sashal(a)kernel.org> Signed-off-by: Li Nan <linan122(a)huawei.com> --- drivers/md/raid5.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index 1c2e2ff162dc..b2b35cdabac5 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -5817,7 +5817,9 @@ static sector_t reshape_request(struct mddev *mddev, sector_t sector_nr, int *sk safepos = conf->reshape_safe; sector_div(safepos, data_disks); if (mddev->reshape_backwards) { - BUG_ON(writepos < reshape_sectors); + if (WARN_ON(writepos < reshape_sectors)) + return MaxSector; + writepos -= reshape_sectors; readpos += reshape_sectors; safepos += reshape_sectors; @@ -5835,14 +5837,18 @@ static sector_t reshape_request(struct mddev *mddev, sector_t sector_nr, int *sk * to set 'stripe_addr' which is where we will write to. */ if (mddev->reshape_backwards) { - BUG_ON(conf->reshape_progress == 0); + if (WARN_ON(conf->reshape_progress == 0)) + return MaxSector; + stripe_addr = writepos; - BUG_ON((mddev->dev_sectors & - ~((sector_t)reshape_sectors - 1)) - - reshape_sectors - stripe_addr - != sector_nr); + if (WARN_ON((mddev->dev_sectors & + ~((sector_t)reshape_sectors - 1)) - + reshape_sectors - stripe_addr != sector_nr)) + return MaxSector; } else { - BUG_ON(writepos != sector_nr + reshape_sectors); + if (WARN_ON(writepos != sector_nr + reshape_sectors)) + return MaxSector; + stripe_addr = sector_nr; } -- 2.39.2

2 1

[PATCH OLK-5.10] md/raid5: avoid BUG_ON() while continue reshape after reassembling
by Li Nan 04 Sep '24

04 Sep '24

From: Yu Kuai <yukuai3(a)huawei.com> stable inclusion from stable-v5.10.224 commit c384dd4f1fb3b14a2fd199360701cc163ea88705 category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/IAMNBN CVE: CVE-2024-43914 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id… -------------------------------- [ Upstream commit 305a5170dc5cf3d395bb4c4e9239bca6d0b54b49 ] Currently, mdadm support --revert-reshape to abort the reshape while reassembling, as the test 07revert-grow. However, following BUG_ON() can be triggerred by the test: kernel BUG at drivers/md/raid5.c:6278! invalid opcode: 0000 [#1] PREEMPT SMP PTI irq event stamp: 158985 CPU: 6 PID: 891 Comm: md0_reshape Not tainted 6.9.0-03335-g7592a0b0049a #94 RIP: 0010:reshape_request+0x3f1/0xe60 Call Trace: <TASK> raid5_sync_request+0x43d/0x550 md_do_sync+0xb7a/0x2110 md_thread+0x294/0x2b0 kthread+0x147/0x1c0 ret_from_fork+0x59/0x70 ret_from_fork_asm+0x1a/0x30 </TASK> Root cause is that --revert-reshape update the raid_disks from 5 to 4, while reshape position is still set, and after reassembling the array, reshape position will be read from super block, then during reshape the checking of 'writepos' that is caculated by old reshape position will fail. Fix this panic the easy way first, by converting the BUG_ON() to WARN_ON(), and stop the reshape if checkings fail. Noted that mdadm must fix --revert-shape as well, and probably md/raid should enhance metadata validation as well, however this means reassemble will fail and there must be user tools to fix the wrong metadata. Signed-off-by: Yu Kuai <yukuai3(a)huawei.com> Signed-off-by: Song Liu <song(a)kernel.org> Link: https://lore.kernel.org/r/20240611132251.1967786-13-yukuai1@huaweicloud.com Signed-off-by: Sasha Levin <sashal(a)kernel.org> Signed-off-by: Li Nan <linan122(a)huawei.com> --- drivers/md/raid5.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index 3cb90d7e88d9..126b9ecfe750 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -5997,7 +5997,9 @@ static sector_t reshape_request(struct mddev *mddev, sector_t sector_nr, int *sk safepos = conf->reshape_safe; sector_div(safepos, data_disks); if (mddev->reshape_backwards) { - BUG_ON(writepos < reshape_sectors); + if (WARN_ON(writepos < reshape_sectors)) + return MaxSector; + writepos -= reshape_sectors; readpos += reshape_sectors; safepos += reshape_sectors; @@ -6015,14 +6017,18 @@ static sector_t reshape_request(struct mddev *mddev, sector_t sector_nr, int *sk * to set 'stripe_addr' which is where we will write to. */ if (mddev->reshape_backwards) { - BUG_ON(conf->reshape_progress == 0); + if (WARN_ON(conf->reshape_progress == 0)) + return MaxSector; + stripe_addr = writepos; - BUG_ON((mddev->dev_sectors & - ~((sector_t)reshape_sectors - 1)) - - reshape_sectors - stripe_addr - != sector_nr); + if (WARN_ON((mddev->dev_sectors & + ~((sector_t)reshape_sectors - 1)) - + reshape_sectors - stripe_addr != sector_nr)) + return MaxSector; } else { - BUG_ON(writepos != sector_nr + reshape_sectors); + if (WARN_ON(writepos != sector_nr + reshape_sectors)) + return MaxSector; + stripe_addr = sector_nr; } -- 2.39.2

2 1

[PATCH openEuler-22.03-LTS-SP1] md/raid5: avoid BUG_ON() while continue reshape after reassembling
by Li Nan 04 Sep '24

04 Sep '24

From: Yu Kuai <yukuai3(a)huawei.com> stable inclusion from stable-v5.10.224 commit c384dd4f1fb3b14a2fd199360701cc163ea88705 category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/IAMNBN CVE: CVE-2024-43914 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id… -------------------------------- [ Upstream commit 305a5170dc5cf3d395bb4c4e9239bca6d0b54b49 ] Currently, mdadm support --revert-reshape to abort the reshape while reassembling, as the test 07revert-grow. However, following BUG_ON() can be triggerred by the test: kernel BUG at drivers/md/raid5.c:6278! invalid opcode: 0000 [#1] PREEMPT SMP PTI irq event stamp: 158985 CPU: 6 PID: 891 Comm: md0_reshape Not tainted 6.9.0-03335-g7592a0b0049a #94 RIP: 0010:reshape_request+0x3f1/0xe60 Call Trace: <TASK> raid5_sync_request+0x43d/0x550 md_do_sync+0xb7a/0x2110 md_thread+0x294/0x2b0 kthread+0x147/0x1c0 ret_from_fork+0x59/0x70 ret_from_fork_asm+0x1a/0x30 </TASK> Root cause is that --revert-reshape update the raid_disks from 5 to 4, while reshape position is still set, and after reassembling the array, reshape position will be read from super block, then during reshape the checking of 'writepos' that is caculated by old reshape position will fail. Fix this panic the easy way first, by converting the BUG_ON() to WARN_ON(), and stop the reshape if checkings fail. Noted that mdadm must fix --revert-shape as well, and probably md/raid should enhance metadata validation as well, however this means reassemble will fail and there must be user tools to fix the wrong metadata. Signed-off-by: Yu Kuai <yukuai3(a)huawei.com> Signed-off-by: Song Liu <song(a)kernel.org> Link: https://lore.kernel.org/r/20240611132251.1967786-13-yukuai1@huaweicloud.com Signed-off-by: Sasha Levin <sashal(a)kernel.org> Signed-off-by: Li Nan <linan122(a)huawei.com> --- drivers/md/raid5.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index 3cb90d7e88d9..126b9ecfe750 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -5997,7 +5997,9 @@ static sector_t reshape_request(struct mddev *mddev, sector_t sector_nr, int *sk safepos = conf->reshape_safe; sector_div(safepos, data_disks); if (mddev->reshape_backwards) { - BUG_ON(writepos < reshape_sectors); + if (WARN_ON(writepos < reshape_sectors)) + return MaxSector; + writepos -= reshape_sectors; readpos += reshape_sectors; safepos += reshape_sectors; @@ -6015,14 +6017,18 @@ static sector_t reshape_request(struct mddev *mddev, sector_t sector_nr, int *sk * to set 'stripe_addr' which is where we will write to. */ if (mddev->reshape_backwards) { - BUG_ON(conf->reshape_progress == 0); + if (WARN_ON(conf->reshape_progress == 0)) + return MaxSector; + stripe_addr = writepos; - BUG_ON((mddev->dev_sectors & - ~((sector_t)reshape_sectors - 1)) - - reshape_sectors - stripe_addr - != sector_nr); + if (WARN_ON((mddev->dev_sectors & + ~((sector_t)reshape_sectors - 1)) - + reshape_sectors - stripe_addr != sector_nr)) + return MaxSector; } else { - BUG_ON(writepos != sector_nr + reshape_sectors); + if (WARN_ON(writepos != sector_nr + reshape_sectors)) + return MaxSector; + stripe_addr = sector_nr; } -- 2.39.2

2 1

[PATCH OLK-6.6] net: missing check virtio
by Zhang Changzhong 04 Sep '24

04 Sep '24

From: Denis Arefev <arefev(a)swemel.ru> stable inclusion from stable-v6.6.44 commit 90d41ebe0cd4635f6410471efc1dd71b33e894cf category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/IAKQ33 CVE: CVE-2024-43817 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id… -------------------------------- [ Upstream commit e269d79c7d35aa3808b1f3c1737d63dab504ddc8 ] Two missing check in virtio_net_hdr_to_skb() allowed syzbot to crash kernels again 1. After the skb_segment function the buffer may become non-linear (nr_frags != 0), but since the SKBTX_SHARED_FRAG flag is not set anywhere the __skb_linearize function will not be executed, then the buffer will remain non-linear. Then the condition (offset >= skb_headlen(skb)) becomes true, which causes WARN_ON_ONCE in skb_checksum_help. 2. The struct sk_buff and struct virtio_net_hdr members must be mathematically related. (gso_size) must be greater than (needed) otherwise WARN_ON_ONCE. (remainder) must be greater than (needed) otherwise WARN_ON_ONCE. (remainder) may be 0 if division is without remainder. offset+2 (4191) > skb_headlen() (1116) WARNING: CPU: 1 PID: 5084 at net/core/dev.c:3303 skb_checksum_help+0x5e2/0x740 net/core/dev.c:3303 Modules linked in: CPU: 1 PID: 5084 Comm: syz-executor336 Not tainted 6.7.0-rc3-syzkaller-00014-gdf60cee26a2e #0 Hardware name: Google Compute Engine/Google Compute Engine, BIOS Google 11/10/2023 RIP: 0010:skb_checksum_help+0x5e2/0x740 net/core/dev.c:3303 Code: 89 e8 83 e0 07 83 c0 03 38 d0 7c 08 84 d2 0f 85 52 01 00 00 44 89 e2 2b 53 74 4c 89 ee 48 c7 c7 40 57 e9 8b e8 af 8f dd f8 90 <0f> 0b 90 90 e9 87 fe ff ff e8 40 0f 6e f9 e9 4b fa ff ff 48 89 ef RSP: 0018:ffffc90003a9f338 EFLAGS: 00010286 RAX: 0000000000000000 RBX: ffff888025125780 RCX: ffffffff814db209 RDX: ffff888015393b80 RSI: ffffffff814db216 RDI: 0000000000000001 RBP: ffff8880251257f4 R08: 0000000000000001 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000001 R12: 000000000000045c R13: 000000000000105f R14: ffff8880251257f0 R15: 000000000000105d FS: 0000555555c24380(0000) GS:ffff8880b9900000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 000000002000f000 CR3: 0000000023151000 CR4: 00000000003506f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: <TASK> ip_do_fragment+0xa1b/0x18b0 net/ipv4/ip_output.c:777 ip_fragment.constprop.0+0x161/0x230 net/ipv4/ip_output.c:584 ip_finish_output_gso net/ipv4/ip_output.c:286 [inline] __ip_finish_output net/ipv4/ip_output.c:308 [inline] __ip_finish_output+0x49c/0x650 net/ipv4/ip_output.c:295 ip_finish_output+0x31/0x310 net/ipv4/ip_output.c:323 NF_HOOK_COND include/linux/netfilter.h:303 [inline] ip_output+0x13b/0x2a0 net/ipv4/ip_output.c:433 dst_output include/net/dst.h:451 [inline] ip_local_out+0xaf/0x1a0 net/ipv4/ip_output.c:129 iptunnel_xmit+0x5b4/0x9b0 net/ipv4/ip_tunnel_core.c:82 ipip6_tunnel_xmit net/ipv6/sit.c:1034 [inline] sit_tunnel_xmit+0xed2/0x28f0 net/ipv6/sit.c:1076 __netdev_start_xmit include/linux/netdevice.h:4940 [inline] netdev_start_xmit include/linux/netdevice.h:4954 [inline] xmit_one net/core/dev.c:3545 [inline] dev_hard_start_xmit+0x13d/0x6d0 net/core/dev.c:3561 __dev_queue_xmit+0x7c1/0x3d60 net/core/dev.c:4346 dev_queue_xmit include/linux/netdevice.h:3134 [inline] packet_xmit+0x257/0x380 net/packet/af_packet.c:276 packet_snd net/packet/af_packet.c:3087 [inline] packet_sendmsg+0x24ca/0x5240 net/packet/af_packet.c:3119 sock_sendmsg_nosec net/socket.c:730 [inline] __sock_sendmsg+0xd5/0x180 net/socket.c:745 __sys_sendto+0x255/0x340 net/socket.c:2190 __do_sys_sendto net/socket.c:2202 [inline] __se_sys_sendto net/socket.c:2198 [inline] __x64_sys_sendto+0xe0/0x1b0 net/socket.c:2198 do_syscall_x64 arch/x86/entry/common.c:51 [inline] do_syscall_64+0x40/0x110 arch/x86/entry/common.c:82 entry_SYSCALL_64_after_hwframe+0x63/0x6b Found by Linux Verification Center (linuxtesting.org) with Syzkaller Fixes: 0f6925b3e8da ("virtio_net: Do not pull payload in skb->head") Signed-off-by: Denis Arefev <arefev(a)swemel.ru> Message-Id: <20240613095448.27118-1-arefev(a)swemel.ru> Signed-off-by: Michael S. Tsirkin <mst(a)redhat.com> Signed-off-by: Sasha Levin <sashal(a)kernel.org> Signed-off-by: Zhang Changzhong <zhangchangzhong(a)huawei.com> --- include/linux/virtio_net.h | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/include/linux/virtio_net.h b/include/linux/virtio_net.h index 6c395a2..c824c52 100644 --- a/include/linux/virtio_net.h +++ b/include/linux/virtio_net.h @@ -56,6 +56,7 @@ static inline int virtio_net_hdr_to_skb(struct sk_buff *skb, unsigned int thlen = 0; unsigned int p_off = 0; unsigned int ip_proto; + u64 ret, remainder, gso_size; if (hdr->gso_type != VIRTIO_NET_HDR_GSO_NONE) { switch (hdr->gso_type & ~VIRTIO_NET_HDR_GSO_ECN) { @@ -98,6 +99,16 @@ static inline int virtio_net_hdr_to_skb(struct sk_buff *skb, u32 off = __virtio16_to_cpu(little_endian, hdr->csum_offset); u32 needed = start + max_t(u32, thlen, off + sizeof(__sum16)); + if (hdr->gso_size) { + gso_size = __virtio16_to_cpu(little_endian, hdr->gso_size); + ret = div64_u64_rem(skb->len, gso_size, &remainder); + if (!(ret && (hdr->gso_size > needed) && + ((remainder > needed) || (remainder == 0)))) { + return -EINVAL; + } + skb_shinfo(skb)->tx_flags |= SKBFL_SHARED_FRAG; + } + if (!pskb_may_pull(skb, needed)) return -EINVAL; -- 2.9.5

2 1

[PATCH openEuler-22.03-LTS-SP1] net: missing check virtio
by Zhang Changzhong 04 Sep '24

04 Sep '24

From: Denis Arefev <arefev(a)swemel.ru> mainline inclusion from mainline-v6.11-rc1 commit e269d79c7d35aa3808b1f3c1737d63dab504ddc8 category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/IAKQ33 CVE: CVE-2024-43817 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?… -------------------------------- Two missing check in virtio_net_hdr_to_skb() allowed syzbot to crash kernels again 1. After the skb_segment function the buffer may become non-linear (nr_frags != 0), but since the SKBTX_SHARED_FRAG flag is not set anywhere the __skb_linearize function will not be executed, then the buffer will remain non-linear. Then the condition (offset >= skb_headlen(skb)) becomes true, which causes WARN_ON_ONCE in skb_checksum_help. 2. The struct sk_buff and struct virtio_net_hdr members must be mathematically related. (gso_size) must be greater than (needed) otherwise WARN_ON_ONCE. (remainder) must be greater than (needed) otherwise WARN_ON_ONCE. (remainder) may be 0 if division is without remainder. offset+2 (4191) > skb_headlen() (1116) WARNING: CPU: 1 PID: 5084 at net/core/dev.c:3303 skb_checksum_help+0x5e2/0x740 net/core/dev.c:3303 Modules linked in: CPU: 1 PID: 5084 Comm: syz-executor336 Not tainted 6.7.0-rc3-syzkaller-00014-gdf60cee26a2e #0 Hardware name: Google Compute Engine/Google Compute Engine, BIOS Google 11/10/2023 RIP: 0010:skb_checksum_help+0x5e2/0x740 net/core/dev.c:3303 Code: 89 e8 83 e0 07 83 c0 03 38 d0 7c 08 84 d2 0f 85 52 01 00 00 44 89 e2 2b 53 74 4c 89 ee 48 c7 c7 40 57 e9 8b e8 af 8f dd f8 90 <0f> 0b 90 90 e9 87 fe ff ff e8 40 0f 6e f9 e9 4b fa ff ff 48 89 ef RSP: 0018:ffffc90003a9f338 EFLAGS: 00010286 RAX: 0000000000000000 RBX: ffff888025125780 RCX: ffffffff814db209 RDX: ffff888015393b80 RSI: ffffffff814db216 RDI: 0000000000000001 RBP: ffff8880251257f4 R08: 0000000000000001 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000001 R12: 000000000000045c R13: 000000000000105f R14: ffff8880251257f0 R15: 000000000000105d FS: 0000555555c24380(0000) GS:ffff8880b9900000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 000000002000f000 CR3: 0000000023151000 CR4: 00000000003506f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: <TASK> ip_do_fragment+0xa1b/0x18b0 net/ipv4/ip_output.c:777 ip_fragment.constprop.0+0x161/0x230 net/ipv4/ip_output.c:584 ip_finish_output_gso net/ipv4/ip_output.c:286 [inline] __ip_finish_output net/ipv4/ip_output.c:308 [inline] __ip_finish_output+0x49c/0x650 net/ipv4/ip_output.c:295 ip_finish_output+0x31/0x310 net/ipv4/ip_output.c:323 NF_HOOK_COND include/linux/netfilter.h:303 [inline] ip_output+0x13b/0x2a0 net/ipv4/ip_output.c:433 dst_output include/net/dst.h:451 [inline] ip_local_out+0xaf/0x1a0 net/ipv4/ip_output.c:129 iptunnel_xmit+0x5b4/0x9b0 net/ipv4/ip_tunnel_core.c:82 ipip6_tunnel_xmit net/ipv6/sit.c:1034 [inline] sit_tunnel_xmit+0xed2/0x28f0 net/ipv6/sit.c:1076 __netdev_start_xmit include/linux/netdevice.h:4940 [inline] netdev_start_xmit include/linux/netdevice.h:4954 [inline] xmit_one net/core/dev.c:3545 [inline] dev_hard_start_xmit+0x13d/0x6d0 net/core/dev.c:3561 __dev_queue_xmit+0x7c1/0x3d60 net/core/dev.c:4346 dev_queue_xmit include/linux/netdevice.h:3134 [inline] packet_xmit+0x257/0x380 net/packet/af_packet.c:276 packet_snd net/packet/af_packet.c:3087 [inline] packet_sendmsg+0x24ca/0x5240 net/packet/af_packet.c:3119 sock_sendmsg_nosec net/socket.c:730 [inline] __sock_sendmsg+0xd5/0x180 net/socket.c:745 __sys_sendto+0x255/0x340 net/socket.c:2190 __do_sys_sendto net/socket.c:2202 [inline] __se_sys_sendto net/socket.c:2198 [inline] __x64_sys_sendto+0xe0/0x1b0 net/socket.c:2198 do_syscall_x64 arch/x86/entry/common.c:51 [inline] do_syscall_64+0x40/0x110 arch/x86/entry/common.c:82 entry_SYSCALL_64_after_hwframe+0x63/0x6b Found by Linux Verification Center (linuxtesting.org) with Syzkaller Fixes: 0f6925b3e8da ("virtio_net: Do not pull payload in skb->head") Signed-off-by: Denis Arefev <arefev(a)swemel.ru> Message-Id: <20240613095448.27118-1-arefev(a)swemel.ru> Signed-off-by: Michael S. Tsirkin <mst(a)redhat.com> Signed-off-by: Zhang Changzhong <zhangchangzhong(a)huawei.com> --- include/linux/virtio_net.h | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/include/linux/virtio_net.h b/include/linux/virtio_net.h index a960de6..b6a9d07 100644 --- a/include/linux/virtio_net.h +++ b/include/linux/virtio_net.h @@ -51,6 +51,7 @@ static inline int virtio_net_hdr_to_skb(struct sk_buff *skb, unsigned int thlen = 0; unsigned int p_off = 0; unsigned int ip_proto; + u64 ret, remainder, gso_size; if (hdr->gso_type != VIRTIO_NET_HDR_GSO_NONE) { switch (hdr->gso_type & ~VIRTIO_NET_HDR_GSO_ECN) { @@ -87,6 +88,16 @@ static inline int virtio_net_hdr_to_skb(struct sk_buff *skb, u32 off = __virtio16_to_cpu(little_endian, hdr->csum_offset); u32 needed = start + max_t(u32, thlen, off + sizeof(__sum16)); + if (hdr->gso_size) { + gso_size = __virtio16_to_cpu(little_endian, hdr->gso_size); + ret = div64_u64_rem(skb->len, gso_size, &remainder); + if (!(ret && (hdr->gso_size > needed) && + ((remainder > needed) || (remainder == 0)))) { + return -EINVAL; + } + skb_shinfo(skb)->tx_flags |= SKBFL_SHARED_FRAG; + } + if (!pskb_may_pull(skb, needed)) return -EINVAL; -- 2.9.5

2 1

[PATCH OLK-5.10] net: missing check virtio
by Zhang Changzhong 04 Sep '24

04 Sep '24

From: Denis Arefev <arefev(a)swemel.ru> mainline inclusion from mainline-v6.11-rc1 commit e269d79c7d35aa3808b1f3c1737d63dab504ddc8 category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/IAKQ33 CVE: CVE-2024-43817 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?… -------------------------------- Two missing check in virtio_net_hdr_to_skb() allowed syzbot to crash kernels again 1. After the skb_segment function the buffer may become non-linear (nr_frags != 0), but since the SKBTX_SHARED_FRAG flag is not set anywhere the __skb_linearize function will not be executed, then the buffer will remain non-linear. Then the condition (offset >= skb_headlen(skb)) becomes true, which causes WARN_ON_ONCE in skb_checksum_help. 2. The struct sk_buff and struct virtio_net_hdr members must be mathematically related. (gso_size) must be greater than (needed) otherwise WARN_ON_ONCE. (remainder) must be greater than (needed) otherwise WARN_ON_ONCE. (remainder) may be 0 if division is without remainder. offset+2 (4191) > skb_headlen() (1116) WARNING: CPU: 1 PID: 5084 at net/core/dev.c:3303 skb_checksum_help+0x5e2/0x740 net/core/dev.c:3303 Modules linked in: CPU: 1 PID: 5084 Comm: syz-executor336 Not tainted 6.7.0-rc3-syzkaller-00014-gdf60cee26a2e #0 Hardware name: Google Compute Engine/Google Compute Engine, BIOS Google 11/10/2023 RIP: 0010:skb_checksum_help+0x5e2/0x740 net/core/dev.c:3303 Code: 89 e8 83 e0 07 83 c0 03 38 d0 7c 08 84 d2 0f 85 52 01 00 00 44 89 e2 2b 53 74 4c 89 ee 48 c7 c7 40 57 e9 8b e8 af 8f dd f8 90 <0f> 0b 90 90 e9 87 fe ff ff e8 40 0f 6e f9 e9 4b fa ff ff 48 89 ef RSP: 0018:ffffc90003a9f338 EFLAGS: 00010286 RAX: 0000000000000000 RBX: ffff888025125780 RCX: ffffffff814db209 RDX: ffff888015393b80 RSI: ffffffff814db216 RDI: 0000000000000001 RBP: ffff8880251257f4 R08: 0000000000000001 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000001 R12: 000000000000045c R13: 000000000000105f R14: ffff8880251257f0 R15: 000000000000105d FS: 0000555555c24380(0000) GS:ffff8880b9900000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 000000002000f000 CR3: 0000000023151000 CR4: 00000000003506f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: <TASK> ip_do_fragment+0xa1b/0x18b0 net/ipv4/ip_output.c:777 ip_fragment.constprop.0+0x161/0x230 net/ipv4/ip_output.c:584 ip_finish_output_gso net/ipv4/ip_output.c:286 [inline] __ip_finish_output net/ipv4/ip_output.c:308 [inline] __ip_finish_output+0x49c/0x650 net/ipv4/ip_output.c:295 ip_finish_output+0x31/0x310 net/ipv4/ip_output.c:323 NF_HOOK_COND include/linux/netfilter.h:303 [inline] ip_output+0x13b/0x2a0 net/ipv4/ip_output.c:433 dst_output include/net/dst.h:451 [inline] ip_local_out+0xaf/0x1a0 net/ipv4/ip_output.c:129 iptunnel_xmit+0x5b4/0x9b0 net/ipv4/ip_tunnel_core.c:82 ipip6_tunnel_xmit net/ipv6/sit.c:1034 [inline] sit_tunnel_xmit+0xed2/0x28f0 net/ipv6/sit.c:1076 __netdev_start_xmit include/linux/netdevice.h:4940 [inline] netdev_start_xmit include/linux/netdevice.h:4954 [inline] xmit_one net/core/dev.c:3545 [inline] dev_hard_start_xmit+0x13d/0x6d0 net/core/dev.c:3561 __dev_queue_xmit+0x7c1/0x3d60 net/core/dev.c:4346 dev_queue_xmit include/linux/netdevice.h:3134 [inline] packet_xmit+0x257/0x380 net/packet/af_packet.c:276 packet_snd net/packet/af_packet.c:3087 [inline] packet_sendmsg+0x24ca/0x5240 net/packet/af_packet.c:3119 sock_sendmsg_nosec net/socket.c:730 [inline] __sock_sendmsg+0xd5/0x180 net/socket.c:745 __sys_sendto+0x255/0x340 net/socket.c:2190 __do_sys_sendto net/socket.c:2202 [inline] __se_sys_sendto net/socket.c:2198 [inline] __x64_sys_sendto+0xe0/0x1b0 net/socket.c:2198 do_syscall_x64 arch/x86/entry/common.c:51 [inline] do_syscall_64+0x40/0x110 arch/x86/entry/common.c:82 entry_SYSCALL_64_after_hwframe+0x63/0x6b Found by Linux Verification Center (linuxtesting.org) with Syzkaller Fixes: 0f6925b3e8da ("virtio_net: Do not pull payload in skb->head") Signed-off-by: Denis Arefev <arefev(a)swemel.ru> Message-Id: <20240613095448.27118-1-arefev(a)swemel.ru> Signed-off-by: Michael S. Tsirkin <mst(a)redhat.com> Signed-off-by: Zhang Changzhong <zhangchangzhong(a)huawei.com> --- include/linux/virtio_net.h | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/include/linux/virtio_net.h b/include/linux/virtio_net.h index 6047058..29b19d0 100644 --- a/include/linux/virtio_net.h +++ b/include/linux/virtio_net.h @@ -51,6 +51,7 @@ static inline int virtio_net_hdr_to_skb(struct sk_buff *skb, unsigned int thlen = 0; unsigned int p_off = 0; unsigned int ip_proto; + u64 ret, remainder, gso_size; if (hdr->gso_type != VIRTIO_NET_HDR_GSO_NONE) { switch (hdr->gso_type & ~VIRTIO_NET_HDR_GSO_ECN) { @@ -87,6 +88,16 @@ static inline int virtio_net_hdr_to_skb(struct sk_buff *skb, u32 off = __virtio16_to_cpu(little_endian, hdr->csum_offset); u32 needed = start + max_t(u32, thlen, off + sizeof(__sum16)); + if (hdr->gso_size) { + gso_size = __virtio16_to_cpu(little_endian, hdr->gso_size); + ret = div64_u64_rem(skb->len, gso_size, &remainder); + if (!(ret && (hdr->gso_size > needed) && + ((remainder > needed) || (remainder == 0)))) { + return -EINVAL; + } + skb_shinfo(skb)->tx_flags |= SKBFL_SHARED_FRAG; + } + if (!pskb_may_pull(skb, needed)) return -EINVAL; -- 2.9.5

2 1

[PATCH openEuler-22.03-LTS-SP1] wifi: cfg80211: Avoid Wdiscarded-qualifiers warning
by Yue Haibing 04 Sep '24

04 Sep '24

hulk inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/IAGX1C -------------------------------- Commit 591481cccbad trigger a warning: In file included from ./include/net/genetlink.h:6, from net/wireless/nl80211.c:25: net/wireless/nl80211.c:692:69: warning: initialization discards 'const' qualifier from pointer target type [-Wdiscarded-qualifiers] 692 | [NL80211_ATTR_TXQ_QUANTUM] = NLA_POLICY_FULL_RANGE(NLA_U32, &q_range), | ^ ./include/net/netlink.h:407:18: note: in definition of macro 'NLA_POLICY_FULL_RANGE' 407 | .range = _range, \ | ^~~~~~ Remove const qualifier to fix this. Fixes: 591481cccbad ("wifi: cfg80211: restrict NL80211_ATTR_TXQ_QUANTUM values") Signed-off-by: Yue Haibing <yuehaibing(a)huawei.com> --- net/wireless/nl80211.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/net/wireless/nl80211.c b/net/wireless/nl80211.c index 89b94d482fcf..674a0d54aff8 100644 --- a/net/wireless/nl80211.c +++ b/net/wireless/nl80211.c @@ -403,7 +403,7 @@ nl80211_unsol_bcast_probe_resp_policy[NL80211_UNSOL_BCAST_PROBE_RESP_ATTR_MAX + .len = IEEE80211_MAX_DATA_LEN } }; -static const struct netlink_range_validation q_range = { +static struct netlink_range_validation q_range = { .max = INT_MAX, }; -- 2.34.1

2 1

[PATCH OLK-6.6] f2fs: fix null reference error when checking end of zone
by Kaixiong Yu 04 Sep '24

04 Sep '24

From: Daejun Park <daejun7.park(a)samsung.com> stable inclusion from stable-v6.10.3 commit 381cbe85592c78fbaeb3e770e3e9f3bfa3e67efb category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/381cbe85592c78fbaeb3e770e3e9f… CVE: CVE-2024-43857 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id… -------------------------------- [ Upstream commit c82bc1ab2a8a5e73d9728e80c4c2ed87e8921a38 ] This patch fixes a potentially null pointer being accessed by is_end_zone_blkaddr() that checks the last block of a zone when f2fs is mounted as a single device. Fixes: e067dc3c6b9c ("f2fs: maintain six open zones for zoned devices") Signed-off-by: Daejun Park <daejun7.park(a)samsung.com> Reviewed-by: Chao Yu <chao(a)kernel.org> Reviewed-by: Daeho Jeong <daehojeong(a)google.com> Signed-off-by: Jaegeuk Kim <jaegeuk(a)kernel.org> Signed-off-by: Sasha Levin <sashal(a)kernel.org> Conflicts: fs/f2fs/data.c [Since hulk-6.6 does not merge stable inclusion commit 7437bb73f087 ("block: remove support for the host aware zone model")] Signed-off-by: Kaixiong Yu <yukaixiong(a)huawei.com> --- fs/f2fs/data.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c index 84fc87018180..0acafb505efc 100644 --- a/fs/f2fs/data.c +++ b/fs/f2fs/data.c @@ -924,6 +924,7 @@ int f2fs_merge_page_bio(struct f2fs_io_info *fio) #ifdef CONFIG_BLK_DEV_ZONED static bool is_end_zone_blkaddr(struct f2fs_sb_info *sbi, block_t blkaddr) { + struct block_device *bdev = sbi->sb->s_bdev; int devi = 0; if (f2fs_is_multi_device(sbi)) { @@ -934,8 +935,9 @@ static bool is_end_zone_blkaddr(struct f2fs_sb_info *sbi, block_t blkaddr) return false; } blkaddr -= FDEV(devi).start_blk; + bdev = FDEV(devi).bdev; } - return bdev_zoned_model(FDEV(devi).bdev) == BLK_ZONED_HM && + return bdev_zoned_model(bdev) == BLK_ZONED_HM && f2fs_blkz_is_seq(sbi, devi, blkaddr) && (blkaddr % sbi->blocks_per_blkz == sbi->blocks_per_blkz - 1); } -- 2.25.1

2 1

[PATCH openEuler-22.03-LTS-SP1] btrfs: do not start relocation until in progress drops are done
by Long Li 04 Sep '24

04 Sep '24

From: Josef Bacik <josef(a)toxicpanda.com> stable inclusion from stable-v5.15.26 commit 6599d5e8bd758d897fd2ef4dc388ae50278b1f7e category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/IALPSL CVE: CVE-2022-48901 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=… -------------------------------- commit b4be6aefa73c9a6899ef3ba9c5faaa8a66e333ef upstream. We hit a bug with a recovering relocation on mount for one of our file systems in production. I reproduced this locally by injecting errors into snapshot delete with balance running at the same time. This presented as an error while looking up an extent item WARNING: CPU: 5 PID: 1501 at fs/btrfs/extent-tree.c:866 lookup_inline_extent_backref+0x647/0x680 CPU: 5 PID: 1501 Comm: btrfs-balance Not tainted 5.16.0-rc8+ #8 RIP: 0010:lookup_inline_extent_backref+0x647/0x680 RSP: 0018:ffffae0a023ab960 EFLAGS: 00010202 RAX: 0000000000000001 RBX: 0000000000000000 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 000000000000000c RDI: 0000000000000000 RBP: ffff943fd2a39b60 R08: 0000000000000000 R09: 0000000000000001 R10: 0001434088152de0 R11: 0000000000000000 R12: 0000000001d05000 R13: ffff943fd2a39b60 R14: ffff943fdb96f2a0 R15: ffff9442fc923000 FS: 0000000000000000(0000) GS:ffff944e9eb40000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f1157b1fca8 CR3: 000000010f092000 CR4: 0000000000350ee0 Call Trace: <TASK> insert_inline_extent_backref+0x46/0xd0 __btrfs_inc_extent_ref.isra.0+0x5f/0x200 ? btrfs_merge_delayed_refs+0x164/0x190 __btrfs_run_delayed_refs+0x561/0xfa0 ? btrfs_search_slot+0x7b4/0xb30 ? btrfs_update_root+0x1a9/0x2c0 btrfs_run_delayed_refs+0x73/0x1f0 ? btrfs_update_root+0x1a9/0x2c0 btrfs_commit_transaction+0x50/0xa50 ? btrfs_update_reloc_root+0x122/0x220 prepare_to_merge+0x29f/0x320 relocate_block_group+0x2b8/0x550 btrfs_relocate_block_group+0x1a6/0x350 btrfs_relocate_chunk+0x27/0xe0 btrfs_balance+0x777/0xe60 balance_kthread+0x35/0x50 ? btrfs_balance+0xe60/0xe60 kthread+0x16b/0x190 ? set_kthread_struct+0x40/0x40 ret_from_fork+0x22/0x30 </TASK> Normally snapshot deletion and relocation are excluded from running at the same time by the fs_info->cleaner_mutex. However if we had a pending balance waiting to get the ->cleaner_mutex, and a snapshot deletion was running, and then the box crashed, we would come up in a state where we have a half deleted snapshot. Again, in the normal case the snapshot deletion needs to complete before relocation can start, but in this case relocation could very well start before the snapshot deletion completes, as we simply add the root to the dead roots list and wait for the next time the cleaner runs to clean up the snapshot. Fix this by setting a bit on the fs_info if we have any DEAD_ROOT's that had a pending drop_progress key. If they do then we know we were in the middle of the drop operation and set a flag on the fs_info. Then balance can wait until this flag is cleared to start up again. If there are DEAD_ROOT's that don't have a drop_progress set then we're safe to start balance right away as we'll be properly protected by the cleaner_mutex. CC: stable(a)vger.kernel.org # 5.10+ Reviewed-by: Filipe Manana <fdmanana(a)suse.com> Signed-off-by: Josef Bacik <josef(a)toxicpanda.com> Reviewed-by: David Sterba <dsterba(a)suse.com> Signed-off-by: David Sterba <dsterba(a)suse.com> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> Conflicts: fs/btrfs/disk-io.c fs/btrfs/ctree.h fs/btrfs/transaction.h [conflicts due to 8cd2908846d1 ("btrfs: clear oneshot options on mount and remount") not mergered] Signed-off-by: Long Li <leo.lilong(a)huawei.com> --- fs/btrfs/ctree.h | 10 ++++++++++ fs/btrfs/disk-io.c | 10 ++++++++++ fs/btrfs/extent-tree.c | 10 ++++++++++ fs/btrfs/relocation.c | 13 +++++++++++++ fs/btrfs/root-tree.c | 15 +++++++++++++++ fs/btrfs/transaction.c | 33 ++++++++++++++++++++++++++++++++- fs/btrfs/transaction.h | 1 + 7 files changed, 91 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 84ac3642023f..90b46ac13003 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -567,6 +567,9 @@ enum { /* Indicate that we can't trust the free space tree for caching yet */ BTRFS_FS_FREE_SPACE_TREE_UNTRUSTED, + + /* Indicate we have half completed snapshot deletions pending. */ + BTRFS_FS_UNFINISHED_DROPS, }; /* @@ -1033,8 +1036,15 @@ enum { BTRFS_ROOT_HAS_LOG_TREE, /* Qgroup flushing is in progress */ BTRFS_ROOT_QGROUP_FLUSHING, + /* This root has a drop operation that was started previously. */ + BTRFS_ROOT_UNFINISHED_DROP, }; +static inline void btrfs_wake_unfinished_drop(struct btrfs_fs_info *fs_info) +{ + clear_and_wake_up_bit(BTRFS_FS_UNFINISHED_DROPS, &fs_info->flags); +} + /* * Record swapped tree blocks of a subvolume tree for delayed subtree trace * code. For detail check comment in fs/btrfs/qgroup.c. diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index b9a3bdacdede..05bc52003e19 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -3427,6 +3427,10 @@ int __cold open_ctree(struct super_block *sb, struct btrfs_fs_devices *fs_device } set_bit(BTRFS_FS_OPEN, &fs_info->flags); + /* Kick the cleaner thread so it'll start deleting snapshots. */ + if (test_bit(BTRFS_FS_UNFINISHED_DROPS, &fs_info->flags)) + wake_up_process(fs_info->cleaner_kthread); + /* * backuproot only affect mount behavior, and if open_ctree succeeded, * no need to keep the flag @@ -4092,6 +4096,12 @@ void __cold close_ctree(struct btrfs_fs_info *fs_info) */ kthread_park(fs_info->cleaner_kthread); + /* + * If we had UNFINISHED_DROPS we could still be processing them, so + * clear that bit and wake up relocation so it can stop. + */ + btrfs_wake_unfinished_drop(fs_info); + /* wait for the qgroup rescan worker to stop */ btrfs_qgroup_wait_for_completion(fs_info, false); diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 284294620e9f..578b217bc4ea 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -5353,6 +5353,7 @@ int btrfs_drop_snapshot(struct btrfs_root *root, int update_ref, int for_reloc) int ret; int level; bool root_dropped = false; + bool unfinished_drop = false; btrfs_debug(fs_info, "Drop subvolume %llu", root->root_key.objectid); @@ -5395,6 +5396,8 @@ int btrfs_drop_snapshot(struct btrfs_root *root, int update_ref, int for_reloc) * already dropped. */ set_bit(BTRFS_ROOT_DELETING, &root->state); + unfinished_drop = test_bit(BTRFS_ROOT_UNFINISHED_DROP, &root->state); + if (btrfs_disk_key_objectid(&root_item->drop_progress) == 0) { level = btrfs_header_level(root->node); path->nodes[level] = btrfs_lock_root_node(root); @@ -5571,6 +5574,13 @@ int btrfs_drop_snapshot(struct btrfs_root *root, int update_ref, int for_reloc) kfree(wc); btrfs_free_path(path); out: + /* + * We were an unfinished drop root, check to see if there are any + * pending, and if not clear and wake up any waiters. + */ + if (!err && unfinished_drop) + btrfs_maybe_wake_unfinished_drop(fs_info); + /* * So if we need to stop dropping the snapshot for whatever reason we * need to make sure to add it back to the dead root list so that we diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c index c21545c5b34b..cf7a8fcea4d6 100644 --- a/fs/btrfs/relocation.c +++ b/fs/btrfs/relocation.c @@ -3595,6 +3595,19 @@ int btrfs_relocate_block_group(struct btrfs_fs_info *fs_info, u64 group_start) int rw = 0; int err = 0; + /* + * This only gets set if we had a half-deleted snapshot on mount. We + * cannot allow relocation to start while we're still trying to clean up + * these pending deletions. + */ + ret = wait_on_bit(&fs_info->flags, BTRFS_FS_UNFINISHED_DROPS, TASK_INTERRUPTIBLE); + if (ret) + return ret; + + /* We may have been woken up by close_ctree, so bail if we're closing. */ + if (btrfs_fs_closing(fs_info)) + return -EINTR; + bg = btrfs_lookup_block_group(fs_info, group_start); if (!bg) return -ENOENT; diff --git a/fs/btrfs/root-tree.c b/fs/btrfs/root-tree.c index cd40e7a92552..7bdaccb58ec0 100644 --- a/fs/btrfs/root-tree.c +++ b/fs/btrfs/root-tree.c @@ -280,6 +280,21 @@ int btrfs_find_orphan_roots(struct btrfs_fs_info *fs_info) WARN_ON(!test_bit(BTRFS_ROOT_ORPHAN_ITEM_INSERTED, &root->state)); if (btrfs_root_refs(&root->root_item) == 0) { + struct btrfs_key drop_key; + + btrfs_disk_key_to_cpu(&drop_key, &root->root_item.drop_progress); + /* + * If we have a non-zero drop_progress then we know we + * made it partly through deleting this snapshot, and + * thus we need to make sure we block any balance from + * happening until this snapshot is completely dropped. + */ + if (drop_key.objectid != 0 || drop_key.type != 0 || + drop_key.offset != 0) { + set_bit(BTRFS_FS_UNFINISHED_DROPS, &fs_info->flags); + set_bit(BTRFS_ROOT_UNFINISHED_DROP, &root->state); + } + set_bit(BTRFS_ROOT_DEAD_TREE, &root->state); btrfs_add_dead_root(root); } diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index a6ddf91d5c40..cf7e8a20bb02 100644 --- a/fs/btrfs/transaction.c +++ b/fs/btrfs/transaction.c @@ -1294,6 +1294,32 @@ static noinline int commit_cowonly_roots(struct btrfs_trans_handle *trans) return 0; } +/* + * If we had a pending drop we need to see if there are any others left in our + * dead roots list, and if not clear our bit and wake any waiters. + */ +void btrfs_maybe_wake_unfinished_drop(struct btrfs_fs_info *fs_info) +{ + /* + * We put the drop in progress roots at the front of the list, so if the + * first entry doesn't have UNFINISHED_DROP set we can wake everybody + * up. + */ + spin_lock(&fs_info->trans_lock); + if (!list_empty(&fs_info->dead_roots)) { + struct btrfs_root *root = list_first_entry(&fs_info->dead_roots, + struct btrfs_root, + root_list); + if (test_bit(BTRFS_ROOT_UNFINISHED_DROP, &root->state)) { + spin_unlock(&fs_info->trans_lock); + return; + } + } + spin_unlock(&fs_info->trans_lock); + + btrfs_wake_unfinished_drop(fs_info); +} + /* * dead roots are old snapshots that need to be deleted. This allocates * a dirty root struct and adds it into the list of dead roots that need to @@ -1306,7 +1332,12 @@ void btrfs_add_dead_root(struct btrfs_root *root) spin_lock(&fs_info->trans_lock); if (list_empty(&root->root_list)) { btrfs_grab_root(root); - list_add_tail(&root->root_list, &fs_info->dead_roots); + + /* We want to process the partially complete drops first. */ + if (test_bit(BTRFS_ROOT_UNFINISHED_DROP, &root->state)) + list_add(&root->root_list, &fs_info->dead_roots); + else + list_add_tail(&root->root_list, &fs_info->dead_roots); } spin_unlock(&fs_info->trans_lock); } diff --git a/fs/btrfs/transaction.h b/fs/btrfs/transaction.h index eb26eb068fe8..0599e3994633 100644 --- a/fs/btrfs/transaction.h +++ b/fs/btrfs/transaction.h @@ -216,6 +216,7 @@ int btrfs_wait_for_commit(struct btrfs_fs_info *fs_info, u64 transid); void btrfs_add_dead_root(struct btrfs_root *root); int btrfs_defrag_root(struct btrfs_root *root); +void btrfs_maybe_wake_unfinished_drop(struct btrfs_fs_info *fs_info); int btrfs_clean_one_deleted_snapshot(struct btrfs_root *root); int btrfs_commit_transaction(struct btrfs_trans_handle *trans); int btrfs_commit_transaction_async(struct btrfs_trans_handle *trans, -- 2.39.2

2 1

[PATCH OLK-5.10] btrfs: do not start relocation until in progress drops are done
by Long Li 04 Sep '24

04 Sep '24

From: Josef Bacik <josef(a)toxicpanda.com> stable inclusion from stable-v5.15.26 commit 6599d5e8bd758d897fd2ef4dc388ae50278b1f7e category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/IALPSL CVE: CVE-2022-48901 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=… -------------------------------- commit b4be6aefa73c9a6899ef3ba9c5faaa8a66e333ef upstream. We hit a bug with a recovering relocation on mount for one of our file systems in production. I reproduced this locally by injecting errors into snapshot delete with balance running at the same time. This presented as an error while looking up an extent item WARNING: CPU: 5 PID: 1501 at fs/btrfs/extent-tree.c:866 lookup_inline_extent_backref+0x647/0x680 CPU: 5 PID: 1501 Comm: btrfs-balance Not tainted 5.16.0-rc8+ #8 RIP: 0010:lookup_inline_extent_backref+0x647/0x680 RSP: 0018:ffffae0a023ab960 EFLAGS: 00010202 RAX: 0000000000000001 RBX: 0000000000000000 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 000000000000000c RDI: 0000000000000000 RBP: ffff943fd2a39b60 R08: 0000000000000000 R09: 0000000000000001 R10: 0001434088152de0 R11: 0000000000000000 R12: 0000000001d05000 R13: ffff943fd2a39b60 R14: ffff943fdb96f2a0 R15: ffff9442fc923000 FS: 0000000000000000(0000) GS:ffff944e9eb40000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f1157b1fca8 CR3: 000000010f092000 CR4: 0000000000350ee0 Call Trace: <TASK> insert_inline_extent_backref+0x46/0xd0 __btrfs_inc_extent_ref.isra.0+0x5f/0x200 ? btrfs_merge_delayed_refs+0x164/0x190 __btrfs_run_delayed_refs+0x561/0xfa0 ? btrfs_search_slot+0x7b4/0xb30 ? btrfs_update_root+0x1a9/0x2c0 btrfs_run_delayed_refs+0x73/0x1f0 ? btrfs_update_root+0x1a9/0x2c0 btrfs_commit_transaction+0x50/0xa50 ? btrfs_update_reloc_root+0x122/0x220 prepare_to_merge+0x29f/0x320 relocate_block_group+0x2b8/0x550 btrfs_relocate_block_group+0x1a6/0x350 btrfs_relocate_chunk+0x27/0xe0 btrfs_balance+0x777/0xe60 balance_kthread+0x35/0x50 ? btrfs_balance+0xe60/0xe60 kthread+0x16b/0x190 ? set_kthread_struct+0x40/0x40 ret_from_fork+0x22/0x30 </TASK> Normally snapshot deletion and relocation are excluded from running at the same time by the fs_info->cleaner_mutex. However if we had a pending balance waiting to get the ->cleaner_mutex, and a snapshot deletion was running, and then the box crashed, we would come up in a state where we have a half deleted snapshot. Again, in the normal case the snapshot deletion needs to complete before relocation can start, but in this case relocation could very well start before the snapshot deletion completes, as we simply add the root to the dead roots list and wait for the next time the cleaner runs to clean up the snapshot. Fix this by setting a bit on the fs_info if we have any DEAD_ROOT's that had a pending drop_progress key. If they do then we know we were in the middle of the drop operation and set a flag on the fs_info. Then balance can wait until this flag is cleared to start up again. If there are DEAD_ROOT's that don't have a drop_progress set then we're safe to start balance right away as we'll be properly protected by the cleaner_mutex. CC: stable(a)vger.kernel.org # 5.10+ Reviewed-by: Filipe Manana <fdmanana(a)suse.com> Signed-off-by: Josef Bacik <josef(a)toxicpanda.com> Reviewed-by: David Sterba <dsterba(a)suse.com> Signed-off-by: David Sterba <dsterba(a)suse.com> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> Conflicts: fs/btrfs/disk-io.c fs/btrfs/ctree.h fs/btrfs/transaction.h [conflicts due to 8cd2908846d1 ("btrfs: clear oneshot options on mount and remount") not mergered] Signed-off-by: Long Li <leo.lilong(a)huawei.com> --- fs/btrfs/ctree.h | 10 ++++++++++ fs/btrfs/disk-io.c | 10 ++++++++++ fs/btrfs/extent-tree.c | 10 ++++++++++ fs/btrfs/relocation.c | 13 +++++++++++++ fs/btrfs/root-tree.c | 15 +++++++++++++++ fs/btrfs/transaction.c | 33 ++++++++++++++++++++++++++++++++- fs/btrfs/transaction.h | 1 + 7 files changed, 91 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 482842ac397b..abc1e72a4464 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -565,6 +565,9 @@ enum { /* Indicate that we can't trust the free space tree for caching yet */ BTRFS_FS_FREE_SPACE_TREE_UNTRUSTED, + + /* Indicate we have half completed snapshot deletions pending. */ + BTRFS_FS_UNFINISHED_DROPS, }; /* @@ -1031,8 +1034,15 @@ enum { BTRFS_ROOT_HAS_LOG_TREE, /* Qgroup flushing is in progress */ BTRFS_ROOT_QGROUP_FLUSHING, + /* This root has a drop operation that was started previously. */ + BTRFS_ROOT_UNFINISHED_DROP, }; +static inline void btrfs_wake_unfinished_drop(struct btrfs_fs_info *fs_info) +{ + clear_and_wake_up_bit(BTRFS_FS_UNFINISHED_DROPS, &fs_info->flags); +} + /* * Record swapped tree blocks of a subvolume tree for delayed subtree trace * code. For detail check comment in fs/btrfs/qgroup.c. diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 707344288b1c..b923011dec85 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -3444,6 +3444,10 @@ int __cold open_ctree(struct super_block *sb, struct btrfs_fs_devices *fs_device } set_bit(BTRFS_FS_OPEN, &fs_info->flags); + /* Kick the cleaner thread so it'll start deleting snapshots. */ + if (test_bit(BTRFS_FS_UNFINISHED_DROPS, &fs_info->flags)) + wake_up_process(fs_info->cleaner_kthread); + /* * backuproot only affect mount behavior, and if open_ctree succeeded, * no need to keep the flag @@ -4109,6 +4113,12 @@ void __cold close_ctree(struct btrfs_fs_info *fs_info) */ kthread_park(fs_info->cleaner_kthread); + /* + * If we had UNFINISHED_DROPS we could still be processing them, so + * clear that bit and wake up relocation so it can stop. + */ + btrfs_wake_unfinished_drop(fs_info); + /* wait for the qgroup rescan worker to stop */ btrfs_qgroup_wait_for_completion(fs_info, false); diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 3ba43a40032c..1cf787209798 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -5365,6 +5365,7 @@ int btrfs_drop_snapshot(struct btrfs_root *root, int update_ref, int for_reloc) int ret; int level; bool root_dropped = false; + bool unfinished_drop = false; btrfs_debug(fs_info, "Drop subvolume %llu", root->root_key.objectid); @@ -5407,6 +5408,8 @@ int btrfs_drop_snapshot(struct btrfs_root *root, int update_ref, int for_reloc) * already dropped. */ set_bit(BTRFS_ROOT_DELETING, &root->state); + unfinished_drop = test_bit(BTRFS_ROOT_UNFINISHED_DROP, &root->state); + if (btrfs_disk_key_objectid(&root_item->drop_progress) == 0) { level = btrfs_header_level(root->node); path->nodes[level] = btrfs_lock_root_node(root); @@ -5583,6 +5586,13 @@ int btrfs_drop_snapshot(struct btrfs_root *root, int update_ref, int for_reloc) kfree(wc); btrfs_free_path(path); out: + /* + * We were an unfinished drop root, check to see if there are any + * pending, and if not clear and wake up any waiters. + */ + if (!err && unfinished_drop) + btrfs_maybe_wake_unfinished_drop(fs_info); + /* * So if we need to stop dropping the snapshot for whatever reason we * need to make sure to add it back to the dead root list so that we diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c index 93db4486a943..bdfa4a93d373 100644 --- a/fs/btrfs/relocation.c +++ b/fs/btrfs/relocation.c @@ -3601,6 +3601,19 @@ int btrfs_relocate_block_group(struct btrfs_fs_info *fs_info, u64 group_start) int rw = 0; int err = 0; + /* + * This only gets set if we had a half-deleted snapshot on mount. We + * cannot allow relocation to start while we're still trying to clean up + * these pending deletions. + */ + ret = wait_on_bit(&fs_info->flags, BTRFS_FS_UNFINISHED_DROPS, TASK_INTERRUPTIBLE); + if (ret) + return ret; + + /* We may have been woken up by close_ctree, so bail if we're closing. */ + if (btrfs_fs_closing(fs_info)) + return -EINTR; + bg = btrfs_lookup_block_group(fs_info, group_start); if (!bg) return -ENOENT; diff --git a/fs/btrfs/root-tree.c b/fs/btrfs/root-tree.c index cd40e7a92552..7bdaccb58ec0 100644 --- a/fs/btrfs/root-tree.c +++ b/fs/btrfs/root-tree.c @@ -280,6 +280,21 @@ int btrfs_find_orphan_roots(struct btrfs_fs_info *fs_info) WARN_ON(!test_bit(BTRFS_ROOT_ORPHAN_ITEM_INSERTED, &root->state)); if (btrfs_root_refs(&root->root_item) == 0) { + struct btrfs_key drop_key; + + btrfs_disk_key_to_cpu(&drop_key, &root->root_item.drop_progress); + /* + * If we have a non-zero drop_progress then we know we + * made it partly through deleting this snapshot, and + * thus we need to make sure we block any balance from + * happening until this snapshot is completely dropped. + */ + if (drop_key.objectid != 0 || drop_key.type != 0 || + drop_key.offset != 0) { + set_bit(BTRFS_FS_UNFINISHED_DROPS, &fs_info->flags); + set_bit(BTRFS_ROOT_UNFINISHED_DROP, &root->state); + } + set_bit(BTRFS_ROOT_DEAD_TREE, &root->state); btrfs_add_dead_root(root); } diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index 89117cc2cf05..7d0ea8aeadb0 100644 --- a/fs/btrfs/transaction.c +++ b/fs/btrfs/transaction.c @@ -1301,6 +1301,32 @@ static noinline int commit_cowonly_roots(struct btrfs_trans_handle *trans) return 0; } +/* + * If we had a pending drop we need to see if there are any others left in our + * dead roots list, and if not clear our bit and wake any waiters. + */ +void btrfs_maybe_wake_unfinished_drop(struct btrfs_fs_info *fs_info) +{ + /* + * We put the drop in progress roots at the front of the list, so if the + * first entry doesn't have UNFINISHED_DROP set we can wake everybody + * up. + */ + spin_lock(&fs_info->trans_lock); + if (!list_empty(&fs_info->dead_roots)) { + struct btrfs_root *root = list_first_entry(&fs_info->dead_roots, + struct btrfs_root, + root_list); + if (test_bit(BTRFS_ROOT_UNFINISHED_DROP, &root->state)) { + spin_unlock(&fs_info->trans_lock); + return; + } + } + spin_unlock(&fs_info->trans_lock); + + btrfs_wake_unfinished_drop(fs_info); +} + /* * dead roots are old snapshots that need to be deleted. This allocates * a dirty root struct and adds it into the list of dead roots that need to @@ -1313,7 +1339,12 @@ void btrfs_add_dead_root(struct btrfs_root *root) spin_lock(&fs_info->trans_lock); if (list_empty(&root->root_list)) { btrfs_grab_root(root); - list_add_tail(&root->root_list, &fs_info->dead_roots); + + /* We want to process the partially complete drops first. */ + if (test_bit(BTRFS_ROOT_UNFINISHED_DROP, &root->state)) + list_add(&root->root_list, &fs_info->dead_roots); + else + list_add_tail(&root->root_list, &fs_info->dead_roots); } spin_unlock(&fs_info->trans_lock); } diff --git a/fs/btrfs/transaction.h b/fs/btrfs/transaction.h index eb26eb068fe8..0599e3994633 100644 --- a/fs/btrfs/transaction.h +++ b/fs/btrfs/transaction.h @@ -216,6 +216,7 @@ int btrfs_wait_for_commit(struct btrfs_fs_info *fs_info, u64 transid); void btrfs_add_dead_root(struct btrfs_root *root); int btrfs_defrag_root(struct btrfs_root *root); +void btrfs_maybe_wake_unfinished_drop(struct btrfs_fs_info *fs_info); int btrfs_clean_one_deleted_snapshot(struct btrfs_root *root); int btrfs_commit_transaction(struct btrfs_trans_handle *trans); int btrfs_commit_transaction_async(struct btrfs_trans_handle *trans, -- 2.39.2

2 1

[PATCH openEuler-22.03-LTS-SP1] nfc: pn533: Wait for out_urb's completion in pn533_usb_send_frame()
by Kaixiong Yu 04 Sep '24

04 Sep '24

From: Minsuk Kang <linuxlovemin(a)yonsei.ac.kr> stable inclusion from stable-v4.19.270 commit 321db5131c92983dac4f3338e8fbb6df214238c0 category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/IALIN1 CVE: CVE-2023-52907 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id… -------------------------------- [ Upstream commit 9dab880d675b9d0dd56c6428e4e8352a3339371d ] Fix a use-after-free that occurs in hcd when in_urb sent from pn533_usb_send_frame() is completed earlier than out_urb. Its callback frees the skb data in pn533_send_async_complete() that is used as a transfer buffer of out_urb. Wait before sending in_urb until the callback of out_urb is called. To modify the callback of out_urb alone, separate the complete function of out_urb and ack_urb. Found by a modified version of syzkaller. BUG: KASAN: use-after-free in dummy_timer Call Trace: memcpy (mm/kasan/shadow.c:65) dummy_perform_transfer (drivers/usb/gadget/udc/dummy_hcd.c:1352) transfer (drivers/usb/gadget/udc/dummy_hcd.c:1453) dummy_timer (drivers/usb/gadget/udc/dummy_hcd.c:1972) arch_static_branch (arch/x86/include/asm/jump_label.h:27) static_key_false (include/linux/jump_label.h:207) timer_expire_exit (include/trace/events/timer.h:127) call_timer_fn (kernel/time/timer.c:1475) expire_timers (kernel/time/timer.c:1519) __run_timers (kernel/time/timer.c:1790) run_timer_softirq (kernel/time/timer.c:1803) Fixes: c46ee38620a2 ("NFC: pn533: add NXP pn533 nfc device driver") Signed-off-by: Minsuk Kang <linuxlovemin(a)yonsei.ac.kr> Signed-off-by: David S. Miller <davem(a)davemloft.net> Signed-off-by: Sasha Levin <sashal(a)kernel.org> Signed-off-by: Kaixiong Yu <yukaixiong(a)huawei.com> --- drivers/nfc/pn533/usb.c | 44 ++++++++++++++++++++++++++++++++++++++--- 1 file changed, 41 insertions(+), 3 deletions(-) diff --git a/drivers/nfc/pn533/usb.c b/drivers/nfc/pn533/usb.c index 84f2983bf384..57b07446bb76 100644 --- a/drivers/nfc/pn533/usb.c +++ b/drivers/nfc/pn533/usb.c @@ -153,10 +153,17 @@ static int pn533_usb_send_ack(struct pn533 *dev, gfp_t flags) return usb_submit_urb(phy->ack_urb, flags); } +struct pn533_out_arg { + struct pn533_usb_phy *phy; + struct completion done; +}; + static int pn533_usb_send_frame(struct pn533 *dev, struct sk_buff *out) { struct pn533_usb_phy *phy = dev->phy; + struct pn533_out_arg arg; + void *cntx; int rc; if (phy->priv == NULL) @@ -168,10 +175,17 @@ static int pn533_usb_send_frame(struct pn533 *dev, print_hex_dump_debug("PN533 TX: ", DUMP_PREFIX_NONE, 16, 1, out->data, out->len, false); + init_completion(&arg.done); + cntx = phy->out_urb->context; + phy->out_urb->context = &arg; + rc = usb_submit_urb(phy->out_urb, GFP_KERNEL); if (rc) return rc; + wait_for_completion(&arg.done); + phy->out_urb->context = cntx; + if (dev->protocol_type == PN533_PROTO_REQ_RESP) { /* request for response for sent packet directly */ rc = pn533_submit_urb_for_response(phy, GFP_KERNEL); @@ -412,7 +426,31 @@ static int pn533_acr122_poweron_rdr(struct pn533_usb_phy *phy) return arg.rc; } -static void pn533_send_complete(struct urb *urb) +static void pn533_out_complete(struct urb *urb) +{ + struct pn533_out_arg *arg = urb->context; + struct pn533_usb_phy *phy = arg->phy; + + switch (urb->status) { + case 0: + break; /* success */ + case -ECONNRESET: + case -ENOENT: + dev_dbg(&phy->udev->dev, + "The urb has been stopped (status %d)\n", + urb->status); + break; + case -ESHUTDOWN: + default: + nfc_err(&phy->udev->dev, + "Urb failure (status %d)\n", + urb->status); + } + + complete(&arg->done); +} + +static void pn533_ack_complete(struct urb *urb) { struct pn533_usb_phy *phy = urb->context; @@ -500,10 +538,10 @@ static int pn533_usb_probe(struct usb_interface *interface, usb_fill_bulk_urb(phy->out_urb, phy->udev, usb_sndbulkpipe(phy->udev, out_endpoint), - NULL, 0, pn533_send_complete, phy); + NULL, 0, pn533_out_complete, phy); usb_fill_bulk_urb(phy->ack_urb, phy->udev, usb_sndbulkpipe(phy->udev, out_endpoint), - NULL, 0, pn533_send_complete, phy); + NULL, 0, pn533_ack_complete, phy); switch (id->driver_info) { case PN533_DEVICE_STD: -- 2.25.1

2 1

[PATCH openEuler-1.0-LTS] net: hns3: do not allow call hns3_nic_net_open repeatedly
by Xiang Yang 04 Sep '24

04 Sep '24

From: Jian Shen <shenjian15(a)huawei.com> mainline inclusion from mainline-v5.15-rc4 commit 5b09e88e1bf7fe86540fab4b5f3eece8abead39e category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/I9R4O1 CVE: CVE-2021-47400 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?… -------------------------------- hns3_nic_net_open() is not allowed to called repeatly, but there is no checking for this. When doing device reset and setup tc concurrently, there is a small oppotunity to call hns3_nic_net_open repeatedly, and cause kernel bug by calling napi_enable twice. The calltrace information is like below: [ 3078.222780] ------------[ cut here ]------------ [ 3078.230255] kernel BUG at net/core/dev.c:6991! [ 3078.236224] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP [ 3078.243431] Modules linked in: hns3 hclgevf hclge hnae3 vfio_iommu_type1 vfio_pci vfio_virqfd vfio pv680_mii(O) [ 3078.258880] CPU: 0 PID: 295 Comm: kworker/u8:5 Tainted: G O 5.14.0-rc4+ #1 [ 3078.269102] Hardware name: , BIOS KpxxxFPGA 1P B600 V181 08/12/2021 [ 3078.276801] Workqueue: hclge hclge_service_task [hclge] [ 3078.288774] pstate: 60400009 (nZCv daif +PAN -UAO -TCO BTYPE=--) [ 3078.296168] pc : napi_enable+0x80/0x84 tc qdisc sho[w 3d0e7v8 .e3t0h218 79] lr : hns3_nic_net_open+0x138/0x510 [hns3] [ 3078.314771] sp : ffff8000108abb20 [ 3078.319099] x29: ffff8000108abb20 x28: 0000000000000000 x27: ffff0820a8490300 [ 3078.329121] x26: 0000000000000001 x25: ffff08209cfc6200 x24: 0000000000000000 [ 3078.339044] x23: ffff0820a8490300 x22: ffff08209cd76000 x21: ffff0820abfe3880 [ 3078.349018] x20: 0000000000000000 x19: ffff08209cd76900 x18: 0000000000000000 [ 3078.358620] x17: 0000000000000000 x16: ffffc816e1727a50 x15: 0000ffff8f4ff930 [ 3078.368895] x14: 0000000000000000 x13: 0000000000000000 x12: 0000259e9dbeb6b4 [ 3078.377987] x11: 0096a8f7e764eb40 x10: 634615ad28d3eab5 x9 : ffffc816ad8885b8 [ 3078.387091] x8 : ffff08209cfc6fb8 x7 : ffff0820ac0da058 x6 : ffff0820a8490344 [ 3078.396356] x5 : 0000000000000140 x4 : 0000000000000003 x3 : ffff08209cd76938 [ 3078.405365] x2 : 0000000000000000 x1 : 0000000000000010 x0 : ffff0820abfe38a0 [ 3078.414657] Call trace: [ 3078.418517] napi_enable+0x80/0x84 [ 3078.424626] hns3_reset_notify_up_enet+0x78/0xd0 [hns3] [ 3078.433469] hns3_reset_notify+0x64/0x80 [hns3] [ 3078.441430] hclge_notify_client+0x68/0xb0 [hclge] [ 3078.450511] hclge_reset_rebuild+0x524/0x884 [hclge] [ 3078.458879] hclge_reset_service_task+0x3c4/0x680 [hclge] [ 3078.467470] hclge_service_task+0xb0/0xb54 [hclge] [ 3078.475675] process_one_work+0x1dc/0x48c [ 3078.481888] worker_thread+0x15c/0x464 [ 3078.487104] kthread+0x160/0x170 [ 3078.492479] ret_from_fork+0x10/0x18 [ 3078.498785] Code: c8027c81 35ffffa2 d50323bf d65f03c0 (d4210000) [ 3078.506889] ---[ end trace 8ebe0340a1b0fb44 ]--- Once hns3_nic_net_open() is excute success, the flag HNS3_NIC_STATE_DOWN will be cleared. So add checking for this flag, directly return when HNS3_NIC_STATE_DOWN is no set. Fixes: e888402789b9 ("net: hns3: call hns3_nic_net_open() while doing HNAE3_UP_CLIENT") Signed-off-by: Jian Shen <shenjian15(a)huawei.com> Signed-off-by: Guangbin Huang <huangguangbin2(a)huawei.com> Signed-off-by: David S. Miller <davem(a)davemloft.net> Signed-off-by: Xiang Yang <xiangyang3(a)huawei.com> --- drivers/net/ethernet/hisilicon/hns3/hns3_enet.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c index 20703ce86281..89f1d0597844 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c @@ -438,6 +438,11 @@ static int hns3_nic_net_open(struct net_device *netdev) if (hns3_nic_resetting(netdev)) return -EBUSY; + if (!test_bit(HNS3_NIC_STATE_DOWN, &priv->state)) { + netdev_warn(netdev, "net open repeatedly!\n"); + return 0; + } + netif_carrier_off(netdev); ret = hns3_nic_set_real_num_queue(netdev); -- 2.34.1

2 1

[PATCH OLK-5.10] dma: fix call order in dmam_free_coherent
by Kaixiong Yu 04 Sep '24

04 Sep '24

From: Lance Richardson <rlance(a)google.com> stable inclusion from stable-v6.6.44 commit 1fe97f68fce1ba24bf823bfb0eb0956003473130 category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/IAKQ2W CVE: CVE-2024-43856 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id… -------------------------------- [ Upstream commit 28e8b7406d3a1f5329a03aa25a43aa28e087cb20 ] dmam_free_coherent() frees a DMA allocation, which makes the freed vaddr available for reuse, then calls devres_destroy() to remove and free the data structure used to track the DMA allocation. Between the two calls, it is possible for a concurrent task to make an allocation with the same vaddr and add it to the devres list. If this happens, there will be two entries in the devres list with the same vaddr and devres_destroy() can free the wrong entry, triggering the WARN_ON() in dmam_match. Fix by destroying the devres entry before freeing the DMA allocation. Tested: kokonut //net/encryption http://sponge2/b9145fe6-0f72-4325-ac2f-a84d81075b03 Fixes: 9ac7849e35f7 ("devres: device resource management") Signed-off-by: Lance Richardson <rlance(a)google.com> Signed-off-by: Christoph Hellwig <hch(a)lst.de> Signed-off-by: Sasha Levin <sashal(a)kernel.org> Signed-off-by: Kaixiong Yu <yukaixiong(a)huawei.com> --- kernel/dma/mapping.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/dma/mapping.c b/kernel/dma/mapping.c index 51bb8fa8eb89..453c0fbe87ff 100644 --- a/kernel/dma/mapping.c +++ b/kernel/dma/mapping.c @@ -60,8 +60,8 @@ void dmam_free_coherent(struct device *dev, size_t size, void *vaddr, { struct dma_devres match_data = { size, vaddr, dma_handle }; - dma_free_coherent(dev, size, vaddr, dma_handle); WARN_ON(devres_destroy(dev, dmam_release, dmam_match, &match_data)); + dma_free_coherent(dev, size, vaddr, dma_handle); } EXPORT_SYMBOL(dmam_free_coherent); -- 2.25.1

2 1

[PATCH openEuler-1.0-LTS] usb: gadget: f_ncm: fix potential NULL ptr deref in ncm_bitrate()
by Wang Hai 04 Sep '24

04 Sep '24

From: Maciej Żenczykowski <maze(a)google.com> stable inclusion from stable-v4.19.271 commit 63d161f29cd39c050e8873aa36e0c9fc013bb763 category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/IALIDP CVE: CVE-2023-52894 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id… -------------------------------- commit c6ec929595c7443250b2a4faea988c62019d5cd2 upstream. In Google internal bug 265639009 we've received an (as yet) unreproducible crash report from an aarch64 GKI 5.10.149-android13 running device. AFAICT the source code is at: https://android.googlesource.com/kernel/common/+/refs/tags/ASB-2022-12-05_1… The call stack is: ncm_close() -> ncm_notify() -> ncm_do_notify() with the crash at: ncm_do_notify+0x98/0x270 Code: 79000d0b b9000a6c f940012a f9400269 (b9405d4b) Which I believe disassembles to (I don't know ARM assembly, but it looks sane enough to me...): // halfword (16-bit) store presumably to event->wLength (at offset 6 of struct usb_cdc_notification) 0B 0D 00 79 strh w11, [x8, #6] // word (32-bit) store presumably to req->Length (at offset 8 of struct usb_request) 6C 0A 00 B9 str w12, [x19, #8] // x10 (NULL) was read here from offset 0 of valid pointer x9 // IMHO we're reading 'cdev->gadget' and getting NULL // gadget is indeed at offset 0 of struct usb_composite_dev 2A 01 40 F9 ldr x10, [x9] // loading req->buf pointer, which is at offset 0 of struct usb_request 69 02 40 F9 ldr x9, [x19] // x10 is null, crash, appears to be attempt to read cdev->gadget->max_speed 4B 5D 40 B9 ldr w11, [x10, #0x5c] which seems to line up with ncm_do_notify() case NCM_NOTIFY_SPEED code fragment: event->wLength = cpu_to_le16(8); req->length = NCM_STATUS_BYTECOUNT; /* SPEED_CHANGE data is up/down speeds in bits/sec */ data = req->buf + sizeof *event; data[0] = cpu_to_le32(ncm_bitrate(cdev->gadget)); My analysis of registers and NULL ptr deref crash offset (Unable to handle kernel NULL pointer dereference at virtual address 000000000000005c) heavily suggests that the crash is due to 'cdev->gadget' being NULL when executing: data[0] = cpu_to_le32(ncm_bitrate(cdev->gadget)); which calls: ncm_bitrate(NULL) which then calls: gadget_is_superspeed(NULL) which reads ((struct usb_gadget *)NULL)->max_speed and hits a panic. AFAICT, if I'm counting right, the offset of max_speed is indeed 0x5C. (remember there's a GKI KABI reservation of 16 bytes in struct work_struct) It's not at all clear to me how this is all supposed to work... but returning 0 seems much better than panic-ing... Cc: Felipe Balbi <balbi(a)kernel.org> Cc: Lorenzo Colitti <lorenzo(a)google.com> Cc: Carlos Llamas <cmllamas(a)google.com> Cc: stable(a)vger.kernel.org Signed-off-by: Maciej Żenczykowski <maze(a)google.com> Cc: stable <stable(a)kernel.org> Link: https://lore.kernel.org/r/20230117131839.1138208-1-maze@google.com Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> Conflicts: drivers/usb/gadget/function/f_ncm.c [Context conflict] Signed-off-by: Wang Hai <wanghai38(a)huawei.com> --- drivers/usb/gadget/function/f_ncm.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/usb/gadget/function/f_ncm.c b/drivers/usb/gadget/function/f_ncm.c index 3da7e6511fc6..6031e96cf351 100644 --- a/drivers/usb/gadget/function/f_ncm.c +++ b/drivers/usb/gadget/function/f_ncm.c @@ -85,7 +85,9 @@ static inline struct f_ncm *func_to_ncm(struct usb_function *f) /* peak (theoretical) bulk transfer rate in bits-per-second */ static inline unsigned ncm_bitrate(struct usb_gadget *g) { - if (gadget_is_superspeed(g) && g->speed == USB_SPEED_SUPER) + if (!g) + return 0; + else if (gadget_is_superspeed(g) && g->speed == USB_SPEED_SUPER) return 13 * 1024 * 8 * 1000 * 8; else if (gadget_is_dualspeed(g) && g->speed == USB_SPEED_HIGH) return 13 * 512 * 8 * 1000 * 8; -- 2.17.1

2 1

[PATCH OLK-5.10] btrfs: do not WARN_ON() if we have PageError set
by Yu Kuai 04 Sep '24

04 Sep '24

From: Josef Bacik <josef(a)toxicpanda.com> mainline inclusion from mainline-v5.17-rc7 commit a50e1fcbc9b85fd4e95b89a75c0884cb032a3e06 category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/IALPSO CVE: CVE-2022-48902 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?… ------------------------------------------- Whenever we do any extent buffer operations we call assert_eb_page_uptodate() to complain loudly if we're operating on an non-uptodate page. Our overnight tests caught this warning earlier this week WARNING: CPU: 1 PID: 553508 at fs/btrfs/extent_io.c:6849 assert_eb_page_uptodate+0x3f/0x50 CPU: 1 PID: 553508 Comm: kworker/u4:13 Tainted: G W 5.17.0-rc3+ #564 Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.13.0-2.fc32 04/01/2014 Workqueue: btrfs-cache btrfs_work_helper RIP: 0010:assert_eb_page_uptodate+0x3f/0x50 RSP: 0018:ffffa961440a7c68 EFLAGS: 00010246 RAX: 0017ffffc0002112 RBX: ffffe6e74453f9c0 RCX: 0000000000001000 RDX: ffffe6e74467c887 RSI: ffffe6e74453f9c0 RDI: ffff8d4c5efc2fc0 RBP: 0000000000000d56 R08: ffff8d4d4a224000 R09: 0000000000000000 R10: 00015817fa9d1ef0 R11: 000000000000000c R12: 00000000000007b1 R13: ffff8d4c5efc2fc0 R14: 0000000001500000 R15: 0000000001cb1000 FS: 0000000000000000(0000) GS:ffff8d4dbbd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007ff31d3448d8 CR3: 0000000118be8004 CR4: 0000000000370ee0 Call Trace: extent_buffer_test_bit+0x3f/0x70 free_space_test_bit+0xa6/0xc0 load_free_space_tree+0x1f6/0x470 caching_thread+0x454/0x630 ? rcu_read_lock_sched_held+0x12/0x60 ? rcu_read_lock_sched_held+0x12/0x60 ? rcu_read_lock_sched_held+0x12/0x60 ? lock_release+0x1f0/0x2d0 btrfs_work_helper+0xf2/0x3e0 ? lock_release+0x1f0/0x2d0 ? finish_task_switch.isra.0+0xf9/0x3a0 process_one_work+0x26d/0x580 ? process_one_work+0x580/0x580 worker_thread+0x55/0x3b0 ? process_one_work+0x580/0x580 kthread+0xf0/0x120 ? kthread_complete_and_exit+0x20/0x20 ret_from_fork+0x1f/0x30 This was partially fixed by c2e39305299f01 ("btrfs: clear extent buffer uptodate when we fail to write it"), however all that fix did was keep us from finding extent buffers after a failed writeout. It didn't keep us from continuing to use a buffer that we already had found. In this case we're searching the commit root to cache the block group, so we can start committing the transaction and switch the commit root and then start writing. After the switch we can look up an extent buffer that hasn't been written yet and start processing that block group. Then we fail to write that block out and clear Uptodate on the page, and then we start spewing these errors. Normally we're protected by the tree lock to a certain degree here. If we read a block we have that block read locked, and we block the writer from locking the block before we submit it for the write. However this isn't necessarily fool proof because the read could happen before we do the submit_bio and after we locked and unlocked the extent buffer. Also in this particular case we have path->skip_locking set, so that won't save us here. We'll simply get a block that was valid when we read it, but became invalid while we were using it. What we really want is to catch the case where we've "read" a block but it's not marked Uptodate. On read we ClearPageError(), so if we're !Uptodate and !Error we know we didn't do the right thing for reading the page. Fix this by checking !Uptodate && !Error, this way we will not complain if our buffer gets invalidated while we're using it, and we'll maintain the spirit of the check which is to make sure we have a fully in-cache block while we're messing with it. CC: stable(a)vger.kernel.org # 5.4+ Signed-off-by: Josef Bacik <josef(a)toxicpanda.com> Signed-off-by: David Sterba <dsterba(a)suse.com> Conflicts: fs/btrfs/extent_io.c [1. commit a1d767c11cca ("btrfs: introduce helpers for subpage uptodate status") is not backported, subpage is not supported. 2. commit b8f957715eae ("btrfs: support page uptodate assertions in subpage mode") is not backported, the helper assert_eb_page_uptodate() is not introduced.] Signed-off-by: Yu Kuai <yukuai3(a)huawei.com> --- fs/btrfs/extent_io.c | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index 104e2f1fe4f7..6bcaf539f894 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -5734,7 +5734,7 @@ void write_extent_buffer_chunk_tree_uuid(const struct extent_buffer *eb, { char *kaddr; - WARN_ON(!PageUptodate(eb->pages[0])); + WARN_ON(!PageUptodate(eb->pages[0]) && !PageError(eb->pages[0])); kaddr = page_address(eb->pages[0]); memcpy(kaddr + offsetof(struct btrfs_header, chunk_tree_uuid), srcv, BTRFS_FSID_SIZE); @@ -5744,7 +5744,7 @@ void write_extent_buffer_fsid(const struct extent_buffer *eb, const void *srcv) { char *kaddr; - WARN_ON(!PageUptodate(eb->pages[0])); + WARN_ON(!PageUptodate(eb->pages[0]) && !PageError(eb->pages[0])); kaddr = page_address(eb->pages[0]); memcpy(kaddr + offsetof(struct btrfs_header, fsid), srcv, BTRFS_FSID_SIZE); @@ -5767,7 +5767,7 @@ void write_extent_buffer(const struct extent_buffer *eb, const void *srcv, while (len > 0) { page = eb->pages[i]; - WARN_ON(!PageUptodate(page)); + WARN_ON(!PageUptodate(page) && !PageError(page)); cur = min(len, PAGE_SIZE - offset); kaddr = page_address(page); @@ -5796,7 +5796,7 @@ void memzero_extent_buffer(const struct extent_buffer *eb, unsigned long start, while (len > 0) { page = eb->pages[i]; - WARN_ON(!PageUptodate(page)); + WARN_ON(!PageUptodate(page) && !PageError(page)); cur = min(len, PAGE_SIZE - offset); kaddr = page_address(page); @@ -5844,7 +5844,7 @@ void copy_extent_buffer(const struct extent_buffer *dst, while (len > 0) { page = dst->pages[i]; - WARN_ON(!PageUptodate(page)); + WARN_ON(!PageUptodate(page) && !PageError(page)); cur = min(len, (unsigned long)(PAGE_SIZE - offset)); @@ -5906,7 +5906,8 @@ int extent_buffer_test_bit(const struct extent_buffer *eb, unsigned long start, eb_bitmap_offset(eb, start, nr, &i, &offset); page = eb->pages[i]; - WARN_ON(!PageUptodate(page)); + WARN_ON(!PageUptodate(page) && !PageError(page)); + kaddr = page_address(page); return 1U & (kaddr[offset] >> (nr & (BITS_PER_BYTE - 1))); } @@ -5931,7 +5932,7 @@ void extent_buffer_bitmap_set(const struct extent_buffer *eb, unsigned long star eb_bitmap_offset(eb, start, pos, &i, &offset); page = eb->pages[i]; - WARN_ON(!PageUptodate(page)); + WARN_ON(!PageUptodate(page) && !PageError(page)); kaddr = page_address(page); while (len >= bits_to_set) { @@ -5942,7 +5943,7 @@ void extent_buffer_bitmap_set(const struct extent_buffer *eb, unsigned long star if (++offset >= PAGE_SIZE && len > 0) { offset = 0; page = eb->pages[++i]; - WARN_ON(!PageUptodate(page)); + WARN_ON(!PageUptodate(page) && !PageError(page)); kaddr = page_address(page); } } @@ -5974,7 +5975,7 @@ void extent_buffer_bitmap_clear(const struct extent_buffer *eb, eb_bitmap_offset(eb, start, pos, &i, &offset); page = eb->pages[i]; - WARN_ON(!PageUptodate(page)); + WARN_ON(!PageUptodate(page) && !PageError(page)); kaddr = page_address(page); while (len >= bits_to_clear) { @@ -5985,7 +5986,7 @@ void extent_buffer_bitmap_clear(const struct extent_buffer *eb, if (++offset >= PAGE_SIZE && len > 0) { offset = 0; page = eb->pages[++i]; - WARN_ON(!PageUptodate(page)); + WARN_ON(!PageUptodate(page) && !PageError(page)); kaddr = page_address(page); } } -- 2.39.2

2 1

[PATCH OLK-6.6 0/4] mainline patches
by Chen Ridong 04 Sep '24

04 Sep '24

Chen Ridong (1): cgroup/cpuset: fix panic caused by partcmd_update Waiman Long (3): cgroup/cpuset: Optimize isolated partition only generate_sched_domains() calls cgroup/cpuset: Fix remote root partition creation problem cgroup/cpuset: Clear effective_xcpus on cpus_allowed clearing only if cpus.exclusive not set kernel/cgroup/cpuset.c | 68 +++++++++++++++++++++++++++++++++--------- 1 file changed, 54 insertions(+), 14 deletions(-) -- 2.34.1

2 6

[PATCH OLK-5.10 0/3] btrfs: do not WARN_ON() if we have PageError set
by Yu Kuai 04 Sep '24

04 Sep '24

Josef Bacik (1): btrfs: do not WARN_ON() if we have PageError set Qu Wenruo (2): btrfs: handle sectorsize < PAGE_SIZE case for extent buffer accessors btrfs: support page uptodate assertions in subpage mode fs/btrfs/ctree.c | 3 +- fs/btrfs/ctree.h | 38 ++++++++++++- fs/btrfs/extent_io.c | 116 +++++++++++++++++++++++++++------------- fs/btrfs/struct-funcs.c | 18 ++++--- 4 files changed, 127 insertions(+), 48 deletions(-) -- 2.39.2

2 4

[PATCH OLK-6.6] ext4: Track data blocks freeing operation in journal
by Zhihao Cheng 04 Sep '24

04 Sep '24

Offering: HULK hulk inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I9DN5Z -------------------------------- Since commit 7f6416dcd4a3f ("ext4: implement writeback iomap path"), the order mode is removed in iomap framework, which lets ext4_mb_clear_bb() free data blocks immediately. It may cause stale data read from truncated file in power-cut case. Following is details: P1 P2 vfs_truncate(file A) ext4_setattr EXT4_I(inode)->i_disksize = attr->ia_size // record in journal ext4_truncate ext4_mb_clear_bb mb_free_blocks // free block i vfs_write(file B) // get block i and writeback >> powercut << In the next mount, inode size and extent tree is stale(before truncated), the content in block i is file B. Fix the problem by tracking free data blocks in journal for iomap/non-writeback case. Fixes: 7f6416dcd4a3 ("ext4: implement writeback iomap path") Signed-off-by: Zhihao Cheng <chengzhihao1(a)huawei.com> Signed-off-by: Zhihao Cheng <chengzhihao(a)huaweicloud.com> --- fs/ext4/ext4_jbd2.c | 6 ------ fs/ext4/ext4_jbd2.h | 7 +++++++ 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/fs/ext4/ext4_jbd2.c b/fs/ext4/ext4_jbd2.c index 94c8073b49e7..d1a2e6624401 100644 --- a/fs/ext4/ext4_jbd2.c +++ b/fs/ext4/ext4_jbd2.c @@ -11,12 +11,6 @@ int ext4_inode_journal_mode(struct inode *inode) { if (EXT4_JOURNAL(inode) == NULL) return EXT4_INODE_WRITEBACK_DATA_MODE; /* writeback */ - /* - * Ordered mode is no longer needed for the inode that use the - * iomap path, always use writeback mode. - */ - if (ext4_test_inode_state(inode, EXT4_STATE_BUFFERED_IOMAP)) - return EXT4_INODE_WRITEBACK_DATA_MODE; /* writeback */ /* We do not support data journalling with delayed allocation */ if (!S_ISREG(inode->i_mode) || ext4_test_inode_flag(inode, EXT4_INODE_EA_INODE) || diff --git a/fs/ext4/ext4_jbd2.h b/fs/ext4/ext4_jbd2.h index 0c77697d5e90..c52d1caf6622 100644 --- a/fs/ext4/ext4_jbd2.h +++ b/fs/ext4/ext4_jbd2.h @@ -467,6 +467,13 @@ static inline int ext4_should_journal_data(struct inode *inode) static inline int ext4_should_order_data(struct inode *inode) { + /* + * Ordered mode is no longer needed for the inode that use the + * iomap path, always use writeback mode. + */ + if (ext4_test_inode_state(inode, EXT4_STATE_BUFFERED_IOMAP)) + return 0; /* writeback */ + return ext4_inode_journal_mode(inode) & EXT4_INODE_ORDERED_DATA_MODE; } -- 2.31.1

2 1

【重要】openEuler 24.03 LTS SP1版本需求收集中，请大家提交需要合入的特性到release plan
by Sujinling 04 Sep '24

04 Sep '24

大家好， openEuler 24.03 LTS SP1是基于6.6内核的24.03-LTS版本增强扩展版本，按release-plan计划启动需求收集，欢迎各sig maintainer、伙伴和社区开发者们积极反馈和交流。请大家提交需要合入的特性清单到release plan上，感谢！ openEuler 24.03 LTS SP1版本release plan详细版本计划： https://gitee.com/openeuler/release-management/blob/master/openEuler-24.03-… 需求申请流程链接，请按照流程步骤提交，issue 类型选择需求， issue 标题以 [openEuler-24.03 LTS SP1] 开头： https://gitee.com/openeuler/release-management/blob/master/Goverance/openEu… Thanks & best regards, 苏锦铃

1 1

[PATCH openEuler-22.03-LTS-SP1] fuse: Initialize beyond-EOF page contents before setting uptodate
by Yifan Qiao 04 Sep '24

04 Sep '24

From: Jann Horn <jannh(a)google.com> stable inclusion from stable-v6.6.48 commit ac42e0f0eb66af966015ee33fd355bc6f5d80cd6 category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/IAOAMF CVE: CVE-2024-44947 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id… -------------------------------- commit 3c0da3d163eb32f1f91891efaade027fa9b245b9 upstream. fuse_notify_store(), unlike fuse_do_readpage(), does not enable page zeroing (because it can be used to change partial page contents). So fuse_notify_store() must be more careful to fully initialize page contents (including parts of the page that are beyond end-of-file) before marking the page uptodate. The current code can leave beyond-EOF page contents uninitialized, which makes these uninitialized page contents visible to userspace via mmap(). This is an information leak, but only affects systems which do not enable init-on-alloc (via CONFIG_INIT_ON_ALLOC_DEFAULT_ON=y or the corresponding kernel command line parameter). Link: https://bugs.chromium.org/p/project-zero/issues/detail?id=2574 Cc: stable(a)kernel.org Fixes: a1d75f258230 ("fuse: add store request") Signed-off-by: Jann Horn <jannh(a)google.com> Signed-off-by: Linus Torvalds <torvalds(a)linux-foundation.org> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> Signed-off-by: Yifan Qiao <qiaoyifan4(a)huawei.com> --- fs/fuse/dev.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/fs/fuse/dev.c b/fs/fuse/dev.c index 8ac91ba05d6d..e6cbed7aedcb 100644 --- a/fs/fuse/dev.c +++ b/fs/fuse/dev.c @@ -1627,9 +1627,11 @@ static int fuse_notify_store(struct fuse_conn *fc, unsigned int size, this_num = min_t(unsigned, num, PAGE_SIZE - offset); err = fuse_copy_page(cs, &page, offset, this_num, 0); - if (!err && offset == 0 && - (this_num == PAGE_SIZE || file_size == end)) + if (!PageUptodate(page) && !err && offset == 0 && + (this_num == PAGE_SIZE || file_size == end)) { + zero_user_segment(page, this_num, PAGE_SIZE); SetPageUptodate(page); + } unlock_page(page); put_page(page); -- 2.39.2

2 1

[PATCH OLK-5.10] fuse: Initialize beyond-EOF page contents before setting uptodate
by Yifan Qiao 04 Sep '24

04 Sep '24

From: Jann Horn <jannh(a)google.com> stable inclusion from stable-v6.6.48 commit ac42e0f0eb66af966015ee33fd355bc6f5d80cd6 category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/IAOAMF CVE: CVE-2024-44947 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id… -------------------------------- commit 3c0da3d163eb32f1f91891efaade027fa9b245b9 upstream. fuse_notify_store(), unlike fuse_do_readpage(), does not enable page zeroing (because it can be used to change partial page contents). So fuse_notify_store() must be more careful to fully initialize page contents (including parts of the page that are beyond end-of-file) before marking the page uptodate. The current code can leave beyond-EOF page contents uninitialized, which makes these uninitialized page contents visible to userspace via mmap(). This is an information leak, but only affects systems which do not enable init-on-alloc (via CONFIG_INIT_ON_ALLOC_DEFAULT_ON=y or the corresponding kernel command line parameter). Link: https://bugs.chromium.org/p/project-zero/issues/detail?id=2574 Cc: stable(a)kernel.org Fixes: a1d75f258230 ("fuse: add store request") Signed-off-by: Jann Horn <jannh(a)google.com> Signed-off-by: Linus Torvalds <torvalds(a)linux-foundation.org> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> Signed-off-by: Yifan Qiao <qiaoyifan4(a)huawei.com> --- fs/fuse/dev.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/fs/fuse/dev.c b/fs/fuse/dev.c index 8ac91ba05d6d..e6cbed7aedcb 100644 --- a/fs/fuse/dev.c +++ b/fs/fuse/dev.c @@ -1627,9 +1627,11 @@ static int fuse_notify_store(struct fuse_conn *fc, unsigned int size, this_num = min_t(unsigned, num, PAGE_SIZE - offset); err = fuse_copy_page(cs, &page, offset, this_num, 0); - if (!err && offset == 0 && - (this_num == PAGE_SIZE || file_size == end)) + if (!PageUptodate(page) && !err && offset == 0 && + (this_num == PAGE_SIZE || file_size == end)) { + zero_user_segment(page, this_num, PAGE_SIZE); SetPageUptodate(page); + } unlock_page(page); put_page(page); -- 2.39.2

2 1

[PATCH OLK-6.6] fuse: Initialize beyond-EOF page contents before setting uptodate
by Yifan Qiao 04 Sep '24

04 Sep '24

From: Jann Horn <jannh(a)google.com> stable inclusion from stable-v6.6.48 commit ac42e0f0eb66af966015ee33fd355bc6f5d80cd6 category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/IAOAMF CVE: CVE-2024-44947 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id… -------------------------------- commit 3c0da3d163eb32f1f91891efaade027fa9b245b9 upstream. fuse_notify_store(), unlike fuse_do_readpage(), does not enable page zeroing (because it can be used to change partial page contents). So fuse_notify_store() must be more careful to fully initialize page contents (including parts of the page that are beyond end-of-file) before marking the page uptodate. The current code can leave beyond-EOF page contents uninitialized, which makes these uninitialized page contents visible to userspace via mmap(). This is an information leak, but only affects systems which do not enable init-on-alloc (via CONFIG_INIT_ON_ALLOC_DEFAULT_ON=y or the corresponding kernel command line parameter). Link: https://bugs.chromium.org/p/project-zero/issues/detail?id=2574 Cc: stable(a)kernel.org Fixes: a1d75f258230 ("fuse: add store request") Signed-off-by: Jann Horn <jannh(a)google.com> Signed-off-by: Linus Torvalds <torvalds(a)linux-foundation.org> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> Signed-off-by: Yifan Qiao <qiaoyifan4(a)huawei.com> --- fs/fuse/dev.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/fs/fuse/dev.c b/fs/fuse/dev.c index 1a8f82f478cb..8573d79ef29c 100644 --- a/fs/fuse/dev.c +++ b/fs/fuse/dev.c @@ -1618,9 +1618,11 @@ static int fuse_notify_store(struct fuse_conn *fc, unsigned int size, this_num = min_t(unsigned, num, PAGE_SIZE - offset); err = fuse_copy_page(cs, &page, offset, this_num, 0); - if (!err && offset == 0 && - (this_num == PAGE_SIZE || file_size == end)) + if (!PageUptodate(page) && !err && offset == 0 && + (this_num == PAGE_SIZE || file_size == end)) { + zero_user_segment(page, this_num, PAGE_SIZE); SetPageUptodate(page); + } unlock_page(page); put_page(page); -- 2.39.2

2 1

[PATCH openEuler-1.0-LTS V2] cifs: Fix pages leak when cifs_writedata allocate fails in cifs_writedata_direct_alloc()
by Zizhi Wo 04 Sep '24

04 Sep '24

hulk inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/IAOH1I -------------------------------- The function cifs_writedata_alloc() first allocates pages and then executes cifs_writedata_direct_alloc(). If the subsequent allocation of wdata fails, the previously allocated pages are not released, leading to a memory leak: hex dump (first 8 bytes): 00 00 00 00 00 00 00 00 ........ backtrace: [<00000000c2b9cca5>] cifs_write_from_iter.isra.0+0x103/0x730 [<0000000069a4b2e6>] cifs_user_writev+0x138/0x280 [<00000000bfdbeacd>] __vfs_write+0x12b/0x190 [<00000000460567cf>] vfs_write+0xac/0x1a0 [<00000000f92e5dfc>] ksys_write+0x43/0xb0 [<0000000094ddd57e>] do_syscall_64+0x63/0x220 [<0000000055ce26ff>] entry_SYSCALL_64_after_hwframe+0x5c/0xc1 [<00000000359223fa>] 0xffffffffffffffff This issue can be avoided by promptly using kvfree. Fixes: 8e7360f67e75 ("CIFS: Add support for direct pages in wdata") Signed-off-by: Zizhi Wo <wozizhi(a)huawei.com> --- fs/cifs/cifssmb.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/cifs/cifssmb.c b/fs/cifs/cifssmb.c index cb70f0c6aa1b..7950c906e7b3 100644 --- a/fs/cifs/cifssmb.c +++ b/fs/cifs/cifssmb.c @@ -2129,6 +2129,8 @@ cifs_writedata_direct_alloc(struct page **pages, work_func_t complete) INIT_LIST_HEAD(&wdata->list); init_completion(&wdata->done); INIT_WORK(&wdata->work, complete); + } else { + kvfree(pages); } return wdata; } -- 2.39.2

2 1

[PATCH OLK-5.10] dmaengine: idxd: Prevent use after free on completion memory
by Liu Mingrui 04 Sep '24

04 Sep '24

From: Reinette Chatre <reinette.chatre(a)intel.com> stable inclusion from stable-v6.1.8 commit b9e8e3fcfec625fc1c2f68f684448aeeb882625b category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/IALILE CVE: CVE-2022-48867 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id… -------------------------------- commit 1beeec45f9ac31eba52478379f70a5fa9c2ad005 upstream. On driver unload any pending descriptors are flushed at the time the interrupt is freed: idxd_dmaengine_drv_remove() -> drv_disable_wq() -> idxd_wq_free_irq() -> idxd_flush_pending_descs(). If there are any descriptors present that need to be flushed this flow triggers a "not present" page fault as below: BUG: unable to handle page fault for address: ff391c97c70c9040 #PF: supervisor read access in kernel mode #PF: error_code(0x0000) - not-present page The address that triggers the fault is the address of the descriptor that was freed moments earlier via: drv_disable_wq()->idxd_wq_free_resources() Fix the use after free by freeing the descriptors after any possible usage. This is done after idxd_wq_reset() to ensure that the memory remains accessible during possible completion writes by the device. Fixes: 63c14ae6c161 ("dmaengine: idxd: refactor wq driver enable/disable operations") Suggested-by: Dave Jiang <dave.jiang(a)intel.com> Signed-off-by: Reinette Chatre <reinette.chatre(a)intel.com> Reviewed-by: Dave Jiang <dave.jiang(a)intel.com> Reviewed-by: Fenghua Yu <fenghua.yu(a)intel.com> Cc: stable(a)vger.kernel.org Link: https://lore.kernel.org/r/6c4657d9cff0a0a00501a7b928297ac966e9ec9d.16704524… Signed-off-by: Vinod Koul <vkoul(a)kernel.org> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> Signed-off-by: Liu Mingrui <liumingrui(a)huawei.com> --- drivers/dma/idxd/device.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/dma/idxd/device.c b/drivers/dma/idxd/device.c index 06f5d3783d77..300ee3b5cf63 100644 --- a/drivers/dma/idxd/device.c +++ b/drivers/dma/idxd/device.c @@ -1408,11 +1408,11 @@ void drv_disable_wq(struct idxd_wq *wq) dev_warn(dev, "Clients has claim on wq %d: %d\n", wq->id, idxd_wq_refcount(wq)); - idxd_wq_free_resources(wq); idxd_wq_unmap_portal(wq); idxd_wq_drain(wq); idxd_wq_free_irq(wq); idxd_wq_reset(wq); + idxd_wq_free_resources(wq); percpu_ref_exit(&wq->wq_active); wq->type = IDXD_WQT_NONE; wq->client_count = 0; -- 2.25.1

2 1

[PATCH OLK-5.10] dmaengine: idxd: Prevent use after free on completion memory
by Liu Mingrui 04 Sep '24

04 Sep '24

From: Reinette Chatre <reinette.chatre(a)intel.com> stable inclusion from stable-v6.1.8~66 commit b9e8e3fcfec625fc1c2f68f684448aeeb882625b category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/IALILE CVE: CVE-2022-48867 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id… -------------------------------- commit 1beeec45f9ac31eba52478379f70a5fa9c2ad005 upstream. On driver unload any pending descriptors are flushed at the time the interrupt is freed: idxd_dmaengine_drv_remove() -> drv_disable_wq() -> idxd_wq_free_irq() -> idxd_flush_pending_descs(). If there are any descriptors present that need to be flushed this flow triggers a "not present" page fault as below: BUG: unable to handle page fault for address: ff391c97c70c9040 #PF: supervisor read access in kernel mode #PF: error_code(0x0000) - not-present page The address that triggers the fault is the address of the descriptor that was freed moments earlier via: drv_disable_wq()->idxd_wq_free_resources() Fix the use after free by freeing the descriptors after any possible usage. This is done after idxd_wq_reset() to ensure that the memory remains accessible during possible completion writes by the device. Fixes: 63c14ae6c161 ("dmaengine: idxd: refactor wq driver enable/disable operations") Suggested-by: Dave Jiang <dave.jiang(a)intel.com> Signed-off-by: Reinette Chatre <reinette.chatre(a)intel.com> Reviewed-by: Dave Jiang <dave.jiang(a)intel.com> Reviewed-by: Fenghua Yu <fenghua.yu(a)intel.com> Cc: stable(a)vger.kernel.org Link: https://lore.kernel.org/r/6c4657d9cff0a0a00501a7b928297ac966e9ec9d.16704524… Signed-off-by: Vinod Koul <vkoul(a)kernel.org> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> Signed-off-by: Liu Mingrui <liumingrui(a)huawei.com> --- drivers/dma/idxd/device.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/dma/idxd/device.c b/drivers/dma/idxd/device.c index 06f5d3783d77..300ee3b5cf63 100644 --- a/drivers/dma/idxd/device.c +++ b/drivers/dma/idxd/device.c @@ -1408,11 +1408,11 @@ void drv_disable_wq(struct idxd_wq *wq) dev_warn(dev, "Clients has claim on wq %d: %d\n", wq->id, idxd_wq_refcount(wq)); - idxd_wq_free_resources(wq); idxd_wq_unmap_portal(wq); idxd_wq_drain(wq); idxd_wq_free_irq(wq); idxd_wq_reset(wq); + idxd_wq_free_resources(wq); percpu_ref_exit(&wq->wq_active); wq->type = IDXD_WQT_NONE; wq->client_count = 0; -- 2.25.1

2 1

[PATCH openEuler-22.03-LTS-SP1 0/1] fix CVE-2023-45896
by Lin Yujun 04 Sep '24

04 Sep '24

Konstantin Komarov (1): fs/ntfs3: Add more attributes checks in mi_enum_attr() fs/ntfs3/record.c | 63 +++++++++++++++++++++++++++++++++++++---------- 1 file changed, 50 insertions(+), 13 deletions(-) -- 2.34.1

2 2

[PATCH OLK-5.10 0/1] fix CVE-2023-45896
by Lin Yujun 04 Sep '24

04 Sep '24

Konstantin Komarov (1): fs/ntfs3: Add more attributes checks in mi_enum_attr() fs/ntfs3/record.c | 63 +++++++++++++++++++++++++++++++++++++---------- 1 file changed, 50 insertions(+), 13 deletions(-) -- 2.34.1

2 4

[PATCH OLK-5.10] hfs: fix to initialize fields of hfs_inode_info after hfs_alloc_inode()
by Liu Mingrui 04 Sep '24

04 Sep '24

From: Chao Yu <chao(a)kernel.org> stable inclusion from stable-v5.10.224 commit 10f7163bfb5f8b4e0c9c05a939f20b8540e33c65 category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/IAKPRV CVE: CVE-2024-42311 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id… -------------------------------- commit 26a2ed107929a855155429b11e1293b83e6b2a8b upstream. Syzbot reports uninitialized value access issue as below: loop0: detected capacity change from 0 to 64 ===================================================== BUG: KMSAN: uninit-value in hfs_revalidate_dentry+0x307/0x3f0 fs/hfs/sysdep.c:30 hfs_revalidate_dentry+0x307/0x3f0 fs/hfs/sysdep.c:30 d_revalidate fs/namei.c:862 [inline] lookup_fast+0x89e/0x8e0 fs/namei.c:1649 walk_component fs/namei.c:2001 [inline] link_path_walk+0x817/0x1480 fs/namei.c:2332 path_lookupat+0xd9/0x6f0 fs/namei.c:2485 filename_lookup+0x22e/0x740 fs/namei.c:2515 user_path_at_empty+0x8b/0x390 fs/namei.c:2924 user_path_at include/linux/namei.h:57 [inline] do_mount fs/namespace.c:3689 [inline] __do_sys_mount fs/namespace.c:3898 [inline] __se_sys_mount+0x66b/0x810 fs/namespace.c:3875 __x64_sys_mount+0xe4/0x140 fs/namespace.c:3875 do_syscall_x64 arch/x86/entry/common.c:52 [inline] do_syscall_64+0xcf/0x1e0 arch/x86/entry/common.c:83 entry_SYSCALL_64_after_hwframe+0x63/0x6b BUG: KMSAN: uninit-value in hfs_ext_read_extent fs/hfs/extent.c:196 [inline] BUG: KMSAN: uninit-value in hfs_get_block+0x92d/0x1620 fs/hfs/extent.c:366 hfs_ext_read_extent fs/hfs/extent.c:196 [inline] hfs_get_block+0x92d/0x1620 fs/hfs/extent.c:366 block_read_full_folio+0x4ff/0x11b0 fs/buffer.c:2271 hfs_read_folio+0x55/0x60 fs/hfs/inode.c:39 filemap_read_folio+0x148/0x4f0 mm/filemap.c:2426 do_read_cache_folio+0x7c8/0xd90 mm/filemap.c:3553 do_read_cache_page mm/filemap.c:3595 [inline] read_cache_page+0xfb/0x2f0 mm/filemap.c:3604 read_mapping_page include/linux/pagemap.h:755 [inline] hfs_btree_open+0x928/0x1ae0 fs/hfs/btree.c:78 hfs_mdb_get+0x260c/0x3000 fs/hfs/mdb.c:204 hfs_fill_super+0x1fb1/0x2790 fs/hfs/super.c:406 mount_bdev+0x628/0x920 fs/super.c:1359 hfs_mount+0xcd/0xe0 fs/hfs/super.c:456 legacy_get_tree+0x167/0x2e0 fs/fs_context.c:610 vfs_get_tree+0xdc/0x5d0 fs/super.c:1489 do_new_mount+0x7a9/0x16f0 fs/namespace.c:3145 path_mount+0xf98/0x26a0 fs/namespace.c:3475 do_mount fs/namespace.c:3488 [inline] __do_sys_mount fs/namespace.c:3697 [inline] __se_sys_mount+0x919/0x9e0 fs/namespace.c:3674 __ia32_sys_mount+0x15b/0x1b0 fs/namespace.c:3674 do_syscall_32_irqs_on arch/x86/entry/common.c:112 [inline] __do_fast_syscall_32+0xa2/0x100 arch/x86/entry/common.c:178 do_fast_syscall_32+0x37/0x80 arch/x86/entry/common.c:203 do_SYSENTER_32+0x1f/0x30 arch/x86/entry/common.c:246 entry_SYSENTER_compat_after_hwframe+0x70/0x82 Uninit was created at: __alloc_pages+0x9a6/0xe00 mm/page_alloc.c:4590 __alloc_pages_node include/linux/gfp.h:238 [inline] alloc_pages_node include/linux/gfp.h:261 [inline] alloc_slab_page mm/slub.c:2190 [inline] allocate_slab mm/slub.c:2354 [inline] new_slab+0x2d7/0x1400 mm/slub.c:2407 ___slab_alloc+0x16b5/0x3970 mm/slub.c:3540 __slab_alloc mm/slub.c:3625 [inline] __slab_alloc_node mm/slub.c:3678 [inline] slab_alloc_node mm/slub.c:3850 [inline] kmem_cache_alloc_lru+0x64d/0xb30 mm/slub.c:3879 alloc_inode_sb include/linux/fs.h:3018 [inline] hfs_alloc_inode+0x5a/0xc0 fs/hfs/super.c:165 alloc_inode+0x83/0x440 fs/inode.c:260 new_inode_pseudo fs/inode.c:1005 [inline] new_inode+0x38/0x4f0 fs/inode.c:1031 hfs_new_inode+0x61/0x1010 fs/hfs/inode.c:186 hfs_mkdir+0x54/0x250 fs/hfs/dir.c:228 vfs_mkdir+0x49a/0x700 fs/namei.c:4126 do_mkdirat+0x529/0x810 fs/namei.c:4149 __do_sys_mkdirat fs/namei.c:4164 [inline] __se_sys_mkdirat fs/namei.c:4162 [inline] __x64_sys_mkdirat+0xc8/0x120 fs/namei.c:4162 do_syscall_x64 arch/x86/entry/common.c:52 [inline] do_syscall_64+0xcf/0x1e0 arch/x86/entry/common.c:83 entry_SYSCALL_64_after_hwframe+0x63/0x6b It missed to initialize .tz_secondswest, .cached_start and .cached_blocks fields in struct hfs_inode_info after hfs_alloc_inode(), fix it. Cc: stable(a)vger.kernel.org Reported-by: syzbot+3ae6be33a50b5aae4dab(a)syzkaller.appspotmail.com Closes: https://lore.kernel.org/linux-fsdevel/0000000000005ad04005ee48897f@google.c… Signed-off-by: Chao Yu <chao(a)kernel.org> Link: https://lore.kernel.org/r/20240616013841.2217-1-chao@kernel.org Signed-off-by: Christian Brauner <brauner(a)kernel.org> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> Signed-off-by: Liu Mingrui <liumingrui(a)huawei.com> --- fs/hfs/inode.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/fs/hfs/inode.c b/fs/hfs/inode.c index 6fbde990e056..3d59cdb31278 100644 --- a/fs/hfs/inode.c +++ b/fs/hfs/inode.c @@ -200,6 +200,7 @@ struct inode *hfs_new_inode(struct inode *dir, const struct qstr *name, umode_t HFS_I(inode)->flags = 0; HFS_I(inode)->rsrc_inode = NULL; HFS_I(inode)->fs_blocks = 0; + HFS_I(inode)->tz_secondswest = sys_tz.tz_minuteswest * 60; if (S_ISDIR(mode)) { inode->i_size = 2; HFS_SB(sb)->folder_count++; @@ -275,6 +276,8 @@ void hfs_inode_read_fork(struct inode *inode, struct hfs_extent *ext, for (count = 0, i = 0; i < 3; i++) count += be16_to_cpu(ext[i].count); HFS_I(inode)->first_blocks = count; + HFS_I(inode)->cached_start = 0; + HFS_I(inode)->cached_blocks = 0; inode->i_size = HFS_I(inode)->phys_size = log_size; HFS_I(inode)->fs_blocks = (log_size + sb->s_blocksize - 1) >> sb->s_blocksize_bits; -- 2.25.1

2 1

[openeuler:OLK-5.10 29999/30000] drivers/acpi/acpi_memhotplug.c:236: undefined reference to `hmat_restore_target'
by kernel test robot 04 Sep '24

04 Sep '24

tree: https://gitee.com/openeuler/kernel.git OLK-5.10 head: b7bed6628b750ffd687d1da0a170dece4b0c08bd commit: 2e1b00fcf1e3152a1e73846f5f9ec37cef088a65 [29999/30000] ACPI/HMAT: Add missing locality information for hot-added device config: x86_64-randconfig-013-20240903 (https://download.01.org/0day-ci/archive/20240904/202409040825.59qJfROR-lkp@…) compiler: gcc-11 (Debian 11.3.0-12) 11.3.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240904/202409040825.59qJfROR-lkp@…) If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot <lkp(a)intel.com> | Closes: https://lore.kernel.org/oe-kbuild-all/202409040825.59qJfROR-lkp@intel.com/ All errors (new ones prefixed by >>): ld: drivers/acpi/acpi_memhotplug.o: in function `acpi_memory_enable_device': >> drivers/acpi/acpi_memhotplug.c:236: undefined reference to `hmat_restore_target' vim +236 drivers/acpi/acpi_memhotplug.c 171 172 static int acpi_memory_enable_device(struct acpi_memory_device *mem_device) 173 { 174 acpi_handle handle = mem_device->device->handle; 175 int result, num_enabled = 0; 176 struct acpi_memory_info *info; 177 int node; 178 179 node = acpi_get_node(handle); 180 /* 181 * Tell the VM there is more memory here... 182 * Note: Assume that this function returns zero on success 183 * We don't have memory-hot-add rollback function,now. 184 * (i.e. memory-hot-remove function) 185 */ 186 list_for_each_entry(info, &mem_device->res_list, list) { 187 if (info->enabled) { /* just sanity check...*/ 188 num_enabled++; 189 continue; 190 } 191 /* 192 * If the memory block size is zero, please ignore it. 193 * Don't try to do the following memory hotplug flowchart. 194 */ 195 if (!info->length) 196 continue; 197 if (node < 0) 198 node = memory_add_physaddr_to_nid(info->start_addr); 199 200 result = __add_memory(node, info->start_addr, info->length, 201 MHP_NONE); 202 203 /* 204 * If the memory block has been used by the kernel, add_memory() 205 * returns -EEXIST. If add_memory() returns the other error, it 206 * means that this memory block is not used by the kernel. 207 */ 208 if (result && result != -EEXIST) 209 continue; 210 211 result = acpi_bind_memory_blocks(info, mem_device->device); 212 if (result) { 213 acpi_unbind_memory_blocks(info); 214 return -ENODEV; 215 } 216 217 info->enabled = 1; 218 219 /* 220 * Add num_enable even if add_memory() returns -EEXIST, so the 221 * device is bound to this driver. 222 */ 223 224 hotplug_mdev[node] = mem_device->device; 225 num_enabled++; 226 } 227 if (acpi_has_method(handle, "_HMA")) { 228 acpi_status status; 229 struct acpi_buffer buffer = { ACPI_ALLOCATE_BUFFER, NULL }; 230 231 status = acpi_evaluate_object(handle, "_HMA", NULL, &buffer); 232 if (ACPI_SUCCESS(status) && buffer.length) { 233 union acpi_object *obj = buffer.pointer; 234 235 if (!obj->buffer.length) > 236 hmat_restore_target(node); 237 } 238 } 239 240 if (!num_enabled) { 241 dev_err(&mem_device->device->dev, "add_memory failed\n"); 242 return -EINVAL; 243 } 244 /* 245 * Sometimes the memory device will contain several memory blocks. 246 * When one memory block is hot-added to the system memory, it will 247 * be regarded as a success. 248 * Otherwise if the last memory block can't be hot-added to the system 249 * memory, it will be failure and the memory device can't be bound with 250 * driver. 251 */ 252 return 0; 253 } 254 -- 0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki

1 0

[openeuler:OLK-5.10 29716/30000] drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.c:1195:5: warning: no previous prototype for function 'hclge_tm_vf_tc_dwrr_cfg'
by kernel test robot 04 Sep '24

04 Sep '24

tree: https://gitee.com/openeuler/kernel.git OLK-5.10 head: b7bed6628b750ffd687d1da0a170dece4b0c08bd commit: 518ea89ec98b78c749c27d6ba7792e1bdb5779ed [29716/30000] net: hns3: add support for vf multiple tcs config: x86_64-allyesconfig (https://download.01.org/0day-ci/archive/20240904/202409040605.jzWfVsDp-lkp@…) compiler: clang version 18.1.5 (https://github.com/llvm/llvm-project 617a15a9eac96088ae5e9134248d8236e34b91b1) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240904/202409040605.jzWfVsDp-lkp@…) If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot <lkp(a)intel.com> | Closes: https://lore.kernel.org/oe-kbuild-all/202409040605.jzWfVsDp-lkp@intel.com/ All warnings (new ones prefixed by >>): >> drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.c:1195:5: warning: no previous prototype for function 'hclge_tm_vf_tc_dwrr_cfg' [-Wmissing-prototypes] 1195 | int hclge_tm_vf_tc_dwrr_cfg(struct hclge_vport *vport) | ^ drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.c:1195:1: note: declare 'static' if the function is not intended to be used outside of this translation unit 1195 | int hclge_tm_vf_tc_dwrr_cfg(struct hclge_vport *vport) | ^ | static >> drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.c:2323:5: warning: no previous prototype for function 'hclge_mbx_set_vf_multi_tc' [-Wmissing-prototypes] 2323 | int hclge_mbx_set_vf_multi_tc(struct hclge_vport *vport, | ^ drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.c:2323:1: note: declare 'static' if the function is not intended to be used outside of this translation unit 2323 | int hclge_mbx_set_vf_multi_tc(struct hclge_vport *vport, | ^ | static 2 warnings generated. vim +/hclge_tm_vf_tc_dwrr_cfg +1195 drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.c 1194 > 1195 int hclge_tm_vf_tc_dwrr_cfg(struct hclge_vport *vport) 1196 { 1197 struct hnae3_knic_private_info *kinfo = &vport->nic.kinfo; 1198 struct hclge_dev *hdev = vport->back; 1199 struct hclge_pg_info *pg_info; 1200 u8 dwrr; 1201 int ret; 1202 u32 i; 1203 1204 for (i = 0; i < kinfo->tc_info.max_tc; i++) { 1205 pg_info = &hdev->tm_info.pg_info[hdev->tm_info.tc_info[i].pgid]; 1206 dwrr = i < kinfo->tc_info.num_tc ? pg_info->tc_dwrr[i] : 0; 1207 ret = hclge_tm_qs_weight_cfg(hdev, vport->qs_offset + i, dwrr); 1208 if (ret) 1209 return ret; 1210 } 1211 1212 return 0; 1213 } 1214 -- 0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki

1 0

[openeuler:OLK-5.10 16259/30000] mm/memcontrol.c:5402:40: error: implicit declaration of function 'ksm_process_profit'
by kernel test robot 04 Sep '24

04 Sep '24

tree: https://gitee.com/openeuler/kernel.git OLK-5.10 head: b7bed6628b750ffd687d1da0a170dece4b0c08bd commit: 0f6fb3570be40b783d8c756d64297be1a8e3d2b3 [16259/30000] memcg: support ksm merge any mode per cgroup config: x86_64-buildonly-randconfig-004-20240903 (https://download.01.org/0day-ci/archive/20240904/202409040111.5iXDLbsV-lkp@…) compiler: gcc-12 (Debian 12.2.0-14) 12.2.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240904/202409040111.5iXDLbsV-lkp@…) If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot <lkp(a)intel.com> | Closes: https://lore.kernel.org/oe-kbuild-all/202409040111.5iXDLbsV-lkp@intel.com/ All errors (new ones prefixed by >>): mm/memcontrol.c: In function 'memory_ksm_show': >> mm/memcontrol.c:5402:40: error: implicit declaration of function 'ksm_process_profit' [-Werror=implicit-function-declaration] 5402 | ksm_process_profits += ksm_process_profit(mm); | ^~~~~~~~~~~~~~~~~~ cc1: some warnings being treated as errors vim +/ksm_process_profit +5402 mm/memcontrol.c 5379 5380 static int memory_ksm_show(struct seq_file *m, void *v) 5381 { 5382 unsigned long ksm_merging_pages = 0; 5383 unsigned long ksm_rmap_items = 0; 5384 long ksm_process_profits = 0; 5385 unsigned int tasks = 0; 5386 struct task_struct *task; 5387 struct mm_struct *mm; 5388 struct css_task_iter it; 5389 struct mem_cgroup *memcg = mem_cgroup_from_seq(m); 5390 5391 css_task_iter_start(&memcg->css, CSS_TASK_ITER_PROCS, &it); 5392 while ((task = css_task_iter_next(&it))) { 5393 mm = get_task_mm(task); 5394 if (!mm) 5395 continue; 5396 5397 if (test_bit(MMF_VM_MERGE_ANY, &mm->flags)) 5398 tasks++; 5399 5400 ksm_rmap_items += mm->ksm_rmap_items; 5401 ksm_merging_pages += mm->ksm_merging_pages; > 5402 ksm_process_profits += ksm_process_profit(mm); 5403 mmput(mm); 5404 } 5405 css_task_iter_end(&it); 5406 5407 seq_printf(m, "merge any state: %d\n", READ_ONCE(memcg->ksm_merge_any)); 5408 seq_printf(m, "merge any tasks: %u\n", tasks); 5409 seq_printf(m, "ksm_rmap_items %lu\n", ksm_rmap_items); 5410 seq_printf(m, "ksm_merging_pages %lu\n", ksm_merging_pages); 5411 seq_printf(m, "ksm_process_profits %ld\n", ksm_process_profits); 5412 return 0; 5413 } 5414 -- 0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki

1 0

[openeuler:OLK-5.10 29379/30000] drivers/net/can/spi/mcp251xfd/mcp251xfd-core.c:2327:6: warning: no previous prototype for function 'mcp251xfd_tx_obj_write_sync'
by kernel test robot 04 Sep '24

04 Sep '24

tree: https://gitee.com/openeuler/kernel.git OLK-5.10 head: b7bed6628b750ffd687d1da0a170dece4b0c08bd commit: b4eaf3e808dc2abcab55bc72df3b0d15961f33f0 [29379/30000] can: mcp251xfd: fix infinite loop when xmit fails config: x86_64-allyesconfig (https://download.01.org/0day-ci/archive/20240904/202409040151.sxh3qZ5K-lkp@…) compiler: clang version 18.1.5 (https://github.com/llvm/llvm-project 617a15a9eac96088ae5e9134248d8236e34b91b1) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240904/202409040151.sxh3qZ5K-lkp@…) If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot <lkp(a)intel.com> | Closes: https://lore.kernel.org/oe-kbuild-all/202409040151.sxh3qZ5K-lkp@intel.com/ All warnings (new ones prefixed by >>): >> drivers/net/can/spi/mcp251xfd/mcp251xfd-core.c:2327:6: warning: no previous prototype for function 'mcp251xfd_tx_obj_write_sync' [-Wmissing-prototypes] 2327 | void mcp251xfd_tx_obj_write_sync(struct work_struct *work) | ^ drivers/net/can/spi/mcp251xfd/mcp251xfd-core.c:2327:1: note: declare 'static' if the function is not intended to be used outside of this translation unit 2327 | void mcp251xfd_tx_obj_write_sync(struct work_struct *work) | ^ | static drivers/net/can/spi/mcp251xfd/mcp251xfd-core.c:511:1: warning: unused function 'mcp251xfd_chip_set_mode_nowait' [-Wunused-function] 511 | mcp251xfd_chip_set_mode_nowait(const struct mcp251xfd_priv *priv, | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2 warnings generated. vim +/mcp251xfd_tx_obj_write_sync +2327 drivers/net/can/spi/mcp251xfd/mcp251xfd-core.c 2326 > 2327 void mcp251xfd_tx_obj_write_sync(struct work_struct *work) 2328 { 2329 struct mcp251xfd_priv *priv = container_of(work, struct mcp251xfd_priv, 2330 tx_work); 2331 struct mcp251xfd_tx_obj *tx_obj = priv->tx_work_obj; 2332 struct mcp251xfd_tx_ring *tx_ring = priv->tx; 2333 int err; 2334 2335 err = spi_sync(priv->spi, &tx_obj->msg); 2336 if (err) 2337 mcp251xfd_tx_failure_drop(priv, tx_ring, err); 2338 } 2339 -- 0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki

1 0