From: Vijayanand Jitta <vjitta@codeaurora.org> mainline inclusion from mainline-v5.12-rc1 commit 64427985c76fcb54c783de617edf353009499a03 category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I9GSSR Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- Fix the below ignoring return value warning for kstrtobool in is_stack_depot_disabled function. lib/stackdepot.c: In function 'is_stack_depot_disabled': lib/stackdepot.c:154:2: warning: ignoring return value of 'kstrtobool' declared with attribute 'warn_unused_result' [-Wunused-result] Link: https://lkml.kernel.org/r/1612163048-28026-1-git-send-email-vjitta@codeauror... Fixes: b9779abb09a8 ("lib: stackdepot: add support to disable stack depot") Signed-off-by: Vijayanand Jitta <vjitta@codeaurora.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com> --- lib/stackdepot.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 33e22f20ffa4..fba6332e645d 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -152,8 +152,10 @@ static struct stack_record **stack_table; static int __init is_stack_depot_disabled(char *str) { - kstrtobool(str, &stack_depot_disable); - if (stack_depot_disable) { + int ret; + + ret = kstrtobool(str, &stack_depot_disable); + if (!ret && stack_depot_disable) { pr_info("Stack Depot is disabled\n"); stack_table = NULL; } -- 2.25.1

From: Vlastimil Babka <vbabka@suse.cz> mainline inclusion from mainline-v5.19-rc1 commit a5f1783be29adae15666fd803efd7d2979130869 category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I9GSSR Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- In a later patch we want to add stackdepot support for object owner tracking in slub caches, which is enabled by slub_debug boot parameter. This creates a bootstrap problem as some caches are created early in boot when slab_is_available() is false and thus stack_depot_init() tries to use memblock. But, as reported by Hyeonggon Yoo [1] we are already beyond memblock_free_all(). Ideally memblock allocation should fail, yet it succeeds, but later the system crashes, which is a separately handled issue. To resolve this boostrap issue in a robust way, this patch adds another way to request stack_depot_early_init(), which happens at a well-defined point of time. In addition to build-time CONFIG_STACKDEPOT_ALWAYS_INIT, code that's e.g. processing boot parameters (which happens early enough) can call a new function stack_depot_want_early_init(), which sets a flag that stack_depot_early_init() will check. In this patch we also convert page_owner to this approach. While it doesn't have the bootstrap issue as slub, it's also a functionality enabled by a boot param and can thus request stack_depot_early_init() with memblock allocation instead of later initialization with kvmalloc(). As suggested by Mike, make stack_depot_early_init() only attempt memblock allocation and stack_depot_init() only attempt kvmalloc(). Also change the latter to kvcalloc(). In both cases we can lose the explicit array zeroing, which the allocations do already. As suggested by Marco, provide empty implementations of the init functions for !CONFIG_STACKDEPOT builds to simplify the callers. [1] https://lore.kernel.org/all/YhnUcqyeMgCrWZbd@ip-172-31-19-208.ap-northeast-1... Reported-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Suggested-by: Mike Rapoport <rppt@linux.ibm.com> Suggested-by: Marco Elver <elver@google.com> Signed-off-by: Vlastimil Babka <vbabka@suse.cz> Reviewed-by: Marco Elver <elver@google.com> Reviewed-and-tested-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Reviewed-by: Mike Rapoport <rppt@linux.ibm.com> Acked-by: David Rientjes <rientjes@google.com> Conflicts: include/linux/stackdepot.h lib/stackdepot.c mm/page_owner.c [context conflicts] Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com> --- include/linux/stackdepot.h | 25 +++++++++++--- lib/stackdepot.c | 67 +++++++++++++++++++++++++------------- mm/page_owner.c | 5 +-- 3 files changed, 69 insertions(+), 28 deletions(-) diff --git a/include/linux/stackdepot.h b/include/linux/stackdepot.h index 30dfc8549b5f..563e92754035 100644 --- a/include/linux/stackdepot.h +++ b/include/linux/stackdepot.h @@ -14,18 +14,35 @@ typedef u32 depot_stack_handle_t; /* - * Every user of stack depot has to call this during its own init when it's - * decided that it will be calling stack_depot_save() later. + * Every user of stack depot has to call stack_depot_init() during its own init + * when it's decided that it will be calling stack_depot_save() later. This is + * recommended for e.g. modules initialized later in the boot process, when + * slab_is_available() is true. * * The alternative is to select STACKDEPOT_ALWAYS_INIT to have stack depot * enabled as part of mm_init(), for subsystems where it's known at compile time * that stack depot will be used. + * + * Another alternative is to call stack_depot_want_early_init(), when the + * decision to use stack depot is taken e.g. when evaluating kernel boot + * parameters, which precedes the enablement point in mm_init(). + * + * stack_depot_init() and stack_depot_want_early_init() can be called regardless + * of CONFIG_STACKDEPOT and are no-op when disabled. The actual save/fetch/print + * functions should only be called from code that makes sure CONFIG_STACKDEPOT + * is enabled. */ +#ifdef CONFIG_STACKDEPOT int stack_depot_init(void); +void __init stack_depot_want_early_init(void); -#ifdef CONFIG_STACKDEPOT_ALWAYS_INIT -static inline int stack_depot_early_init(void) { return stack_depot_init(); } +/* This is supposed to be called only from mm_init() */ +int __init stack_depot_early_init(void); #else +static inline int stack_depot_init(void) { return 0; } + +static inline void stack_depot_want_early_init(void) { } + static inline int stack_depot_early_init(void) { return 0; } #endif diff --git a/lib/stackdepot.c b/lib/stackdepot.c index ede2a433055c..8aed500d5ee8 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -66,6 +66,9 @@ struct stack_record { unsigned long entries[1]; /* Variable-sized array of entries. */ }; +static bool __stack_depot_want_early_init __initdata = IS_ENABLED(CONFIG_STACKDEPOT_ALWAYS_INIT); +static bool __stack_depot_early_init_passed __initdata; + static void *stack_slabs[STACK_ALLOC_MAX_SLABS]; static int depot_index; @@ -164,38 +167,58 @@ static int __init is_stack_depot_disabled(char *str) } early_param("stack_depot_disable", is_stack_depot_disabled); -/* - * __ref because of memblock_alloc(), which will not be actually called after - * the __init code is gone, because at that point slab_is_available() is true - */ -__ref int stack_depot_init(void) +void __init stack_depot_want_early_init(void) +{ + /* Too late to request early init now */ + WARN_ON(__stack_depot_early_init_passed); + + __stack_depot_want_early_init = true; +} + +int __init stack_depot_early_init(void) +{ + size_t size; + + /* This is supposed to be called only once, from mm_init() */ + if (WARN_ON(__stack_depot_early_init_passed)) + return 0; + + __stack_depot_early_init_passed = true; + + if (!__stack_depot_want_early_init || stack_depot_disable) + return 0; + + size = (STACK_HASH_SIZE * sizeof(struct stack_record *)); + pr_info("Stack Depot early init allocating hash table with memblock_alloc, %zu bytes\n", + size); + stack_table = memblock_alloc(size, SMP_CACHE_BYTES); + + if (!stack_table) { + pr_err("Stack Depot hash table allocation failed, disabling\n"); + stack_depot_disable = true; + return -ENOMEM; + } + + return 0; +} + +int stack_depot_init(void) { static DEFINE_MUTEX(stack_depot_init_mutex); + int ret = 0; mutex_lock(&stack_depot_init_mutex); if (!stack_depot_disable && !stack_table) { - size_t size = (STACK_HASH_SIZE * sizeof(struct stack_record *)); - int i; - - if (slab_is_available()) { - pr_info("Stack Depot allocating hash table with kvmalloc\n"); - stack_table = kvmalloc(size, GFP_KERNEL); - } else { - pr_info("Stack Depot allocating hash table with memblock_alloc\n"); - stack_table = memblock_alloc(size, SMP_CACHE_BYTES); - } - if (stack_table) { - for (i = 0; i < STACK_HASH_SIZE; i++) - stack_table[i] = NULL; - } else { + pr_info("Stack Depot allocating hash table with kvcalloc\n"); + stack_table = kvcalloc(STACK_HASH_SIZE, sizeof(struct stack_record *), GFP_KERNEL); + if (!stack_table) { pr_err("Stack Depot hash table allocation failed, disabling\n"); stack_depot_disable = true; - mutex_unlock(&stack_depot_init_mutex); - return -ENOMEM; + ret = -ENOMEM; } } mutex_unlock(&stack_depot_init_mutex); - return 0; + return ret; } EXPORT_SYMBOL_GPL(stack_depot_init); diff --git a/mm/page_owner.c b/mm/page_owner.c index 61ba829e895c..a80917074147 100644 --- a/mm/page_owner.c +++ b/mm/page_owner.c @@ -48,6 +48,9 @@ static int __init early_page_owner_param(char *buf) if (strcmp(buf, "on") == 0) page_owner_enabled = true; + if (page_owner_enabled) + stack_depot_want_early_init(); + return 0; } early_param("page_owner", early_page_owner_param); @@ -86,8 +89,6 @@ static void init_page_owner(void) if (!page_owner_enabled) return; - stack_depot_init(); - register_dummy_stack(); register_failure_stack(); register_early_stack(); -- 2.25.1

From: Yogesh Lal <ylal@codeaurora.org> mainline inclusion from mainline-v5.12-rc1 commit d262093656a0eec6d6114a3178a9d887fddd0ded category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I9GSSR Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- Use CONFIG_STACK_HASH_ORDER to configure STACK_HASH_SIZE. Aim is to have configurable value for STACK_HASH_SIZE, so depend on use case one can configure it. One example is of Page Owner, CONFIG_PAGE_OWNER works only if page_owner=on via kernel parameter on CONFIG_PAGE_OWNER configured system. Thus, unless admin enable it via command line option, the stackdepot will just waste 8M memory without any customer. Making it configurable and use lower value helps to enable features like CONFIG_PAGE_OWNER without any significant overhead. Link: https://lkml.kernel.org/r/1611749198-24316-1-git-send-email-vjitta@codeauror... Signed-off-by: Yogesh Lal <ylal@codeaurora.org> Signed-off-by: Vinayak Menon <vinmenon@codeaurora.org> Signed-off-by: Vijayanand Jitta <vjitta@codeaurora.org> Reviewed-by: Minchan Kim <minchan@kernel.org> Reviewed-by: Alexander Potapenko <glider@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Conflicts: lib/Kconfig [context conflicts] Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com> --- lib/Kconfig | 9 +++++++++ lib/stackdepot.c | 3 +-- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/lib/Kconfig b/lib/Kconfig index 497304f1647c..06a796a94828 100644 --- a/lib/Kconfig +++ b/lib/Kconfig @@ -664,6 +664,15 @@ config STACKDEPOT bool select STACKTRACE +config STACK_HASH_ORDER + int "stack depot hash size (12 => 4KB, 20 => 1024KB)" + range 12 20 + default 20 + depends on STACKDEPOT + help + Select the hash size as a power of 2 for the stackdepot hash table. + Choose a lower value to reduce the memory impact. + config STACKDEPOT_ALWAYS_INIT bool select STACKDEPOT diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 8aed500d5ee8..e3275f07076c 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -146,8 +146,7 @@ static struct stack_record *depot_alloc_stack(unsigned long *entries, int size, return stack; } -#define STACK_HASH_ORDER 20 -#define STACK_HASH_SIZE (1L << STACK_HASH_ORDER) +#define STACK_HASH_SIZE (1L << CONFIG_STACK_HASH_ORDER) #define STACK_HASH_MASK (STACK_HASH_SIZE - 1) #define STACK_HASH_SEED 0x9747b28c -- 2.25.1

From: Sergei Trofimovich <slyfox@gentoo.org> mainline inclusion from mainline-v5.13-rc1 commit 8e9b16c47680f6e7d6e5864a37f313f905a91cf5 category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I9GSSR Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- Before the change page_owner recursion was detected via fetching backtrace and inspecting it for current instruction pointer. It has a few problems: - it is slightly slow as it requires extra backtrace and a linear stack scan of the result - it is too late to check if backtrace fetching required memory allocation itself (ia64's unwinder requires it). To simplify recursion tracking let's use page_owner recursion flag in 'struct task_struct'. The change make page_owner=on work on ia64 by avoiding infinite recursion in: kmalloc() -> __set_page_owner() -> save_stack() -> unwind() [ia64-specific] -> build_script() -> kmalloc() -> __set_page_owner() [we short-circuit here] -> save_stack() -> unwind() [recursion] Link: https://lkml.kernel.org/r/20210402115342.1463781-1-slyfox@gentoo.org Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org> Reviewed-by: Andrew Morton <akpm@linux-foundation.org> Acked-by: Vlastimil Babka <vbabka@suse.cz> Cc: Ingo Molnar <mingo@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Vincent Guittot <vincent.guittot@linaro.org> Cc: Dietmar Eggemann <dietmar.eggemann@arm.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Ben Segall <bsegall@google.com> Cc: Mel Gorman <mgorman@suse.de> Cc: Daniel Bristot de Oliveira <bristot@redhat.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Conflicts: include/linux/sched.h [This patch adds a bit field in struct task_struct. Checked by pahole, there are holes to store the bit. Use macro KABI_FILL_HOLE to avoid kabi breakage warning.] Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com> --- include/linux/sched.h | 3 +++ mm/page_owner.c | 32 ++++++++++---------------------- 2 files changed, 13 insertions(+), 22 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index b4ab407cab37..40022e4a48a6 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -884,6 +884,9 @@ struct task_struct { #ifdef CONFIG_IOMMU_SVA KABI_FILL_HOLE(unsigned pasid_activated:1) #endif +#ifdef CONFIG_PAGE_OWNER + KABI_FILL_HOLE(unsigned in_page_owner:1) +#endif unsigned long atomic_flags; /* Flags requiring atomic access. */ diff --git a/mm/page_owner.c b/mm/page_owner.c index a80917074147..15f2b8f1e0c5 100644 --- a/mm/page_owner.c +++ b/mm/page_owner.c @@ -107,42 +107,30 @@ static inline struct page_owner *get_page_owner(struct page_ext *page_ext) return (void *)page_ext + page_owner_ops.offset; } -static inline bool check_recursive_alloc(unsigned long *entries, - unsigned int nr_entries, - unsigned long ip) -{ - unsigned int i; - - for (i = 0; i < nr_entries; i++) { - if (entries[i] == ip) - return true; - } - return false; -} - static noinline depot_stack_handle_t save_stack(gfp_t flags) { unsigned long entries[PAGE_OWNER_STACK_DEPTH]; depot_stack_handle_t handle; unsigned int nr_entries; - nr_entries = stack_trace_save(entries, ARRAY_SIZE(entries), 2); - /* - * We need to check recursion here because our request to - * stackdepot could trigger memory allocation to save new - * entry. New memory allocation would reach here and call - * stack_depot_save_entries() again if we don't catch it. There is - * still not enough memory in stackdepot so it would try to - * allocate memory again and loop forever. + * Avoid recursion. + * + * Sometimes page metadata allocation tracking requires more + * memory to be allocated: + * - when new stack trace is saved to stack depot + * - when backtrace itself is calculated (ia64) */ - if (check_recursive_alloc(entries, nr_entries, _RET_IP_)) + if (current->in_page_owner) return dummy_handle; + current->in_page_owner = 1; + nr_entries = stack_trace_save(entries, ARRAY_SIZE(entries), 2); handle = stack_depot_save(entries, nr_entries, flags); if (!handle) handle = failure_handle; + current->in_page_owner = 0; return handle; } -- 2.25.1

From: Waiman Long <longman@redhat.com> mainline inclusion from mainline-v5.18-rc1 commit 3ebc439761273274ea00258da84d997841f01e72 category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I9GSSR Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- The snprintf() function can return a length greater than the given input size. That will require a check for buffer overrun after each invocation of snprintf(). scnprintf(), on the other hand, will never return a greater length. By using scnprintf() in selected places, we can avoid some buffer overrun checks except after stack_depot_snprint() and after the last snprintf(). Link: https://lkml.kernel.org/r/20220202203036.744010-3-longman@redhat.com Signed-off-by: Waiman Long <longman@redhat.com> Acked-by: David Rientjes <rientjes@google.com> Reviewed-by: Sergey Senozhatsky <senozhatsky@chromium.org> Acked-by: Rafael Aquini <aquini@redhat.com> Acked-by: Mike Rapoport <rppt@linux.ibm.com> Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Cc: Ira Weiny <ira.weiny@intel.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Michal Hocko <mhocko@kernel.org> Cc: Petr Mladek <pmladek@suse.com> Cc: Rasmus Villemoes <linux@rasmusvillemoes.dk> Cc: Roman Gushchin <roman.gushchin@linux.dev> Cc: Steven Rostedt (Google) <rostedt@goodmis.org> Cc: Vladimir Davydov <vdavydov.dev@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Conflicts: mm/page_owner.c [context conflicts] Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com> --- mm/page_owner.c | 14 +++----------- 1 file changed, 3 insertions(+), 11 deletions(-) diff --git a/mm/page_owner.c b/mm/page_owner.c index bd1b0e9e11de..46521c006665 100644 --- a/mm/page_owner.c +++ b/mm/page_owner.c @@ -350,19 +350,16 @@ print_page_owner(char __user *buf, size_t count, unsigned long pfn, if (!kbuf) return -ENOMEM; - ret = snprintf(kbuf, count, + ret = scnprintf(kbuf, count, "Page allocated via order %u, mask %#x(%pGg), pid %d, ts %llu ns, free_ts %llu ns\n", page_owner->order, page_owner->gfp_mask, &page_owner->gfp_mask, page_owner->pid, page_owner->ts_nsec, page_owner->free_ts_nsec); - if (ret >= count) - goto err; - /* Print information relevant to grouping pages by mobility */ pageblock_mt = get_pageblock_migratetype(page); page_mt = gfp_migratetype(page_owner->gfp_mask); - ret += snprintf(kbuf + ret, count - ret, + ret += scnprintf(kbuf + ret, count - ret, "PFN %lu type %s Block %lu type %s Flags %#lx(%pGp)\n", pfn, migratetype_names[page_mt], @@ -370,20 +367,15 @@ print_page_owner(char __user *buf, size_t count, unsigned long pfn, migratetype_names[pageblock_mt], page->flags, &page->flags); - if (ret >= count) - goto err; - nr_entries = stack_depot_fetch(handle, &entries); ret += stack_trace_snprint(kbuf + ret, count - ret, entries, nr_entries, 0); if (ret >= count) goto err; if (page_owner->last_migrate_reason != -1) { - ret += snprintf(kbuf + ret, count - ret, + ret += scnprintf(kbuf + ret, count - ret, "Page has been migrated, last migrate reason: %s\n", migrate_reason_names[page_owner->last_migrate_reason]); - if (ret >= count) - goto err; } ret += snprintf(kbuf + ret, count - ret, "\n"); -- 2.25.1

From: Waiman Long <longman@redhat.com> mainline inclusion from mainline-v5.18-rc1 commit 865ed6a3278654ce4a55eb74c5283eeb82ad4699 category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I9GSSR Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- The page_owner information currently includes the pid of the calling task. That is useful as long as the task is still running. Otherwise, the number is meaningless. To have more information about the allocating tasks that had exited by the time the page_owner information is retrieved, we need to store the command name of the task. Add a new comm field into page_owner structure to store the command name and display it when the page_owner information is retrieved. Link: https://lkml.kernel.org/r/20220202203036.744010-5-longman@redhat.com Signed-off-by: Waiman Long <longman@redhat.com> Acked-by: Rafael Aquini <aquini@redhat.com> Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Cc: David Rientjes <rientjes@google.com> Cc: Ira Weiny <ira.weiny@intel.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Michal Hocko <mhocko@kernel.org> Cc: Mike Rapoport <rppt@kernel.org> Cc: Petr Mladek <pmladek@suse.com> Cc: Rasmus Villemoes <linux@rasmusvillemoes.dk> Cc: Roman Gushchin <roman.gushchin@linux.dev> Cc: Sergey Senozhatsky <senozhatsky@chromium.org> Cc: Steven Rostedt (Google) <rostedt@goodmis.org> Cc: Vladimir Davydov <vdavydov.dev@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Conflicts: mm/page_owner.c [context conflicts] Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com> --- mm/page_owner.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/mm/page_owner.c b/mm/page_owner.c index 08f8a861b25c..0b4b5b204257 100644 --- a/mm/page_owner.c +++ b/mm/page_owner.c @@ -29,6 +29,7 @@ struct page_owner { depot_stack_handle_t free_handle; u64 ts_nsec; u64 free_ts_nsec; + char comm[TASK_COMM_LEN]; pid_t pid; }; @@ -172,6 +173,8 @@ static inline void __set_page_owner_handle(struct page *page, page_owner->last_migrate_reason = -1; page_owner->pid = current->pid; page_owner->ts_nsec = local_clock(); + strlcpy(page_owner->comm, current->comm, + sizeof(page_owner->comm)); __set_bit(PAGE_EXT_OWNER, &page_ext->flags); __set_bit(PAGE_EXT_OWNER_ALLOCATED, &page_ext->flags); @@ -239,6 +242,7 @@ void __copy_page_owner(struct page *oldpage, struct page *newpage) new_page_owner->pid = old_page_owner->pid; new_page_owner->ts_nsec = old_page_owner->ts_nsec; new_page_owner->free_ts_nsec = old_page_owner->ts_nsec; + strcpy(new_page_owner->comm, old_page_owner->comm); /* * We don't clear the bit on the oldpage as it's going to be freed @@ -391,10 +395,11 @@ print_page_owner(char __user *buf, size_t count, unsigned long pfn, return -ENOMEM; ret = scnprintf(kbuf, count, - "Page allocated via order %u, mask %#x(%pGg), pid %d, ts %llu ns, free_ts %llu ns\n", + "Page allocated via order %u, mask %#x(%pGg), pid %d (%s), ts %llu ns, free_ts %llu ns\n", page_owner->order, page_owner->gfp_mask, &page_owner->gfp_mask, page_owner->pid, - page_owner->ts_nsec, page_owner->free_ts_nsec); + page_owner->comm, page_owner->ts_nsec, + page_owner->free_ts_nsec); /* Print information relevant to grouping pages by mobility */ pageblock_mt = get_pageblock_migratetype(page); @@ -464,9 +469,10 @@ void __dump_page_owner(struct page *page) else pr_alert("page_owner tracks the page as freed\n"); - pr_alert("page last allocated via order %u, migratetype %s, gfp_mask %#x(%pGg), pid %d, ts %llu, free_ts %llu\n", + pr_alert("page last allocated via order %u, migratetype %s, gfp_mask %#x(%pGg), pid %d (%s), ts %llu, free_ts %llu\n", page_owner->order, migratetype_names[mt], gfp_mask, &gfp_mask, - page_owner->pid, page_owner->ts_nsec, page_owner->free_ts_nsec); + page_owner->pid, page_owner->comm, page_owner->ts_nsec, + page_owner->free_ts_nsec); handle = READ_ONCE(page_owner->handle); if (!handle) { -- 2.25.1

From: Eric Dumazet <edumazet@google.com> mainline inclusion from mainline-v5.19-rc1 commit cd8c1fd8cdd14158f2d8bea2d1bfe8015dccfa3a category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I9GSSR Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- current->comm[] is not a string (no guarantee for a zero byte in it). strlcpy(s1, s2, l) is calling strlen(s2), potentially causing out-of-bound access, as reported by syzbot: detected buffer overflow in __fortify_strlen ------------[ cut here ]------------ kernel BUG at lib/string_helpers.c:980! invalid opcode: 0000 [#1] PREEMPT SMP KASAN CPU: 0 PID: 4087 Comm: dhcpcd-run-hooks Not tainted 5.18.0-rc3-syzkaller-01537-g20b87e7c29df #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 RIP: 0010:fortify_panic+0x18/0x1a lib/string_helpers.c:980 Code: 8c e8 c5 ba e1 fa e9 23 0f bf fa e8 0b 5d 8c f8 eb db 55 48 89 fd e8 e0 49 40 f8 48 89 ee 48 c7 c7 80 f5 26 8a e8 99 09 f1 ff <0f> 0b e8 ca 49 40 f8 48 8b 54 24 18 4c 89 f1 48 c7 c7 00 00 27 8a RSP: 0018:ffffc900000074a8 EFLAGS: 00010286 RAX: 000000000000002c RBX: ffff88801226b728 RCX: 0000000000000000 RDX: ffff8880198e0000 RSI: ffffffff81600458 RDI: fffff52000000e87 RBP: ffffffff89da2aa0 R08: 000000000000002c R09: 0000000000000000 R10: ffffffff815fae2e R11: 0000000000000000 R12: ffff88801226b700 R13: ffff8880198e0830 R14: 0000000000000000 R15: 0000000000000000 FS: 0000000000000000(0000) GS:ffff8880b9c00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f5876ad6ff8 CR3: 000000001a48c000 CR4: 00000000003506f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600 Call Trace: <IRQ> __fortify_strlen include/linux/fortify-string.h:128 [inline] strlcpy include/linux/fortify-string.h:143 [inline] __set_page_owner_handle+0x2b1/0x3e0 mm/page_owner.c:171 __set_page_owner+0x3e/0x50 mm/page_owner.c:190 prep_new_page mm/page_alloc.c:2441 [inline] get_page_from_freelist+0xba2/0x3e00 mm/page_alloc.c:4182 __alloc_pages+0x1b2/0x500 mm/page_alloc.c:5408 alloc_pages+0x1aa/0x310 mm/mempolicy.c:2272 alloc_slab_page mm/slub.c:1799 [inline] allocate_slab+0x26c/0x3c0 mm/slub.c:1944 new_slab mm/slub.c:2004 [inline] ___slab_alloc+0x8df/0xf20 mm/slub.c:3005 __slab_alloc.constprop.0+0x4d/0xa0 mm/slub.c:3092 slab_alloc_node mm/slub.c:3183 [inline] slab_alloc mm/slub.c:3225 [inline] __kmem_cache_alloc_lru mm/slub.c:3232 [inline] kmem_cache_alloc+0x360/0x3b0 mm/slub.c:3242 dst_alloc+0x146/0x1f0 net/core/dst.c:92 Link: https://lkml.kernel.org/r/20220509145949.265184-1-eric.dumazet@gmail.com Fixes: 865ed6a32786 ("mm/page_owner: record task command name") Signed-off-by: Eric Dumazet <edumazet@google.com> Reported-by: syzbot <syzkaller@googlegroups.com> Acked-by: Waiman Long <longman@redhat.com> Acked-by: Shakeel Butt <shakeelb@google.com> Cc: <stable@vger.kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com> --- mm/page_owner.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/page_owner.c b/mm/page_owner.c index ce44df1d603a..84f6d08632c3 100644 --- a/mm/page_owner.c +++ b/mm/page_owner.c @@ -175,7 +175,7 @@ static inline void __set_page_owner_handle(struct page *page, page_owner->pid = current->pid; page_owner->tgid = current->tgid; page_owner->ts_nsec = local_clock(); - strlcpy(page_owner->comm, current->comm, + strscpy(page_owner->comm, current->comm, sizeof(page_owner->comm)); __set_bit(PAGE_EXT_OWNER, &page_ext->flags); __set_bit(PAGE_EXT_OWNER_ALLOCATED, &page_ext->flags); -- 2.25.1

From: Zhenhua Huang <quic_zhenhuah@quicinc.com> mainline inclusion from mainline-v6.1-rc1 commit 0bba9af03d55d2cc1aa7616a8b9e522ceb49d180 category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I9GSSR Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- Remove an expensive and unnecessary operation as PCP pages are safely skipped when reading page owner.PCP pages can be skipped because PAGE_EXT_OWNER_ALLOCATED is cleared. With draining PCP pages, these pages are moved to buddy list so they can be identified as buddy pages and skipped quickly. Although it improved efficiency of PFN walker, the drain is guaranteed expensive that is unlikely to be offset by a slight increase in efficiency when skipping free pages. PAGE_EXT_OWNER_ALLOCATED is cleared in the page owner reset path below: free_unref_page -> free_unref_page_prepare -> free_pcp_prepare -> free_pages_prepare which do page owner reset -> free_unref_page_commit which add pages into pcp list Link: https://lkml.kernel.org/r/1662704326-15899-1-git-send-email-quic_zhenhuah@qu... Link: https://lkml.kernel.org/r/1662633204-10044-1-git-send-email-quic_zhenhuah@qu... Link: https://lkml.kernel.org/r/1662537673-9392-1-git-send-email-quic_zhenhuah@qui... Signed-off-by: Zhenhua Huang <quic_zhenhuah@quicinc.com> Acked-by: Mel Gorman <mgorman@techsingularity.net> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com> --- mm/page_owner.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/mm/page_owner.c b/mm/page_owner.c index 6aad07c996b3..ea1a1e4063b2 100644 --- a/mm/page_owner.c +++ b/mm/page_owner.c @@ -538,8 +538,6 @@ read_page_owner(struct file *file, char __user *buf, size_t count, loff_t *ppos) while (!pfn_valid(pfn) && (pfn & (MAX_ORDER_NR_PAGES - 1)) != 0) pfn++; - drain_all_pages(NULL); - /* Find an allocated page */ for (; pfn < max_pfn; pfn++) { /* -- 2.25.1

From: Charan Teja Kalla <quic_charante@quicinc.com> mainline inclusion from mainline-v6.1-rc7 commit ed86b74874f839f0e579bdf92ea0a5aabdfabebb category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I9GSSR Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- Fix the below compiler warnings reported with 'make W=1 mm/'. mm/page_ext.c:178: warning: Function parameter or member 'page_ext' not described in 'page_ext_put'. [quic_pkondeti@quicinc.com: better patch title] Link: https://lkml.kernel.org/r/1667884582-2465-1-git-send-email-quic_charante@qui... Fixes: b1d5488a252dc9 ("mm: fix use-after free of page_ext after race with memory-offline") Signed-off-by: Charan Teja Kalla <quic_charante@quicinc.com> Reported-by: Vlastimil Babka <vbabka@suse.cz> Tested-by: Vlastimil Babka <vbabka@suse.cz> Cc: Pavan Kondeti <quic_pkondeti@quicinc.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com> --- mm/page_ext.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/page_ext.c b/mm/page_ext.c index c40d3f7456df..d521c93b3ed6 100644 --- a/mm/page_ext.c +++ b/mm/page_ext.c @@ -154,7 +154,7 @@ struct page_ext *page_ext_get(struct page *page) /** * page_ext_put() - Working with page extended information is done. - * @page_ext - Page extended information received from page_ext_get(). + * @page_ext: Page extended information received from page_ext_get(). * * The page extended information of the page may not be valid after this * function is called. -- 2.25.1

From: Zhenliang Wei <weizhenliang@huawei.com> mainline inclusion from mainline-v5.16-rc1 commit f7df2b1cf03af680354bd4300f48f7ea11316ce8 category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I9GSSR Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- When viewing page owner information, we may be more concerned about the total memory rather than the times of stack appears. Therefore, the following adjustments are made: 1. Added the statistics on the total number of pages. 2. Added the optional parameter "-m" to configure the program to sort by memory (total pages). The general output of page_owner is as follows: Page allocated via order XXX, ... PFN XXX ... // Detailed stack Page allocated via order XXX, ... PFN XXX ... // Detailed stack The original page_owner_sort ignores PFN rows, puts the remaining rows in buf, counts the times of buf, and finally sorts them according to the times. General output: XXX times: Page allocated via order XXX, ... // Detailed stack Now, we use regexp to extract the page order value from the buf, and count the total pages for the buf. General output: XXX times, XXX pages: Page allocated via order XXX, ... // Detailed stack By default, it is still sorted by the times of buf; If you want to sort by the pages nums of buf, use the new -m parameter. Link: https://lkml.kernel.org/r/1631678242-41033-1-git-send-email-weizhenliang@hua... Signed-off-by: Zhenliang Wei <weizhenliang@huawei.com> Cc: Tang Bin <tangbin@cmss.chinamobile.com> Cc: Zhang Shengju <zhangshengju@cmss.chinamobile.com> Cc: Zhenliang Wei <weizhenliang@huawei.com> Cc: Xiaoming Ni <nixiaoming@huawei.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com> --- Documentation/vm/page_owner.rst | 23 +++++++- tools/vm/page_owner_sort.c | 94 +++++++++++++++++++++++++++++---- 2 files changed, 107 insertions(+), 10 deletions(-) diff --git a/Documentation/vm/page_owner.rst b/Documentation/vm/page_owner.rst index 2175465c9bf2..9837fc8147dd 100644 --- a/Documentation/vm/page_owner.rst +++ b/Documentation/vm/page_owner.rst @@ -85,5 +85,26 @@ Usage cat /sys/kernel/debug/page_owner > page_owner_full.txt ./page_owner_sort page_owner_full.txt sorted_page_owner.txt + The general output of ``page_owner_full.txt`` is as follows: + + Page allocated via order XXX, ... + PFN XXX ... + // Detailed stack + + Page allocated via order XXX, ... + PFN XXX ... + // Detailed stack + + The ``page_owner_sort`` tool ignores ``PFN`` rows, puts the remaining rows + in buf, uses regexp to extract the page order value, counts the times + and pages of buf, and finally sorts them according to the times. + See the result about who allocated each page - in the ``sorted_page_owner.txt``. + in the ``sorted_page_owner.txt``. General output: + + XXX times, XXX pages: + Page allocated via order XXX, ... + // Detailed stack + + By default, ``page_owner_sort`` is sorted according to the times of buf. + If you want to sort by the pages nums of buf, use the ``-m`` parameter. diff --git a/tools/vm/page_owner_sort.c b/tools/vm/page_owner_sort.c index 0e75f22c9475..9ebb84a9c731 100644 --- a/tools/vm/page_owner_sort.c +++ b/tools/vm/page_owner_sort.c @@ -5,6 +5,8 @@ * Example use: * cat /sys/kernel/debug/page_owner > page_owner_full.txt * ./page_owner_sort page_owner_full.txt sorted_page_owner.txt + * Or sort by total memory: + * ./page_owner_sort -m page_owner_full.txt sorted_page_owner.txt * * See Documentation/vm/page_owner.rst */ @@ -16,14 +18,18 @@ #include <fcntl.h> #include <unistd.h> #include <string.h> +#include <regex.h> +#include <errno.h> struct block_list { char *txt; int len; int num; + int page_num; }; - +static int sort_by_memory; +static regex_t order_pattern; static struct block_list *list; static int list_size; static int max_size; @@ -59,12 +65,50 @@ static int compare_num(const void *p1, const void *p2) return l2->num - l1->num; } +static int compare_page_num(const void *p1, const void *p2) +{ + const struct block_list *l1 = p1, *l2 = p2; + + return l2->page_num - l1->page_num; +} + +static int get_page_num(char *buf) +{ + int err, val_len, order_val; + char order_str[4] = {0}; + char *endptr; + regmatch_t pmatch[2]; + + err = regexec(&order_pattern, buf, 2, pmatch, REG_NOTBOL); + if (err != 0 || pmatch[1].rm_so == -1) { + printf("no order pattern in %s\n", buf); + return 0; + } + val_len = pmatch[1].rm_eo - pmatch[1].rm_so; + if (val_len > 2) /* max_order should not exceed 2 digits */ + goto wrong_order; + + memcpy(order_str, buf + pmatch[1].rm_so, val_len); + + errno = 0; + order_val = strtol(order_str, &endptr, 10); + if (errno != 0 || endptr == order_str || *endptr != '\0') + goto wrong_order; + + return 1 << order_val; + +wrong_order: + printf("wrong order in follow buf:\n%s\n", buf); + return 0; +} + static void add_list(char *buf, int len) { if (list_size != 0 && len == list[list_size-1].len && memcmp(buf, list[list_size-1].txt, len) == 0) { list[list_size-1].num++; + list[list_size-1].page_num += get_page_num(buf); return; } if (list_size == max_size) { @@ -74,6 +118,7 @@ static void add_list(char *buf, int len) list[list_size].txt = malloc(len+1); list[list_size].len = len; list[list_size].num = 1; + list[list_size].page_num = get_page_num(buf); memcpy(list[list_size].txt, buf, len); list[list_size].txt[len] = 0; list_size++; @@ -85,6 +130,13 @@ static void add_list(char *buf, int len) #define BUF_SIZE (128 * 1024) +static void usage(void) +{ + printf("Usage: ./page_owner_sort [-m] <input> <output>\n" + "-m Sort by total memory. If this option is unset, sort by times\n" + ); +} + int main(int argc, char **argv) { FILE *fin, *fout; @@ -92,21 +144,39 @@ int main(int argc, char **argv) int ret, i, count; struct block_list *list2; struct stat st; + int err; + int opt; - if (argc < 3) { - printf("Usage: ./program <input> <output>\n"); - perror("open: "); + while ((opt = getopt(argc, argv, "m")) != -1) + switch (opt) { + case 'm': + sort_by_memory = 1; + break; + default: + usage(); + exit(1); + } + + if (optind >= (argc - 1)) { + usage(); exit(1); } - fin = fopen(argv[1], "r"); - fout = fopen(argv[2], "w"); + fin = fopen(argv[optind], "r"); + fout = fopen(argv[optind + 1], "w"); if (!fin || !fout) { - printf("Usage: ./program <input> <output>\n"); + usage(); perror("open: "); exit(1); } + err = regcomp(&order_pattern, "order\\s*([0-9]*),", REG_EXTENDED|REG_NEWLINE); + if (err != 0 || order_pattern.re_nsub != 1) { + printf("%s: Invalid pattern 'order\\s*([0-9]*),' code %d\n", + argv[0], err); + exit(1); + } + fstat(fileno(fin), &st); max_size = st.st_size / 100; /* hack ... */ @@ -145,13 +215,19 @@ int main(int argc, char **argv) list2[count++] = list[i]; } else { list2[count-1].num += list[i].num; + list2[count-1].page_num += list[i].page_num; } } - qsort(list2, count, sizeof(list[0]), compare_num); + if (sort_by_memory) + qsort(list2, count, sizeof(list[0]), compare_page_num); + else + qsort(list2, count, sizeof(list[0]), compare_num); for (i = 0; i < count; i++) - fprintf(fout, "%d times:\n%s\n", list2[i].num, list2[i].txt); + fprintf(fout, "%d times, %d pages:\n%s\n", + list2[i].num, list2[i].page_num, list2[i].txt); + regfree(&order_pattern); return 0; } -- 2.25.1

From: Sean Anderson <seanga2@gmail.com> mainline inclusion from mainline-v5.18-rc1 commit 82f5ebc2beb3c803bf17bead4e38c1d5bf2c8d78 category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I9GSSR Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- This adds the ability to sort by stacktraces. This is helpful when comparing multiple dumps of page_owner taken at different times, since blocks will not be reordered if they were allocated/free'd. Link: https://lkml.kernel.org/r/20211124193709.1805776-2-seanga2@gmail.com Signed-off-by: Sean Anderson <seanga2@gmail.com> Cc: Zhenliang Wei <weizhenliang@huawei.com> Cc: Changhee Han <ch0.han@lge.com> Cc: Tang Bin <tangbin@cmss.chinamobile.com> Cc: Zhang Shengju <zhangshengju@cmss.chinamobile.com> Cc: Stephen Rothwell <sfr@canb.auug.org.au> Cc: Yinan Zhang <zhangyinan2019@email.szu.edu.cn> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com> --- tools/vm/page_owner_sort.c | 23 ++++++++++++++--------- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/tools/vm/page_owner_sort.c b/tools/vm/page_owner_sort.c index 5582d8454d3b..1b2acf02d3cd 100644 --- a/tools/vm/page_owner_sort.c +++ b/tools/vm/page_owner_sort.c @@ -29,7 +29,6 @@ struct block_list { int page_num; }; -static int sort_by_memory; static regex_t order_pattern; static struct block_list *list; static int list_size; @@ -134,13 +133,16 @@ static void add_list(char *buf, int len) static void usage(void) { - printf("Usage: ./page_owner_sort [-m] <input> <output>\n" - "-m Sort by total memory. If this option is unset, sort by times\n" + printf("Usage: ./page_owner_sort [OPTIONS] <input> <output>\n" + "-m Sort by total memory.\n" + "-s Sort by the stack trace.\n" + "-t Sort by times (default).\n" ); } int main(int argc, char **argv) { + int (*cmp)(const void *, const void *) = compare_num; FILE *fin, *fout; char *buf; int ret, i, count; @@ -149,10 +151,16 @@ int main(int argc, char **argv) int err; int opt; - while ((opt = getopt(argc, argv, "m")) != -1) + while ((opt = getopt(argc, argv, "mst")) != -1) switch (opt) { case 'm': - sort_by_memory = 1; + cmp = compare_page_num; + break; + case 's': + cmp = compare_stacktrace; + break; + case 't': + cmp = compare_num; break; default: usage(); @@ -221,10 +229,7 @@ int main(int argc, char **argv) } } - if (sort_by_memory) - qsort(list2, count, sizeof(list[0]), compare_page_num); - else - qsort(list2, count, sizeof(list[0]), compare_num); + qsort(list2, count, sizeof(list[0]), cmp); for (i = 0; i < count; i++) fprintf(fout, "%d times, %d pages:\n%s\n", -- 2.25.1

From: Chongxi Zhao <zhaochongxi2019@email.szu.edu.cn> mainline inclusion from mainline-v5.18-rc1 commit 8f9c447e2e2b53c2db4fac85fc42ecada8b39e52 category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I9GSSR Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- When viewing the page owner information, we expect that the information can be sorted by PID, so that we can quickly combine PID with the program to check the information together. We also expect that the information can be sorted by time. Time sorting helps to view the running status of the program according to the time interval when the program hangs up. Finally, we hope to pass the page_ owner_ Sort. C can reduce part of the output and only output the plate information whose memory has not been released, which can make us locate the problem of the program faster. Therefore, the following adjustments have been made: 1. Add the static functions search_pattern and check_regcomp to improve the cleanliness. 2. Add member attributes and their corresponding sorting methods. In terms of comparison time, int will overflow because the data of ull is too large, so the ternary operator is used 3. Add the -f parameter to filter out the information of blocks whose memory has not been released Link: https://lkml.kernel.org/r/20211206165653.5093-1-zhaochongxi2019@email.szu.ed... Signed-off-by: Chongxi Zhao <zhaochongxi2019@email.szu.edu.cn> Reviewed-by: Sean Anderson <seanga2@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com> --- tools/vm/page_owner_sort.c | 177 +++++++++++++++++++++++++++++++------ 1 file changed, 148 insertions(+), 29 deletions(-) diff --git a/tools/vm/page_owner_sort.c b/tools/vm/page_owner_sort.c index 492be7f752c0..c9fedc1806d5 100644 --- a/tools/vm/page_owner_sort.c +++ b/tools/vm/page_owner_sort.c @@ -20,6 +20,7 @@ #include <string.h> #include <regex.h> #include <errno.h> +#include <linux/types.h> struct block_list { char *txt; @@ -27,9 +28,15 @@ struct block_list { int len; int num; int page_num; + pid_t pid; + __u64 ts_nsec; + __u64 free_ts_nsec; }; static regex_t order_pattern; +static regex_t pid_pattern; +static regex_t ts_nsec_pattern; +static regex_t free_ts_nsec_pattern; static struct block_list *list; static int list_size; static int max_size; @@ -79,34 +86,124 @@ static int compare_page_num(const void *p1, const void *p2) return l2->page_num - l1->page_num; } -static int get_page_num(char *buf) +static int compare_pid(const void *p1, const void *p2) { - int err, val_len, order_val; - char order_str[4] = {0}; - char *endptr; + const struct block_list *l1 = p1, *l2 = p2; + + return l1->pid - l2->pid; +} + +static int compare_ts(const void *p1, const void *p2) +{ + const struct block_list *l1 = p1, *l2 = p2; + + return l1->ts_nsec < l2->ts_nsec ? -1 : 1; +} + +static int compare_free_ts(const void *p1, const void *p2) +{ + const struct block_list *l1 = p1, *l2 = p2; + + return l1->free_ts_nsec < l2->free_ts_nsec ? -1 : 1; +} + +static int search_pattern(regex_t *pattern, char *pattern_str, char *buf) +{ + int err, val_len; regmatch_t pmatch[2]; - err = regexec(&order_pattern, buf, 2, pmatch, REG_NOTBOL); + err = regexec(pattern, buf, 2, pmatch, REG_NOTBOL); if (err != 0 || pmatch[1].rm_so == -1) { - printf("no order pattern in %s\n", buf); - return 0; + printf("no matching pattern in %s\n", buf); + return -1; } val_len = pmatch[1].rm_eo - pmatch[1].rm_so; - if (val_len > 2) /* max_order should not exceed 2 digits */ - goto wrong_order; - memcpy(order_str, buf + pmatch[1].rm_so, val_len); + memcpy(pattern_str, buf + pmatch[1].rm_so, val_len); + + return 0; +} + +static void check_regcomp(regex_t *pattern, const char *regex) +{ + int err; + + err = regcomp(pattern, regex, REG_EXTENDED | REG_NEWLINE); + if (err != 0 || pattern->re_nsub != 1) { + printf("Invalid pattern %s code %d\n", regex, err); + exit(1); + } +} + +# define FIELD_BUFF 25 + +static int get_page_num(char *buf) +{ + int order_val; + char order_str[FIELD_BUFF] = {0}; + char *endptr; + search_pattern(&order_pattern, order_str, buf); errno = 0; order_val = strtol(order_str, &endptr, 10); - if (errno != 0 || endptr == order_str || *endptr != '\0') - goto wrong_order; + if (order_val > 64 || errno != 0 || endptr == order_str || *endptr != '\0') { + printf("wrong order in follow buf:\n%s\n", buf); + return 0; + } return 1 << order_val; +} -wrong_order: - printf("wrong order in follow buf:\n%s\n", buf); - return 0; +static pid_t get_pid(char *buf) +{ + pid_t pid; + char pid_str[FIELD_BUFF] = {0}; + char *endptr; + + search_pattern(&pid_pattern, pid_str, buf); + errno = 0; + pid = strtol(pid_str, &endptr, 10); + if (errno != 0 || endptr == pid_str || *endptr != '\0') { + printf("wrong/invalid pid in follow buf:\n%s\n", buf); + return -1; + } + + return pid; + +} + +static __u64 get_ts_nsec(char *buf) +{ + __u64 ts_nsec; + char ts_nsec_str[FIELD_BUFF] = {0}; + char *endptr; + + search_pattern(&ts_nsec_pattern, ts_nsec_str, buf); + errno = 0; + ts_nsec = strtoull(ts_nsec_str, &endptr, 10); + if (errno != 0 || endptr == ts_nsec_str || *endptr != '\0') { + printf("wrong ts_nsec in follow buf:\n%s\n", buf); + return -1; + } + + return ts_nsec; +} + +static __u64 get_free_ts_nsec(char *buf) +{ + __u64 free_ts_nsec; + char free_ts_nsec_str[FIELD_BUFF] = {0}; + char *endptr; + + search_pattern(&free_ts_nsec_pattern, free_ts_nsec_str, buf); + errno = 0; + free_ts_nsec = strtoull(free_ts_nsec_str, &endptr, 10); + if (errno != 0 || endptr == free_ts_nsec_str || *endptr != '\0') { + printf("wrong free_ts_nsec in follow buf:\n%s\n", buf); + return -1; + } + + return free_ts_nsec; } static void add_list(char *buf, int len) @@ -129,6 +226,11 @@ static void add_list(char *buf, int len) memcpy(list[list_size].txt, buf, len); list[list_size].txt[len] = 0; list[list_size].stacktrace = strchr(list[list_size].txt, '\n') ?: ""; + list[list_size].pid = get_pid(buf); + list[list_size].ts_nsec = get_ts_nsec(buf); + list[list_size].free_ts_nsec = get_free_ts_nsec(buf); + memcpy(list[list_size].txt, buf, len); + list[list_size].txt[len] = 0; list_size++; if (list_size % 1000 == 0) { printf("loaded %d\r", list_size); @@ -144,6 +246,9 @@ static void usage(void) "-m Sort by total memory.\n" "-s Sort by the stack trace.\n" "-t Sort by times (default).\n" + "-p Sort by pid.\n" + "-a Sort by memory allocate time.\n" + "-r Sort by memory release time.\n" "-c cull by comparing stacktrace instead of total block.\n" ); } @@ -152,28 +257,40 @@ int main(int argc, char **argv) { int (*cmp)(const void *, const void *) = compare_num; int cull_st = 0; + int filter = 0; FILE *fin, *fout; char *buf; int ret, i, count; struct block_list *list2; struct stat st; - int err; int opt; - while ((opt = getopt(argc, argv, "mstc")) != -1) + while ((opt = getopt(argc, argv, "acfmprst")) != -1) switch (opt) { + case 'a': + cmp = compare_ts; + break; + case 'c': + cull_st = 1; + break; + case 'f': + filter = 1; + break; case 'm': cmp = compare_page_num; break; + case 'p': + cmp = compare_pid; + break; + case 'r': + cmp = compare_free_ts; + break; case 's': cmp = compare_stacktrace; break; case 't': cmp = compare_num; break; - case 'c': - cull_st = 1; - break; default: usage(); exit(1); @@ -192,13 +309,10 @@ int main(int argc, char **argv) exit(1); } - err = regcomp(&order_pattern, "order\\s*([0-9]*),", REG_EXTENDED|REG_NEWLINE); - if (err != 0 || order_pattern.re_nsub != 1) { - printf("%s: Invalid pattern 'order\\s*([0-9]*),' code %d\n", - argv[0], err); - exit(1); - } - + check_regcomp(&order_pattern, "order\\s*([0-9]*),"); + check_regcomp(&pid_pattern, "pid\\s*([0-9]*),"); + check_regcomp(&ts_nsec_pattern, "ts\\s*([0-9]*)\\s*ns,"); + check_regcomp(&free_ts_nsec_pattern, "free_ts\\s*([0-9]*)\\s*ns"); fstat(fileno(fin), &st); max_size = st.st_size / 100; /* hack ... */ @@ -248,10 +362,15 @@ int main(int argc, char **argv) qsort(list2, count, sizeof(list[0]), cmp); - for (i = 0; i < count; i++) + for (i = 0; i < count; i++) { + if (filter == 1 && list2[i].free_ts_nsec != 0) + continue; fprintf(fout, "%d times, %d pages:\n%s\n", list2[i].num, list2[i].page_num, list2[i].txt); - + } regfree(&order_pattern); + regfree(&pid_pattern); + regfree(&ts_nsec_pattern); + regfree(&free_ts_nsec_pattern); return 0; } -- 2.25.1

From: Yixuan Cao <caoyixuan2019@email.szu.edu.cn> mainline inclusion from mainline-v5.18-rc1 commit 41ed64347b5db8f6c9359d4f87f768db1a83bd79 category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I9GSSR Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- I noticed that there is two invalid lines of duplicate code. It's better to delete it. Link: https://lkml.kernel.org/r/20211213095743.3630-1-caoyixuan2019@email.szu.edu.... Signed-off-by: Yixuan Cao <caoyixuan2019@email.szu.edu.cn> Cc: Mark Brown <broonie@kernel.org> Cc: Sean Anderson <seanga2@gmail.com> Cc: Zhenliang Wei <weizhenliang@huawei.com> Cc: Tang Bin <tangbin@cmss.chinamobile.com> Cc: Yinan Zhang <zhangyinan2019@email.szu.edu.cn> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com> --- tools/vm/page_owner_sort.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/tools/vm/page_owner_sort.c b/tools/vm/page_owner_sort.c index 284a5070402c..c8ec2d6b314d 100644 --- a/tools/vm/page_owner_sort.c +++ b/tools/vm/page_owner_sort.c @@ -227,8 +227,6 @@ static void add_list(char *buf, int len) list[list_size].pid = get_pid(buf); list[list_size].ts_nsec = get_ts_nsec(buf); list[list_size].free_ts_nsec = get_free_ts_nsec(buf); - memcpy(list[list_size].txt, buf, len); - list[list_size].txt[len] = 0; list_size++; if (list_size % 1000 == 0) { printf("loaded %d\r", list_size); -- 2.25.1

From: Jiajian Ye <yejiajian2018@email.szu.edu.cn> mainline inclusion from mainline-v5.18-rc1 commit 56465a38305f22bca3469c2738d7320a0c333e72 category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I9GSSR Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- Add a security check after using malloc() to allocate memory. Link: https://lkml.kernel.org/r/20220301151438.166118-2-yejiajian2018@email.szu.ed... Signed-off-by: Jiajian Ye <yejiajian2018@email.szu.edu.cn> Cc: Stephen Rothwell <sfr@canb.auug.org.au> Cc: Yinan Zhang <zhangyinan2019@email.szu.edu.cn> Cc: Yixuan Cao <caoyixuan2019@email.szu.edu.cn> Cc: Zhenliang Wei <weizhenliang@huawei.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com> --- tools/vm/page_owner_sort.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/tools/vm/page_owner_sort.c b/tools/vm/page_owner_sort.c index 79d69c3b84ed..69fb6ca7c0b7 100644 --- a/tools/vm/page_owner_sort.c +++ b/tools/vm/page_owner_sort.c @@ -217,7 +217,13 @@ static void add_list(char *buf, int len) printf("max_size too small??\n"); exit(1); } + list[list_size].txt = malloc(len+1); + if (!list[list_size].txt) { + printf("Out of memory\n"); + exit(1); + } + list[list_size].len = len; list[list_size].num = 1; list[list_size].page_num = get_page_num(buf); -- 2.25.1

From: Jiajian Ye <yejiajian2018@email.szu.edu.cn> mainline inclusion from mainline-v5.18-rc1 commit cf3c2c8678a0b21052d00b64d7a5903f3b1d1197 category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I9GSSR Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- When the "page owner" information is read, the information sorted by TGID is expected. As a result, the following adjustments have been made: 1. Add a new -P option to sort the information of blocks by TGID in ascending order. 2. Adjust the order of member variables in block_list strust to avoid one 4 byte hole. 3. Add -P option explanation in the document. Link: https://lkml.kernel.org/r/20220301151438.166118-3-yejiajian2018@email.szu.ed... Signed-off-by: Jiajian Ye <yejiajian2018@email.szu.edu.cn> Cc: Stephen Rothwell <sfr@canb.auug.org.au> Cc: Yixuan Cao <caoyixuan2019@email.szu.edu.cn> Cc: Zhenliang Wei <weizhenliang@huawei.com> Cc: Yinan Zhang <zhangyinan2019@email.szu.edu.cn> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com> --- Documentation/vm/page_owner.rst | 1 + tools/vm/page_owner_sort.c | 40 ++++++++++++++++++++++++++++++--- 2 files changed, 38 insertions(+), 3 deletions(-) diff --git a/Documentation/vm/page_owner.rst b/Documentation/vm/page_owner.rst index 602cf6eefcb5..b83332fd5eed 100644 --- a/Documentation/vm/page_owner.rst +++ b/Documentation/vm/page_owner.rst @@ -116,6 +116,7 @@ Usage -a Sort by memory allocation time. -m Sort by total memory. -p Sort by pid. + -P Sort by tgid. -r Sort by memory release time. -s Sort by stack trace. -t Sort by times (default). diff --git a/tools/vm/page_owner_sort.c b/tools/vm/page_owner_sort.c index 69fb6ca7c0b7..d166f2f900eb 100644 --- a/tools/vm/page_owner_sort.c +++ b/tools/vm/page_owner_sort.c @@ -25,16 +25,18 @@ struct block_list { char *txt; char *stacktrace; + __u64 ts_nsec; + __u64 free_ts_nsec; int len; int num; int page_num; pid_t pid; - __u64 ts_nsec; - __u64 free_ts_nsec; + pid_t tgid; }; static regex_t order_pattern; static regex_t pid_pattern; +static regex_t tgid_pattern; static regex_t ts_nsec_pattern; static regex_t free_ts_nsec_pattern; static struct block_list *list; @@ -91,6 +93,13 @@ static int compare_pid(const void *p1, const void *p2) return l1->pid - l2->pid; } +static int compare_tgid(const void *p1, const void *p2) +{ + const struct block_list *l1 = p1, *l2 = p2; + + return l1->tgid - l2->tgid; +} + static int compare_ts(const void *p1, const void *p2) { const struct block_list *l1 = p1, *l2 = p2; @@ -170,6 +179,24 @@ static pid_t get_pid(char *buf) } +static pid_t get_tgid(char *buf) +{ + pid_t tgid; + char tgid_str[FIELD_BUFF] = {0}; + char *endptr; + + search_pattern(&tgid_pattern, tgid_str, buf); + errno = 0; + tgid = strtol(tgid_str, &endptr, 10); + if (errno != 0 || endptr == tgid_str || *endptr != '\0') { + printf("wrong/invalid tgid in follow buf:\n%s\n", buf); + return -1; + } + + return tgid; + +} + static __u64 get_ts_nsec(char *buf) { __u64 ts_nsec; @@ -231,6 +258,7 @@ static void add_list(char *buf, int len) list[list_size].txt[len] = 0; list[list_size].stacktrace = strchr(list[list_size].txt, '\n') ?: ""; list[list_size].pid = get_pid(buf); + list[list_size].tgid = get_tgid(buf); list[list_size].ts_nsec = get_ts_nsec(buf); list[list_size].free_ts_nsec = get_free_ts_nsec(buf); list_size++; @@ -249,6 +277,7 @@ static void usage(void) "-s Sort by the stack trace.\n" "-t Sort by times (default).\n" "-p Sort by pid.\n" + "-P Sort by tgid.\n" "-a Sort by memory allocate time.\n" "-r Sort by memory release time.\n" "-c Cull by comparing stacktrace instead of total block.\n" @@ -268,7 +297,7 @@ int main(int argc, char **argv) struct stat st; int opt; - while ((opt = getopt(argc, argv, "acfmprst")) != -1) + while ((opt = getopt(argc, argv, "acfmprstP")) != -1) switch (opt) { case 'a': cmp = compare_ts; @@ -294,6 +323,9 @@ int main(int argc, char **argv) case 't': cmp = compare_num; break; + case 'P': + cmp = compare_tgid; + break; default: usage(); exit(1); @@ -314,6 +346,7 @@ int main(int argc, char **argv) check_regcomp(&order_pattern, "order\\s*([0-9]*),"); check_regcomp(&pid_pattern, "pid\\s*([0-9]*),"); + check_regcomp(&tgid_pattern, "tgid\\s*([0-9]*) "); check_regcomp(&ts_nsec_pattern, "ts\\s*([0-9]*)\\s*ns,"); check_regcomp(&free_ts_nsec_pattern, "free_ts\\s*([0-9]*)\\s*ns"); fstat(fileno(fin), &st); @@ -373,6 +406,7 @@ int main(int argc, char **argv) } regfree(&order_pattern); regfree(&pid_pattern); + regfree(&tgid_pattern); regfree(&ts_nsec_pattern); regfree(&free_ts_nsec_pattern); return 0; -- 2.25.1

From: Jiajian Ye <yejiajian2018@email.szu.edu.cn> mainline inclusion from mainline-v5.18-rc1 commit 194d52d771b8f7cf5bcf0f81f87dd76e492c355c category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I9GSSR Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- When viewing page owner information, we may also need to the block to be sorted by task command name. Therefore, the following adjustments are made: 1. Add a member variable to record task command name of block. 2. Add a new -n option to sort the information of blocks by task command name. 3. Add -n option explanation in the document. Link: https://lkml.kernel.org/r/20220306030640.43054-2-yejiajian2018@email.szu.edu... Signed-off-by: Jiajian Ye <yejiajian2018@email.szu.edu.cn> Cc: Stephen Rothwell <sfr@canb.auug.org.au> Cc: Sean Anderson <seanga2@gmail.com> Cc: Yixuan Cao <caoyixuan2019@email.szu.edu.cn> Cc: Zhenliang Wei <weizhenliang@huawei.com> Cc: <zhaochongxi2019@email.szu.edu.cn> Cc: <hanshenghong2019@email.szu.edu.cn> Cc: <zhangyinan2019@email.szu.edu.cn> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com> --- Documentation/vm/page_owner.rst | 1 + tools/vm/page_owner_sort.c | 35 ++++++++++++++++++++++++++++++++- 2 files changed, 35 insertions(+), 1 deletion(-) diff --git a/Documentation/vm/page_owner.rst b/Documentation/vm/page_owner.rst index b83332fd5eed..73a10b2f368c 100644 --- a/Documentation/vm/page_owner.rst +++ b/Documentation/vm/page_owner.rst @@ -117,6 +117,7 @@ Usage -m Sort by total memory. -p Sort by pid. -P Sort by tgid. + -n Sort by task command name. -r Sort by memory release time. -s Sort by stack trace. -t Sort by times (default). diff --git a/tools/vm/page_owner_sort.c b/tools/vm/page_owner_sort.c index b8d2867b5b18..e508abd8f665 100644 --- a/tools/vm/page_owner_sort.c +++ b/tools/vm/page_owner_sort.c @@ -24,6 +24,7 @@ struct block_list { char *txt; + char *comm; // task command name char *stacktrace; __u64 ts_nsec; __u64 free_ts_nsec; @@ -37,6 +38,7 @@ struct block_list { static regex_t order_pattern; static regex_t pid_pattern; static regex_t tgid_pattern; +static regex_t comm_pattern; static regex_t ts_nsec_pattern; static regex_t free_ts_nsec_pattern; static struct block_list *list; @@ -102,6 +104,13 @@ static int compare_tgid(const void *p1, const void *p2) return l1->tgid - l2->tgid; } +static int compare_comm(const void *p1, const void *p2) +{ + const struct block_list *l1 = p1, *l2 = p2; + + return strcmp(l1->comm, l2->comm); +} + static int compare_ts(const void *p1, const void *p2) { const struct block_list *l1 = p1, *l2 = p2; @@ -145,6 +154,7 @@ static void check_regcomp(regex_t *pattern, const char *regex) } # define FIELD_BUFF 25 +# define TASK_COMM_LEN 16 static int get_page_num(char *buf) { @@ -233,6 +243,22 @@ static __u64 get_free_ts_nsec(char *buf) return free_ts_nsec; } +static char *get_comm(char *buf) +{ + char *comm_str = malloc(TASK_COMM_LEN); + + memset(comm_str, 0, TASK_COMM_LEN); + + search_pattern(&comm_pattern, comm_str, buf); + errno = 0; + if (errno != 0) { + printf("wrong comm in follow buf:\n%s\n", buf); + return NULL; + } + + return comm_str; +} + static void add_list(char *buf, int len) { if (list_size != 0 && @@ -266,6 +292,7 @@ static void add_list(char *buf, int len) list[list_size].stacktrace++; list[list_size].pid = get_pid(buf); list[list_size].tgid = get_tgid(buf); + list[list_size].comm = get_comm(buf); list[list_size].ts_nsec = get_ts_nsec(buf); list_size++; if (list_size % 1000 == 0) { @@ -284,6 +311,7 @@ static void usage(void) "-t Sort by times (default).\n" "-p Sort by pid.\n" "-P Sort by tgid.\n" + "-n Sort by task command name.\n" "-a Sort by memory allocate time.\n" "-r Sort by memory release time.\n" "-c Cull by comparing stacktrace instead of total block.\n" @@ -300,7 +328,7 @@ int main(int argc, char **argv) struct stat st; int opt; - while ((opt = getopt(argc, argv, "acfmprstP")) != -1) + while ((opt = getopt(argc, argv, "acfmnprstP")) != -1) switch (opt) { case 'a': cmp = compare_ts; @@ -329,6 +357,9 @@ int main(int argc, char **argv) case 'P': cmp = compare_tgid; break; + case 'n': + cmp = compare_comm; + break; default: usage(); exit(1); @@ -350,6 +381,7 @@ int main(int argc, char **argv) check_regcomp(&order_pattern, "order\\s*([0-9]*),"); check_regcomp(&pid_pattern, "pid\\s*([0-9]*),"); check_regcomp(&tgid_pattern, "tgid\\s*([0-9]*) "); + check_regcomp(&comm_pattern, "tgid\\s*[0-9]*\\s*\\((.*)\\),\\s*ts"); check_regcomp(&ts_nsec_pattern, "ts\\s*([0-9]*)\\s*ns,"); check_regcomp(&free_ts_nsec_pattern, "free_ts\\s*([0-9]*)\\s*ns"); fstat(fileno(fin), &st); @@ -408,6 +440,7 @@ int main(int argc, char **argv) regfree(&order_pattern); regfree(&pid_pattern); regfree(&tgid_pattern); + regfree(&comm_pattern); regfree(&ts_nsec_pattern); regfree(&free_ts_nsec_pattern); return 0; -- 2.25.1

From: Jiajian Ye <yejiajian2018@email.szu.edu.cn> mainline inclusion from mainline-v5.18-rc1 commit 9c8a0a8e599f4a949ef18207ba495fb557dd1016 category: feature bugzilla: 188978 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- When viewing page owner information, we may want to cull blocks of information with our own rules. So it is important to enhance culling function to provide the support for customizing culling rules. Therefore, following adjustments are made: 1. Add --cull option to support the culling of blocks of information with user-defined culling rules. ./page_owner_sort <input> <output> --cull=<rules> ./page_owner_sort <input> <output> --cull <rules> <rules> is a single argument in the form of a comma-separated list to specify individual culling rules, by the sequence of keys k1,k2, .... Mixed use of abbreviated and complete-form of keys is allowed. For reference, please see the document(Documentation/vm/page_owner.rst). Now, assuming two blocks in the input file are as follows: Page allocated via order 0, mask xxxx, pid 1, tgid 1 (task_name_demo) PFN xxxx prep_new_page+0xd0/0xf8 get_page_from_freelist+0x4a0/0x1290 __alloc_pages+0x168/0x340 alloc_pages+0xb0/0x158 Page allocated via order 0, mask xxxx, pid 32, tgid 32 (task_name_demo) PFN xxxx prep_new_page+0xd0/0xf8 get_page_from_freelist+0x4a0/0x1290 __alloc_pages+0x168/0x340 alloc_pages+0xb0/0x158 If we want to cull the blocks by stacktrace and task command name, we can use this command: ./page_owner_sort <input> <output> --cull=stacktrace,name The output would be like: 2 times, 2 pages, task_comm_name: task_name_demo prep_new_page+0xd0/0xf8 get_page_from_freelist+0x4a0/0x1290 __alloc_pages+0x168/0x340 alloc_pages+0xb0/0x158 As we can see, these two blocks are culled successfully, for they share the same pid and task command name. However, if we want to cull the blocks by pid, stacktrace and task command name, we can this command: ./page_owner_sort <input> <output> --cull=stacktrace,name,pid The output would be like: 1 times, 1 pages, PID 1, task_comm_name: task_name_demo prep_new_page+0xd0/0xf8 get_page_from_freelist+0x4a0/0x1290 __alloc_pages+0x168/0x340 alloc_pages+0xb0/0x158 1 times, 1 pages, PID 32, task_comm_name: task_name_demo prep_new_page+0xd0/0xf8 get_page_from_freelist+0x4a0/0x1290 __alloc_pages+0x168/0x340 alloc_pages+0xb0/0x158 As we can see, these two blocks are failed to cull, for their PIDs are different. 2. Add explanations of --cull options to the document. This work is coauthored by Yixuan Cao Shenghong Han Yinan Zhang Chongxi Zhao Yuhong Feng Link: https://lkml.kernel.org/r/20220312145834.624-1-yejiajian2018@email.szu.edu.c... Signed-off-by: Jiajian Ye <yejiajian2018@email.szu.edu.cn> Cc: Yixuan Cao <caoyixuan2019@email.szu.edu.cn> Cc: Shenghong Han <hanshenghong2019@email.szu.edu.cn> Cc: Yinan Zhang <zhangyinan2019@email.szu.edu.cn> Cc: Chongxi Zhao <zhaochongxi2019@email.szu.edu.cn> Cc: Yuhong Feng <yuhongf@szu.edu.cn> Cc: Stephen Rothwell <sfr@canb.auug.org.au> Cc: Sean Anderson <seanga2@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com> --- Documentation/vm/page_owner.rst | 31 ++++++- tools/vm/page_owner_sort.c | 150 +++++++++++++++++++++++++++----- 2 files changed, 158 insertions(+), 23 deletions(-) diff --git a/Documentation/vm/page_owner.rst b/Documentation/vm/page_owner.rst index 4014bfb529b2..23565062ead1 100644 --- a/Documentation/vm/page_owner.rst +++ b/Documentation/vm/page_owner.rst @@ -126,11 +126,38 @@ Usage Cull: -c Cull by comparing stacktrace instead of total block. + --cull <rules> + Specify culling rules.Culling syntax is key[,key[,...]].Choose a + multi-letter key from the **STANDARD FORMAT SPECIFIERS** section. + + + <rules> is a single argument in the form of a comma-separated list, + which offers a way to specify individual culling rules. The recognized + keywords are described in the **STANDARD FORMAT SPECIFIERS** section below. + <rules> can be specified by the sequence of keys k1,k2, ..., as described in + the STANDARD SORT KEYS section below. Mixed use of abbreviated and + complete-form of keys is allowed. + + + Examples: + ./page_owner_sort <input> <output> --cull=stacktrace + ./page_owner_sort <input> <output> --cull=st,pid,name + ./page_owner_sort <input> <output> --cull=n,f Filter: -f Filter out the information of blocks whose memory has not been released. Select: - --pid <PID> Select by pid. + --pid <PID> Select by pid. --tgid <TGID> Select by tgid. - --name <command> Select by task command name. \ No newline at end of file + --name <command> Select by task command name. + +STANDARD FORMAT SPECIFIERS +========================== + + KEY LONG DESCRIPTION + p pid process ID + tg tgid thread group ID + n name task command name + f free whether the page has been released or not + st stacktrace stace trace of the page allocation \ No newline at end of file diff --git a/tools/vm/page_owner_sort.c b/tools/vm/page_owner_sort.c index e873eff84462..7679335fce5b 100644 --- a/tools/vm/page_owner_sort.c +++ b/tools/vm/page_owner_sort.c @@ -44,7 +44,14 @@ enum FILTER_BIT { FILTER_UNRELEASE = 1<<1, FILTER_PID = 1<<2, FILTER_TGID = 1<<3, - FILTER_TASK_COMM_NAME = 1<<4 + FILTER_COMM = 1<<4 +}; +enum CULL_BIT { + CULL_UNRELEASE = 1<<1, + CULL_PID = 1<<2, + CULL_TGID = 1<<3, + CULL_COMM = 1<<4, + CULL_STACKTRACE = 1<<5 }; struct filter_condition { pid_t tgid; @@ -61,7 +68,7 @@ static regex_t free_ts_nsec_pattern; static struct block_list *list; static int list_size; static int max_size; -static int cull_st; +static int cull; static int filter; int read_block(char *buf, int buf_size, FILE *fin) @@ -142,6 +149,36 @@ static int compare_free_ts(const void *p1, const void *p2) return l1->free_ts_nsec < l2->free_ts_nsec ? -1 : 1; } + +static int compare_release(const void *p1, const void *p2) +{ + const struct block_list *l1 = p1, *l2 = p2; + + if (!l1->free_ts_nsec && !l2->free_ts_nsec) + return 0; + if (l1->free_ts_nsec && l2->free_ts_nsec) + return 0; + return l1->free_ts_nsec ? 1 : -1; +} + + +static int compare_cull_condition(const void *p1, const void *p2) +{ + if (cull == 0) + return compare_txt(p1, p2); + if ((cull & CULL_STACKTRACE) && compare_stacktrace(p1, p2)) + return compare_stacktrace(p1, p2); + if ((cull & CULL_PID) && compare_pid(p1, p2)) + return compare_pid(p1, p2); + if ((cull & CULL_TGID) && compare_tgid(p1, p2)) + return compare_tgid(p1, p2); + if ((cull & CULL_COMM) && compare_comm(p1, p2)) + return compare_comm(p1, p2); + if ((cull & CULL_UNRELEASE) && compare_release(p1, p2)) + return compare_release(p1, p2); + return 0; +} + static int search_pattern(regex_t *pattern, char *pattern_str, char *buf) { int err, val_len; @@ -170,6 +207,38 @@ static void check_regcomp(regex_t *pattern, const char *regex) } } +static char **explode(char sep, const char *str, int *size) +{ + int count = 0, len = strlen(str); + int lastindex = -1, j = 0; + + for (int i = 0; i < len; i++) + if (str[i] == sep) + count++; + char **ret = calloc(++count, sizeof(char *)); + + for (int i = 0; i < len; i++) { + if (str[i] == sep) { + ret[j] = calloc(i - lastindex, sizeof(char)); + memcpy(ret[j++], str + lastindex + 1, i - lastindex - 1); + lastindex = i; + } + } + if (lastindex <= len - 1) { + ret[j] = calloc(len - lastindex, sizeof(char)); + memcpy(ret[j++], str + lastindex + 1, strlen(str) - 1 - lastindex); + } + *size = j; + return ret; +} + +static void free_explode(char **arr, int size) +{ + for (int i = 0; i < size; i++) + free(arr[i]); + free(arr); +} + # define FIELD_BUFF 25 static int get_page_num(char *buf) @@ -277,16 +346,16 @@ static char *get_comm(char *buf) static bool is_need(char *buf) { - if ((filter & FILTER_UNRELEASE) != 0 && get_free_ts_nsec(buf) != 0) + if ((filter & FILTER_UNRELEASE) && get_free_ts_nsec(buf) != 0) return false; - if ((filter & FILTER_PID) != 0 && get_pid(buf) != fc.pid) + if ((filter & FILTER_PID) && get_pid(buf) != fc.pid) return false; - if ((filter & FILTER_TGID) != 0 && get_tgid(buf) != fc.tgid) + if ((filter & FILTER_TGID) && get_tgid(buf) != fc.tgid) return false; char *comm = get_comm(buf); - if ((filter & FILTER_TASK_COMM_NAME) != 0 && + if ((filter & FILTER_COMM) && strncmp(comm, fc.comm, TASK_COMM_LEN) != 0) { free(comm); return false; @@ -335,6 +404,30 @@ static void add_list(char *buf, int len) } } +static bool parse_cull_args(const char *arg_str) +{ + int size = 0; + char **args = explode(',', arg_str, &size); + + for (int i = 0; i < size; ++i) + if (!strcmp(args[i], "pid") || !strcmp(args[i], "p")) + cull |= CULL_PID; + else if (!strcmp(args[i], "tgid") || !strcmp(args[i], "tg")) + cull |= CULL_TGID; + else if (!strcmp(args[i], "name") || !strcmp(args[i], "n")) + cull |= CULL_COMM; + else if (!strcmp(args[i], "stacktrace") || !strcmp(args[i], "st")) + cull |= CULL_STACKTRACE; + else if (!strcmp(args[i], "free") || !strcmp(args[i], "f")) + cull |= CULL_UNRELEASE; + else { + free_explode(args, size); + return false; + } + free_explode(args, size); + return true; +} + #define BUF_SIZE (128 * 1024) static void usage(void) @@ -353,6 +446,7 @@ static void usage(void) "--pid <PID>\tSelect by pid. This selects the information of blocks whose process ID number equals to <PID>.\n" "--tgid <TGID>\tSelect by tgid. This selects the information of blocks whose Thread Group ID number equals to <TGID>.\n" "--name <command>\n\t\tSelect by command name. This selects the information of blocks whose command name identical to <command>.\n" + "--cull <rules>\tCull by user-defined rules. <rules> is a single argument in the form of a comma-separated list with some common fields predefined\n" ); } @@ -368,6 +462,7 @@ int main(int argc, char **argv) { "pid", required_argument, NULL, 1 }, { "tgid", required_argument, NULL, 2 }, { "name", required_argument, NULL, 3 }, + { "cull", required_argument, NULL, 4 }, { 0, 0, 0, 0}, }; @@ -377,7 +472,7 @@ int main(int argc, char **argv) cmp = compare_ts; break; case 'c': - cull_st = 1; + cull = cull | CULL_STACKTRACE; break; case 'f': filter = filter | FILTER_UNRELEASE; @@ -422,10 +517,17 @@ int main(int argc, char **argv) } break; case 3: - filter = filter | FILTER_TASK_COMM_NAME; + filter = filter | FILTER_COMM; strncpy(fc.comm, optarg, TASK_COMM_LEN); fc.comm[TASK_COMM_LEN-1] = '\0'; break; + case 4: + if (!parse_cull_args(optarg)) { + printf("wrong argument after --cull in from the command line:%s\n", + optarg); + exit(1); + } + break; default: usage(); exit(1); @@ -472,20 +574,13 @@ int main(int argc, char **argv) printf("sorting ....\n"); - if (cull_st == 1) - qsort(list, list_size, sizeof(list[0]), compare_stacktrace); - else - qsort(list, list_size, sizeof(list[0]), compare_txt); - - + qsort(list, list_size, sizeof(list[0]), compare_cull_condition); printf("culling\n"); - long offset = cull_st ? &list[0].stacktrace - &list[0].txt : 0; - for (i = count = 0; i < list_size; i++) { if (count == 0 || - strcmp(*(&list[count-1].txt+offset), *(&list[i].txt+offset)) != 0) { + compare_cull_condition((void *)(&list[count-1]), (void *)(&list[i])) != 0) { list[count++] = list[i]; } else { list[count-1].num += list[i].num; @@ -496,12 +591,25 @@ int main(int argc, char **argv) qsort(list, count, sizeof(list[0]), cmp); for (i = 0; i < count; i++) { - if (cull_st == 0) + if (cull == 0) fprintf(fout, "%d times, %d pages:\n%s\n", list[i].num, list[i].page_num, list[i].txt); - else - fprintf(fout, "%d times, %d pages:\n%s\n", - list[i].num, list[i].page_num, list[i].stacktrace); + else { + fprintf(fout, "%d times, %d pages", + list[i].num, list[i].page_num); + if (cull & CULL_PID || filter & FILTER_PID) + fprintf(fout, ", PID %d", list[i].pid); + if (cull & CULL_TGID || filter & FILTER_TGID) + fprintf(fout, ", TGID %d", list[i].pid); + if (cull & CULL_COMM || filter & FILTER_COMM) + fprintf(fout, ", task_comm_name: %s", list[i].comm); + if (cull & CULL_UNRELEASE) + fprintf(fout, " (%s)", + list[i].free_ts_nsec ? "UNRELEASED" : "RELEASED"); + if (cull & CULL_STACKTRACE) + fprintf(fout, ":\n%s", list[i].stacktrace); + fprintf(fout, "\n"); + } } regfree(&order_pattern); regfree(&pid_pattern); -- 2.25.1

From: Yinan Zhang <zhangyinan2019@email.szu.edu.cn> mainline inclusion from mainline-v5.18-rc1 commit c89b3ad2dea254ad17ae2585b17c2cf9f78e64d9 category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I9GSSR Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- -c option has been removed from page_owner_sort.c. Remove the usage of -c option from Documentation. This work is coauthored by Shenghong Han Yixuan Cao Chongxi Zhao Jiajian Ye Yuhong Feng Yongqiang Liu Link: https://lkml.kernel.org/r/20220326085920.1470081-2-zhangyinan2019@email.szu.... Signed-off-by: Yinan Zhang <zhangyinan2019@email.szu.edu.cn> Cc: Jonathan Corbet <corbet@lwn.net> Cc: Stephen Rothwell <sfr@canb.auug.org.au> Cc: Sean Anderson <seanga2@gmail.com> Cc: Tang Bin <tangbin@cmss.chinamobile.com> Cc: Zhenliang Wei <weizhenliang@huawei.com> Cc: Georgi Djakov <georgi.djakov@linaro.org> Cc: Chongxi Zhao <zhaochongxi2019@email.szu.edu.cn> Cc: Jiajian Ye <yejiajian2018@email.szu.edu.cn> Cc: Yixuan Cao <caoyixuan2019@email.szu.edu.cn> Cc: Yuhong Feng <yuhongf@szu.edu.cn> Cc: Yongqiang Liu <liuyongqiang13@huawei.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com> --- Documentation/vm/page_owner.rst | 1 - 1 file changed, 1 deletion(-) diff --git a/Documentation/vm/page_owner.rst b/Documentation/vm/page_owner.rst index 23565062ead1..c0dda23aa7d8 100644 --- a/Documentation/vm/page_owner.rst +++ b/Documentation/vm/page_owner.rst @@ -125,7 +125,6 @@ Usage additional function: Cull: - -c Cull by comparing stacktrace instead of total block. --cull <rules> Specify culling rules.Culling syntax is key[,key[,...]].Choose a multi-letter key from the **STANDARD FORMAT SPECIFIERS** section. -- 2.25.1

From: Jiajian Ye <yejiajian2018@email.szu.edu.cn> mainline inclusion from mainline-v5.19-rc1 commit 75382a2dca0e9e9e57e88b479cf537549461a934 category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I9GSSR Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- When viewing page owner information, we may want to select blocks whose PID/TGID/TASK_COMM_NAME appears in a user-specified list for data analysis and aggregation. But currently page_owner_sort only supports selecting blocks associated with only one specified PID/TGID/TASK_COMM_NAME. Therefore, following adjustments are made to fix the problem: 1. Enhance selecting function to support the selection of multiple PIDs/TGIDs/TASK_COMM_NAMEs. The enhanced usages are as follows: --pid <pidlist> Select by pid. This selects the blocks whose PID numbers appear in <pidlist>. --tgid <tgidlist> Select by tgid. This selects the blocks whose TGID numbers appear in <tgidlist>. --name <cmdlist> Select by task command name. This selects the blocks whose task command name appear in <cmdlist>. Where <pidlist>, <tgidlist>, <cmdlist> are single arguments in the form of a comma-separated list,which offers a way to specify individual selecting rules. For example, if you want to select blocks whose tgids are 1, 2 or 3, you have to use 4 commands as follows: ./page_owner_sort <input> <output1> --tgid=1 ./page_owner_sort <input> <output2> --tgid=2 ./page_owner_sort <input> <output3> --tgid=3 cat <output1> <output2> <output3> > <output> With this patch, you can use only 1 command to obtain the same result as above: ./page_owner_sort <input> <output1> --tgid=1,2,3 2. Update explanations of --pid, --tgid and --name in the function usage() and the document(Documents/vm/page_owner.rst). This work is coauthored by Yixuan Cao Shenghong Han Yinan Zhang Chongxi Zhao Yuhong Feng Yongqiang Liu Link: https://lkml.kernel.org/r/20220401024856.767-2-yejiajian2018@email.szu.edu.c... Signed-off-by: Jiajian Ye <yejiajian2018@email.szu.edu.cn> Cc: Chongxi Zhao <zhaochongxi2019@email.szu.edu.cn> Cc: Shenghong Han <hanshenghong2019@email.szu.edu.cn> Cc: Yinan Zhang <zhangyinan2019@email.szu.edu.cn> Cc: Yixuan Cao <caoyixuan2019@email.szu.edu.cn> Cc: Yongqiang Liu <liuyongqiang13@huawei.com> Cc: Yuhong Feng <yuhongf@szu.edu.cn> Cc: Haowen Bai <baihaowen@meizu.com> Cc: Sean Anderson <seanga2@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com> --- Documentation/vm/page_owner.rst | 20 ++++++--- tools/vm/page_owner_sort.c | 78 ++++++++++++++++++++++++--------- 2 files changed, 72 insertions(+), 26 deletions(-) diff --git a/Documentation/vm/page_owner.rst b/Documentation/vm/page_owner.rst index c0dda23aa7d8..4712159e75b9 100644 --- a/Documentation/vm/page_owner.rst +++ b/Documentation/vm/page_owner.rst @@ -129,7 +129,6 @@ Usage Specify culling rules.Culling syntax is key[,key[,...]].Choose a multi-letter key from the **STANDARD FORMAT SPECIFIERS** section. - <rules> is a single argument in the form of a comma-separated list, which offers a way to specify individual culling rules. The recognized keywords are described in the **STANDARD FORMAT SPECIFIERS** section below. @@ -137,7 +136,6 @@ Usage the STANDARD SORT KEYS section below. Mixed use of abbreviated and complete-form of keys is allowed. - Examples: ./page_owner_sort <input> <output> --cull=stacktrace ./page_owner_sort <input> <output> --cull=st,pid,name @@ -147,9 +145,21 @@ Usage -f Filter out the information of blocks whose memory has not been released. Select: - --pid <PID> Select by pid. - --tgid <TGID> Select by tgid. - --name <command> Select by task command name. + --pid <pidlist> Select by pid. This selects the blocks whose process ID + numbers appear in <pidlist>. + --tgid <tgidlist> Select by tgid. This selects the blocks whose thread + group ID numbers appear in <tgidlist>. + --name <cmdlist> Select by task command name. This selects the blocks whose + task command name appear in <cmdlist>. + + <pidlist>, <tgidlist>, <cmdlist> are single arguments in the form of a comma-separated list, + which offers a way to specify individual selecting rules. + + + Examples: + ./page_owner_sort <input> <output> --pid=1 + ./page_owner_sort <input> <output> --tgid=1,2,3 + ./page_owner_sort <input> <output> --name name1,name2s STANDARD FORMAT SPECIFIERS ========================== diff --git a/tools/vm/page_owner_sort.c b/tools/vm/page_owner_sort.c index 6771003ed5f1..16fb034c6a4e 100644 --- a/tools/vm/page_owner_sort.c +++ b/tools/vm/page_owner_sort.c @@ -54,9 +54,12 @@ enum CULL_BIT { CULL_STACKTRACE = 1<<5 }; struct filter_condition { - pid_t tgid; - pid_t pid; - char comm[TASK_COMM_LEN]; + pid_t *tgids; + int tgids_size; + pid_t *pids; + int pids_size; + char **comms; + int comms_size; }; static struct filter_condition fc; static regex_t order_pattern; @@ -149,7 +152,6 @@ static int compare_free_ts(const void *p1, const void *p2) return l1->free_ts_nsec < l2->free_ts_nsec ? -1 : 1; } - static int compare_release(const void *p1, const void *p2) { const struct block_list *l1 = p1, *l2 = p2; @@ -161,7 +163,6 @@ static int compare_release(const void *p1, const void *p2) return l1->free_ts_nsec ? 1 : -1; } - static int compare_cull_condition(const void *p1, const void *p2) { if (cull == 0) @@ -344,22 +345,40 @@ static char *get_comm(char *buf) return comm_str; } +static bool match_num_list(int num, int *list, int list_size) +{ + for (int i = 0; i < list_size; ++i) + if (list[i] == num) + return true; + return false; +} + +static bool match_str_list(const char *str, char **list, int list_size) +{ + for (int i = 0; i < list_size; ++i) + if (!strcmp(list[i], str)) + return true; + return false; +} + static bool is_need(char *buf) { if ((filter & FILTER_UNRELEASE) && get_free_ts_nsec(buf) != 0) return false; - if ((filter & FILTER_PID) && get_pid(buf) != fc.pid) + if ((filter & FILTER_PID) && !match_num_list(get_pid(buf), fc.pids, fc.pids_size)) return false; - if ((filter & FILTER_TGID) && get_tgid(buf) != fc.tgid) + if ((filter & FILTER_TGID) && + !match_num_list(get_tgid(buf), fc.tgids, fc.tgids_size)) return false; char *comm = get_comm(buf); if ((filter & FILTER_COMM) && - strncmp(comm, fc.comm, TASK_COMM_LEN) != 0) { + !match_str_list(comm, fc.comms, fc.comms_size)) { free(comm); return false; } + free(comm); return true; } @@ -428,6 +447,27 @@ static bool parse_cull_args(const char *arg_str) return true; } +static int *parse_nums_list(char *arg_str, int *list_size) +{ + int size = 0; + char **args = explode(',', arg_str, &size); + int *list = calloc(size, sizeof(int)); + + errno = 0; + for (int i = 0; i < size; ++i) { + char *endptr = NULL; + + list[i] = strtol(args[i], &endptr, 10); + if (errno != 0 || endptr == args[i] || *endptr != '\0') { + free(list); + return NULL; + } + } + *list_size = size; + free_explode(args, size); + return list; +} + #define BUF_SIZE (128 * 1024) static void usage(void) @@ -442,9 +482,9 @@ static void usage(void) "-a\t\tSort by memory allocate time.\n" "-r\t\tSort by memory release time.\n" "-f\t\tFilter out the information of blocks whose memory has been released.\n" - "--pid <PID>\tSelect by pid. This selects the information of blocks whose process ID number equals to <PID>.\n" - "--tgid <TGID>\tSelect by tgid. This selects the information of blocks whose Thread Group ID number equals to <TGID>.\n" - "--name <command>\n\t\tSelect by command name. This selects the information of blocks whose command name identical to <command>.\n" + "--pid <pidlist>\tSelect by pid. This selects the information of blocks whose process ID numbers appear in <pidlist>.\n" + "--tgid <tgidlist>\tSelect by tgid. This selects the information of blocks whose Thread Group ID numbers appear in <tgidlist>.\n" + "--name <cmdlist>\n\t\tSelect by command name. This selects the information of blocks whose command name appears in <cmdlist>.\n" "--cull <rules>\tCull by user-defined rules. <rules> is a single argument in the form of a comma-separated list with some common fields predefined\n" ); } @@ -453,7 +493,7 @@ int main(int argc, char **argv) { int (*cmp)(const void *, const void *) = compare_num; FILE *fin, *fout; - char *buf, *endptr; + char *buf; int ret, i, count; struct stat st; int opt; @@ -496,9 +536,8 @@ int main(int argc, char **argv) break; case 1: filter = filter | FILTER_PID; - errno = 0; - fc.pid = strtol(optarg, &endptr, 10); - if (errno != 0 || endptr == optarg || *endptr != '\0') { + fc.pids = parse_nums_list(optarg, &fc.pids_size); + if (fc.pids == NULL) { fprintf(stderr, "wrong/invalid pid in from the command line:%s\n", optarg); exit(1); @@ -506,9 +545,8 @@ int main(int argc, char **argv) break; case 2: filter = filter | FILTER_TGID; - errno = 0; - fc.tgid = strtol(optarg, &endptr, 10); - if (errno != 0 || endptr == optarg || *endptr != '\0') { + fc.tgids = parse_nums_list(optarg, &fc.tgids_size); + if (fc.tgids == NULL) { fprintf(stderr, "wrong/invalid tgid in from the command line:%s\n", optarg); exit(1); @@ -516,8 +554,7 @@ int main(int argc, char **argv) break; case 3: filter = filter | FILTER_COMM; - strncpy(fc.comm, optarg, TASK_COMM_LEN); - fc.comm[TASK_COMM_LEN-1] = '\0'; + fc.comms = explode(',', optarg, &fc.comms_size); break; case 4: if (!parse_cull_args(optarg)) { @@ -564,7 +601,6 @@ int main(int argc, char **argv) ret = read_block(buf, BUF_SIZE, fin); if (ret < 0) break; - add_list(buf, ret); } -- 2.25.1

From: Jiajian Ye <yejiajian2018@email.szu.edu.cn> mainline inclusion from mainline-v5.19-rc1 commit ebbeae36387ccf1326c896167872c3acf6c3c956 category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I9GSSR Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- When viewing page owner information, we may want to sort blocks of information by multiple keys, since one single key does not uniquely identify a block. Therefore, following adjustments are made: 1. Add a new --sort option to support sorting blocks of information by multiple keys. ./page_owner_sort <input> <output> --sort=<order> ./page_owner_sort <input> <output> --sort <order> <order> is a single argument in the form of a comma-separated list, which offers a way to specify sorting order. Sorting syntax is [+|-]key[,[+|-]key[,...]]. The ascending or descending order can be specified by adding the + (ascending, default) or - (descend -ing) prefix to the key: ./page_owner_sort <input> <output> [option] --sort -key1,+key2,key3... For example, to sort the blocks first by task command name in lexicographic order and then by pid in ascending numerical order, use the following: ./page_owner_sort <input> <output> --sort=name,+pid To sort the blocks first by pid in ascending order and then by timestamp of the page when it is allocated in descending order, use the following: ./page_owner_sort <input> <output> --sort=pid,-alloc_ts 2. Add explanations of a newly added --sort option in the function usage() and the document(Documentation/vm/page_owner.rst). This work is coauthored by Yixuan Cao Shenghong Han Yinan Zhang Chongxi Zhao Yuhong Feng Yongqiang Liu Link: https://lkml.kernel.org/r/20220401024856.767-3-yejiajian2018@email.szu.edu.c... Signed-off-by: Jiajian Ye <yejiajian2018@email.szu.edu.cn> Cc: Chongxi Zhao <zhaochongxi2019@email.szu.edu.cn> Cc: Shenghong Han <hanshenghong2019@email.szu.edu.cn> Cc: Yinan Zhang <zhangyinan2019@email.szu.edu.cn> Cc: Yixuan Cao <caoyixuan2019@email.szu.edu.cn> Cc: Yongqiang Liu <liuyongqiang13@huawei.com> Cc: Yuhong Feng <yuhongf@szu.edu.cn> Cc: Haowen Bai <baihaowen@meizu.com> Cc: Sean Anderson <seanga2@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com> --- Documentation/vm/page_owner.rst | 23 ++++- tools/vm/page_owner_sort.c | 164 +++++++++++++++++++++++++++----- 2 files changed, 164 insertions(+), 23 deletions(-) diff --git a/Documentation/vm/page_owner.rst b/Documentation/vm/page_owner.rst index 9ab61f75819f..54df46b554ca 100644 --- a/Documentation/vm/page_owner.rst +++ b/Documentation/vm/page_owner.rst @@ -121,6 +121,14 @@ Usage -r Sort by memory release time. -s Sort by stack trace. -t Sort by times (default). + --sort <order> Specify sorting order. Sorting syntax is [+|-]key[,[+|-]key[,...]]. + Choose a key from the **STANDARD FORMAT SPECIFIERS** section. The "+" is + optional since default direction is increasing numerical or lexicographic + order. Mixed use of abbreviated and complete-form of keys is allowed. + + Examples: + ./page_owner_sort <input> <output> --sort=n,+pid,-tgid + ./page_owner_sort <input> <output> --sort=at additional function:: @@ -165,9 +173,22 @@ STANDARD FORMAT SPECIFIERS ========================== :: +For --sort option: + + KEY LONG DESCRIPTION + p pid process ID + tg tgid thread group ID + n name task command name + st stacktrace stack trace of the page allocation + T txt full text of block + ft free_ts timestamp of the page when it was released + at alloc_ts timestamp of the page when it was allocated + +For --curl option: + KEY LONG DESCRIPTION p pid process ID tg tgid thread group ID n name task command name f free whether the page has been released or not - st stacktrace stace trace of the page allocation \ No newline at end of file + st stacktrace stack trace of the page allocation diff --git a/tools/vm/page_owner_sort.c b/tools/vm/page_owner_sort.c index 16fb034c6a4e..beca990707fb 100644 --- a/tools/vm/page_owner_sort.c +++ b/tools/vm/page_owner_sort.c @@ -53,15 +53,29 @@ enum CULL_BIT { CULL_COMM = 1<<4, CULL_STACKTRACE = 1<<5 }; +enum ARG_TYPE { + ARG_TXT, ARG_COMM, ARG_STACKTRACE, ARG_ALLOC_TS, ARG_FREE_TS, + ARG_CULL_TIME, ARG_PAGE_NUM, ARG_PID, ARG_TGID, ARG_UNKNOWN, ARG_FREE +}; +enum SORT_ORDER { + SORT_ASC = 1, + SORT_DESC = -1, +}; struct filter_condition { - pid_t *tgids; - int tgids_size; pid_t *pids; - int pids_size; + pid_t *tgids; char **comms; + int pids_size; + int tgids_size; int comms_size; }; +struct sort_condition { + int (**cmps)(const void *, const void *); + int *signs; + int size; +}; static struct filter_condition fc; +static struct sort_condition sc; static regex_t order_pattern; static regex_t pid_pattern; static regex_t tgid_pattern; @@ -107,14 +121,14 @@ static int compare_num(const void *p1, const void *p2) { const struct block_list *l1 = p1, *l2 = p2; - return l2->num - l1->num; + return l1->num - l2->num; } static int compare_page_num(const void *p1, const void *p2) { const struct block_list *l1 = p1, *l2 = p2; - return l2->page_num - l1->page_num; + return l1->page_num - l2->page_num; } static int compare_pid(const void *p1, const void *p2) @@ -180,6 +194,16 @@ static int compare_cull_condition(const void *p1, const void *p2) return 0; } +static int compare_sort_condition(const void *p1, const void *p2) +{ + int cmp = 0; + + for (int i = 0; i < sc.size; ++i) + if (cmp == 0) + cmp = sc.signs[i] * sc.cmps[i](p1, p2); + return cmp; +} + static int search_pattern(regex_t *pattern, char *pattern_str, char *buf) { int err, val_len; @@ -345,6 +369,29 @@ static char *get_comm(char *buf) return comm_str; } +static int get_arg_type(const char *arg) +{ + if (!strcmp(arg, "pid") || !strcmp(arg, "p")) + return ARG_PID; + else if (!strcmp(arg, "tgid") || !strcmp(arg, "tg")) + return ARG_TGID; + else if (!strcmp(arg, "name") || !strcmp(arg, "n")) + return ARG_COMM; + else if (!strcmp(arg, "stacktrace") || !strcmp(arg, "st")) + return ARG_STACKTRACE; + else if (!strcmp(arg, "free") || !strcmp(arg, "f")) + return ARG_FREE; + else if (!strcmp(arg, "txt") || !strcmp(arg, "T")) + return ARG_TXT; + else if (!strcmp(arg, "free_ts") || !strcmp(arg, "ft")) + return ARG_FREE_TS; + else if (!strcmp(arg, "alloc_ts") || !strcmp(arg, "at")) + return ARG_ALLOC_TS; + else { + return ARG_UNKNOWN; + } +} + static bool match_num_list(int num, int *list, int list_size) { for (int i = 0; i < list_size; ++i) @@ -428,21 +475,86 @@ static bool parse_cull_args(const char *arg_str) int size = 0; char **args = explode(',', arg_str, &size); - for (int i = 0; i < size; ++i) - if (!strcmp(args[i], "pid") || !strcmp(args[i], "p")) + for (int i = 0; i < size; ++i) { + int arg_type = get_arg_type(args[i]); + + if (arg_type == ARG_PID) cull |= CULL_PID; - else if (!strcmp(args[i], "tgid") || !strcmp(args[i], "tg")) + else if (arg_type == ARG_TGID) cull |= CULL_TGID; - else if (!strcmp(args[i], "name") || !strcmp(args[i], "n")) + else if (arg_type == ARG_COMM) cull |= CULL_COMM; - else if (!strcmp(args[i], "stacktrace") || !strcmp(args[i], "st")) + else if (arg_type == ARG_STACKTRACE) cull |= CULL_STACKTRACE; - else if (!strcmp(args[i], "free") || !strcmp(args[i], "f")) + else if (arg_type == ARG_FREE) cull |= CULL_UNRELEASE; else { free_explode(args, size); return false; } + } + free_explode(args, size); + return true; +} + +static void set_single_cmp(int (*cmp)(const void *, const void *), int sign) +{ + if (sc.signs == NULL || sc.size < 1) + sc.signs = calloc(1, sizeof(int)); + sc.signs[0] = sign; + if (sc.cmps == NULL || sc.size < 1) + sc.cmps = calloc(1, sizeof(int *)); + sc.cmps[0] = cmp; + sc.size = 1; +} + +static bool parse_sort_args(const char *arg_str) +{ + int size = 0; + + if (sc.size != 0) { /* reset sort_condition */ + free(sc.signs); + free(sc.cmps); + size = 0; + } + + char **args = explode(',', arg_str, &size); + + sc.signs = calloc(size, sizeof(int)); + sc.cmps = calloc(size, sizeof(int *)); + for (int i = 0; i < size; ++i) { + int offset = 0; + + sc.signs[i] = SORT_ASC; + if (args[i][0] == '-' || args[i][0] == '+') { + if (args[i][0] == '-') + sc.signs[i] = SORT_DESC; + offset = 1; + } + + int arg_type = get_arg_type(args[i]+offset); + + if (arg_type == ARG_PID) + sc.cmps[i] = compare_pid; + else if (arg_type == ARG_TGID) + sc.cmps[i] = compare_tgid; + else if (arg_type == ARG_COMM) + sc.cmps[i] = compare_comm; + else if (arg_type == ARG_STACKTRACE) + sc.cmps[i] = compare_stacktrace; + else if (arg_type == ARG_ALLOC_TS) + sc.cmps[i] = compare_ts; + else if (arg_type == ARG_FREE_TS) + sc.cmps[i] = compare_free_ts; + else if (arg_type == ARG_TXT) + sc.cmps[i] = compare_txt; + else { + free_explode(args, size); + sc.size = 0; + return false; + } + } + sc.size = size; free_explode(args, size); return true; } @@ -485,13 +597,13 @@ static void usage(void) "--pid <pidlist>\tSelect by pid. This selects the information of blocks whose process ID numbers appear in <pidlist>.\n" "--tgid <tgidlist>\tSelect by tgid. This selects the information of blocks whose Thread Group ID numbers appear in <tgidlist>.\n" "--name <cmdlist>\n\t\tSelect by command name. This selects the information of blocks whose command name appears in <cmdlist>.\n" - "--cull <rules>\tCull by user-defined rules. <rules> is a single argument in the form of a comma-separated list with some common fields predefined\n" + "--cull <rules>\tCull by user-defined rules.<rules> is a single argument in the form of a comma-separated list with some common fields predefined\n" + "--sort <order>\tSpecify sort order as: [+|-]key[,[+|-]key[,...]]\n" ); } int main(int argc, char **argv) { - int (*cmp)(const void *, const void *) = compare_num; FILE *fin, *fout; char *buf; int ret, i, count; @@ -502,37 +614,38 @@ int main(int argc, char **argv) { "tgid", required_argument, NULL, 2 }, { "name", required_argument, NULL, 3 }, { "cull", required_argument, NULL, 4 }, + { "sort", required_argument, NULL, 5 }, { 0, 0, 0, 0}, }; while ((opt = getopt_long(argc, argv, "afmnprstP", longopts, NULL)) != -1) switch (opt) { case 'a': - cmp = compare_ts; + set_single_cmp(compare_ts, SORT_ASC); break; case 'f': filter = filter | FILTER_UNRELEASE; break; case 'm': - cmp = compare_page_num; + set_single_cmp(compare_page_num, SORT_DESC); break; case 'p': - cmp = compare_pid; + set_single_cmp(compare_pid, SORT_ASC); break; case 'r': - cmp = compare_free_ts; + set_single_cmp(compare_free_ts, SORT_ASC); break; case 's': - cmp = compare_stacktrace; + set_single_cmp(compare_stacktrace, SORT_ASC); break; case 't': - cmp = compare_num; + set_single_cmp(compare_num, SORT_DESC); break; case 'P': - cmp = compare_tgid; + set_single_cmp(compare_tgid, SORT_ASC); break; case 'n': - cmp = compare_comm; + set_single_cmp(compare_comm, SORT_ASC); break; case 1: filter = filter | FILTER_PID; @@ -563,6 +676,13 @@ int main(int argc, char **argv) exit(1); } break; + case 5: + if (!parse_sort_args(optarg)) { + fprintf(stderr, "wrong argument after --sort option:%s\n", + optarg); + exit(1); + } + break; default: usage(); exit(1); @@ -622,7 +742,7 @@ int main(int argc, char **argv) } } - qsort(list, count, sizeof(list[0]), cmp); + qsort(list, count, sizeof(list[0]), compare_sort_condition); for (i = 0; i < count; i++) { if (cull == 0) -- 2.25.1

From: Yixuan Cao <caoyixuan2019@email.szu.edu.cn> mainline inclusion from mainline-v5.19-rc1 commit f09654bb88127473b4baf3bc0b68d4d4695aca7b category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I9GSSR Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- An application is suspected of having memory leak when its memory consumption is high and keeps increasing. There are several commonly used memory allocators: slab, cma, vmalloc, etc. The memory leak identification can be sped up if the page information allocated by an allocator can be analyzed separately. This patch provides supports for memory allocator labelling for slab, vmalloc, and cma. The pages allocated by slab and cma can be confirmed from the "PFN" line according to the kernel codes, and the label of the vmalloc allocator can be obtained by analyzing the stack trace. Thanks for Vlastimil Babka's constructive suggestions. Based on Yinan Zhang's study, the call chain of vmalloc() is vmalloc() -> ... -> __vmalloc_node_range() -> __vmalloc_area_node(). __vmalloc_area_node() requests memory through the interface of buddy allocation system. In the current version, __vmalloc_area_node() uses four interfaces: alloc_pages_bulk_array_mempolicy(), alloc_pages_bulk_array_node(), alloc_pages() and alloc_pages_node(). By disassembling the code, we find that __vmalloc_area_node() is expanded in __vmalloc_node_range(). So __vmalloc_area_node is not in the stack trace. On the test machine, the stack trace of pages allocated by vmalloc has the following four forms: __alloc_pages_bulk+0x230/0x6a0 __vmalloc_node_range+0x19c/0x598 alloc_pages_bulk_array_mempolicy+0xbc/0x278 __vmalloc_node_range+0x1e8/0x598 __alloc_pages+0x160/0x2b0 __vmalloc_node_range+0x234/0x598 alloc_pages+0xac/0x150 __vmalloc_node_range+0x44c/0x598 Therefore, in two consecutive lines of stacktrace, if the first line contains the word "alloc_pages" and the second line contains the word "__vmalloc_node_range", it can be determined that the page is allocated by vmalloc. And the function offset and size are not the same on different machines, so there is no need to match them. At the same time, this patch updates the --cull and --sort options to support allocator-based merge statistics and sorting. The added functions are fully compatible with the original work. When using, you can use "allocator", or abbreviated as "ator". Relevant updates have also been made in the documentation(Documentation/vm/page_owner.rst). Example: ./page_owner_sort <input> <output> --cull=st,pid,name,allocator ./page_owner_sort <input> <output> --sort=ator,pid,name This work is coauthored by Jiajian Ye, Yinan Zhang, Shenghong Han, Chongxi Zhao, Yuhong Feng and Yongqiang Liu. Link: https://lkml.kernel.org/r/20220410132932.9402-1-caoyixuan2019@email.szu.edu.... Signed-off-by: Yixuan Cao <caoyixuan2019@email.szu.edu.cn> Cc: Chongxi Zhao <zhaochongxi2019@email.szu.edu.cn> Cc: Haowen Bai <baihaowen@meizu.com> Cc: Jiajian Ye <yejiajian2018@email.szu.edu.cn> Cc: Sean Anderson <seanga2@gmail.com> Cc: Shenghong Han <hanshenghong2019@email.szu.edu.cn> Cc: Yinan Zhang <zhangyinan2019@email.szu.edu.cn> Cc: Yongqiang Liu <liuyongqiang13@huawei.com> Cc: Yuhong Feng <yuhongf@szu.edu.cn> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com> --- Documentation/vm/page_owner.rst | 2 + tools/vm/page_owner_sort.c | 112 +++++++++++++++++++++++++++----- 2 files changed, 99 insertions(+), 15 deletions(-) diff --git a/Documentation/vm/page_owner.rst b/Documentation/vm/page_owner.rst index 54df46b554ca..3a7f8dad9c1e 100644 --- a/Documentation/vm/page_owner.rst +++ b/Documentation/vm/page_owner.rst @@ -183,6 +183,7 @@ For --sort option: T txt full text of block ft free_ts timestamp of the page when it was released at alloc_ts timestamp of the page when it was allocated + ator allocator memory allocator for pages For --curl option: @@ -192,3 +193,4 @@ For --curl option: n name task command name f free whether the page has been released or not st stacktrace stack trace of the page allocation + ator allocator memory allocator for pages diff --git a/tools/vm/page_owner_sort.c b/tools/vm/page_owner_sort.c index a32e446e5bb2..fa2e4d2a9d68 100644 --- a/tools/vm/page_owner_sort.c +++ b/tools/vm/page_owner_sort.c @@ -39,6 +39,7 @@ struct block_list { int page_num; pid_t pid; pid_t tgid; + int allocator; }; enum FILTER_BIT { FILTER_UNRELEASE = 1<<1, @@ -51,11 +52,19 @@ enum CULL_BIT { CULL_PID = 1<<2, CULL_TGID = 1<<3, CULL_COMM = 1<<4, - CULL_STACKTRACE = 1<<5 + CULL_STACKTRACE = 1<<5, + CULL_ALLOCATOR = 1<<6 +}; +enum ALLOCATOR_BIT { + ALLOCATOR_CMA = 1<<1, + ALLOCATOR_SLAB = 1<<2, + ALLOCATOR_VMALLOC = 1<<3, + ALLOCATOR_OTHERS = 1<<4 }; enum ARG_TYPE { ARG_TXT, ARG_COMM, ARG_STACKTRACE, ARG_ALLOC_TS, ARG_FREE_TS, - ARG_CULL_TIME, ARG_PAGE_NUM, ARG_PID, ARG_TGID, ARG_UNKNOWN, ARG_FREE + ARG_CULL_TIME, ARG_PAGE_NUM, ARG_PID, ARG_TGID, ARG_UNKNOWN, ARG_FREE, + ARG_ALLOCATOR }; enum SORT_ORDER { SORT_ASC = 1, @@ -89,15 +98,20 @@ static int cull; static int filter; static bool debug_on; -int read_block(char *buf, int buf_size, FILE *fin) +static void set_single_cmp(int (*cmp)(const void *, const void *), int sign); + +int read_block(char *buf, char *ext_buf, int buf_size, FILE *fin) { char *curr = buf, *const buf_end = buf + buf_size; while (buf_end - curr > 1 && fgets(curr, buf_end - curr, fin)) { - if (*curr == '\n') /* empty line */ + if (*curr == '\n') { /* empty line */ return curr - buf; - if (!strncmp(curr, "PFN", 3)) + } + if (!strncmp(curr, "PFN", 3)) { + strcpy(ext_buf, curr); continue; + } curr += strlen(curr); } @@ -146,6 +160,13 @@ static int compare_tgid(const void *p1, const void *p2) return l1->tgid - l2->tgid; } +static int compare_allocator(const void *p1, const void *p2) +{ + const struct block_list *l1 = p1, *l2 = p2; + + return l1->allocator - l2->allocator; +} + static int compare_comm(const void *p1, const void *p2) { const struct block_list *l1 = p1, *l2 = p2; @@ -192,6 +213,8 @@ static int compare_cull_condition(const void *p1, const void *p2) return compare_comm(p1, p2); if ((cull & CULL_UNRELEASE) && compare_release(p1, p2)) return compare_release(p1, p2); + if ((cull & CULL_ALLOCATOR) && compare_allocator(p1, p2)) + return compare_allocator(p1, p2); return 0; } @@ -395,11 +418,42 @@ static int get_arg_type(const char *arg) return ARG_FREE_TS; else if (!strcmp(arg, "alloc_ts") || !strcmp(arg, "at")) return ARG_ALLOC_TS; + else if (!strcmp(arg, "allocator") || !strcmp(arg, "ator")) + return ARG_ALLOCATOR; else { return ARG_UNKNOWN; } } +static int get_allocator(const char *buf, const char *migrate_info) +{ + char *tmp, *first_line, *second_line; + int allocator = 0; + + if (strstr(migrate_info, "CMA")) + allocator |= ALLOCATOR_CMA; + if (strstr(migrate_info, "slab")) + allocator |= ALLOCATOR_SLAB; + tmp = strstr(buf, "__vmalloc_node_range"); + if (tmp) { + second_line = tmp; + while (*tmp != '\n') + tmp--; + tmp--; + while (*tmp != '\n') + tmp--; + first_line = ++tmp; + tmp = strstr(tmp, "alloc_pages"); + if (tmp) { + if (tmp && first_line <= tmp && tmp < second_line) + allocator |= ALLOCATOR_VMALLOC; + } + } + if (allocator == 0) + allocator = ALLOCATOR_OTHERS; + return allocator; +} + static bool match_num_list(int num, int *list, int list_size) { for (int i = 0; i < list_size; ++i) @@ -437,7 +491,7 @@ static bool is_need(char *buf) return true; } -static void add_list(char *buf, int len) +static void add_list(char *buf, int len, char *ext_buf) { if (list_size != 0 && len == list[list_size-1].len && @@ -471,6 +525,7 @@ static void add_list(char *buf, int len) list[list_size].stacktrace++; list[list_size].ts_nsec = get_ts_nsec(buf); list[list_size].free_ts_nsec = get_free_ts_nsec(buf); + list[list_size].allocator = get_allocator(buf, ext_buf); list_size++; if (list_size % 1000 == 0) { printf("loaded %d\r", list_size); @@ -496,12 +551,16 @@ static bool parse_cull_args(const char *arg_str) cull |= CULL_STACKTRACE; else if (arg_type == ARG_FREE) cull |= CULL_UNRELEASE; + else if (arg_type == ARG_ALLOCATOR) + cull |= CULL_ALLOCATOR; else { free_explode(args, size); return false; } } free_explode(args, size); + if (sc.size == 0) + set_single_cmp(compare_num, SORT_DESC); return true; } @@ -556,6 +615,8 @@ static bool parse_sort_args(const char *arg_str) sc.cmps[i] = compare_free_ts; else if (arg_type == ARG_TXT) sc.cmps[i] = compare_txt; + else if (arg_type == ARG_ALLOCATOR) + sc.cmps[i] = compare_allocator; else { free_explode(args, size); sc.size = 0; @@ -588,6 +649,19 @@ static int *parse_nums_list(char *arg_str, int *list_size) return list; } +static void print_allocator(FILE *out, int allocator) +{ + fprintf(out, "allocated by "); + if (allocator & ALLOCATOR_CMA) + fprintf(out, "CMA "); + if (allocator & ALLOCATOR_SLAB) + fprintf(out, "SLAB "); + if (allocator & ALLOCATOR_VMALLOC) + fprintf(out, "VMALLOC "); + if (allocator & ALLOCATOR_OTHERS) + fprintf(out, "OTHERS "); +} + #define BUF_SIZE (128 * 1024) static void usage(void) @@ -614,8 +688,8 @@ static void usage(void) int main(int argc, char **argv) { FILE *fin, *fout; - char *buf; - int ret, i, count; + char *buf, *ext_buf; + int i, count; struct stat st; int opt; struct option longopts[] = { @@ -724,16 +798,18 @@ int main(int argc, char **argv) list = malloc(max_size * sizeof(*list)); buf = malloc(BUF_SIZE); - if (!list || !buf) { + ext_buf = malloc(BUF_SIZE); + if (!list || !buf || !ext_buf) { fprintf(stderr, "Out of memory\n"); exit(1); } for ( ; ; ) { - ret = read_block(buf, BUF_SIZE, fin); - if (ret < 0) + int buf_len = read_block(buf, ext_buf, BUF_SIZE, fin); + + if (buf_len < 0) break; - add_list(buf, ret); + add_list(buf, buf_len, ext_buf); } printf("loaded %d\n", list_size); @@ -757,9 +833,11 @@ int main(int argc, char **argv) qsort(list, count, sizeof(list[0]), compare_sort_condition); for (i = 0; i < count; i++) { - if (cull == 0) - fprintf(fout, "%d times, %d pages:\n%s\n", - list[i].num, list[i].page_num, list[i].txt); + if (cull == 0) { + fprintf(fout, "%d times, %d pages, ", list[i].num, list[i].page_num); + print_allocator(fout, list[i].allocator); + fprintf(fout, ":\n%s\n", list[i].txt); + } else { fprintf(fout, "%d times, %d pages", list[i].num, list[i].page_num); @@ -769,6 +847,10 @@ int main(int argc, char **argv) fprintf(fout, ", TGID %d", list[i].pid); if (cull & CULL_COMM || filter & FILTER_COMM) fprintf(fout, ", task_comm_name: %s", list[i].comm); + if (cull & CULL_ALLOCATOR) { + fprintf(fout, ", "); + print_allocator(fout, list[i].allocator); + } if (cull & CULL_UNRELEASE) fprintf(fout, " (%s)", list[i].free_ts_nsec ? "UNRELEASED" : "RELEASED"); -- 2.25.1

From: Yixuan Cao <caoyixuan2019@email.szu.edu.cn> mainline inclusion from mainline-v6.0-rc1 commit 9b7a4039d6856f66521486da68c76838929039eb category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I9GSSR Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- I noticed one more indentation than necessary in is_need(). Link: https://lkml.kernel.org/r/20220717195506.7602-1-caoyixuan2019@email.szu.edu.... Signed-off-by: Yixuan Cao <caoyixuan2019@email.szu.edu.cn> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com> --- tools/vm/page_owner_sort.c | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/tools/vm/page_owner_sort.c b/tools/vm/page_owner_sort.c index c149427eb1c9..69ef83ef7100 100644 --- a/tools/vm/page_owner_sort.c +++ b/tools/vm/page_owner_sort.c @@ -470,23 +470,23 @@ static bool match_str_list(const char *str, char **list, int list_size) static bool is_need(char *buf) { - if ((filter & FILTER_UNRELEASE) && get_free_ts_nsec(buf) != 0) - return false; - if ((filter & FILTER_PID) && !match_num_list(get_pid(buf), fc.pids, fc.pids_size)) - return false; - if ((filter & FILTER_TGID) && - !match_num_list(get_tgid(buf), fc.tgids, fc.tgids_size)) - return false; - - char *comm = get_comm(buf); - - if ((filter & FILTER_COMM) && - !match_str_list(comm, fc.comms, fc.comms_size)) { - free(comm); - return false; - } + if ((filter & FILTER_UNRELEASE) && get_free_ts_nsec(buf) != 0) + return false; + if ((filter & FILTER_PID) && !match_num_list(get_pid(buf), fc.pids, fc.pids_size)) + return false; + if ((filter & FILTER_TGID) && + !match_num_list(get_tgid(buf), fc.tgids, fc.tgids_size)) + return false; + + char *comm = get_comm(buf); + + if ((filter & FILTER_COMM) && + !match_str_list(comm, fc.comms, fc.comms_size)) { free(comm); - return true; + return false; + } + free(comm); + return true; } static void add_list(char *buf, int len, char *ext_buf) -- 2.25.1

From: Jianlin Lv <iecedge@gmail.com> mainline inclusion from mainline-v6.3-rc1 commit ef1faf0e370a8e33fe625088ddc5fde02cf8c4c4 category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I9GSSR Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- Although when a process terminates, the kernel will removes memory associated with that process, It's neither good style nor proper design to leave it to kernel. This patch free allocated memory before process exit. Link: https://lkml.kernel.org/r/20221219164917.14132-1-iecedge@gmail.com Signed-off-by: Jianlin Lv <iecedge@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com> --- tools/vm/page_owner_sort.c | 65 ++++++++++++++++++++++++++------------ 1 file changed, 45 insertions(+), 20 deletions(-) diff --git a/tools/vm/page_owner_sort.c b/tools/vm/page_owner_sort.c index 4252fd1d347d..00b5856f0b2b 100644 --- a/tools/vm/page_owner_sort.c +++ b/tools/vm/page_owner_sort.c @@ -246,15 +246,16 @@ static int search_pattern(regex_t *pattern, char *pattern_str, char *buf) return 0; } -static void check_regcomp(regex_t *pattern, const char *regex) +static bool check_regcomp(regex_t *pattern, const char *regex) { int err; err = regcomp(pattern, regex, REG_EXTENDED | REG_NEWLINE); if (err != 0 || pattern->re_nsub != 1) { fprintf(stderr, "Invalid pattern %s code %d\n", regex, err); - exit(1); + return false; } + return true; } static char **explode(char sep, const char *str, int *size) @@ -494,28 +495,28 @@ static bool is_need(char *buf) return true; } -static void add_list(char *buf, int len, char *ext_buf) +static bool add_list(char *buf, int len, char *ext_buf) { if (list_size != 0 && len == list[list_size-1].len && memcmp(buf, list[list_size-1].txt, len) == 0) { list[list_size-1].num++; list[list_size-1].page_num += get_page_num(buf); - return; + return true; } if (list_size == max_size) { fprintf(stderr, "max_size too small??\n"); - exit(1); + return false; } if (!is_need(buf)) - return; + return true; list[list_size].pid = get_pid(buf); list[list_size].tgid = get_tgid(buf); list[list_size].comm = get_comm(buf); list[list_size].txt = malloc(len+1); if (!list[list_size].txt) { fprintf(stderr, "Out of memory\n"); - exit(1); + return false; } memcpy(list[list_size].txt, buf, len); list[list_size].txt[len] = 0; @@ -534,6 +535,7 @@ static void add_list(char *buf, int len, char *ext_buf) printf("loaded %d\r", list_size); fflush(stdout); } + return true; } static bool parse_cull_args(const char *arg_str) @@ -790,12 +792,19 @@ int main(int argc, char **argv) exit(1); } - check_regcomp(&order_pattern, "order\\s*([0-9]*),"); - check_regcomp(&pid_pattern, "pid\\s*([0-9]*),"); - check_regcomp(&tgid_pattern, "tgid\\s*([0-9]*) "); - check_regcomp(&comm_pattern, "tgid\\s*[0-9]*\\s*\\((.*)\\),\\s*ts"); - check_regcomp(&ts_nsec_pattern, "ts\\s*([0-9]*)\\s*ns,"); - check_regcomp(&free_ts_nsec_pattern, "free_ts\\s*([0-9]*)\\s*ns"); + if (!check_regcomp(&order_pattern, "order\\s*([0-9]*),")) + goto out_order; + if (!check_regcomp(&pid_pattern, "pid\\s*([0-9]*),")) + goto out_pid; + if (!check_regcomp(&tgid_pattern, "tgid\\s*([0-9]*) ")) + goto out_tgid; + if (!check_regcomp(&comm_pattern, "tgid\\s*[0-9]*\\s*\\((.*)\\),\\s*ts")) + goto out_comm; + if (!check_regcomp(&ts_nsec_pattern, "ts\\s*([0-9]*)\\s*ns,")) + goto out_ts; + if (!check_regcomp(&free_ts_nsec_pattern, "free_ts\\s*([0-9]*)\\s*ns")) + goto out_free_ts; + fstat(fileno(fin), &st); max_size = st.st_size / 100; /* hack ... */ @@ -804,7 +813,7 @@ int main(int argc, char **argv) ext_buf = malloc(BUF_SIZE); if (!list || !buf || !ext_buf) { fprintf(stderr, "Out of memory\n"); - exit(1); + goto out_free; } for ( ; ; ) { @@ -812,7 +821,8 @@ int main(int argc, char **argv) if (buf_len < 0) break; - add_list(buf, buf_len, ext_buf); + if (!add_list(buf, buf_len, ext_buf)) + goto out_free; } printf("loaded %d\n", list_size); @@ -862,11 +872,26 @@ int main(int argc, char **argv) fprintf(fout, "\n"); } } - regfree(&order_pattern); - regfree(&pid_pattern); - regfree(&tgid_pattern); - regfree(&comm_pattern); - regfree(&ts_nsec_pattern); + +out_free: + if (ext_buf) + free(ext_buf); + if (buf) + free(buf); + if (list) + free(list); +out_free_ts: regfree(&free_ts_nsec_pattern); +out_ts: + regfree(&ts_nsec_pattern); +out_comm: + regfree(&comm_pattern); +out_tgid: + regfree(&tgid_pattern); +out_pid: + regfree(&pid_pattern); +out_order: + regfree(&order_pattern); + return 0; } -- 2.25.1

From: Akira Yokosawa <akiyks@gmail.com> mainline inclusion from mainline-v5.19-rc1 commit d1ed51fcdbd69be3729f6e249b61cc73fb3b2dd8 category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I9GSSR Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- A semantic conflict between commit 5603f9bdea68 ("docs: vm/page_owner: use literal blocks for param description") and a change queued for v5.19 authored by Jiajian Ye ("tools/vm/page_owner_sort.c: support sorting blocks by multiple keys") results in a warning from "make htmldocs" saying: [...]/vm/page_owner.rst:176: WARNING: Literal block expected; none found. This is because a literal block in ReST ends at a line which has the same indent as the paragraph preceding it. In this case the one with no indent. Indent the two "For --xxxx option:" lines by two columns and make the whole section a literal block. While at it, fix indents by white spaces of "ator" keys. Link: https://lkml.kernel.org/r/fdfecc82-d41e-6d8a-738d-4beb6faa27fb@gmail.com Signed-of-by: Akira Yokosawa <akiyks@gmail.com> Reported-by: Shenghong Han <hanshenghong2019@email.szu.edu.cn> Cc: Jiajian Ye <yejiajian2018@email.szu.edu.cn> Cc: Chongxi Zhao <zhaochongxi2019@email.szu.edu.cn> Cc: Yinan Zhang <zhangyinan2019@email.szu.edu.cn> Cc: Yixuan Cao <caoyixuan2019@email.szu.edu.cn> Cc: Yongqiang Liu <liuyongqiang13@huawei.com> Cc: Yuhong Feng <yuhongf@szu.edu.cn> Cc: Haowen Bai <baihaowen@meizu.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Conflicts: Documentation/vm/page_owner.rst [context conflicts] Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com> --- Documentation/vm/page_owner.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/Documentation/vm/page_owner.rst b/Documentation/vm/page_owner.rst index 3a7f8dad9c1e..380e9162cd5a 100644 --- a/Documentation/vm/page_owner.rst +++ b/Documentation/vm/page_owner.rst @@ -173,7 +173,7 @@ STANDARD FORMAT SPECIFIERS ========================== :: -For --sort option: + For --sort option: KEY LONG DESCRIPTION p pid process ID @@ -183,9 +183,9 @@ For --sort option: T txt full text of block ft free_ts timestamp of the page when it was released at alloc_ts timestamp of the page when it was allocated - ator allocator memory allocator for pages + ator allocator memory allocator for pages -For --curl option: + For --curl option: KEY LONG DESCRIPTION p pid process ID @@ -193,4 +193,4 @@ For --curl option: n name task command name f free whether the page has been released or not st stacktrace stack trace of the page allocation - ator allocator memory allocator for pages + ator allocator memory allocator for pages -- 2.25.1

From: Jonathan Corbet <corbet@lwn.net> mainline inclusion from mainline-v5.18-rc1 commit 18ab307823bb643fc985d316448f2d70eb1cb7c3 category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I9GSSR Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- Commit f7df2b1cf03a ("tools/vm/page_owner_sort.c: count and sort by mem") added a literal text block without the necessary markup, leading to these warnings in the docs build: Documentation/vm/page_owner.rst:92: WARNING: Unexpected indentation. Documentation/vm/page_owner.rst:96: WARNING: Unexpected indentation. Documentation/vm/page_owner.rst:107: WARNING: Unexpected indentation. Add the necessary colons and make the build quieter. Signed-off-by: Jonathan Corbet <corbet@lwn.net> Conflicts: Documentation/vm/page_owner.rst [context conflicts] Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com> --- Documentation/vm/page_owner.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Documentation/vm/page_owner.rst b/Documentation/vm/page_owner.rst index 5b9043fd9452..b5d71ebb2727 100644 --- a/Documentation/vm/page_owner.rst +++ b/Documentation/vm/page_owner.rst @@ -85,7 +85,7 @@ Usage cat /sys/kernel/debug/page_owner > page_owner_full.txt ./page_owner_sort page_owner_full.txt sorted_page_owner.txt - The general output of ``page_owner_full.txt`` is as follows: + The general output of ``page_owner_full.txt`` is as follows:: Page allocated via order XXX, ... PFN XXX ... @@ -100,7 +100,7 @@ Usage and pages of buf, and finally sorts them according to the parameter(s). See the result about who allocated each page - in the ``sorted_page_owner.txt``. General output: + in the ``sorted_page_owner.txt``. General output:: XXX times, XXX pages: Page allocated via order XXX, ... -- 2.25.1

Offering: HULK hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I9GSSR ---------------------------------------- Add a new debufgs interface page_owner_filter, to filter pages not allocated by modules. This will save some space and time when uses only care pages allocated by modules. Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com> --- mm/page_owner.c | 4 +++ mm/page_owner.h | 11 +++++++ mm/page_owner_module.c | 68 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 83 insertions(+) diff --git a/mm/page_owner.c b/mm/page_owner.c index 3e3cf55d270e..c060ec8585da 100644 --- a/mm/page_owner.c +++ b/mm/page_owner.c @@ -604,6 +604,9 @@ read_page_owner(struct file *file, char __user *buf, size_t count, loff_t *ppos) if (!handle) goto ext_put_continue; + if (po_is_filtered(page_owner)) + goto ext_put_continue; + /* Record the next PFN to read in the file offset */ *ppos = (pfn - min_low_pfn) + 1; @@ -727,6 +730,7 @@ static int __init pageowner_init(void) debugfs_create_file("page_owner", 0400, NULL, NULL, &proc_page_owner_operations); + po_module_stat_init(); return 0; } late_initcall(pageowner_init) diff --git a/mm/page_owner.h b/mm/page_owner.h index 18ea05e999e7..c516de29ba6a 100644 --- a/mm/page_owner.h +++ b/mm/page_owner.h @@ -28,6 +28,8 @@ struct page_owner { void po_find_module_name_with_update(depot_stack_handle_t handle, char *mod_name, size_t size); void po_set_module_name(struct page_owner *page_owner, char *mod_name); int po_module_name_snprint(struct page_owner *page_owner, char *kbuf, size_t size); +void po_module_stat_init(void); +bool po_is_filtered(struct page_owner *page_owner); static inline void po_copy_module_name(struct page_owner *dst, struct page_owner *src) @@ -54,6 +56,15 @@ static inline int po_module_name_snprint(struct page_owner *page_owner, static inline void po_copy_module_name(struct page_owner *dst, struct page_owner *src) { } + +static inline void po_module_stat_init(void) +{ +} + +static inline bool po_is_filtered(struct page_owner *page_owner) +{ + return false; +} #endif #endif diff --git a/mm/page_owner_module.c b/mm/page_owner_module.c index 2a2becf975da..cc9150d5b6ff 100644 --- a/mm/page_owner_module.c +++ b/mm/page_owner_module.c @@ -11,6 +11,12 @@ #include "page_owner.h" +#define PAGE_OWNER_FILTER_BUF_SIZE 16 +#define PAGE_OWNER_NONE_FILTER 0 +#define PAGE_OWNER_MODULE_FILTER 1 + +static unsigned int page_owner_filter = PAGE_OWNER_NONE_FILTER; + void po_find_module_name_with_update(depot_stack_handle_t handle, char *mod_name, size_t size) { int i; @@ -68,3 +74,65 @@ int po_module_name_snprint(struct page_owner *page_owner, return 0; } + +static ssize_t read_page_owner_filter(struct file *file, + char __user *user_buf, size_t count, loff_t *ppos) +{ + char kbuf[PAGE_OWNER_FILTER_BUF_SIZE]; + int kcount; + + if (page_owner_filter & PAGE_OWNER_MODULE_FILTER) + kcount = snprintf(kbuf, sizeof(kbuf), "module\n"); + else + kcount = snprintf(kbuf, sizeof(kbuf), "none\n"); + + return simple_read_from_buffer(user_buf, count, ppos, kbuf, kcount); +} + +static ssize_t write_page_owner_filter(struct file *file, + const char __user *user_buf, size_t count, loff_t *ppos) +{ + char kbuf[PAGE_OWNER_FILTER_BUF_SIZE]; + char *p_kbuf; + size_t kbuf_size; + + kbuf_size = min(count, sizeof(kbuf) - 1); + if (copy_from_user(kbuf, user_buf, kbuf_size)) + return -EFAULT; + + kbuf[kbuf_size] = '\0'; + p_kbuf = strstrip(kbuf); + + if (!strcmp(p_kbuf, "module")) + page_owner_filter = PAGE_OWNER_MODULE_FILTER; + else if (!strcmp(p_kbuf, "none")) + page_owner_filter = PAGE_OWNER_NONE_FILTER; + else + return -EINVAL; + + return count; +} + +static const struct file_operations page_owner_filter_ops = { + .read = read_page_owner_filter, + .write = write_page_owner_filter, + .llseek = default_llseek, +}; + +bool po_is_filtered(struct page_owner *page_owner) +{ + if (unlikely(!page_owner)) + return false; + + if (page_owner_filter & PAGE_OWNER_MODULE_FILTER && + !po_is_module(page_owner)) + return true; + + return false; +} + +void po_module_stat_init(void) +{ + debugfs_create_file("page_owner_filter", 0600, NULL, NULL, + &page_owner_filter_ops); +} -- 2.25.1

Offering: HULK hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I9GSSR ---------------------------------------- This patch adds a new debugfs interface page_owner_module_show_max to allow users to configure how much modules are dumped when oom occurs. Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com> --- mm/page_owner_module.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/mm/page_owner_module.c b/mm/page_owner_module.c index b64cfb4a1671..a8a3066b02d7 100644 --- a/mm/page_owner_module.c +++ b/mm/page_owner_module.c @@ -276,6 +276,21 @@ static struct notifier_block po_oom_nb = { .priority = 0 }; +static int po_module_topn_set(void *data, u64 val) +{ + po_module_topn = val; + return 0; +} + +static int po_module_topn_get(void *data, u64 *val) +{ + *val = po_module_topn; + return 0; +} + +DEFINE_SIMPLE_ATTRIBUTE(po_module_topn_fops, po_module_topn_get, + po_module_topn_set, "%llu\n"); + void po_module_stat_init(void) { int ret; @@ -293,4 +308,5 @@ void po_module_stat_init(void) if (ret) pr_warn("Failed to register page owner oom notifier\n"); + debugfs_create_file("page_owner_module_show_max", 0600, NULL, NULL, &po_module_topn_fops); } -- 2.25.1

Offering: HULK hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I9GSSR ---------------------------------------- When a module is unloaded, if it doesn't free all pages, a memory leak occurs. Move the po_module node to leaked_po_module_list, and report the leak when oom occurs and users read page_owner_module_stats interface. Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com> --- mm/page_owner_module.c | 100 +++++++++++++++++++++++++++++++++-------- 1 file changed, 82 insertions(+), 18 deletions(-) diff --git a/mm/page_owner_module.c b/mm/page_owner_module.c index c184b103f6ae..cb878bd3ddcb 100644 --- a/mm/page_owner_module.c +++ b/mm/page_owner_module.c @@ -11,6 +11,7 @@ #include <linux/list_sort.h> #include <linux/oom.h> #include <linux/slab.h> +#include <linux/sched/clock.h> #include "page_owner.h" @@ -28,7 +29,15 @@ struct po_module { long nr_pages_used; }; +struct leaked_po_module { + struct list_head list; + char module_name[MODULE_NAME_LEN]; + long nr_pages_used; + u64 unload_ns; +}; + LIST_HEAD(po_module_list); +LIST_HEAD(leaked_po_module_list); DEFINE_SPINLOCK(po_module_list_lock); static unsigned int po_module_topn = PO_MODULE_DEFAULT_TOPN; @@ -210,6 +219,24 @@ static int po_module_coming(struct module *mod) return 0; } +static void create_leaked_node(struct po_module *po_mod) +{ + struct leaked_po_module *leaked_po_mod; + unsigned long flags; + + leaked_po_mod = kmalloc(sizeof(struct leaked_po_module), GFP_KERNEL); + if (!leaked_po_mod) + return; + + leaked_po_mod->unload_ns = local_clock(); + strscpy(leaked_po_mod->module_name, po_mod->mod->name, MODULE_NAME_LEN); + leaked_po_mod->nr_pages_used = po_mod->nr_pages_used; + INIT_LIST_HEAD(&leaked_po_mod->list); + spin_lock_irqsave(&po_module_list_lock, flags); + list_add_tail(&leaked_po_mod->list, &leaked_po_module_list); + spin_unlock_irqrestore(&po_module_list_lock, flags); +} + static void po_module_going(struct module *mod) { struct po_module *po_mod; @@ -219,6 +246,10 @@ static void po_module_going(struct module *mod) po_mod = po_find_module(mod); list_del(&po_mod->list); spin_unlock_irqrestore(&po_module_list_lock, flags); + + if (unlikely(po_mod->nr_pages_used)) + create_leaked_node(po_mod); + kfree(po_mod); } @@ -245,10 +276,52 @@ static struct notifier_block po_module_nb = { .priority = 0 }; +static void print_list(unsigned int nr, struct seq_file *m) +{ + struct po_module *po_mod; + + lockdep_assert_held(&po_module_list_lock); + + if (list_empty(&po_module_list)) + return; + + list_sort(NULL, &po_module_list, po_module_cmp); + list_for_each_entry(po_mod, &po_module_list, list) { + if (m) + seq_printf(m, "%s %ld\n", po_mod->mod->name, + po_mod->nr_pages_used); + else + pr_info("\tModule %s allocated %ld pages\n", + po_mod->mod->name, po_mod->nr_pages_used); + --nr; + if (!nr) + break; + } +} + +static void print_leaked_list(struct seq_file *m) +{ + struct leaked_po_module *leaked_po_mod; + + lockdep_assert_held(&po_module_list_lock); + + if (list_empty(&leaked_po_module_list)) + return; + + list_for_each_entry(leaked_po_mod, &leaked_po_module_list, list) { + if (m) + seq_printf(m, "[unloaded %llu]%s %ld\n", leaked_po_mod->unload_ns, + leaked_po_mod->module_name, leaked_po_mod->nr_pages_used); + else + pr_info("\t[unloaded %llu]Module %s allocated %ld pages\n", + leaked_po_mod->unload_ns, leaked_po_mod->module_name, + leaked_po_mod->nr_pages_used); + } +} + static int po_oom_notify(struct notifier_block *self, unsigned long val, void *data) { - struct po_module *po_mod; unsigned long flags; unsigned int nr = po_module_topn; int ret = notifier_from_errno(0); @@ -257,15 +330,11 @@ static int po_oom_notify(struct notifier_block *self, return ret; spin_lock_irqsave(&po_module_list_lock, flags); - list_sort(NULL, &po_module_list, po_module_cmp); pr_info("Top modules allocating pages:\n"); - list_for_each_entry(po_mod, &po_module_list, list) { - pr_info("\tModule %s allocated %ld pages\n", po_mod->mod->name, - po_mod->nr_pages_used); - --nr; - if (!nr) - break; - } + + print_list(nr, NULL); + print_leaked_list(NULL); + spin_unlock_irqrestore(&po_module_list_lock, flags); return ret; @@ -293,7 +362,6 @@ DEFINE_SIMPLE_ATTRIBUTE(po_module_topn_fops, po_module_topn_get, static int page_owner_module_stats_show(struct seq_file *m, void *v) { - struct po_module *po_mod; unsigned long flags; unsigned int nr = po_module_topn; @@ -301,14 +369,10 @@ static int page_owner_module_stats_show(struct seq_file *m, void *v) return 0; spin_lock_irqsave(&po_module_list_lock, flags); - list_sort(NULL, &po_module_list, po_module_cmp); - list_for_each_entry(po_mod, &po_module_list, list) { - seq_printf(m, "%s %ld\n", po_mod->mod->name, - po_mod->nr_pages_used); - --nr; - if (!nr) - break; - } + + print_list(nr, m); + print_leaked_list(m); + spin_unlock_irqrestore(&po_module_list_lock, flags); return 0; } -- 2.25.1

Offering: HULK hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I9GSSR ---------------------------------------- Add -M and --module options to support selecting modules. -M option allows to filtering all the items except the ones allcating by modules. --module option allows to only select one or more modules. For example, --module=mod1,mod2 will only select items allocating by mod1 or mod2. Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com> --- tools/vm/page_owner_sort.c | 84 +++++++++++++++++++++++++++++++++----- 1 file changed, 73 insertions(+), 11 deletions(-) diff --git a/tools/vm/page_owner_sort.c b/tools/vm/page_owner_sort.c index 0e1db628291c..884cc60be1ab 100644 --- a/tools/vm/page_owner_sort.c +++ b/tools/vm/page_owner_sort.c @@ -27,6 +27,7 @@ #define true 1 #define false 0 #define TASK_COMM_LEN 16 +#define MODULE_NAME_LEN (64 - sizeof(unsigned long)) struct block_list { char *txt; @@ -40,12 +41,14 @@ struct block_list { pid_t pid; pid_t tgid; int allocator; + char *module; }; enum FILTER_BIT { FILTER_UNRELEASE = 1<<1, FILTER_PID = 1<<2, FILTER_TGID = 1<<3, - FILTER_COMM = 1<<4 + FILTER_COMM = 1<<4, + FILTER_MODULE = 1<<5 }; enum CULL_BIT { CULL_UNRELEASE = 1<<1, @@ -74,9 +77,11 @@ struct filter_condition { pid_t *pids; pid_t *tgids; char **comms; + char **modules; int pids_size; int tgids_size; int comms_size; + int modules_size; }; struct sort_condition { int (**cmps)(const void *, const void *); @@ -91,6 +96,7 @@ static regex_t tgid_pattern; static regex_t comm_pattern; static regex_t ts_nsec_pattern; static regex_t free_ts_nsec_pattern; +static regex_t module_pattern; static struct block_list *list; static int list_size; static int max_size; @@ -100,10 +106,12 @@ static bool debug_on; static void set_single_cmp(int (*cmp)(const void *, const void *), int sign); -int read_block(char *buf, char *ext_buf, int buf_size, FILE *fin) +int read_block(char *buf, char *ext_buf, char *mod_buf, int buf_size, FILE *fin) { char *curr = buf, *const buf_end = buf + buf_size; + char *mod_string = "Page allocated by module"; + mod_buf[0] = '\0'; while (buf_end - curr > 1 && fgets(curr, buf_end - curr, fin)) { if (*curr == '\n') { /* empty line */ return curr - buf; @@ -112,6 +120,10 @@ int read_block(char *buf, char *ext_buf, int buf_size, FILE *fin) strcpy(ext_buf, curr); continue; } + if (!strncmp(curr, mod_string, strlen(mod_string))) { + strcpy(mod_buf, curr); + continue; + } curr += strlen(curr); } @@ -401,6 +413,16 @@ static char *get_comm(char *buf) return comm_str; } +static char *get_module(char *buf) +{ + char *mod = malloc(MODULE_NAME_LEN); + + memset(mod, 0, MODULE_NAME_LEN); + search_pattern(&module_pattern, mod, buf); + + return mod; +} + static int get_arg_type(const char *arg) { if (!strcmp(arg, "pid") || !strcmp(arg, "p")) @@ -469,7 +491,24 @@ static bool match_str_list(const char *str, char **list, int list_size) return false; } -static bool is_need(char *buf) +static bool is_module_filtered(char *mod_buf) +{ + char *mod = get_module(mod_buf); + int ret = true; + + if (!strlen(mod)) + goto out; + + if (fc.modules_size == 0 || + match_str_list(mod, fc.modules, fc.modules_size)) + ret = false; + +out: + free(mod); + return ret; +} + +static bool is_need(char *buf, char *mod_buf) { __u64 ts_nsec, free_ts_nsec; @@ -484,6 +523,9 @@ static bool is_need(char *buf) !match_num_list(get_tgid(buf), fc.tgids, fc.tgids_size)) return false; + if ((filter & FILTER_MODULE) && is_module_filtered(mod_buf)) + return false; + char *comm = get_comm(buf); if ((filter & FILTER_COMM) && @@ -495,7 +537,7 @@ static bool is_need(char *buf) return true; } -static bool add_list(char *buf, int len, char *ext_buf) +static bool add_list(char *buf, int len, char *ext_buf, char *mod_buf) { if (list_size != 0 && len == list[list_size-1].len && @@ -508,7 +550,7 @@ static bool add_list(char *buf, int len, char *ext_buf) fprintf(stderr, "max_size too small??\n"); return false; } - if (!is_need(buf)) + if (!is_need(buf, mod_buf)) return true; list[list_size].pid = get_pid(buf); list[list_size].tgid = get_tgid(buf); @@ -530,6 +572,7 @@ static bool add_list(char *buf, int len, char *ext_buf) list[list_size].ts_nsec = get_ts_nsec(buf); list[list_size].free_ts_nsec = get_free_ts_nsec(buf); list[list_size].allocator = get_allocator(buf, ext_buf); + list[list_size].module = get_module(mod_buf); list_size++; if (list_size % 1000 == 0) { printf("loaded %d\r", list_size); @@ -681,19 +724,21 @@ static void usage(void) "-a\t\tSort by memory allocate time.\n" "-r\t\tSort by memory release time.\n" "-f\t\tFilter out the information of blocks whose memory has been released.\n" + "-M\t\tFilter out the information of blocks whose memory isn't allocated by modules.\n" "-d\t\tPrint debug information.\n" "--pid <pidlist>\tSelect by pid. This selects the information of blocks whose process ID numbers appear in <pidlist>.\n" "--tgid <tgidlist>\tSelect by tgid. This selects the information of blocks whose Thread Group ID numbers appear in <tgidlist>.\n" "--name <cmdlist>\n\t\tSelect by command name. This selects the information of blocks whose command name appears in <cmdlist>.\n" "--cull <rules>\tCull by user-defined rules.<rules> is a single argument in the form of a comma-separated list with some common fields predefined\n" "--sort <order>\tSpecify sort order as: [+|-]key[,[+|-]key[,...]]\n" + "--module <modulelist>\tSelect by module. This selects the information of blocks whose memory is allocated by modules appear in <modulelist>.\n" ); } int main(int argc, char **argv) { FILE *fin, *fout; - char *buf, *ext_buf; + char *buf, *ext_buf, *mod_buf; int i, count; struct stat st; int opt; @@ -703,10 +748,11 @@ int main(int argc, char **argv) { "name", required_argument, NULL, 3 }, { "cull", required_argument, NULL, 4 }, { "sort", required_argument, NULL, 5 }, + { "module", required_argument, NULL, 6 }, { 0, 0, 0, 0}, }; - while ((opt = getopt_long(argc, argv, "adfmnprstP", longopts, NULL)) != -1) + while ((opt = getopt_long(argc, argv, "adfmnprstPM", longopts, NULL)) != -1) switch (opt) { case 'a': set_single_cmp(compare_ts, SORT_ASC); @@ -738,6 +784,11 @@ int main(int argc, char **argv) case 'n': set_single_cmp(compare_comm, SORT_ASC); break; + case 'M': + filter = filter | FILTER_MODULE; + fc.modules_size = 0; + fc.modules = NULL; + break; case 1: filter = filter | FILTER_PID; fc.pids = parse_nums_list(optarg, &fc.pids_size); @@ -774,6 +825,10 @@ int main(int argc, char **argv) exit(1); } break; + case 6: + filter = filter | FILTER_MODULE; + fc.modules = explode(',', optarg, &fc.modules_size); + break; default: usage(); exit(1); @@ -804,6 +859,8 @@ int main(int argc, char **argv) goto out_ts; if (!check_regcomp(&free_ts_nsec_pattern, "free_ts\\s*([0-9]*)\\s*ns")) goto out_free_ts; + if (!check_regcomp(&module_pattern, "Page allocated by module (.*)")) + goto out_module; fstat(fileno(fin), &st); max_size = st.st_size / 100; /* hack ... */ @@ -811,17 +868,18 @@ int main(int argc, char **argv) list = malloc(max_size * sizeof(*list)); buf = malloc(BUF_SIZE); ext_buf = malloc(BUF_SIZE); - if (!list || !buf || !ext_buf) { + mod_buf = malloc(BUF_SIZE); + if (!list || !buf || !ext_buf || !mod_buf) { fprintf(stderr, "Out of memory\n"); goto out_free; } for ( ; ; ) { - int buf_len = read_block(buf, ext_buf, BUF_SIZE, fin); + int buf_len = read_block(buf, ext_buf, mod_buf, BUF_SIZE, fin); if (buf_len < 0) break; - if (!add_list(buf, buf_len, ext_buf)) + if (!add_list(buf, buf_len, ext_buf, mod_buf)) goto out_free; } @@ -848,8 +906,10 @@ int main(int argc, char **argv) for (i = 0; i < count; i++) { if (cull == 0) { fprintf(fout, "%d times, %d pages, ", list[i].num, list[i].page_num); + if (strlen(list[i].module) != 0) + fprintf(fout, "allocated by module %s, ", list[i].module); print_allocator(fout, list[i].allocator); - fprintf(fout, ":\n%s\n", list[i].txt); + fprintf(fout, " :\n%s\n", list[i].txt); } else { fprintf(fout, "%d times, %d pages", @@ -880,6 +940,8 @@ int main(int argc, char **argv) free(buf); if (list) free(list); +out_module: + regfree(&module_pattern); out_free_ts: regfree(&free_ts_nsec_pattern); out_ts: -- 2.25.1

Offering: HULK hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I9GSSR ---------------------------------------- This patch adds new argument (i.e. source) for --sort option, allows the items sorted by module name. Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com> --- tools/vm/page_owner_sort.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tools/vm/page_owner_sort.c b/tools/vm/page_owner_sort.c index 16ddff316f1b..b4936e78d7d5 100644 --- a/tools/vm/page_owner_sort.c +++ b/tools/vm/page_owner_sort.c @@ -679,6 +679,8 @@ static bool parse_sort_args(const char *arg_str) sc.cmps[i] = compare_txt; else if (arg_type == ARG_ALLOCATOR) sc.cmps[i] = compare_allocator; + else if (arg_type == ARG_MODULE) + sc.cmps[i] = compare_module; else { free_explode(args, size); sc.size = 0; -- 2.25.1