From: Chengming Zhou zhouchengming@bytedance.com
mainline inclusion from mainline-v6.8-rc1 commit 213094b5d1af7e6ab294a6d8f3b50cafb72642ae category: performance bugzilla: https://gitee.com/openeuler/kernel/issues/I9CSFJ CVE: NA
Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i...
--------------------------------
We will have unfrozen slabs out of the node partial list later, so we need a freeze_slab() function to freeze the partial slab and get its freelist.
Signed-off-by: Chengming Zhou zhouchengming@bytedance.com Reviewed-by: Vlastimil Babka vbabka@suse.cz Tested-by: Hyeonggon Yoo 42.hyeyoo@gmail.com Reviewed-by: Hyeonggon Yoo 42.hyeyoo@gmail.com Signed-off-by: Vlastimil Babka vbabka@suse.cz Signed-off-by: ZhangPeng zhangpeng362@huawei.com --- mm/slub.c | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+)
diff --git a/mm/slub.c b/mm/slub.c index dcb47aa54a87..c1dc6d4cf5a9 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -3098,6 +3098,33 @@ static inline void *get_freelist(struct kmem_cache *s, struct slab *slab) return freelist; }
+/* + * Freeze the partial slab and return the pointer to the freelist. + */ +static inline void *freeze_slab(struct kmem_cache *s, struct slab *slab) +{ + struct slab new; + unsigned long counters; + void *freelist; + + do { + freelist = slab->freelist; + counters = slab->counters; + + new.counters = counters; + VM_BUG_ON(new.frozen); + + new.inuse = slab->objects; + new.frozen = 1; + + } while (!slab_update_freelist(s, slab, + freelist, counters, + NULL, new.counters, + "freeze_slab")); + + return freelist; +} + /* * Slow path. The lockless freelist is empty or we need to perform * debugging duties.