[PATCH OLK-6.6 04/12] slub: Prepare __slab_free() for unfrozen partial slab out of node partial list

From: Chengming Zhou zhouchengming@bytedance.com

mainline inclusion from mainline-v6.8-rc1 commit 422e7d54375889484b66962d1dcbc392a6bd9e7a category: performance bugzilla: https://gitee.com/openeuler/kernel/issues/I9CSFJ CVE: NA

Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i...

--------------------------------

Now the partially empty slub will be frozen when taken out of node partial list, so the __slab_free() will know from "was_frozen" that the partially empty slab is not on node partial list and is a cpu or cpu partial slab of some cpu.

But we will change this, make partial slabs leave the node partial list with unfrozen state, so we need to change __slab_free() to use the new slab_test_node_partial() we just introduced.

Signed-off-by: Chengming Zhou zhouchengming@bytedance.com Reviewed-by: Vlastimil Babka vbabka@suse.cz Tested-by: Hyeonggon Yoo 42.hyeyoo@gmail.com Reviewed-by: Hyeonggon Yoo 42.hyeyoo@gmail.com Signed-off-by: Vlastimil Babka vbabka@suse.cz Signed-off-by: ZhangPeng zhangpeng362@huawei.com --- mm/slub.c | 11 +++++++++++ 1 file changed, 11 insertions(+)

diff --git a/mm/slub.c b/mm/slub.c index 3002c670c654..dcb47aa54a87 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -3631,6 +3631,7 @@ static void __slab_free(struct kmem_cache *s, struct slab *slab, unsigned long counters; struct kmem_cache_node *n = NULL; unsigned long flags; + bool on_node_partial;

stat(s, FREE_SLOWPATH);

@@ -3678,6 +3679,7 @@ static void __slab_free(struct kmem_cache *s, struct slab *slab, */ spin_lock_irqsave(&n->list_lock, flags);

+ on_node_partial = slab_test_node_partial(slab); } }

@@ -3706,6 +3708,15 @@ static void __slab_free(struct kmem_cache *s, struct slab *slab, return; }

+ /* + * This slab was partially empty but not on the per-node partial list, + * in which case we shouldn't manipulate its list, just return. + */ + if (prior && !on_node_partial) { + spin_unlock_irqrestore(&n->list_lock, flags); + return; + } + if (unlikely(!new.inuse && n->nr_partial >= s->min_partial)) goto slab_empty;