- Kernel - mailweb.openeuler.org

[PATCH OLK-6.6] mm/huge_memory: fix huge_zero_pfn race
by Zhang Qilong 08 Sep '26

08 Sep '26

From: "Lorenzo Stoakes (ARM)" <ljs(a)kernel.org> stable inclusion from stable-v6.6.153 commit f3a874a903053c53fb53ba287ea9eacda69c68e8 category: bugfix bugzilla: https://atomgit.com/src-openeuler/kernel/issues/18287 CVE: CVE-2026-74632 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id… -------------------------------- commit 33192a26cddea7a7e4ca66e5c3eebd36fa8be2bb upstream. Patch series "mm/huge_memory: fix huge_zero_pfn race", v2. There is a subtle race in the reference-counted huge_zero_folio implementation. The fast path atomic logic fails to account for the fact that the shrinker (which drops the final huge_zero_refcount pin) can overwrite huge_zero_pfn with the ~0UL sentinel value in shrink_huge_zero_folio_scan() after a racing get_huge_zero_folio() installed a valid value there. This results in huge_zero_folio being correctly set but huge_zero_pfn being set incorrectly and thus is_huge_zero_pfn() and consequently is_huge_zero_pmd() will misidentify the huge zero folio as being an ordinary THP folio. This can result in the huge zero folio being split and otherwise treated incorrectly. The solution to this is very subtle as there is an atomic fast path, and thus ordering in weakly ordered architectures has to be treated very carefully. The first commit fixes the issue by introducing a spinlock around huge_zero_[pfn, folio, refcount] write, with careful consideration paid to load/store ordering in the fast path. It is placed first and kept as small as possible so that it can be backported on its own. The second commit is a pure cleanup which reworks the CONFIG_PERSISTENT_HUGE_ZERO_FOLIO logic to better separate the persistent logic from the dynamically allocated one. This patch (of 2): If !CONFIG_PERSISTENT_HUGE_ZERO_FOLIO, the huge_zero_folio is refcounted by huge_zero_refcount and returned by mm_get_huge_zero_folio(). When the caller is done with the huge zero page, its reference count is decremented. Only a shrinker can set the reference count to zero. A race can unfortunately occur between a shrinker decrementing the reference count to zero and a concurrent page fault. This is because shrink_huge_zero_folio_scan() might, if very unlucky, be preempted between setting huge_zero_refcount to zero and writing an invalid value. During this time get_huge_zero_folio() could write to huge_zero_pfn before shrink_huge_zero_folio_scan() resumes. In this event the huge zero folio will be persistently misidentified causing the THP code path to be entered inappropriately for the huge zero folio: CPU 0 CPU 1 =======================================|================================= shrink_huge_zero_folio_scan() | atomic_cmpxchg() sets refcount to 0 | xchg() sets huge_zero_folio to NULL | get_huge_zero_folio() | | atomic_inc_not_zero() -> zero preempted for a long time | Allocate new huge zero folio | | Write valid huge_zero_folio v | Write valid huge_zero_pfn Overwrite huge_zero_pfn with ~0UL <--- Invalid overwrite! This results in is_huge_zero_pfn() and is_huge_zero_pmd() incorrectly returning false for a huge zero page which could result in issues like the huge zero folio being incorrectly split. Note that the issue is with huge_zero_pfn not huge_zero_folio, as get_huge_zero_folio() uses cmpxchg() gated on huge_zero_folio being NULL with a retry loop and shrink_huge_zero_folio_scan() uses xchg() to set huge_zero_folio. Fix the issue by introducing a spinlock, huge_zero_lock, to prevent concurrent write of huge_zero_folio, huge_zero_pfn and huge_zero_refcount. There needs to be significant care taken here to ensure correctness: The fast path in get_huge_zero_folio() uses atomic_inc_not_zero(), which is outside of the critical section, and means huge zero allocation is gated on zero huge_zero_refcount. The fast path doesn't use huge_zero_lock, so the critical section is irrelevant to it. So invariants are required - huge_zero_refcount MUST: * Only be set in the huge_zero_lock critical section to ensure serialisation of huge_zero_pfn, huge_zero_folio and huge_zero_refcount writes. * Be set non-zero only AFTER huge_zero_[pfn, folio] are set to valid values so installation of the huge zero folio on read page fault ensures concurrent is_huge_zero_*() calls correctly identify the huge zero folio. * Be set zero only BEFORE huge_zero_[pfn, folio] are set to NULL and ~0UL respectively, and atomically. Establish these by: * Only setting huge_zero_refcount to zero or an absolute value in the huge_zero_lock critical section in get_huge_zero_folio() and shrink_huge_zero_folio_scan(), and always updating atomically there and elsewhere. * Using atomic_set_release(&huge_zero_refcount) in get_huge_zero_folio() after huge_zero_[pfn, folio] are set. This is paired with atomic_inc_not_zero() to ensure atomic_inc_not_zero() only observes a non-zero value if huge_zero_[pfn, folio] are set. * Using atomic_cmpxchg() in shrink_huge_zero_folio_scan() (as before) to ensure that it is set zero only when equal to 1 and set atomically. * atomic_cmpxchg() being fully ordered ensures this is done prior to huge_zero_[folio, pfn] being set to NULL and ~0UL respectively. Eliminate the retry loop in get_huge_zero_folio() as the atomic_cmpxchg() in shrink_huge_zero_folio_scan() is now performed under the lock, and replace with an equally locked atomic_inc() to set the reference count should the caller be raced on huge zero folio installation. folio_put() naturally implies a full memory barrier so its ordering is maintained correctly. The huge zero folio also cannot be released except when the shrinker does so as it is non-LRU and non-rmappable. Note that only the huge zero shrinker (via shrink_huge_zero_folio_scan()) can actually set huge_zero_refcount to zero, which is the count of mm's which have at least one huge zero folio installed plus one shrinker pin. Additionally convert a BUG_ON() to a VM_WARN_ON_ONCE(). Link: https://lore.kernel.org/20260730-fix-refcounted-huge-zero-v2-0-c5d8a41b317f… Link: https://lore.kernel.org/20260730-fix-refcounted-huge-zero-v2-1-c5d8a41b317f… Fixes: 3b77e8c8cde5 ("mm/thp: make is_huge_zero_pmd() safe and quicker") Signed-off-by: Lorenzo Stoakes (ARM) <ljs(a)kernel.org> Reported-by: Hengbin Zhang <uqbarz(a)gmail.com> Closes: https://lore.kernel.org/linux-mm/20260727154001.4102341-1-uqbarz@gmail.com/ Suggested-by: David Hildenbrand (Arm) <david(a)kernel.org> Acked-by: David Hildenbrand (Arm) <david(a)kernel.org> Cc: Baolin Wang <baolin.wang(a)linux.alibaba.com> Cc: Barry Song <baohua(a)kernel.org> Cc: Dev Jain <dev.jain(a)arm.com> Cc: Hannes Reinecke <hare(a)suse.de> Cc: Hugh Dickins <hughd(a)google.com> Cc: Kiryl Shutsemau <kas(a)kernel.org> Cc: Lance Yang <lance.yang(a)linux.dev> Cc: Liam R. Howlett <liam(a)infradead.org> Cc: Nico Pache <npache(a)redhat.com> Cc: Pankaj Raghav <p.raghav(a)samsung.com> Cc: Ryan Roberts <ryan.roberts(a)arm.com> Cc: Yang Shi <shy828301(a)gmail.com> Cc: Zi Yan <ziy(a)nvidia.com> Cc: <stable(a)vger.kernel.org> Signed-off-by: Andrew Morton <akpm(a)linux-foundation.org> [ adapted folio API (`huge_zero_folio`, `folio_pfn`, `folio_put`) to pre-6.11 page naming (`huge_zero_page`, `page_to_pfn`, `__free_pages`) ] Signed-off-by: Sasha Levin <sashal(a)kernel.org> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> Conflicts: mm/huge_memory.c [Just shrinker name conflicts] Signed-off-by: Zhang Qilong <zhangqilong3(a)huawei.com> --- mm/huge_memory.c | 43 +++++++++++++++++++++++++++++-------------- 1 file changed, 29 insertions(+), 14 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index bb884be9b761..952abb6d00a7 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -38,10 +38,11 @@ #include <linux/numa.h> #include <linux/page_owner.h> #include <linux/sched/sysctl.h> #include <linux/memory-tiers.h> #include <linux/compat.h> +#include <linux/cleanup.h> #include <asm/tlb.h> #include <asm/pgalloc.h> #include "internal.h" #include "swap.h" @@ -75,10 +76,11 @@ static unsigned long deferred_split_count(struct shrinker_v2 *shrink, struct shrink_control *sc); static unsigned long deferred_split_scan(struct shrinker_v2 *shrink, struct shrink_control *sc); static atomic_t huge_zero_refcount; +static DEFINE_SPINLOCK(huge_zero_lock); struct page *huge_zero_page __read_mostly; unsigned long huge_zero_pfn __read_mostly = ~0UL; unsigned long huge_anon_orders_always __read_mostly; unsigned long huge_anon_orders_madvise __read_mostly; unsigned long huge_anon_orders_inherit __read_mostly; @@ -196,31 +198,37 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma, } static bool get_huge_zero_page(void) { struct page *zero_page; -retry: + + /* Paired with atomic_set_release(). */ if (likely(atomic_inc_not_zero(&huge_zero_refcount))) return true; zero_page = alloc_pages((GFP_TRANSHUGE | __GFP_ZERO) & ~__GFP_MOVABLE, HPAGE_PMD_ORDER); if (!zero_page) { count_vm_event(THP_ZERO_PAGE_ALLOC_FAILED); return false; } - preempt_disable(); - if (cmpxchg(&huge_zero_page, NULL, zero_page)) { - preempt_enable(); + + /* Paired with critical section in shrink_huge_zero_page_scan(). */ + spin_lock(&huge_zero_lock); + if (huge_zero_page) { + /* Somebody else already installed it. */ + atomic_inc(&huge_zero_refcount); + spin_unlock(&huge_zero_lock); __free_pages(zero_page, compound_order(zero_page)); - goto retry; + return true; } + WRITE_ONCE(huge_zero_page, zero_page); WRITE_ONCE(huge_zero_pfn, page_to_pfn(zero_page)); + /* Paired with atomic_inc_not_zero(). +1 for shrinker pin. */ + atomic_set_release(&huge_zero_refcount, 2); + spin_unlock(&huge_zero_lock); - /* We take additional reference here. It will be put back by shrinker */ - atomic_set(&huge_zero_refcount, 2); - preempt_enable(); count_vm_event(THP_ZERO_PAGE_ALLOC); return true; } static void put_huge_zero_page(void) @@ -260,19 +268,26 @@ static unsigned long shrink_huge_zero_page_count(struct shrinker_v2 *shrink, } static unsigned long shrink_huge_zero_page_scan(struct shrinker_v2 *shrink, struct shrink_control *sc) { - if (atomic_cmpxchg(&huge_zero_refcount, 1, 0) == 1) { - struct page *zero_page = xchg(&huge_zero_page, NULL); - BUG_ON(zero_page == NULL); + struct page *zero_page; + + /* Paired with critical section in get_huge_zero_page(). */ + scoped_guard(spinlock, &huge_zero_lock) { + /* Paired with atomic_inc_not_zero() in get_huge_zero_page(). */ + if (atomic_cmpxchg(&huge_zero_refcount, 1, 0) != 1) + return 0; + + zero_page = huge_zero_page; + VM_WARN_ON_ONCE(!zero_page); + WRITE_ONCE(huge_zero_page, NULL); WRITE_ONCE(huge_zero_pfn, ~0UL); - __free_pages(zero_page, compound_order(zero_page)); - return HPAGE_PMD_NR; } - return 0; + __free_pages(zero_page, compound_order(zero_page)); + return HPAGE_PMD_NR; } static struct shrinker_v2 *huge_zero_page_shrinker; #ifdef CONFIG_SYSFS -- 2.43.0

2 1

[PATCH OLK-6.6] net: usb: ipheth: fix carrier_work UAF on disconnect
by Zhang Qilong 08 Sep '26

08 Sep '26

From: Doruk Tan Ozturk <doruk(a)0sec.ai> stable inclusion from stable-v6.12.104 commit d07133fe1befae9a1e4c4c5e46ef0b73d2992020 category: bugfix bugzilla: https://atomgit.com/src-openeuler/kernel/issues/18212 CVE: CVE-2026-74677 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id… -------------------------------- commit fde39b8a521780391fb4e5bda2c0aa4928947f12 upstream. ipheth_sndbulk_callback() re-arms the carrier-check work on any non-zero URB status: else schedule_delayed_work(&dev->carrier_work, 0); Nothing ties that to the interface being up, so the work can be armed again after ipheth_close() has already drained it, and stay armed until the netdev whose private area embeds it is freed. On unplug with a TX URB in flight, ipheth_disconnect() drains the work through unregister_netdev() -> ipheth_close() -> cancel_delayed_work_sync() and only then calls ipheth_kill_urbs(). usb_kill_urb() completes the in-flight TX URB with -ENOENT, so ipheth_sndbulk_callback() runs after the drain and re-arms carrier_work. The same completion also re-arms the work if the interface is only brought down while a TX URB is in flight, and ipheth_carrier_check_work() then keeps re-queueing itself once a second. unregister_netdev() does not call ipheth_close() for an already-down interface, so nothing drains it on the later unplug either. In both cases free_netdev() frees the netdev while carrier_work is still pending, and ipheth_carrier_check_work() dereferences freed memory. Tie the work to the interface state instead of chasing the completion: disable it in ipheth_close() and enable it in ipheth_open(), so a schedule_delayed_work() from the URB completion is a no-op whenever the interface is not up. disable_delayed_work_sync() also waits for a running instance, so it fully replaces the cancel_delayed_work_sync() it takes the place of. The work starts out disabled in ipheth_probe() so the enable/disable counts balance from the first open. Reproduced under KASAN on linux-next (next-20260731) with dummy_hcd and raw-gadget standing in for the device, driving the second path above (the interface is already down, so unregister_netdev() does not call ipheth_close()): 15 of 15 unpatched boots report a slab-use-after-free in __run_timers(), freed by ipheth_disconnect() and re-armed from ipheth_sndbulk_callback() via queue_delayed_work_on(). The same trigger on a kernel differing only by this patch reports 0 of 15, and the carrier check still functions across open/close cycles. The reproducer needs an attached USB device that stops draining bulk OUT, plus a link down and unplug, driven as root. It is not a privilege boundary crossing and no exploit primitive was developed. Found by 0sec (https://0sec.ai) Fixes: bb1b40c7cb86 ("usbnet: ipheth: prevent TX queue timeouts when device not ready") Cc: stable(a)vger.kernel.org Signed-off-by: Doruk Tan Ozturk <doruk(a)0sec.ai> Link: https://patch.msgid.link/20260802120602.42595-1-doruk@0sec.ai Signed-off-by: Jakub Kicinski <kuba(a)kernel.org> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> Signed-off-by: Zhang Qilong <zhangqilong3(a)huawei.com> --- drivers/net/usb/ipheth.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/drivers/net/usb/ipheth.c b/drivers/net/usb/ipheth.c index a19789b57190..a86d0a4dbd28 100644 --- a/drivers/net/usb/ipheth.c +++ b/drivers/net/usb/ipheth.c @@ -488,20 +488,25 @@ static int ipheth_open(struct net_device *net) retval = ipheth_rx_submit(dev, GFP_KERNEL); if (retval) return retval; + enable_delayed_work(&dev->carrier_work); schedule_delayed_work(&dev->carrier_work, IPHETH_CARRIER_CHECK_TIMEOUT); return retval; } static int ipheth_close(struct net_device *net) { struct ipheth_device *dev = netdev_priv(net); netif_stop_queue(net); - cancel_delayed_work_sync(&dev->carrier_work); + /* A TX URB can still complete with an error after this point and + * try to re-arm the carrier work. Disable it instead of cancelling + * it, so that such a schedule_delayed_work() is a no-op. + */ + disable_delayed_work_sync(&dev->carrier_work); return 0; } static netdev_tx_t ipheth_tx(struct sk_buff *skb, struct net_device *net) { @@ -631,10 +636,14 @@ static int ipheth_probe(struct usb_interface *intf, dev->rx_buf_len = IPHETH_RX_BUF_SIZE_NCM; dev->rcvbulk_callback = ipheth_rcvbulk_callback_ncm; } INIT_DELAYED_WORK(&dev->carrier_work, ipheth_carrier_check_work); + /* Armed only between ipheth_open() and ipheth_close(). Start out + * disabled so the enable/disable counts balance from the first open. + */ + disable_delayed_work(&dev->carrier_work); retval = ipheth_alloc_urbs(dev); if (retval) { dev_err(&intf->dev, "error allocating urbs: %d\n", retval); goto err_alloc_urbs; -- 2.43.0

2 1

[PATCH OLK-6.6] sctp: fix addr_wq_timer race in sctp_free_addr_wq()
by Zhang Qilong 08 Sep '26

08 Sep '26

From: Xin Long <lucien.xin(a)gmail.com> stable inclusion from stable-v6.18.40 commit a8323fb2ab6cd6978f359daeed6688e0cadf32ba category: bugfix bugzilla: https://atomgit.com/src-openeuler/kernel/issues/17334 CVE: CVE-2026-72383 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id… -------------------------------- [ Upstream commit 976c19de0f22a857ba0112f39635f8fd7a257568 ] sctp_free_addr_wq() previously removed addr_wq_timer using timer_delete() while holding addr_wq_lock. However, timer_delete() does not guarantee that a currently running timer handler has completed. This allows a race with sctp_addr_wq_timeout_handler(), where the handler may still run after addr_waitq has been freed, acquire addr_wq_lock, and access freed memory, leading to a use-after-free. Fix this by calling timer_shutdown_sync() before taking addr_wq_lock. This guarantees that any in-flight timer handler has finished and prevents the timer from being re-armed during teardown, making subsequent cleanup safe. Fixes: 4db67e808640 ("sctp: Make the address lists per network namespace") Reported-by: Sashiko <sashiko-bot(a)kernel.org> Signed-off-by: Xin Long <lucien.xin(a)gmail.com> Link: https://patch.msgid.link/5dc95f295bdb5c3f60e880dd9aa5112dc5c071cc.178275787… Signed-off-by: Jakub Kicinski <kuba(a)kernel.org> Signed-off-by: Sasha Levin <sashal(a)kernel.org> Conflicts: net/sctp/protocol.c [Just context conflicts] Signed-off-by: Zhang Qilong <zhangqilong3(a)huawei.com> --- net/sctp/protocol.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/net/sctp/protocol.c b/net/sctp/protocol.c index 0beb9a82c2a3..a862644a6caa 100644 --- a/net/sctp/protocol.c +++ b/net/sctp/protocol.c @@ -690,12 +690,13 @@ static void sctp_addr_wq_timeout_handler(struct timer_list *t) static void sctp_free_addr_wq(struct net *net) { struct sctp_sockaddr_entry *addrw; struct sctp_sockaddr_entry *temp; + timer_shutdown_sync(&net->sctp.addr_wq_timer); + spin_lock_bh(&net->sctp.addr_wq_lock); - del_timer(&net->sctp.addr_wq_timer); list_for_each_entry_safe(addrw, temp, &net->sctp.addr_waitq, list) { list_del(&addrw->list); kfree(addrw); } spin_unlock_bh(&net->sctp.addr_wq_lock); -- 2.43.0

2 1

[PATCH OLK-6.6 0/3] md/raid1,raid10: fix deadlock and bio accounting in read error path
by Zhou Minqiang 08 Sep '26

08 Sep '26

Splitting a bio while executing in the raid1 thread can lead to recursion, as task->bio_list is NULL in this context. In addition, resubmitting an md_cloned_bio after splitting may lead to a deadlock if the array is suspended before the md driver calls percpu_ref_tryget_live(&mddev->active_io) on it's path to pers->make_request(). Avoid splitting the bio in this context and require that it is either read in full or not at all. This prevents recursion and avoids potential deadlocks during array suspension. Zhou Minqiang (3): md/raid1,raid10: fix deadlock in read error recovery path md/raid1,raid10: fix error-path detection with md_cloned_bio() md/raid1,raid10: fix bio accounting for split md cloned bios drivers/md/md.c | 40 ++++++++++++++++++++++++---------------- drivers/md/md.h | 5 +++++ drivers/md/raid1.c | 15 +++++++++++---- drivers/md/raid10.c | 28 ++++++++++++++++++---------- 4 files changed, 58 insertions(+), 30 deletions(-) -- 2.52.0

2 4

[PATCH OLK-6.6] enic: fix tx_hang_reset use-after-free on device removal
by Jinjiang Tu 08 Sep '26

08 Sep '26

From: Satish Kharat <satishkh(a)cisco.com> stable inclusion from stable-v6.12.104 commit 8619865f34fb3b130b567855382a5c4aadd522b9 category: bugfix bugzilla: https://atomgit.com/src-openeuler/kernel/issues/18260 CVE: CVE-2026-74725 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id… -------------------------------- [ Upstream commit ec680ea4ba1bca92a767fb7e7869758bfdd886e3 ] enic_remove() cancels the reset and change_mtu_work items but does not cancel tx_hang_reset. A TX timeout that fires while the device is being removed can schedule enic_tx_hang_reset() so that it runs after free_netdev(), resulting in a use-after-free. cancel_work_sync() alone is not sufficient here: the still-live watchdog and notify paths can re-schedule these work items in the window between the cancel and unregister_netdev(). Use disable_work_sync(), which cancels the work and blocks any subsequent schedule_work() from requeuing it, and apply it to the reset and change_mtu_work items as well so the same requeue race is closed for all teardown work. Fixes: 937317c7c109 ("enic: do hang reset only in case of tx timeout") Signed-off-by: Satish Kharat <satishkh(a)cisco.com> Link: https://patch.msgid.link/20260728062730.2394873-1-satishkh@cisco.com Signed-off-by: Jakub Kicinski <kuba(a)kernel.org> Signed-off-by: Sasha Levin <sashal(a)kernel.org> Signed-off-by: Jinjiang Tu <tujinjiang(a)huawei.com> --- drivers/net/ethernet/cisco/enic/enic_main.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/cisco/enic/enic_main.c b/drivers/net/ethernet/cisco/enic/enic_main.c index 48701032c20c..11cf6709b7c4 100644 --- a/drivers/net/ethernet/cisco/enic/enic_main.c +++ b/drivers/net/ethernet/cisco/enic/enic_main.c @@ -3016,8 +3016,9 @@ static void enic_remove(struct pci_dev *pdev) if (netdev) { struct enic *enic = netdev_priv(netdev); - cancel_work_sync(&enic->reset); - cancel_work_sync(&enic->change_mtu_work); + disable_work_sync(&enic->reset); + disable_work_sync(&enic->tx_hang_reset); + disable_work_sync(&enic->change_mtu_work); unregister_netdev(netdev); enic_dev_deinit(enic); vnic_dev_close(enic->vdev); -- 2.43.0

2 1

[PATCH OLK-5.10] dm-ioctl: fix a possible overflow in list_version_get_info
by Zhou Minqiang 08 Sep '26

08 Sep '26

mainline inclusion from mainline-v7.2-rc3 commit 76c6f845dc0c614304a6e6ee619b552f97cf24b3 category: bugfix bugzilla: https://atomgit.com/src-openeuler/kernel/issues/17390 CVE: CVE-2026-72106 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?… -------------------------------- commit 76c6f845dc0c614304a6e6ee619b552f97cf24b3 upstream. sizeof(tt->version) is 12 bytes, but the code writes 16 bytes into the output buffer - info->vers->version[0], info->vers->version[1], info->vers->version[2] and info->vers->next. This can cause buffer overflow. Fix this buffer overflow by replacing "sizeof(tt->version)" with "sizeof(struct dm_target_versions)". Signed-off-by: Mikulas Patocka <mpatocka(a)redhat.com> Assisted-by: Claude:claude-opus-4.6 Cc: stable(a)vger.kernel.org Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> Conflicts: drivers/md/dm-ioctl.c [ctx conflicts] Signed-off-by: Zhou Minqiang <zhouminqiang2(a)huawei.com> --- drivers/md/dm-ioctl.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/drivers/md/dm-ioctl.c b/drivers/md/dm-ioctl.c index 3bb9c0dced6e..2665e893cf22 100644 --- a/drivers/md/dm-ioctl.c +++ b/drivers/md/dm-ioctl.c @@ -585,9 +585,7 @@ static void list_version_get_info(struct target_type *tt, void *param) struct vers_iter *info = param; /* Check space - it might have changed since the first iteration */ - if ((char *)info->vers + sizeof(tt->version) + strlen(tt->name) + 1 > - info->end) { - + if ((char *)info->vers + sizeof(struct dm_target_versions) + strlen(tt->name) + 1 > info->end) { info->flags = DM_BUFFER_FULL_FLAG; return; } -- 2.52.0

2 1

[PATCH OLK-5.10 0/3] CVE-2026-74730
by Zhou Minqiang 08 Sep '26

08 Sep '26

CVE-2026-74730 Zhou Minqiang (3): nfs4: take a reference on the nfs_client when running FREE_STATEID NFS: Pin the 'struct nfs_server' during a FREE_STATEID call NFS: Decrement refcounts if allocating nfs_free_stateid_data fails fs/nfs/nfs4proc.c | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-) -- 2.52.0

2 4

[PATCH openEuler-1.0-LTS 0/3] CVE-2026-74730
by Zhou Minqiang 08 Sep '26

08 Sep '26

CVE-2026-74730 Zhou Minqiang (3): nfs4: take a reference on the nfs_client when running FREE_STATEID NFS: Pin the 'struct nfs_server' during a FREE_STATEID call NFS: Decrement refcounts if allocating nfs_free_stateid_data fails fs/nfs/nfs4proc.c | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-) -- 2.52.0

2 4

[PATCH OLK-6.6 0/3] md/raid1,raid10: fix deadlock and bio accounting in read error path
by Zhou Minqiang 08 Sep '26

08 Sep '26

Splitting a bio while executing in the raid1 thread can lead to recursion, as task->bio_list is NULL in this context. In addition, resubmitting an md_cloned_bio after splitting may lead to a deadlock if the array is suspended before the md driver calls percpu_ref_tryget_live(&mddev->active_io) on it's path to pers->make_request(). Avoid splitting the bio in this context and require that it is either read in full or not at all. This prevents recursion and avoids potential deadlocks during array suspension. Zhou Minqiang (3): md/raid1,raid10: fix deadlock in read error recovery path md/raid1,raid10: fix error-path detection with md_cloned_bio() md/raid1,raid10: fix bio accounting for split md cloned bios drivers/md/md.c | 40 ++++++++++++++++++++++++---------------- drivers/md/md.h | 5 +++++ drivers/md/raid1.c | 15 +++++++++++---- drivers/md/raid10.c | 28 ++++++++++++++++++---------- 4 files changed, 58 insertions(+), 30 deletions(-) -- 2.52.0

2 4

[PATCH OLK-6.6] bpf: Fix NMI/tracepoint re-entry deadlock on lru locks
by Chen Yuxi 08 Sep '26

08 Sep '26

From: Mykyta Yatsenko <yatsenko(a)meta.com> mainline inclusion from mainline-v7.2-rc1 commit 89edbdfc5d0308cef57b71359331de5c4ddbf763 category: bugfix bugzilla: https://atomgit.com/src-openeuler/kernel/issues/17962 CVE: CVE-2026-74337 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?… -------------------------------- NMI and tracepoint BPF programs can re-enter the per-CPU or global LRU lock that bpf_lru_pop_free()/push_free() already hold on the same CPU, AA-deadlocking. Lockdep reports "inconsistent {INITIAL USE} -> {IN-NMI}" on &l->lock (syzbot c69a0a2c816716f1e0d5) and "possible recursive locking detected" on &loc_l->lock (syzbot 18b26edb69b2e19f3b33). Prior trylock and rqspinlock based fixes (see links) were nacked because compromised on reliability. This patch converts every LRU lock site to rqspinlock_t and adds a recovery path for some failure windows to avoid node leaks. Failure recovery: - *_pop_free top-level: return NULL; prealloc_lru_pop() already treats that as no-free-element (-ENOMEM). - Cross-CPU steal: skip the victim's locked loc_l, try next CPU. - Post-steal local lock fail: publish stolen node to lockless per-CPU free_llist; next pop on this CPU picks it up. - push_free fail: mark node pending_free=1. __local_list_flush(), __local_list_pop_pending() reclaim the node from pending_list. __bpf_lru_list_shrink_inactive() reclaims the node from inactive list. Nodes from active list are reclaimed by __bpf_lru_list_shrink() or after __bpf_lru_list_rotate_active() demotes it to the inactive. Fixes: 3a08c2fd7634 ("bpf: LRU List") Reported-by: syzbot+c69a0a2c816716f1e0d5(a)syzkaller.appspotmail.com Closes: https://syzkaller.appspot.com/bug?extid=c69a0a2c816716f1e0d5 Reported-by: syzbot+18b26edb69b2e19f3b33(a)syzkaller.appspotmail.com Closes: https://syzkaller.appspot.com/bug?extid=18b26edb69b2e19f3b33 Link: https://lore.kernel.org/bpf/CAPPBnEYO4R+m+SpVc2gNj_x31R6fo1uJvj2bK2YS1P09GW… Link: https://lore.kernel.org/bpf/CAPPBnEZmFA3ab8Uc=PEm0bdojZy=7T_F5_+eyZSHyZR3MB… Link: https://lore.kernel.org/bpf/20251030030010.95352-1-dongml2@chinatelecom.cn/ Link: https://lore.kernel.org/bpf/20260119142120.28170-1-leon.hwang@linux.dev/ Signed-off-by: Mykyta Yatsenko <yatsenko(a)meta.com> Link: https://lore.kernel.org/r/20260607-lru_map_spin-v3-1-bcd9332e911b@meta.com Signed-off-by: Alexei Starovoitov <ast(a)kernel.org> Conflicts: kernel/bpf/bpf_lru_list.c, kernel/bpf/bpf_lru_list.h [Since hulk-6.6 does not include rqspinlock, this patch does not use the method from mainline, and instead adopts the re-entry rejection form.] Signed-off-by: Chen Yuxi <chenyuxi19(a)huawei.com> --- kernel/bpf/bpf_lru_list.c | 268 +++++++++++++++++++++++++++----------- kernel/bpf/bpf_lru_list.h | 33 ++++- 2 files changed, 223 insertions(+), 78 deletions(-) diff --git a/kernel/bpf/bpf_lru_list.c b/kernel/bpf/bpf_lru_list.c index 2d6e1c98d8ad..aac697d1d593 100644 --- a/kernel/bpf/bpf_lru_list.c +++ b/kernel/bpf/bpf_lru_list.c @@ -4,6 +4,7 @@ #include <linux/cpumask.h> #include <linux/spinlock.h> #include <linux/percpu.h> +#include <linux/llist.h> #include "bpf_lru_list.h" @@ -13,29 +14,75 @@ #define PERCPU_FREE_TARGET (4) #define PERCPU_NR_SCANS PERCPU_FREE_TARGET -/* Helpers to get the local list index */ -#define LOCAL_LIST_IDX(t) ((t) - BPF_LOCAL_LIST_T_OFFSET) -#define LOCAL_FREE_LIST_IDX LOCAL_LIST_IDX(BPF_LRU_LOCAL_LIST_T_FREE) -#define LOCAL_PENDING_LIST_IDX LOCAL_LIST_IDX(BPF_LRU_LOCAL_LIST_T_PENDING) #define IS_LOCAL_LIST_TYPE(t) ((t) >= BPF_LOCAL_LIST_T_OFFSET) -static int get_next_cpu(int cpu) +/* + * Per-CPU spinlock that rejects same-CPU re-entry. + * + * Same-CPU re-entry would AA-deadlock a plain raw spinlock, so an owner + * mark rejects it and lets the caller run its recovery path; cross-CPU + * contention blocks on the raw spinlock. + */ +static int bpf_lru_spin_lock_init(struct bpf_lru_spin_lock *l) { - cpu = cpumask_next(cpu, cpu_possible_mask); - if (cpu >= nr_cpu_ids) - cpu = cpumask_first(cpu_possible_mask); - return cpu; + raw_spin_lock_init(&l->lock); + l->nest = alloc_percpu(int); + if (!l->nest) + return -ENOMEM; + return 0; +} + +static void bpf_lru_spin_lock_destroy(struct bpf_lru_spin_lock *l) +{ + free_percpu(l->nest); +} + +static int bpf_lru_spin_lock(struct bpf_lru_spin_lock *l) +{ + preempt_disable(); + if (this_cpu_inc_return(*l->nest) != 1) { + this_cpu_dec(*l->nest); + preempt_enable(); + return -EBUSY; + } + raw_spin_lock(&l->lock); + return 0; +} + +static void bpf_lru_spin_unlock(struct bpf_lru_spin_lock *l) +{ + raw_spin_unlock(&l->lock); + this_cpu_dec(*l->nest); + preempt_enable(); } -/* Local list helpers */ -static struct list_head *local_free_list(struct bpf_lru_locallist *loc_l) +static int bpf_lru_spin_lock_irqsave(struct bpf_lru_spin_lock *l, + unsigned long *flags) { - return &loc_l->lists[LOCAL_FREE_LIST_IDX]; + preempt_disable(); + if (this_cpu_inc_return(*l->nest) != 1) { + this_cpu_dec(*l->nest); + preempt_enable(); + return -EBUSY; + } + raw_spin_lock_irqsave(&l->lock, *flags); + return 0; } -static struct list_head *local_pending_list(struct bpf_lru_locallist *loc_l) +static void bpf_lru_spin_unlock_irqrestore(struct bpf_lru_spin_lock *l, + unsigned long flags) { - return &loc_l->lists[LOCAL_PENDING_LIST_IDX]; + raw_spin_unlock_irqrestore(&l->lock, flags); + this_cpu_dec(*l->nest); + preempt_enable(); +} + +static int get_next_cpu(int cpu) +{ + cpu = cpumask_next(cpu, cpu_possible_mask); + if (cpu >= nr_cpu_ids) + cpu = cpumask_first(cpu_possible_mask); + return cpu; } /* bpf_lru_node helpers */ @@ -80,6 +127,7 @@ static void __bpf_lru_node_move_to_free(struct bpf_lru_list *l, bpf_lru_list_count_dec(l, node->type); node->type = tgt_free_type; + WRITE_ONCE(node->pending_free, 0); list_move(&node->list, free_list); } @@ -95,6 +143,9 @@ static void __bpf_lru_node_move_in(struct bpf_lru_list *l, bpf_lru_list_count_inc(l, tgt_type); node->type = tgt_type; bpf_lru_node_clear_ref(node); + /* Reset pending_free only when moving to the free list */ + if (tgt_type == BPF_LRU_LIST_T_FREE) + WRITE_ONCE(node->pending_free, 0); list_move(&node->list, &l->lists[tgt_type]); } @@ -220,9 +271,11 @@ __bpf_lru_list_shrink_inactive(struct bpf_lru *lru, unsigned int i = 0; list_for_each_entry_safe_reverse(node, tmp_node, inactive, list) { - if (bpf_lru_node_is_ref(node)) { + if (bpf_lru_node_is_ref(node) && + !READ_ONCE(node->pending_free)) { __bpf_lru_node_move(l, node, BPF_LRU_LIST_T_ACTIVE); - } else if (lru->del_from_htab(lru->del_arg, node)) { + } else if (READ_ONCE(node->pending_free) || + lru->del_from_htab(lru->del_arg, node)) { __bpf_lru_node_move_to_free(l, node, free_list, tgt_free_type); if (++nshrinked == tgt_nshrink) @@ -281,7 +334,8 @@ static unsigned int __bpf_lru_list_shrink(struct bpf_lru *lru, list_for_each_entry_safe_reverse(node, tmp_node, force_shrink_list, list) { - if (lru->del_from_htab(lru->del_arg, node)) { + if (READ_ONCE(node->pending_free) || + lru->del_from_htab(lru->del_arg, node)) { __bpf_lru_node_move_to_free(l, node, free_list, tgt_free_type); return 1; @@ -298,8 +352,10 @@ static void __local_list_flush(struct bpf_lru_list *l, struct bpf_lru_node *node, *tmp_node; list_for_each_entry_safe_reverse(node, tmp_node, - local_pending_list(loc_l), list) { - if (bpf_lru_node_is_ref(node)) + &loc_l->pending_list, list) { + if (READ_ONCE(node->pending_free)) + __bpf_lru_node_move_in(l, node, BPF_LRU_LIST_T_FREE); + else if (bpf_lru_node_is_ref(node)) __bpf_lru_node_move_in(l, node, BPF_LRU_LIST_T_ACTIVE); else __bpf_lru_node_move_in(l, node, @@ -315,9 +371,12 @@ static void bpf_lru_list_push_free(struct bpf_lru_list *l, if (WARN_ON_ONCE(IS_LOCAL_LIST_TYPE(node->type))) return; - raw_spin_lock_irqsave(&l->lock, flags); + if (bpf_lru_spin_lock_irqsave(&l->lock, &flags)) { + WRITE_ONCE(node->pending_free, 1); + return; + } __bpf_lru_node_move(l, node, BPF_LRU_LIST_T_FREE); - raw_spin_unlock_irqrestore(&l->lock, flags); + bpf_lru_spin_unlock_irqrestore(&l->lock, flags); } static void bpf_lru_list_pop_free_to_local(struct bpf_lru *lru, @@ -326,8 +385,10 @@ static void bpf_lru_list_pop_free_to_local(struct bpf_lru *lru, struct bpf_lru_list *l = &lru->common_lru.lru_list; struct bpf_lru_node *node, *tmp_node; unsigned int nfree = 0; + LIST_HEAD(tmp_free); - raw_spin_lock(&l->lock); + if (bpf_lru_spin_lock(&l->lock)) + return; __local_list_flush(l, loc_l); @@ -335,7 +396,7 @@ static void bpf_lru_list_pop_free_to_local(struct bpf_lru *lru, list_for_each_entry_safe(node, tmp_node, &l->lists[BPF_LRU_LIST_T_FREE], list) { - __bpf_lru_node_move_to_free(l, node, local_free_list(loc_l), + __bpf_lru_node_move_to_free(l, node, &tmp_free, BPF_LRU_LOCAL_LIST_T_FREE); if (++nfree == lru->target_free) break; @@ -343,10 +404,19 @@ static void bpf_lru_list_pop_free_to_local(struct bpf_lru *lru, if (nfree < lru->target_free) __bpf_lru_list_shrink(lru, l, lru->target_free - nfree, - local_free_list(loc_l), + &tmp_free, BPF_LRU_LOCAL_LIST_T_FREE); - raw_spin_unlock(&l->lock); + bpf_lru_spin_unlock(&l->lock); + + /* + * Transfer the harvested nodes from the temporary list_head into + * the lockless per-CPU free llist. + */ + list_for_each_entry_safe(node, tmp_node, &tmp_free, list) { + list_del(&node->list); + llist_add(&node->llist, &loc_l->free_llist); + } } static void __local_list_add_pending(struct bpf_lru *lru, @@ -358,22 +428,21 @@ static void __local_list_add_pending(struct bpf_lru *lru, *(u32 *)((void *)node + lru->hash_offset) = hash; node->cpu = cpu; node->type = BPF_LRU_LOCAL_LIST_T_PENDING; + WRITE_ONCE(node->pending_free, 0); bpf_lru_node_clear_ref(node); - list_add(&node->list, local_pending_list(loc_l)); + list_add(&node->list, &loc_l->pending_list); } static struct bpf_lru_node * __local_list_pop_free(struct bpf_lru_locallist *loc_l) { - struct bpf_lru_node *node; + struct llist_node *llnode; - node = list_first_entry_or_null(local_free_list(loc_l), - struct bpf_lru_node, - list); - if (node) - list_del(&node->list); + llnode = llist_del_first(&loc_l->free_llist); + if (!llnode) + return NULL; - return node; + return container_of(llnode, struct bpf_lru_node, llist); } static struct bpf_lru_node * @@ -384,10 +453,10 @@ __local_list_pop_pending(struct bpf_lru *lru, struct bpf_lru_locallist *loc_l) ignore_ref: /* Get from the tail (i.e. older element) of the pending list. */ - list_for_each_entry_reverse(node, local_pending_list(loc_l), - list) { + list_for_each_entry_reverse(node, &loc_l->pending_list, list) { if ((!bpf_lru_node_is_ref(node) || force) && - lru->del_from_htab(lru->del_arg, node)) { + (READ_ONCE(node->pending_free) || + lru->del_from_htab(lru->del_arg, node))) { list_del(&node->list); return node; } @@ -412,7 +481,8 @@ static struct bpf_lru_node *bpf_percpu_lru_pop_free(struct bpf_lru *lru, l = per_cpu_ptr(lru->percpu_lru, cpu); - raw_spin_lock_irqsave(&l->lock, flags); + if (bpf_lru_spin_lock_irqsave(&l->lock, &flags)) + return NULL; __bpf_lru_list_rotate(lru, l); @@ -428,7 +498,7 @@ static struct bpf_lru_node *bpf_percpu_lru_pop_free(struct bpf_lru *lru, __bpf_lru_node_move(l, node, BPF_LRU_LIST_T_INACTIVE); } - raw_spin_unlock_irqrestore(&l->lock, flags); + bpf_lru_spin_unlock_irqrestore(&l->lock, flags); return node; } @@ -445,7 +515,8 @@ static struct bpf_lru_node *bpf_common_lru_pop_free(struct bpf_lru *lru, loc_l = per_cpu_ptr(clru->local_list, cpu); - raw_spin_lock_irqsave(&loc_l->lock, flags); + if (bpf_lru_spin_lock_irqsave(&loc_l->lock, &flags)) + return NULL; node = __local_list_pop_free(loc_l); if (!node) { @@ -456,17 +527,22 @@ static struct bpf_lru_node *bpf_common_lru_pop_free(struct bpf_lru *lru, if (node) __local_list_add_pending(lru, loc_l, cpu, node, hash); - raw_spin_unlock_irqrestore(&loc_l->lock, flags); + bpf_lru_spin_unlock_irqrestore(&loc_l->lock, flags); if (node) return node; - /* No free nodes found from the local free list and + /* + * No free nodes found from the local free list and * the global LRU list. * * Steal from the local free/pending list of the * current CPU and remote CPU in RR. It starts * with the loc_l->next_steal CPU. + * + * Acquire the victim's lock before touching either list. On + * acquisition failure (rqspinlock AA or timeout) skip the victim + * and try the next CPU. */ first_steal = loc_l->next_steal; @@ -474,24 +550,36 @@ static struct bpf_lru_node *bpf_common_lru_pop_free(struct bpf_lru *lru, do { steal_loc_l = per_cpu_ptr(clru->local_list, steal); - raw_spin_lock_irqsave(&steal_loc_l->lock, flags); - - node = __local_list_pop_free(steal_loc_l); - if (!node) - node = __local_list_pop_pending(lru, steal_loc_l); - - raw_spin_unlock_irqrestore(&steal_loc_l->lock, flags); + if (!bpf_lru_spin_lock_irqsave(&steal_loc_l->lock, &flags)) { + node = __local_list_pop_free(steal_loc_l); + if (!node) + node = __local_list_pop_pending(lru, steal_loc_l); + bpf_lru_spin_unlock_irqrestore(&steal_loc_l->lock, flags); + } steal = get_next_cpu(steal); } while (!node && steal != first_steal); loc_l->next_steal = steal; - if (node) { - raw_spin_lock_irqsave(&loc_l->lock, flags); - __local_list_add_pending(lru, loc_l, cpu, node, hash); - raw_spin_unlock_irqrestore(&loc_l->lock, flags); + if (!node) + return NULL; + + if (bpf_lru_spin_lock_irqsave(&loc_l->lock, &flags)) { + /* + * The local pending lock can't be acquired (rqspinlock AA + * or timeout). Return the stolen node to the per-CPU + * free_llist instead of orphaning it; the next pop_free on + * this CPU will pick it up. + */ + node->type = BPF_LRU_LOCAL_LIST_T_FREE; + bpf_lru_node_clear_ref(node); + WRITE_ONCE(node->pending_free, 0); + llist_add(&node->llist, &loc_l->free_llist); + return NULL; } + __local_list_add_pending(lru, loc_l, cpu, node, hash); + bpf_lru_spin_unlock_irqrestore(&loc_l->lock, flags); return node; } @@ -519,18 +607,23 @@ static void bpf_common_lru_push_free(struct bpf_lru *lru, loc_l = per_cpu_ptr(lru->common_lru.local_list, node->cpu); - raw_spin_lock_irqsave(&loc_l->lock, flags); + if (bpf_lru_spin_lock_irqsave(&loc_l->lock, &flags)) { + WRITE_ONCE(node->pending_free, 1); + return; + } if (unlikely(node->type != BPF_LRU_LOCAL_LIST_T_PENDING)) { - raw_spin_unlock_irqrestore(&loc_l->lock, flags); + bpf_lru_spin_unlock_irqrestore(&loc_l->lock, flags); goto check_lru_list; } node->type = BPF_LRU_LOCAL_LIST_T_FREE; bpf_lru_node_clear_ref(node); - list_move(&node->list, local_free_list(loc_l)); + list_del(&node->list); - raw_spin_unlock_irqrestore(&loc_l->lock, flags); + bpf_lru_spin_unlock_irqrestore(&loc_l->lock, flags); + + llist_add(&node->llist, &loc_l->free_llist); return; } @@ -546,11 +639,14 @@ static void bpf_percpu_lru_push_free(struct bpf_lru *lru, l = per_cpu_ptr(lru->percpu_lru, node->cpu); - raw_spin_lock_irqsave(&l->lock, flags); + if (bpf_lru_spin_lock_irqsave(&l->lock, &flags)) { + WRITE_ONCE(node->pending_free, 1); + return; + } __bpf_lru_node_move(l, node, BPF_LRU_LIST_T_FREE); - raw_spin_unlock_irqrestore(&l->lock, flags); + bpf_lru_spin_unlock_irqrestore(&l->lock, flags); } void bpf_lru_push_free(struct bpf_lru *lru, struct bpf_lru_node *node) @@ -573,6 +669,7 @@ static void bpf_common_lru_populate(struct bpf_lru *lru, void *buf, node = (struct bpf_lru_node *)(buf + node_offset); node->type = BPF_LRU_LIST_T_FREE; + node->pending_free = 0; bpf_lru_node_clear_ref(node); list_add(&node->list, &l->lists[BPF_LRU_LIST_T_FREE]); buf += elem_size; @@ -602,6 +699,7 @@ static void bpf_percpu_lru_populate(struct bpf_lru *lru, void *buf, node = (struct bpf_lru_node *)(buf + node_offset); node->cpu = cpu; node->type = BPF_LRU_LIST_T_FREE; + node->pending_free = 0; bpf_lru_node_clear_ref(node); list_add(&node->list, &l->lists[BPF_LRU_LIST_T_FREE]); i++; @@ -624,19 +722,17 @@ void bpf_lru_populate(struct bpf_lru *lru, void *buf, u32 node_offset, nr_elems); } -static void bpf_lru_locallist_init(struct bpf_lru_locallist *loc_l, int cpu) +static int bpf_lru_locallist_init(struct bpf_lru_locallist *loc_l, int cpu) { - int i; - - for (i = 0; i < NR_BPF_LRU_LOCAL_LIST_T; i++) - INIT_LIST_HEAD(&loc_l->lists[i]); + INIT_LIST_HEAD(&loc_l->pending_list); + init_llist_head(&loc_l->free_llist); loc_l->next_steal = cpu; - raw_spin_lock_init(&loc_l->lock); + return bpf_lru_spin_lock_init(&loc_l->lock); } -static void bpf_lru_list_init(struct bpf_lru_list *l) +static int bpf_lru_list_init(struct bpf_lru_list *l) { int i; @@ -648,13 +744,15 @@ static void bpf_lru_list_init(struct bpf_lru_list *l) l->next_inactive_rotation = &l->lists[BPF_LRU_LIST_T_INACTIVE]; - raw_spin_lock_init(&l->lock); + return bpf_lru_spin_lock_init(&l->lock); } int bpf_lru_init(struct bpf_lru *lru, bool percpu, u32 hash_offset, del_from_htab_func del_from_htab, void *del_arg) { - int cpu; + int cpu, err; + + lru->percpu = percpu; if (percpu) { lru->percpu_lru = alloc_percpu(struct bpf_lru_list); @@ -665,7 +763,9 @@ int bpf_lru_init(struct bpf_lru *lru, bool percpu, u32 hash_offset, struct bpf_lru_list *l; l = per_cpu_ptr(lru->percpu_lru, cpu); - bpf_lru_list_init(l); + err = bpf_lru_list_init(l); + if (err) + goto out; } lru->nr_scans = PERCPU_NR_SCANS; } else { @@ -679,25 +779,47 @@ int bpf_lru_init(struct bpf_lru *lru, bool percpu, u32 hash_offset, struct bpf_lru_locallist *loc_l; loc_l = per_cpu_ptr(clru->local_list, cpu); - bpf_lru_locallist_init(loc_l, cpu); + err = bpf_lru_locallist_init(loc_l, cpu); + if (err) + goto out; } - bpf_lru_list_init(&clru->lru_list); + err = bpf_lru_list_init(&clru->lru_list); + if (err) + goto out; lru->nr_scans = LOCAL_NR_SCANS; } - lru->percpu = percpu; lru->del_from_htab = del_from_htab; lru->del_arg = del_arg; lru->hash_offset = hash_offset; return 0; +out: + bpf_lru_destroy(lru); + return err; } void bpf_lru_destroy(struct bpf_lru *lru) { - if (lru->percpu) + int cpu; + + if (lru->percpu) { + for_each_possible_cpu(cpu) { + struct bpf_lru_list *l; + + l = per_cpu_ptr(lru->percpu_lru, cpu); + bpf_lru_spin_lock_destroy(&l->lock); + } free_percpu(lru->percpu_lru); - else + } else { + for_each_possible_cpu(cpu) { + struct bpf_lru_locallist *loc_l; + + loc_l = per_cpu_ptr(lru->common_lru.local_list, cpu); + bpf_lru_spin_lock_destroy(&loc_l->lock); + } + bpf_lru_spin_lock_destroy(&lru->common_lru.lru_list.lock); free_percpu(lru->common_lru.local_list); + } } diff --git a/kernel/bpf/bpf_lru_list.h b/kernel/bpf/bpf_lru_list.h index fe2661a58ea9..6830147c7a6a 100644 --- a/kernel/bpf/bpf_lru_list.h +++ b/kernel/bpf/bpf_lru_list.h @@ -6,11 +6,12 @@ #include <linux/cache.h> #include <linux/list.h> +#include <linux/llist.h> +#include <linux/percpu.h> #include <linux/spinlock_types.h> #define NR_BPF_LRU_LIST_T (3) #define NR_BPF_LRU_LIST_COUNT (2) -#define NR_BPF_LRU_LOCAL_LIST_T (2) #define BPF_LOCAL_LIST_T_OFFSET NR_BPF_LRU_LIST_T enum bpf_lru_list_type { @@ -22,10 +23,31 @@ enum bpf_lru_list_type { }; struct bpf_lru_node { - struct list_head list; + /* + * A node is in at most one list at a time. The free path on the + * per-CPU locallist uses an llist, so share storage via a union. + */ + union { + struct list_head list; + struct llist_node llist; + }; u16 cpu; u8 type; u8 ref; + /* + * Marks nodes whose *_push_free() lock acquire failed; reclaimed + * by flush/shrink which honor the flag instead of del_from_htab(). + */ + u8 pending_free; +}; + +/* + * Mutually-exclusive per-CPU spinlock with re-entry rejection + * (see implementation in bpf_lru_list.c). + */ +struct bpf_lru_spin_lock { + raw_spinlock_t lock; + int __percpu *nest; }; struct bpf_lru_list { @@ -34,13 +56,14 @@ struct bpf_lru_list { /* The next inactive list rotation starts from here */ struct list_head *next_inactive_rotation; - raw_spinlock_t lock ____cacheline_aligned_in_smp; + struct bpf_lru_spin_lock lock ____cacheline_aligned_in_smp; }; struct bpf_lru_locallist { - struct list_head lists[NR_BPF_LRU_LOCAL_LIST_T]; + struct list_head pending_list; + struct llist_head free_llist; u16 next_steal; - raw_spinlock_t lock; + struct bpf_lru_spin_lock lock; }; struct bpf_common_lru { -- 2.34.1

2 1