- Kernel - mailweb.openeuler.org

[PATCH OLK-5.10] cifs: fix loff_t underflow in cifs_remap_file_range() when len == 0
by Zizhi Wo 18 Sep '26

18 Sep '26

From: Frank Sorenson <sorenson(a)redhat.com> mainline inclusion from mainline-v7.3-rc1 commit 6c322f5cf7476ded7a9a20f7be72462065a03c68 category: bugfix bugzilla: https://atomgit.com/src-openeuler/kernel/issues/18924 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id… -------------------------------- With len == 0 (clone to EOF), the effective length is computed as: len = src_inode->i_size - off; If off > i_size, this is a negative loff_t, corrupting the ByteCount in the FSCTL_DUPLICATE_EXTENTS_TO_FILE request and inverting the range in filemap_write_and_wait_range(). The existing off >= i_size check fires only after the ioctl has already been sent. Snapshot i_size_read() once for both the bounds check and the length calculation, eliminating the TOCTOU and 32-bit torn-read risk. Reject off > src_size with -EINVAL. Treat off == src_size as a no-op, consistent with __generic_remap_file_range_prep(). Fixes: 04b38d601239 ("vfs: pull btrfs clone API to vfs layer") Cc: stable(a)vger.kernel.org Signed-off-by: Frank Sorenson <sorenson(a)redhat.com> Reviewed-by: Namjae Jeon <linkinjeon(a)kernel.org> Signed-off-by: Paulo Alcantara <pc(a)manguebit.org> Conflicts: fs/cifs/cifsfs.c [5.10 keeps this file at fs/cifs/cifsfs.c and its cifs_remap_file_range() has no unlock label, which this fix references; add unlock: right before unlock_two_nondirectories() so the new error and zero-length exits release the locks and still free the xid via the existing out: path.] Signed-off-by: Zizhi Wo <wozizhi(a)huawei.com> --- fs/cifs/cifsfs.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/fs/cifs/cifsfs.c b/fs/cifs/cifsfs.c index a86a1fb34e59..1d102d8c3ba6 100644 --- a/fs/cifs/cifsfs.c +++ b/fs/cifs/cifsfs.c @@ -1116,12 +1116,23 @@ static loff_t cifs_remap_file_range(struct file *src_file, loff_t off, * checks for proper open modes and file type and if it wants * server could even support copy of range where source = target */ lock_two_nondirectories(target_inode, src_inode); - if (len == 0) - len = src_inode->i_size - off; + if (len == 0) { + loff_t src_size = i_size_read(src_inode); + + if (off > src_size) { + rc = -EINVAL; + goto unlock; + } + len = src_size - off; + if (!len) { + rc = 0; + goto unlock; + } + } cifs_dbg(FYI, "about to flush pages\n"); /* should we flush first and last page first */ truncate_inode_pages_range(&target_inode->i_data, destoff, PAGE_ALIGN(destoff + len)-1); @@ -1133,10 +1144,11 @@ static loff_t cifs_remap_file_range(struct file *src_file, loff_t off, rc = -EOPNOTSUPP; /* force revalidate of size and timestamps of target file now that target is updated on the server */ CIFS_I(target_inode)->time = 0; +unlock: /* although unlocking in the reverse order from locking is not strictly necessary here it is a little cleaner to be consistent */ unlock_two_nondirectories(src_inode, target_inode); out: free_xid(xid); -- 2.52.0

1 0

[PATCH OLK-6.6 0/5] mainline patch submission for OLK-6.6
by Chen Yuxi 18 Sep '26

18 Sep '26

Nicholas Dudar (2): bpf: Reject rdonly/rdwr_buf_size kfunc arguments that exceed u32 max bpf: Reject writes through untrusted BTF pointers Pu Lehui (2): bpf: Fix potential UAF when reading bpf link info bpf: Fix potential UAF in bpf_netns_link_update_prog Sanghyun Park (1): bpf: Fix use-after-free on mm_struct in bpf_find_vma() kernel/bpf/net_namespace.c | 17 ++++++----------- kernel/bpf/syscall.c | 21 +++++++++++++++++---- kernel/bpf/task_iter.c | 36 +++++++++++++++++++++++++++++++++--- kernel/bpf/verifier.c | 13 +++++++++++-- 4 files changed, 67 insertions(+), 20 deletions(-) -- 2.34.1

2 6

[PATCH OLK-6.6 0/5] mainline patch submission for OLK-6.6
by Chen Yuxi 18 Sep '26

18 Sep '26

Nicholas Dudar (2): bpf: Reject rdonly/rdwr_buf_size kfunc arguments that exceed u32 max bpf: Reject writes through untrusted BTF pointers Pu Lehui (2): bpf: Fix potential UAF when reading bpf link info bpf: Fix potential UAF in bpf_netns_link_update_prog Sanghyun Park (1): bpf: Fix use-after-free on mm_struct in bpf_find_vma() kernel/bpf/net_namespace.c | 17 ++++++----------- kernel/bpf/syscall.c | 21 +++++++++++++++++---- kernel/bpf/task_iter.c | 36 +++++++++++++++++++++++++++++++++--- kernel/bpf/verifier.c | 13 +++++++++++-- 4 files changed, 67 insertions(+), 20 deletions(-) -- 2.34.1

2 6

[PATCH openEuler-1.0-LTS] smb: client: clear setuid/setgid bit on write with cifsacl/modefromsid/posix extensions
by Zizhi Wo 18 Sep '26

18 Sep '26

From: Jiangshan Yi <yijiangshan(a)kylinos.cn> stable inclusion from stable-v6.18.51 commit b10015807e4c628095d1d1d1c9307efc8cdd9e1b category: bugfix bugzilla: https://atomgit.com/src-openeuler/kernel/issues/18923 CVE: CVE-2026-89638 Reference: https://web.git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit… -------------------------------- [ Upstream commit b8e5dc4f95e5484159b343903f302eb6d783f2e6 ] When a file has the setuid or setgid bit set and is written to, the VFS strips those bits and issues a setattr with ATTR_KILL_SUID/ATTR_KILL_SGID together with an ATTR_MODE carrying the already-cleared mode. Both cifs_setattr_unix() and cifs_setattr_nounix() unconditionally dropped ATTR_MODE in that case: /* skip mode change if it's just for clearing setuid/setgid */ if (attrs->ia_valid & (ATTR_KILL_SUID|ATTR_KILL_SGID)) attrs->ia_valid &= ~ATTR_MODE; This is fine for the default mount, where the mode is only emulated via the DOS read-only attribute and cannot represent the setuid/setgid bits anyway. However, with the "cifsacl" or "modefromsid" mount options the mode is stored on the server through an ACL (id_mode_to_cifs_acl()), with the SMB3.1.1 POSIX extensions the mode is sent to the server directly, and with the SMB1 Unix extensions (cifs_setattr_unix) the mode is sent via CIFSSMBUnixSetPathInfo(). In all those cases dropping ATTR_MODE means the cleared mode is never pushed to the server, so the setuid/setgid bit survives the write. This is a security issue: on local filesystems the setuid bit is stripped when a file is written, but over these cifs.ko mounts the bit persists on the server, potentially allowing an unexpected privilege escalation on subsequent execution. Fix this in two places: 1. cifs_setattr_nounix(): only take the "skip mode change" shortcut when the mode is emulated via the DOS read-only attribute (i.e. neither cifsacl/modefromsid nor the SMB3.1.1 POSIX extensions are in effect), so that the cleared mode is propagated to the server in the ACL / POSIX cases. 2. cifs_setattr_unix(): this function is only called when Unix extensions are in effect, so the mode is always stored on the server. Remove the shortcut entirely so that the cleared mode is always pushed. Fixes: d32c4f2626ac ("CIFS: ignore mode change if it's just for clearing setuid/setgid bits") Cc: stable(a)vger.kernel.org Signed-off-by: Jiangshan Yi <yijiangshan(a)kylinos.cn> Signed-off-by: Namjae Jeon <linkinjeon(a)kernel.org> Signed-off-by: Paulo Alcantara <pc(a)manguebit.org> Signed-off-by: Sasha Levin <sashal(a)kernel.org> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> Conflicts: fs/cifs/inode.c fs/smb/client/inode.c [1. Commit b8e5dc4f95e5 touches fs/smb/client/inode.c, which this tree still has as fs/cifs/inode.c. The new check used the upstream-local sbflags variable, which does not exist here, so use cifs_sb->mnt_cifs_flags instead. 2. Commit 412094a8fb07 ("smb3: add new mount option to retrieve mode from special ACE") introduce CIFS_MOUNT_MODE_FROM_SID, this version has not merged.] Signed-off-by: Zizhi Wo <wozizhi(a)huawei.com> Signed-off-by: Zizhi Wo <wozizhi(a)huawei.com> --- fs/cifs/inode.c | 29 ++++++++++++++++++++++++----- 1 file changed, 24 insertions(+), 5 deletions(-) diff --git a/fs/cifs/inode.c b/fs/cifs/inode.c index e5d0066f6125..793cd915ec1d 100644 --- a/fs/cifs/inode.c +++ b/fs/cifs/inode.c @@ -2290,12 +2290,27 @@ cifs_setattr_unix(struct dentry *direntry, struct iattr *attrs) rc = cifs_set_file_size(inode, attrs, xid, full_path); if (rc != 0) goto out; } - /* skip mode change if it's just for clearing setuid/setgid */ - if (attrs->ia_valid & (ATTR_KILL_SUID|ATTR_KILL_SGID)) + /* + * Skip the mode change if it is only being done to clear the + * setuid/setgid bits *and* the mode is emulated via the DOS + * read-only attribute (the default, non-ACL case), which cannot + * represent the setuid/setgid bits anyway. + * + * When the mode is instead stored on the server - i.e. with the + * cifsacl or modefromsid mount options (via an ACL) or with the + * SMB3.1.1 POSIX extensions - the cleared mode must be pushed to + * the server. Dropping ATTR_MODE here would leave the setuid/ + * setgid bit set on the server after a write, which is a security + * issue (the bits are not stripped as they are on local + * filesystems). + */ + if ((attrs->ia_valid & (ATTR_KILL_SUID|ATTR_KILL_SGID)) && + !((cifs_sb->mnt_cifs_flags & CIFS_MOUNT_CIFS_ACL) || + cifs_sb_master_tcon(cifs_sb)->posix_extensions)) attrs->ia_valid &= ~ATTR_MODE; args = kmalloc(sizeof(*args), GFP_KERNEL); if (args == NULL) { rc = -ENOMEM; @@ -2461,13 +2476,17 @@ cifs_setattr_nounix(struct dentry *direntry, struct iattr *attrs) } else #endif /* CONFIG_CIFS_ACL */ if (!(cifs_sb->mnt_cifs_flags & CIFS_MOUNT_SET_UID)) attrs->ia_valid &= ~(ATTR_UID | ATTR_GID); - /* skip mode change if it's just for clearing setuid/setgid */ - if (attrs->ia_valid & (ATTR_KILL_SUID|ATTR_KILL_SGID)) - attrs->ia_valid &= ~ATTR_MODE; + /* + * This function is only called when Unix extensions are in effect, + * so the mode is always sent to and stored on the server. Do not + * skip the mode change when clearing setuid/setgid bits: dropping + * ATTR_MODE here would leave those bits set on the server after a + * write, which is a security issue. + */ if (attrs->ia_valid & ATTR_MODE) { mode = attrs->ia_mode; rc = 0; #ifdef CONFIG_CIFS_ACL -- 2.52.0

2 1

[PATCH OLK-5.10] smb: client: clear setuid/setgid bit on write with cifsacl/modefromsid/posix extensions
by Zizhi Wo 18 Sep '26

18 Sep '26

From: Jiangshan Yi <yijiangshan(a)kylinos.cn> stable inclusion from stable-v6.18.51 commit b10015807e4c628095d1d1d1c9307efc8cdd9e1b category: bugfix bugzilla: https://atomgit.com/src-openeuler/kernel/issues/18923 CVE: CVE-2026-89638 Reference: https://web.git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit… -------------------------------- [ Upstream commit b8e5dc4f95e5484159b343903f302eb6d783f2e6 ] When a file has the setuid or setgid bit set and is written to, the VFS strips those bits and issues a setattr with ATTR_KILL_SUID/ATTR_KILL_SGID together with an ATTR_MODE carrying the already-cleared mode. Both cifs_setattr_unix() and cifs_setattr_nounix() unconditionally dropped ATTR_MODE in that case: /* skip mode change if it's just for clearing setuid/setgid */ if (attrs->ia_valid & (ATTR_KILL_SUID|ATTR_KILL_SGID)) attrs->ia_valid &= ~ATTR_MODE; This is fine for the default mount, where the mode is only emulated via the DOS read-only attribute and cannot represent the setuid/setgid bits anyway. However, with the "cifsacl" or "modefromsid" mount options the mode is stored on the server through an ACL (id_mode_to_cifs_acl()), with the SMB3.1.1 POSIX extensions the mode is sent to the server directly, and with the SMB1 Unix extensions (cifs_setattr_unix) the mode is sent via CIFSSMBUnixSetPathInfo(). In all those cases dropping ATTR_MODE means the cleared mode is never pushed to the server, so the setuid/setgid bit survives the write. This is a security issue: on local filesystems the setuid bit is stripped when a file is written, but over these cifs.ko mounts the bit persists on the server, potentially allowing an unexpected privilege escalation on subsequent execution. Fix this in two places: 1. cifs_setattr_nounix(): only take the "skip mode change" shortcut when the mode is emulated via the DOS read-only attribute (i.e. neither cifsacl/modefromsid nor the SMB3.1.1 POSIX extensions are in effect), so that the cleared mode is propagated to the server in the ACL / POSIX cases. 2. cifs_setattr_unix(): this function is only called when Unix extensions are in effect, so the mode is always stored on the server. Remove the shortcut entirely so that the cleared mode is always pushed. Fixes: d32c4f2626ac ("CIFS: ignore mode change if it's just for clearing setuid/setgid bits") Cc: stable(a)vger.kernel.org Signed-off-by: Jiangshan Yi <yijiangshan(a)kylinos.cn> Signed-off-by: Namjae Jeon <linkinjeon(a)kernel.org> Signed-off-by: Paulo Alcantara <pc(a)manguebit.org> Signed-off-by: Sasha Levin <sashal(a)kernel.org> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> Conflicts: fs/cifs/inode.c fs/smb/client/inode.c [Commit b8e5dc4f95e5 touches fs/smb/client/inode.c, which this tree still has as fs/cifs/inode.c. The new check used the upstream-local sbflags variable, which does not exist here, so use cifs_sb->mnt_cifs_flags instead.] Signed-off-by: Zizhi Wo <wozizhi(a)huawei.com> --- fs/cifs/inode.c | 30 +++++++++++++++++++++++++----- 1 file changed, 25 insertions(+), 5 deletions(-) diff --git a/fs/cifs/inode.c b/fs/cifs/inode.c index 045d77a929df..774acee6fb58 100644 --- a/fs/cifs/inode.c +++ b/fs/cifs/inode.c @@ -2662,12 +2662,28 @@ cifs_setattr_unix(struct dentry *direntry, struct iattr *attrs) rc = cifs_set_file_size(inode, attrs, xid, full_path); if (rc != 0) goto out; } - /* skip mode change if it's just for clearing setuid/setgid */ - if (attrs->ia_valid & (ATTR_KILL_SUID|ATTR_KILL_SGID)) + /* + * Skip the mode change if it is only being done to clear the + * setuid/setgid bits *and* the mode is emulated via the DOS + * read-only attribute (the default, non-ACL case), which cannot + * represent the setuid/setgid bits anyway. + * + * When the mode is instead stored on the server - i.e. with the + * cifsacl or modefromsid mount options (via an ACL) or with the + * SMB3.1.1 POSIX extensions - the cleared mode must be pushed to + * the server. Dropping ATTR_MODE here would leave the setuid/ + * setgid bit set on the server after a write, which is a security + * issue (the bits are not stripped as they are on local + * filesystems). + */ + if ((attrs->ia_valid & (ATTR_KILL_SUID|ATTR_KILL_SGID)) && + !((cifs_sb->mnt_cifs_flags & + (CIFS_MOUNT_CIFS_ACL | CIFS_MOUNT_MODE_FROM_SID)) || + cifs_sb_master_tcon(cifs_sb)->posix_extensions)) attrs->ia_valid &= ~ATTR_MODE; args = kmalloc(sizeof(*args), GFP_KERNEL); if (args == NULL) { rc = -ENOMEM; @@ -2848,13 +2864,17 @@ cifs_setattr_nounix(struct dentry *direntry, struct iattr *attrs) } } else if (!(cifs_sb->mnt_cifs_flags & CIFS_MOUNT_SET_UID)) attrs->ia_valid &= ~(ATTR_UID | ATTR_GID); - /* skip mode change if it's just for clearing setuid/setgid */ - if (attrs->ia_valid & (ATTR_KILL_SUID|ATTR_KILL_SGID)) - attrs->ia_valid &= ~ATTR_MODE; + /* + * This function is only called when Unix extensions are in effect, + * so the mode is always sent to and stored on the server. Do not + * skip the mode change when clearing setuid/setgid bits: dropping + * ATTR_MODE here would leave those bits set on the server after a + * write, which is a security issue. + */ if (attrs->ia_valid & ATTR_MODE) { mode = attrs->ia_mode; rc = 0; if ((cifs_sb->mnt_cifs_flags & CIFS_MOUNT_CIFS_ACL) || -- 2.52.0

2 2

[PATCH OLK-6.6] Revert "iomap: guard io_size EOF trim against concurrent truncate underflow"
by Ran Hongyun 18 Sep '26

18 Sep '26

hulk inclusion category: bugfix bugzilla: https://atomgit.com/src-openeuler/kernel/issues/17916 CVE: CVE-2026-72367 -------------------------------- This reverts commit 70ad22e4e8dfd119161fdaebc2bf4007f136516a. The upstream fix (55ec50d046c0) guards against unsigned underflow when `end_pos < io_offset` by clamping io_size to zero. In mainline, `end_pos` is a per-folio snapshot passed as a parameter. commit e728e754141f uses `isize = i_size_read(inode)` instead of `end_pos`: if (pos < isize && pos + len > isize) wpc->ioend->io_size = isize - wpc->ioend->io_offset; Since `io_offset <= pos < isize`, when the branch is taken, the subtraction cannot underflow. Fixes: 70ad22e4e8df ("iomap: guard io_size EOF trim against concurrent truncate underflow") Signed-off-by: Ran Hongyun <ranhongyun1(a)huawei.com> --- fs/iomap/buffered-io.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index baa7bbe02fad..1b9177ad7cdd 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -2002,12 +2002,8 @@ static int iomap_add_to_ioend(struct iomap_writepage_ctx *wpc, * should not be trimmed in such cases. */ wpc->ioend->io_size += len; - if (wpc->ioend->io_offset + wpc->ioend->io_size > isize) { - if (wpc->ioend->io_offset >= isize) - wpc->ioend->io_size = 0; - else - wpc->ioend->io_size = isize - wpc->ioend->io_offset; - } + if (pos < isize && pos + len > isize) + wpc->ioend->io_size = isize - wpc->ioend->io_offset; wbc_account_cgroup_owner(wbc, &folio->page, len); return 0; -- 2.52.0

2 1

[PATCH OLK-6.6 0/2] eventpoll: use spinlock under CONFIG_PREEMPT_RT, rwlock otherwise
by Ran Hongyun 18 Sep '26

18 Sep '26

Commit d26ee5945bbc ("eventpoll: Replace rwlock with spinlock") replaced ep->lock rwlock with spinlock to fix RT priority inversion. However, the rwlock read-side cannot be priority-boosted on PREEMPT_RT (rwbase_rt readers are not rt_mutex owners), while on non-RT kernels preempt_disable already guarantees bounded waiting. So the inversion is RT-only. Replacing rwlock with spinlock unconditionally causes performance regression on non-RT systems with many producers, since producers can no longer hold the read lock concurrently. This series reverts the unconditional replacement, then re-applies it under CONFIG_PREEMPT_RT via ep_lock_* wrapper macros: - RT: spinlock_t (PI chain complete, no inversion) - non-RT: rwlock_t (preserved for producer concurrency) The lockless helpers list_add_tail_lockless() and chain_epi_lockless() are compiled out under CONFIG_PREEMPT_RT. Ran Hongyun (2): Revert "eventpoll: Replace rwlock with spinlock" eventpoll: use spinlock under CONFIG_PREEMPT_RT, rwlock otherwise fs/eventpoll.c | 174 +++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 154 insertions(+), 20 deletions(-) -- 2.52.0

2 1

[PATCH OLK-6.6] md: use bio_alloc when sync_set is not initialized in md_super_write
by Ran Hongyun 18 Sep '26

18 Sep '26

hulk inclusion category: bugfix bugzilla: https://atomgit.com/openeuler/kernel/issues/9983 -------------------------------- Commit 6a5965696856 removed md_bio_alloc_sync and used bio_alloc_bioset instead. But when md_super_write is called after __md_stop has destroyed sync_set, bio_alloc_bioset with &mddev->sync_set triggers WARN_ON_ONCE because the bioset is not initialized. Fall back to bio_alloc (which uses fs_bio_set) when sync_set is not initialized. Fixes: 6a5965696856 ("md: remove md_bio_alloc_sync") Signed-off-by: Ran Hongyun <ranhongyun1(a)huawei.com> --- drivers/md/md.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index 2a32de5f5b64..a501fe675893 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -1075,6 +1075,9 @@ void md_super_write(struct mddev *mddev, struct md_rdev *rdev, * If an error occurred, call md_error */ struct bio *bio; + struct block_device *bdev; + blk_opf_t opf = REQ_OP_WRITE | REQ_SYNC | REQ_IDLE | REQ_META + | REQ_PREFLUSH | REQ_FUA; if (!page) return; @@ -1082,11 +1085,12 @@ void md_super_write(struct mddev *mddev, struct md_rdev *rdev, if (test_bit(Faulty, &rdev->flags)) return; - bio = bio_alloc_bioset(rdev->meta_bdev ? rdev->meta_bdev : rdev->bdev, - 1, - REQ_OP_WRITE | REQ_SYNC | REQ_IDLE | REQ_META - | REQ_PREFLUSH | REQ_FUA, - GFP_NOIO, &mddev->sync_set); + bdev = rdev->meta_bdev ? rdev->meta_bdev : rdev->bdev; + + if (bioset_initialized(&mddev->sync_set)) + bio = bio_alloc_bioset(bdev, 1, opf, GFP_NOIO, &mddev->sync_set); + else + bio = bio_alloc(bdev, 1, opf, GFP_NOIO); atomic_inc(&rdev->nr_pending); -- 2.52.0

2 1

[PATCH OLK-6.6 2/2] eventpoll: use spinlock under CONFIG_PREEMPT_RT, rwlock otherwise
by Ran Hongyun 18 Sep '26

18 Sep '26

hulk inclusion category: bugfix bugzilla: NA -------------------------------- Introduce ep_lock_* wrapper macros that select spinlock_t under CONFIG_PREEMPT_RT and rwlock_t otherwise. The rwlock read-side cannot be priority-boosted on RT (rwbase_rt readers are not rt_mutex owners), causing unbounded priority inversion; spinlock fixes this. On non-RT kernels the rwlock is preserved for producer concurrency. The lockless helpers list_add_tail_lockless() and chain_epi_lockless() are compiled out under CONFIG_PREEMPT_RT since the spinlock serializes all producers. Fixes: a218cc491420 ("epoll: use rwlock in order to reduce ep_poll_callback() contention") Signed-off-by: Ran Hongyun <ranhongyun1(a)huawei.com> --- fs/eventpoll.c | 95 +++++++++++++++++++++++++++++++++++++------------- 1 file changed, 71 insertions(+), 24 deletions(-) diff --git a/fs/eventpoll.c b/fs/eventpoll.c index ba719222a2da..523e69de1e32 100644 --- a/fs/eventpoll.c +++ b/fs/eventpoll.c @@ -39,13 +39,45 @@ #include <linux/rculist.h> #include <net/busy_poll.h> +/* + * Under CONFIG_PREEMPT_RT the ep->lock is a spinlock because the rwlock + * read-side cannot be priority-boosted (rwbase_rt readers are not + * rt_mutex owners), which causes unbounded priority inversion when a + * high-priority consumer (writer) waits on low-priority producers + * (readers). On non-RT kernels the rwlock is preserved so that multiple + * producers can run concurrently, which scales better with large event + * sources. + */ + +#ifdef CONFIG_PREEMPT_RT + +#define ep_lock_t spinlock_t +#define ep_lock_init(lock) spin_lock_init(lock) +#define ep_lock_irq(lock) spin_lock_irq(lock) +#define ep_unlock_irq(lock) spin_unlock_irq(lock) +#define ep_lock_irqsave(lock, flags) spin_lock_irqsave(lock, flags) +#define ep_unlock_irqrestore(lock, flags) \ + spin_unlock_irqrestore(lock, flags) + +#else /* !CONFIG_PREEMPT_RT */ + +#define ep_lock_t rwlock_t +#define ep_lock_init(lock) rwlock_init(lock) +#define ep_lock_irq(lock) write_lock_irq(lock) +#define ep_unlock_irq(lock) write_unlock_irq(lock) +#define ep_lock_irqsave(lock, flags) read_lock_irqsave(lock, flags) +#define ep_unlock_irqrestore(lock, flags) \ + read_unlock_irqrestore(lock, flags) + +#endif /* CONFIG_PREEMPT_RT */ + /* * LOCKING: * There are three level of locking required by epoll : * * 1) epnested_mutex (mutex) * 2) ep->mtx (mutex) - * 3) ep->lock (rwlock) + * 3) ep->lock (rwlock or spinlock, see above) * * The acquire order is the one listed above, from 1 to 3. * We need a rwlock (ep->lock) because we manipulate objects @@ -194,7 +226,7 @@ struct eventpoll { struct list_head rdllist; /* Lock which protects rdllist and ovflist */ - rwlock_t lock; + ep_lock_t lock; /* RB tree root used to store monitored fd structs */ struct rb_root_cached rbr; @@ -628,10 +660,10 @@ static void ep_start_scan(struct eventpoll *ep, struct list_head *txlist) * in a lockless way. */ lockdep_assert_irqs_enabled(); - write_lock_irq(&ep->lock); + ep_lock_irq(&ep->lock); list_splice_init(&ep->rdllist, txlist); WRITE_ONCE(ep->ovflist, NULL); - write_unlock_irq(&ep->lock); + ep_unlock_irq(&ep->lock); } static void ep_done_scan(struct eventpoll *ep, @@ -639,7 +671,7 @@ static void ep_done_scan(struct eventpoll *ep, { struct epitem *epi, *nepi; - write_lock_irq(&ep->lock); + ep_lock_irq(&ep->lock); /* * During the time we spent inside the "sproc" callback, some * other events might have been queued by the poll callback. @@ -680,7 +712,7 @@ static void ep_done_scan(struct eventpoll *ep, wake_up(&ep->wq); } - write_unlock_irq(&ep->lock); + ep_unlock_irq(&ep->lock); } static void epi_rcu_free(struct rcu_head *head) @@ -777,10 +809,10 @@ static bool ep_remove_epi(struct eventpoll *ep, struct epitem *epi) rb_erase_cached(&epi->rbn, &ep->rbr); - write_lock_irq(&ep->lock); + ep_lock_irq(&ep->lock); if (ep_is_linked(epi)) list_del_init(&epi->rdllink); - write_unlock_irq(&ep->lock); + ep_unlock_irq(&ep->lock); wakeup_source_unregister(ep_wakeup_source(epi)); /* @@ -1040,7 +1072,7 @@ static int ep_alloc(struct eventpoll **pep) return -ENOMEM; mutex_init(&ep->mtx); - rwlock_init(&ep->lock); + ep_lock_init(&ep->lock); init_waitqueue_head(&ep->wq); init_waitqueue_head(&ep->poll_wait); INIT_LIST_HEAD(&ep->rdllist); @@ -1127,6 +1159,7 @@ struct file *get_epoll_tfile_raw_ptr(struct file *file, int tfd, } #endif /* CONFIG_KCMP */ +#ifndef CONFIG_PREEMPT_RT /* * Adds a new entry to the tail of the list in a lockless way, i.e. * multiple CPUs are allowed to call this function concurrently. @@ -1204,16 +1237,17 @@ static inline bool chain_epi_lockless(struct epitem *epi) return true; } +#endif /* !CONFIG_PREEMPT_RT */ /* * This is the callback that is passed to the wait queue wakeup * mechanism. It is called by the stored file descriptors when they * have events to report. * - * This callback takes a read lock in order not to contend with concurrent - * events from another file descriptor, thus all modifications to ->rdllist - * or ->ovflist are lockless. Read lock is paired with the write lock from - * ep_start/done_scan(), which stops all list modifications and guarantees - * that lists state is seen correctly. + * Under !CONFIG_PREEMPT_RT this callback takes a read lock in order not + * to contend with concurrent events from another file descriptor, thus + * all modifications to ->rdllist or ->ovflist are lockless. Read lock is + * paired with the write lock from ep_start/done_scan(), which stops all + * list modifications and guarantees that lists state is seen correctly. * * Another thing worth to mention is that ep_poll_callback() can be called * concurrently for the same @epi from different CPUs if poll table was inited @@ -1231,7 +1265,7 @@ static int ep_poll_callback(wait_queue_entry_t *wait, unsigned mode, int sync, v unsigned long flags; int ewake = 0; - read_lock_irqsave(&ep->lock, flags); + ep_lock_irqsave(&ep->lock, flags); ep_set_busy_poll_napi_id(epi); @@ -1260,12 +1294,25 @@ static int ep_poll_callback(wait_queue_entry_t *wait, unsigned mode, int sync, v * chained in ep->ovflist and requeued later on. */ if (READ_ONCE(ep->ovflist) != EP_UNACTIVE_PTR) { +#ifdef CONFIG_PREEMPT_RT + if (epi->next == EP_UNACTIVE_PTR) { + epi->next = READ_ONCE(ep->ovflist); + WRITE_ONCE(ep->ovflist, epi); + ep_pm_stay_awake_rcu(epi); + } +#else if (chain_epi_lockless(epi)) ep_pm_stay_awake_rcu(epi); +#endif } else if (!ep_is_linked(epi)) { /* In the usual case, add event to ready list. */ +#ifdef CONFIG_PREEMPT_RT + list_add_tail(&epi->rdllink, &ep->rdllist); + ep_pm_stay_awake_rcu(epi); +#else if (list_add_tail_lockless(&epi->rdllink, &ep->rdllist)) ep_pm_stay_awake_rcu(epi); +#endif } /* @@ -1298,7 +1345,7 @@ static int ep_poll_callback(wait_queue_entry_t *wait, unsigned mode, int sync, v pwake++; out_unlock: - read_unlock_irqrestore(&ep->lock, flags); + ep_unlock_irqrestore(&ep->lock, flags); /* We have to call this outside the lock */ if (pwake) @@ -1633,7 +1680,7 @@ static int ep_insert(struct eventpoll *ep, const struct epoll_event *event, } /* We have to drop the new item inside our item list to keep track of it */ - write_lock_irq(&ep->lock); + ep_lock_irq(&ep->lock); /* record NAPI ID of new item if present */ ep_set_busy_poll_napi_id(epi); @@ -1650,7 +1697,7 @@ static int ep_insert(struct eventpoll *ep, const struct epoll_event *event, pwake++; } - write_unlock_irq(&ep->lock); + ep_unlock_irq(&ep->lock); /* We have to call this outside the lock */ if (pwake) @@ -1714,7 +1761,7 @@ static int ep_modify(struct eventpoll *ep, struct epitem *epi, * list, push it inside. */ if (ep_item_poll(epi, &pt, 1)) { - write_lock_irq(&ep->lock); + ep_lock_irq(&ep->lock); if (!ep_is_linked(epi)) { list_add_tail(&epi->rdllink, &ep->rdllist); ep_pm_stay_awake(epi); @@ -1725,7 +1772,7 @@ static int ep_modify(struct eventpoll *ep, struct epitem *epi, if (waitqueue_active(&ep->poll_wait)) pwake++; } - write_unlock_irq(&ep->lock); + ep_unlock_irq(&ep->lock); } /* We have to call this outside the lock */ @@ -1958,7 +2005,7 @@ static int ep_poll(struct eventpoll *ep, struct epoll_event __user *events, init_wait(&wait); wait.func = ep_autoremove_wake_function; - write_lock_irq(&ep->lock); + ep_lock_irq(&ep->lock); /* * Barrierless variant, waitqueue_active() is called under * the same lock on wakeup ep_poll_callback() side, so it @@ -1977,7 +2024,7 @@ static int ep_poll(struct eventpoll *ep, struct epoll_event __user *events, if (!eavail) __add_wait_queue_exclusive(&ep->wq, &wait); - write_unlock_irq(&ep->lock); + ep_unlock_irq(&ep->lock); if (!eavail) timed_out = !schedule_hrtimeout_range(to, slack, @@ -1992,7 +2039,7 @@ static int ep_poll(struct eventpoll *ep, struct epoll_event __user *events, eavail = 1; if (!list_empty_careful(&wait.entry)) { - write_lock_irq(&ep->lock); + ep_lock_irq(&ep->lock); /* * If the thread timed out and is not on the wait queue, * it means that the thread was woken up after its @@ -2003,7 +2050,7 @@ static int ep_poll(struct eventpoll *ep, struct epoll_event __user *events, if (timed_out) eavail = list_empty(&wait.entry); __remove_wait_queue(&ep->wq, &wait); - write_unlock_irq(&ep->lock); + ep_unlock_irq(&ep->lock); } } } -- 2.52.0

1 0

[PATCH OLK-6.6 1/2] Revert "eventpoll: Replace rwlock with spinlock"
by Ran Hongyun 18 Sep '26

18 Sep '26

hulk inclusion category: bugfix bugzilla: NA -------------------------------- This reverts commit d26ee5945bbcd8be44bf38853afe5d0b4e2e1e31. Commit d26ee5945bbc ("eventpoll: Replace rwlock with spinlock") replaced rwlock with spinlock to fix priority inversion. However, in eventpoll, the issue only affects RT. To avoid the performance regression on non-RT systems, revert this change in preparation for adding macro isolation to keep using rwlock on non-RT systems. Fixes: d26ee5945bbc ("eventpoll: Replace rwlock with spinlock") Signed-off-by: Ran Hongyun <ranhongyun1(a)huawei.com> --- fs/eventpoll.c | 139 ++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 113 insertions(+), 26 deletions(-) diff --git a/fs/eventpoll.c b/fs/eventpoll.c index 2cfee7d7b50c..ba719222a2da 100644 --- a/fs/eventpoll.c +++ b/fs/eventpoll.c @@ -45,10 +45,10 @@ * * 1) epnested_mutex (mutex) * 2) ep->mtx (mutex) - * 3) ep->lock (spinlock) + * 3) ep->lock (rwlock) * * The acquire order is the one listed above, from 1 to 3. - * We need a spinlock (ep->lock) because we manipulate objects + * We need a rwlock (ep->lock) because we manipulate objects * from inside the poll callback, that might be triggered from * a wake_up() that in turn might be called from IRQ context. * So we can't sleep inside the poll callback and hence we need @@ -194,7 +194,7 @@ struct eventpoll { struct list_head rdllist; /* Lock which protects rdllist and ovflist */ - spinlock_t lock; + rwlock_t lock; /* RB tree root used to store monitored fd structs */ struct rb_root_cached rbr; @@ -628,10 +628,10 @@ static void ep_start_scan(struct eventpoll *ep, struct list_head *txlist) * in a lockless way. */ lockdep_assert_irqs_enabled(); - spin_lock_irq(&ep->lock); + write_lock_irq(&ep->lock); list_splice_init(&ep->rdllist, txlist); WRITE_ONCE(ep->ovflist, NULL); - spin_unlock_irq(&ep->lock); + write_unlock_irq(&ep->lock); } static void ep_done_scan(struct eventpoll *ep, @@ -639,7 +639,7 @@ static void ep_done_scan(struct eventpoll *ep, { struct epitem *epi, *nepi; - spin_lock_irq(&ep->lock); + write_lock_irq(&ep->lock); /* * During the time we spent inside the "sproc" callback, some * other events might have been queued by the poll callback. @@ -680,7 +680,7 @@ static void ep_done_scan(struct eventpoll *ep, wake_up(&ep->wq); } - spin_unlock_irq(&ep->lock); + write_unlock_irq(&ep->lock); } static void epi_rcu_free(struct rcu_head *head) @@ -777,10 +777,10 @@ static bool ep_remove_epi(struct eventpoll *ep, struct epitem *epi) rb_erase_cached(&epi->rbn, &ep->rbr); - spin_lock_irq(&ep->lock); + write_lock_irq(&ep->lock); if (ep_is_linked(epi)) list_del_init(&epi->rdllink); - spin_unlock_irq(&ep->lock); + write_unlock_irq(&ep->lock); wakeup_source_unregister(ep_wakeup_source(epi)); /* @@ -1040,7 +1040,7 @@ static int ep_alloc(struct eventpoll **pep) return -ENOMEM; mutex_init(&ep->mtx); - spin_lock_init(&ep->lock); + rwlock_init(&ep->lock); init_waitqueue_head(&ep->wq); init_waitqueue_head(&ep->poll_wait); INIT_LIST_HEAD(&ep->rdllist); @@ -1127,10 +1127,100 @@ struct file *get_epoll_tfile_raw_ptr(struct file *file, int tfd, } #endif /* CONFIG_KCMP */ +/* + * Adds a new entry to the tail of the list in a lockless way, i.e. + * multiple CPUs are allowed to call this function concurrently. + * + * Beware: it is necessary to prevent any other modifications of the + * existing list until all changes are completed, in other words + * concurrent list_add_tail_lockless() calls should be protected + * with a read lock, where write lock acts as a barrier which + * makes sure all list_add_tail_lockless() calls are fully + * completed. + * + * Also an element can be locklessly added to the list only in one + * direction i.e. either to the tail or to the head, otherwise + * concurrent access will corrupt the list. + * + * Return: %false if element has been already added to the list, %true + * otherwise. + */ +static inline bool list_add_tail_lockless(struct list_head *new, + struct list_head *head) +{ + struct list_head *prev; + + /* + * This is simple 'new->next = head' operation, but cmpxchg() + * is used in order to detect that same element has been just + * added to the list from another CPU: the winner observes + * new->next == new. + */ + if (!try_cmpxchg(&new->next, &new, head)) + return false; + + /* + * Initially ->next of a new element must be updated with the head + * (we are inserting to the tail) and only then pointers are atomically + * exchanged. XCHG guarantees memory ordering, thus ->next should be + * updated before pointers are actually swapped and pointers are + * swapped before prev->next is updated. + */ + + prev = xchg(&head->prev, new); + + /* + * It is safe to modify prev->next and new->prev, because a new element + * is added only to the tail and new->next is updated before XCHG. + */ + + prev->next = new; + new->prev = prev; + + return true; +} + +/* + * Chains a new epi entry to the tail of the ep->ovflist in a lockless way, + * i.e. multiple CPUs are allowed to call this function concurrently. + * + * Return: %false if epi element has been already chained, %true otherwise. + */ +static inline bool chain_epi_lockless(struct epitem *epi) +{ + struct eventpoll *ep = epi->ep; + + /* Fast preliminary check */ + if (epi->next != EP_UNACTIVE_PTR) + return false; + + /* Check that the same epi has not been just chained from another CPU */ + if (cmpxchg(&epi->next, EP_UNACTIVE_PTR, NULL) != EP_UNACTIVE_PTR) + return false; + + /* Atomically exchange tail */ + epi->next = xchg(&ep->ovflist, epi); + + return true; +} + /* * This is the callback that is passed to the wait queue wakeup * mechanism. It is called by the stored file descriptors when they * have events to report. + * + * This callback takes a read lock in order not to contend with concurrent + * events from another file descriptor, thus all modifications to ->rdllist + * or ->ovflist are lockless. Read lock is paired with the write lock from + * ep_start/done_scan(), which stops all list modifications and guarantees + * that lists state is seen correctly. + * + * Another thing worth to mention is that ep_poll_callback() can be called + * concurrently for the same @epi from different CPUs if poll table was inited + * with several wait queues entries. Plural wakeup from different CPUs of a + * single wait queue is serialized by wq.lock, but the case when multiple wait + * queues are used should be detected accordingly. This is detected using + * cmpxchg() operation. */ static int ep_poll_callback(wait_queue_entry_t *wait, unsigned mode, int sync, void *key) { @@ -1141,7 +1231,7 @@ static int ep_poll_callback(wait_queue_entry_t *wait, unsigned mode, int sync, v unsigned long flags; int ewake = 0; - spin_lock_irqsave(&ep->lock, flags); + read_lock_irqsave(&ep->lock, flags); ep_set_busy_poll_napi_id(epi); @@ -1170,15 +1260,12 @@ static int ep_poll_callback(wait_queue_entry_t *wait, unsigned mode, int sync, v * chained in ep->ovflist and requeued later on. */ if (READ_ONCE(ep->ovflist) != EP_UNACTIVE_PTR) { - if (epi->next == EP_UNACTIVE_PTR) { - epi->next = READ_ONCE(ep->ovflist); - WRITE_ONCE(ep->ovflist, epi); + if (chain_epi_lockless(epi)) ep_pm_stay_awake_rcu(epi); - } } else if (!ep_is_linked(epi)) { /* In the usual case, add event to ready list. */ - list_add_tail(&epi->rdllink, &ep->rdllist); - ep_pm_stay_awake_rcu(epi); + if (list_add_tail_lockless(&epi->rdllink, &ep->rdllist)) + ep_pm_stay_awake_rcu(epi); } /* @@ -1211,7 +1298,7 @@ static int ep_poll_callback(wait_queue_entry_t *wait, unsigned mode, int sync, v pwake++; out_unlock: - spin_unlock_irqrestore(&ep->lock, flags); + read_unlock_irqrestore(&ep->lock, flags); /* We have to call this outside the lock */ if (pwake) @@ -1546,7 +1633,7 @@ static int ep_insert(struct eventpoll *ep, const struct epoll_event *event, } /* We have to drop the new item inside our item list to keep track of it */ - spin_lock_irq(&ep->lock); + write_lock_irq(&ep->lock); /* record NAPI ID of new item if present */ ep_set_busy_poll_napi_id(epi); @@ -1563,7 +1650,7 @@ static int ep_insert(struct eventpoll *ep, const struct epoll_event *event, pwake++; } - spin_unlock_irq(&ep->lock); + write_unlock_irq(&ep->lock); /* We have to call this outside the lock */ if (pwake) @@ -1627,7 +1714,7 @@ static int ep_modify(struct eventpoll *ep, struct epitem *epi, * list, push it inside. */ if (ep_item_poll(epi, &pt, 1)) { - spin_lock_irq(&ep->lock); + write_lock_irq(&ep->lock); if (!ep_is_linked(epi)) { list_add_tail(&epi->rdllink, &ep->rdllist); ep_pm_stay_awake(epi); @@ -1638,7 +1725,7 @@ static int ep_modify(struct eventpoll *ep, struct epitem *epi, if (waitqueue_active(&ep->poll_wait)) pwake++; } - spin_unlock_irq(&ep->lock); + write_unlock_irq(&ep->lock); } /* We have to call this outside the lock */ @@ -1871,7 +1958,7 @@ static int ep_poll(struct eventpoll *ep, struct epoll_event __user *events, init_wait(&wait); wait.func = ep_autoremove_wake_function; - spin_lock_irq(&ep->lock); + write_lock_irq(&ep->lock); /* * Barrierless variant, waitqueue_active() is called under * the same lock on wakeup ep_poll_callback() side, so it @@ -1890,7 +1977,7 @@ static int ep_poll(struct eventpoll *ep, struct epoll_event __user *events, if (!eavail) __add_wait_queue_exclusive(&ep->wq, &wait); - spin_unlock_irq(&ep->lock); + write_unlock_irq(&ep->lock); if (!eavail) timed_out = !schedule_hrtimeout_range(to, slack, @@ -1905,7 +1992,7 @@ static int ep_poll(struct eventpoll *ep, struct epoll_event __user *events, eavail = 1; if (!list_empty_careful(&wait.entry)) { - spin_lock_irq(&ep->lock); + write_lock_irq(&ep->lock); /* * If the thread timed out and is not on the wait queue, * it means that the thread was woken up after its @@ -1916,7 +2003,7 @@ static int ep_poll(struct eventpoll *ep, struct epoll_event __user *events, if (timed_out) eavail = list_empty(&wait.entry); __remove_wait_queue(&ep->wq, &wait); - spin_unlock_irq(&ep->lock); + write_unlock_irq(&ep->lock); } } } -- 2.52.0

1 0