From: Jens Axboe axboe@kernel.dk
stable inclusion from stable-v5.10.188 commit 810e401b34c4c4c244d8b93b9947ea5b3d4d49f8 category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I7KXLN CVE: CVE-2023-21400
Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=...
--------------------------------
No direct upstream commit exists for this issue. It was fixed in 5.18 as part of a larger rework of the completion side.
io_commit_cqring() writes the CQ ring tail to make it visible, but it also kicks off any deferred work we have. A ring setup with IOPOLL does not need any locking around the CQ ring updates, as we're always under the ctx uring_lock. But if we have deferred work that needs processing, then io_queue_deferred() assumes that the completion_lock is held, as it is for !IOPOLL.
Add a lockdep assertion to check and document this fact, and have io_iopoll_complete() check if we have deferred work and run that separately with the appropriate lock grabbed.
Cc: stable@vger.kernel.org # 5.10, 5.15 Reported-by: dghost david daviduniverse18@gmail.com Signed-off-by: Jens Axboe axboe@kernel.dk Signed-off-by: Greg Kroah-Hartman gregkh@linuxfoundation.org Signed-off-by: Lin Yujun linyujun809@huawei.com Conflicts: fs/io_uring.c Signed-off-by: Zhihao Cheng chengzhihao1@huawei.com --- v1->v2: Add completion_lock for whole io_commit_cqring in iopoll completion fs/io_uring.c | 6 ++++++ 1 file changed, 6 insertions(+)
diff --git a/fs/io_uring.c b/fs/io_uring.c index ce60df5e4d91..88eca93c55b7 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -1310,6 +1310,8 @@ static void io_kill_timeouts(struct io_ring_ctx *ctx)
static void __io_queue_deferred(struct io_ring_ctx *ctx) { + lockdep_assert_held(&ctx->completion_lock); + do { struct io_defer_entry *de = list_first_entry(&ctx->defer_list, struct io_defer_entry, list); @@ -2154,6 +2156,7 @@ static void io_iopoll_complete(struct io_ring_ctx *ctx, unsigned int *nr_events, struct req_batch rb; struct io_kiocb *req; LIST_HEAD(again); + unsigned long flags;
/* order with ->result store in io_complete_rw_iopoll() */ smp_rmb(); @@ -2181,7 +2184,10 @@ static void io_iopoll_complete(struct io_ring_ctx *ctx, unsigned int *nr_events, io_req_free_batch(&rb, req); }
+ spin_lock_irqsave(&ctx->completion_lock, flags); io_commit_cqring(ctx); + spin_unlock_irqrestore(&ctx->completion_lock, flags); + if (ctx->flags & IORING_SETUP_SQPOLL) io_cqring_ev_posted(ctx); io_req_free_batch_finish(ctx, &rb);