From: Pavel Begunkov asml.silence@gmail.com
mainline inclusion from mainline-v5.13-rc1 commit f70865db5ff35f5ed0c7e9ef63e7cca3d4947f04 category: bugfix bugzilla: 185739 CVE: NA
-----------------------------------------------
Revert of revert of "io_uring: wait potential ->release() on resurrect", which adds a helper for resurrect not racing completion reinit, as was removed because of a strange bug with no clear root or link to the patch.
Was improved, instead of rcu_synchronize(), just wait_for_completion() because we're at 0 refs and it will happen very shortly. Specifically use non-interruptible version to ignore all pending signals that may have ended prior interruptible wait.
This reverts commit cb5e1b81304e089ee3ca948db4d29f71902eb575.
Signed-off-by: Pavel Begunkov asml.silence@gmail.com Link: https://lore.kernel.org/r/7a080c20f686d026efade810b116b72f88abaff9.161810175... Signed-off-by: Jens Axboe axboe@kernel.dk
conflicts: fs/io_uring.c
Signed-off-by: Ye Bin yebin10@huawei.com Reviewed-by: Zhang Yi yi.zhang@huawei.com Signed-off-by: Yang Yingliang yangyingliang@huawei.com --- fs/io_uring.c | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-)
diff --git a/fs/io_uring.c b/fs/io_uring.c index da61eeaf64e88..d07388600bbed 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -8669,6 +8669,18 @@ static bool io_register_op_must_quiesce(int op) } }
+static void io_refs_resurrect(struct percpu_ref *ref, struct completion *compl) +{ + bool got = percpu_ref_tryget(ref); + + /* already at zero, wait for ->release() */ + if (!got) + wait_for_completion(compl); + percpu_ref_resurrect(ref); + if (got) + percpu_ref_put(ref); +} + static int __io_uring_register(struct io_ring_ctx *ctx, unsigned opcode, void __user *arg, unsigned nr_args) __releases(ctx->uring_lock) @@ -8699,9 +8711,8 @@ static int __io_uring_register(struct io_ring_ctx *ctx, unsigned opcode, ret = wait_for_completion_interruptible(&ctx->ref_comp); mutex_lock(&ctx->uring_lock); if (ret) { - percpu_ref_resurrect(&ctx->refs); - ret = -EINTR; - goto out; + io_refs_resurrect(&ctx->refs, &ctx->ref_comp); + return ret; } }
@@ -8772,7 +8783,6 @@ static int __io_uring_register(struct io_ring_ctx *ctx, unsigned opcode, if (io_register_op_must_quiesce(opcode)) { /* bring the ctx back to life */ percpu_ref_reinit(&ctx->refs); -out: reinit_completion(&ctx->ref_comp); } return ret;