From: Pavel Begunkov asml.silence@gmail.com
mainline inclusion from mainline-5.7-rc1 commit 5eae8619907a1389dbd1b4a1049caf52782c0916 category: feature bugzilla: https://bugzilla.openeuler.org/show_bug.cgi?id=27 CVE: NA ---------------------------
IO_WQ_WORK_CB is used only for linked timeouts, which will be armed before the work setup (i.e. mm, override creds, etc). The setup shouldn't take long, so it's ok to arm it a bit later and get rid of IO_WQ_WORK_CB.
Make io-wq call work->func() only once, callbacks will handle the rest. i.e. the linked timeout handler will do the actual issue. And as a bonus, it removes an extra indirect call.
Signed-off-by: Pavel Begunkov asml.silence@gmail.com Signed-off-by: Jens Axboe axboe@kernel.dk Signed-off-by: yangerkun yangerkun@huawei.com Reviewed-by: zhangyi (F) yi.zhang@huawei.com Signed-off-by: Cheng Jian cj.chengjian@huawei.com --- fs/io-wq.c | 3 --- fs/io-wq.h | 1 - fs/io_uring.c | 3 +-- 3 files changed, 1 insertion(+), 6 deletions(-)
diff --git a/fs/io-wq.c b/fs/io-wq.c index 90767828ad01..f3894022d467 100644 --- a/fs/io-wq.c +++ b/fs/io-wq.c @@ -480,9 +480,6 @@ static void io_worker_handle_work(struct io_worker *worker) worker->cur_work = work; spin_unlock_irq(&worker->lock);
- if (work->flags & IO_WQ_WORK_CB) - work->func(&work); - if (work->files && current->files != work->files) { task_lock(current); current->files = work->files; diff --git a/fs/io-wq.h b/fs/io-wq.h index d500d88ab84e..a0978d6958f0 100644 --- a/fs/io-wq.h +++ b/fs/io-wq.h @@ -7,7 +7,6 @@ enum { IO_WQ_WORK_CANCEL = 1, IO_WQ_WORK_HASHED = 4, IO_WQ_WORK_UNBOUND = 32, - IO_WQ_WORK_CB = 128, IO_WQ_WORK_NO_CANCEL = 256, IO_WQ_WORK_CONCURRENT = 512,
diff --git a/fs/io_uring.c b/fs/io_uring.c index 7891229e1b5c..a3f93c6ebe4d 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -2546,7 +2546,7 @@ static void io_link_work_cb(struct io_wq_work **workptr) struct io_kiocb *link = work->data;
io_queue_linked_timeout(link); - work->func = io_wq_submit_work; + io_wq_submit_work(workptr); }
static void io_wq_assign_next(struct io_wq_work **workptr, struct io_kiocb *nxt) @@ -2556,7 +2556,6 @@ static void io_wq_assign_next(struct io_wq_work **workptr, struct io_kiocb *nxt) io_prep_next_work(nxt, &link); *workptr = &nxt->work; if (link) { - nxt->work.flags |= IO_WQ_WORK_CB; nxt->work.func = io_link_work_cb; nxt->work.data = link; }