io_uring support
Aleix Roca Nonell (1): io_uring: fix manual setup of iov_iter for fixed buffers
Arnd Bergmann (2): io_uring: fix big-endian compat signal mask handling io_uring: use __kernel_timespec in timeout ABI
Bart Van Assche (1): percpu-refcount: Introduce percpu_ref_resurrect()
Bijan Mottahedeh (13): io_uring: clear req->result always before issuing a read/write request io_uring: process requests completed with -EAGAIN on poll list io_uring: use proper references for fallback_req locking io_uring: don't use kiocb.private to store buf_index io_uring: add io_statx structure statx: allow system call to be invoked from io_uring io_uring: call statx directly statx: hide interfaces no longer used by io_uring io_uring: validate the full range of provided buffers for access io_uring: add wrappers for memory accounting io_uring: rename ctx->account_mem field io_uring: report pinned memory usage io_uring: separate reporting of ring pages from registered pages
Bob Liu (2): io_uring: clean up io_uring_cancel_files() io_uring: introduce req_need_defer()
Brian Gianforcaro (1): io_uring: fix stale comment and a few typos
Christoph Hellwig (2): fs: add an iopoll method to struct file_operations io_uring: add fsync support
Chucheng Luo (1): io_uring: fix missing 'return' in comment
Colin Ian King (3): io_uring: fix shadowed variable ret return code being not checked io_uring: remove redundant variable pointer nxt and io_wq_assign_next call io_uring: Fix sizeof() mismatch
Damien Le Moal (5): aio: Comment use of IOCB_FLAG_IOPRIO aio flag block: Introduce get_current_ioprio() aio: Fix fallback I/O priority value block: prevent merging of requests with different priorities block: Initialize BIO I/O priority early
Dan Carpenter (3): io-wq: remove extra space characters io_uring: remove unnecessary NULL checks io_uring: fix a use after free in io_async_task_func()
Daniel Xu (1): io_uring: increase IORING_MAX_ENTRIES to 32K
Daniele Albano (1): io_uring: always allow drain/link/hardlink/async sqe flags
Deepa Dinamani (5): signal: Add set_user_sigmask() signal: Add restore_user_sigmask() ppoll: use __kernel_timespec pselect6: use __kernel_timespec io_pgetevents: use __kernel_timespec
Denis Efremov (1): io_uring: use kvfree() in io_sqe_buffer_register()
Dmitrii Dolgov (1): io_uring: add set of tracing events
Dmitry Vyukov (1): io_uring: fix sq array offset calculation
Eric Biggers (1): io_uring: fix memory leak of UNIX domain socket inode
Eric W. Biederman (2): signal/cifs: Fix cifs_put_tcp_session to call send_sig instead of force_sig signal: Allow cifs and drbd to receive their terminating signals
Eugene Syromiatnikov (1): io_uring: fix compat for IORING_REGISTER_FILES_UPDATE
Guoyu Huang (1): io_uring: Fix NULL pointer dereference in loop_rw_iter()
Hillf Danton (6): io-wq: remove unused busy list from io_sqe io-wq: add cond_resched() to worker thread io-uring: drop completion when removing file io-uring: drop 'free_pfile' in struct io_file_put io_uring: add missing finish_wait() in io_sq_thread() io-wq: fix use-after-free in io_wq_worker_running
Hristo Venev (1): io_uring: allocate the two rings together
Hrvoje Zeba (1): io_uring: remove superfluous check for sqe->off in io_accept()
Jackie Liu (17): io_uring: adjust smp_rmb inside io_cqring_events io_uring: use wait_event_interruptible for cq_wait conditional wait io_uring: fix io_sq_thread_stop running in front of io_sq_thread io_uring: fix KASAN use after free in io_sq_wq_submit_work io_uring: fix an issue when IOSQE_IO_LINK is inserted into defer list io_uring: fix wrong sequence setting logic io_uring: add support for link with drain io_uring: use kmemdup instead of kmalloc and memcpy io_uring: fix use-after-free of shadow_req io_uring: fix potential crash issue due to io_get_req failure io_uring: replace s->needs_lock with s->in_async io_uring: set -EINTR directly when a signal wakes up in io_cqring_wait io_uring: remove passed in 'ctx' function parameter ctx if possible io_uring: keep io_put_req only responsible for release and put req io_uring: separate the io_free_req and io_free_req_find_next interface io_uring: remove parameter ctx of io_submit_state_start io_uring: remove io_wq_current_is_worker
Jann Horn (2): io_uring: use kzalloc instead of kcalloc for single-element allocations io-wq: fix handling of NUMA node IDs
Jens Axboe (352): Add io_uring IO interface io_uring: support for IO polling fs: add fget_many() and fput_many() io_uring: use fget/fput_many() for file references io_uring: batch io_kiocb allocation io_uring: add support for pre-mapped user IO buffers net: split out functions related to registering inflight socket files io_uring: add file set registration io_uring: add submission polling io_uring: add io_kiocb ref count io_uring: add support for IORING_OP_POLL io_uring: allow workqueue item to handle multiple buffered requests io_uring: add a few test tools tools/io_uring: remove IOCQE_FLAG_CACHEHIT io_uring: use regular request ref counts io_uring: make io_read/write return an integer io_uring: add prepped flag io_uring: fix fget/fput handling io_uring: fix poll races io_uring: retry bulk slab allocs as single allocs io_uring: fix double free in case of fileset regitration failure io_uring: restrict IORING_SETUP_SQPOLL to root io_uring: park SQPOLL thread if it's percpu io_uring: only test SQPOLL cpu after we've verified it io_uring: drop io_file_put() 'file' argument io_uring: fix possible deadlock between io_uring_{enter,register} io_uring: fix CQ overflow condition io_uring: fail io_uring_register(2) on a dying io_uring instance io_uring: remove 'state' argument from io_{read,write} path io_uring: have submission side sqe errors post a cqe io_uring: drop req submit reference always in async punt fs: add sync_file_range() helper io_uring: add support for marking commands as draining io_uring: add support for IORING_OP_SYNC_FILE_RANGE io_uring: add support for eventfd notifications io_uring: fix failure to verify SQ_AFF cpu io_uring: remove 'ev_flags' argument tools/io_uring: fix Makefile for pthread library link tools/io_uring: sync with liburing io_uring: ensure req->file is cleared on allocation uio: make import_iovec()/compat_import_iovec() return bytes on success io_uring: punt short reads to async context io_uring: add support for sqe links io_uring: add support for sendmsg() io_uring: add support for recvmsg() io_uring: don't use iov_iter_advance() for fixed buffers io_uring: ensure ->list is initialized for poll commands io_uring: fix potential hang with polled IO io_uring: don't enter poll loop if we have CQEs pending io_uring: add need_resched() check in inner poll loop io_uring: expose single mmap capability io_uring: optimize submit_and_wait API io_uring: add io_queue_async_work() helper io_uring: limit parallelism of buffered writes io_uring: extend async work merging io_uring: make sqpoll wakeup possible with getevents io_uring: ensure poll commands clear ->sqe io_uring: use cond_resched() in sqthread io_uring: IORING_OP_TIMEOUT support io_uring: correctly handle non ->{read,write}_iter() file_operations io_uring: make CQ ring wakeups be more efficient io_uring: only flush workqueues on fileset removal io_uring: fix sequence logic for timeout requests io_uring: fix up O_NONBLOCK handling for sockets io_uring: revert "io_uring: optimize submit_and_wait API" io_uring: used cached copies of sq->dropped and cq->overflow io_uring: fix bad inflight accounting for SETUP_IOPOLL|SETUP_SQTHREAD io_uring: don't touch ctx in setup after ring fd install io_uring: run dependent links inline if possible io_uring: allow sparse fixed file sets io_uring: add support for IORING_REGISTER_FILES_UPDATE io_uring: allow application controlled CQ ring size io_uring: add support for absolute timeouts io_uring: add support for canceling timeout requests io-wq: small threadpool implementation for io_uring io_uring: replace workqueue usage with io-wq io_uring: io_uring: add support for async work inheriting files net: add __sys_accept4_file() helper io_uring: add support for IORING_OP_ACCEPT io_uring: protect fixed file indexing with array_index_nospec() io_uring: support for larger fixed file sets io_uring: fix race with canceling timeouts io_uring: io_wq_create() returns an error pointer, not NULL io_uring: ensure we clear io_kiocb->result before each issue io_uring: support for generic async request cancel io_uring: add completion trace event io-wq: use proper nesting IRQ disabling spinlocks for cancel io_uring: enable optimized link handling for IORING_OP_POLL_ADD io_uring: fixup a few spots where link failure isn't flagged io_uring: kill dead REQ_F_LINK_DONE flag io_uring: abstract out io_async_cancel_one() helper io_uring: add support for linked SQE timeouts io_uring: make io_cqring_events() take 'ctx' as argument io_uring: pass in io_kiocb to fill/add CQ handlers io_uring: add support for backlogged CQ ring io-wq: io_wqe_run_queue() doesn't need to use list_empty_careful() io-wq: add support for bounded vs unbunded work io_uring: properly mark async work as bounded vs unbounded io_uring: reduce/pack size of io_ring_ctx io_uring: fix error clear of ->file_table in io_sqe_files_register() io_uring: convert accept4() -ERESTARTSYS into -EINTR io_uring: provide fallback request for OOM situations io_uring: make ASYNC_CANCEL work with poll and timeout io_uring: flag SQPOLL busy condition to userspace io_uring: don't do flush cancel under inflight_lock io_uring: fix -ENOENT issue with linked timer with short timeout io_uring: make timeout sequence == 0 mean no sequence io_uring: use correct "is IO worker" helper io_uring: fix potential deadlock in io_poll_wake() io_uring: check for validity of ->rings in teardown io_wq: add get/put_work handlers to io_wq_create() io-wq: ensure we have a stable view of ->cur_work for cancellations io_uring: ensure registered buffer import returns the IO length io-wq: ensure free/busy list browsing see all items io-wq: remove now redundant struct io_wq_nulls_list io_uring: make POLL_ADD/POLL_REMOVE scale better io_uring: io_async_cancel() should pass in 'nxt' request pointer io_uring: cleanup return values from the queueing functions io_uring: make io_double_put_req() use normal completion path io_uring: make req->timeout be dynamically allocated io_uring: fix sequencing issues with linked timeouts io_uring: remove dead REQ_F_SEQ_PREV flag io_uring: correct poll cancel and linked timeout expiration completion io_uring: request cancellations should break links io-wq: wait for io_wq_create() to setup necessary workers io_uring: io_fail_links() should only consider first linked timeout io_uring: io_allocate_scq_urings() should return a sane state io_uring: allow finding next link independent of req reference count io_uring: close lookup gap for dependent next work io_uring: improve trace_io_uring_defer() trace point io_uring: only return -EBUSY for submit on non-flushed backlog net: add __sys_connect_file() helper io_uring: add support for IORING_OP_CONNECT io-wq: have io_wq_create() take a 'data' argument io_uring: async workers should inherit the user creds io-wq: shrink io_wq_work a bit io_uring: make poll->wait dynamically allocated io_uring: fix missing kmap() declaration on powerpc io_uring: use current task creds instead of allocating a new one io_uring: transform send/recvmsg() -ERESTARTSYS to -EINTR io_uring: add general async offload context io_uring: ensure async punted read/write requests copy iovec net: separate out the msghdr copy from ___sys_{send,recv}msg() net: disallow ancillary data for __sys_{send,recv}msg_file() io_uring: ensure async punted sendmsg/recvmsg requests copy data io_uring: ensure async punted connect requests copy data io_uring: mark us with IORING_FEAT_SUBMIT_STABLE io_uring: handle connect -EINPROGRESS like -EAGAIN io_uring: allow IO_SQE_* flags on IORING_OP_TIMEOUT io_uring: ensure deferred timeouts copy necessary data io-wq: clear node->next on list deletion io_uring: use hash table for poll command lookups io_uring: allow unbreakable links io-wq: remove worker->wait waitqueue io-wq: briefly spin for new work after finishing work io_uring: sqthread should grab ctx->uring_lock for submissions io_uring: deferred send/recvmsg should assign iov io_uring: don't dynamically allocate poll data io_uring: run next sqe inline if possible io_uring: only hash regular files for async work execution io_uring: add sockets to list of files that support non-blocking issue io_uring: ensure we return -EINVAL on unknown opcode io_uring: fix sporadic -EFAULT from IORING_OP_RECVMSG io-wq: re-add io_wq_current_is_worker() io_uring: fix pre-prepped issue with force_nonblock == true io_uring: remove 'sqe' parameter to the OP helpers that take it io_uring: any deferred command must have stable sqe data io_uring: make IORING_POLL_ADD and IORING_POLL_REMOVE deferrable io_uring: make IORING_OP_CANCEL_ASYNC deferrable io_uring: make IORING_OP_TIMEOUT_REMOVE deferrable io_uring: read opcode and user_data from SQE exactly once io_uring: warn about unhandled opcode io_uring: io_wq_submit_work() should not touch req->rw io_uring: use u64_to_user_ptr() consistently io_uring: add and use struct io_rw for read/writes io_uring: move all prep state for IORING_OP_CONNECT to prep handler io_uring: move all prep state for IORING_OP_{SEND,RECV}_MGS to prep handler io_uring: read 'count' for IORING_OP_TIMEOUT in prep handler io_uring: standardize the prep methods io_uring: pass in 'sqe' to the prep handlers io_uring: remove punt of short reads to async context io_uring: don't setup async context for read/write fixed io-wq: cancel work if we fail getting a mm reference io_uring: be consistent in assigning next work from handler io_uring: ensure workqueue offload grabs ring mutex for poll list io_uring: only allow submit from owning task Revert "io_uring: only allow submit from owning task" io_uring: don't cancel all work on process exit io_uring: add support for fallocate() fs: make build_open_flags() available internally io_uring: add support for IORING_OP_OPENAT io-wq: add support for uncancellable work io_uring: add support for IORING_OP_CLOSE io_uring: avoid ring quiesce for fixed file set unregister and update fs: make two stat prep helpers available io_uring: add support for IORING_OP_STATX io-wq: support concurrent non-blocking work io_uring: add IOSQE_ASYNC io_uring: remove two unnecessary function declarations io_uring: add lookup table for various opcode needs io_uring: split overflow state into SQ and CQ side io_uring: improve poll completion performance io_uring: add non-vectored read/write commands io_uring: allow use of offset == -1 to mean file position io_uring: add IORING_OP_FADVISE mm: make do_madvise() available internally io_uring: add IORING_OP_MADVISE io_uring: wrap multi-req freeing in struct req_batch io_uring: extend batch freeing to cover more cases io_uring: add support for IORING_SETUP_CLAMP io_uring: add support for send(2) and recv(2) io_uring: file set registration should use interruptible waits io_uring: change io_ring_ctx bool fields into bit fields io_uring: enable option to only trigger eventfd for async completions io_uring: remove 'fname' from io_open structure io_uring: add opcode to issue trace event io_uring: account fixed file references correctly in batch io_uring: add support for probing opcodes io_uring: file switch work needs to get flushed on exit io_uring: don't attempt to copy iovec for READ/WRITE io-wq: make the io_wq ref counted io_uring/io-wq: don't use static creds/mm assignments io_uring: allow registering credentials io_uring: support using a registered personality for commands io_uring: fix linked command file table usage eventpoll: abstract out epoll_ctl() handler eventpoll: support non-blocking do_epoll_ctl() calls io_uring: add support for epoll_ctl(2) io_uring: add ->show_fdinfo() for the io_uring file descriptor io_uring: prevent potential eventfd recursion on poll io_uring: use the proper helpers for io_send/recv io_uring: don't map read/write iovec potentially twice io_uring: fix sporadic double CQE entry for close io_uring: punt even fadvise() WILLNEED to async context io_uring: spin for sq thread to idle on shutdown io_uring: cleanup fixed file data table references io_uring: statx/openat/openat2 don't support fixed files io_uring: retry raw bdev writes if we hit -EOPNOTSUPP io-wq: add support for inheriting ->fs io_uring: grab ->fs as part of async preparation io_uring: allow AT_FDCWD for non-file openat/openat2/statx io-wq: make io_wqe_cancel_work() take a match handler io-wq: add io_wq_cancel_pid() to cancel based on a specific pid io_uring: cancel pending async work if task exits io_uring: retain sockaddr_storage across send/recvmsg async punt io-wq: don't call kXalloc_node() with non-online node io_uring: prune request from overflow list on flush io_uring: handle multiple personalities in link chains io_uring: fix personality idr leak io-wq: remove spin-for-work optimization io-wq: ensure work->task_pid is cleared on init io_uring: pick up link work on submit reference drop io_uring: import_single_range() returns 0/-ERROR io_uring: drop file set ref put/get on switch io_uring: fix 32-bit compatability with sendmsg/recvmsg io_uring: free fixed_file_data after RCU grace period io_uring: ensure RCU callback ordering with rcu_barrier() io_uring: make sure openat/openat2 honor rlimit nofile io_uring: make sure accept honor rlimit nofile io_uring: consider any io_read/write -EAGAIN as final io_uring: io_accept() should hold on to submit reference on retry io_uring: store io_kiocb in wait->private io_uring: add per-task callback handler io_uring: mark requests that we can do poll async in io_op_defs io_uring: use poll driven retry for files that support it io_uring: buffer registration infrastructure io_uring: add IORING_OP_PROVIDE_BUFFERS io_uring: support buffer selection for OP_READ and OP_RECV io_uring: add IOSQE_BUFFER_SELECT support for IORING_OP_READV net: abstract out normal and compat msghdr import io_uring: add IOSQE_BUFFER_SELECT support for IORING_OP_RECVMSG io_uring: provide means of removing buffers io_uring: add end-of-bits marker and build time verify it io_uring: dual license io_uring.h uapi header io_uring: fix truncated async read/readv and write/writev retry io_uring: honor original task RLIMIT_FSIZE io_uring: retry poll if we got woken with non-matching mask io_uring: grab task reference for poll requests io_uring: use io-wq manager as backup task if task is exiting io_uring: remove bogus RLIMIT_NOFILE check in file registration io_uring: ensure openat sets O_LARGEFILE if needed io_uring: punt final io_ring_ctx wait-and-free to workqueue io_uring: correct O_NONBLOCK check for splice punt io_uring: check for need to re-wait in polled async handling io_uring: io_async_task_func() should check and honor cancelation io_uring: only post events in io_poll_remove_all() if we completed some io_uring: statx must grab the file table for valid fd io_uring: enable poll retry for any file with ->read_iter / ->write_iter io_uring: only force async punt if poll based retry can't handle it io_uring: don't use 'fd' for openat/openat2/statx io_uring: polled fixed file must go through free iteration io_uring: initialize ctx->sqo_wait earlier io_uring: remove dead check in io_splice() io_uring: cancel work if task_work_add() fails io_uring: don't add non-IO requests to iopoll pending list io_uring: remove 'fd is io_uring' from close path io_uring: name sq thread and ref completions io_uring: batch reap of dead file registrations io_uring: allow POLL_ADD with double poll_wait() users io_uring: file registration list and lock optimization io_uring: cleanup io_poll_remove_one() logic io_uring: async task poll trigger cleanup io_uring: disallow close of ring itself io_uring: re-set iov base/len for buffer select retry io_uring: allow O_NONBLOCK async retry io_uring: acquire 'mm' for task_work for SQPOLL io_uring: reap poll completions while waiting for refs to drop on exit io_uring: use signal based task_work running io_uring: fix regression with always ignoring signals in io_cqring_wait() io_uring: account user memory freed when exit has been queued io_uring: ensure double poll additions work with both request types io_uring: use TWA_SIGNAL for task_work uncondtionally io_uring: hold 'ctx' reference around task_work queue + execute io_uring: clear req->result on IOPOLL re-issue io_uring: fix IOPOLL -EAGAIN retries io_uring: always delete double poll wait entry on match io_uring: fix potential ABBA deadlock in ->show_fdinfo() io_uring: use type appropriate io_kiocb handler for double poll io_uring: round-up cq size before comparing with rounded sq size io_uring: remove dead 'ctx' argument and move forward declaration io_uring: don't touch 'ctx' after installing file descriptor io_uring: account locked memory before potential error case io_uring: fix imbalanced sqo_mm accounting io_uring: stash ctx task reference for SQPOLL io_uring: ensure consistent view of original task ->mm from SQPOLL io_uring: allow non-fixed files with SQPOLL io_uring: fail poll arm on queue proc failure io_uring: sanitize double poll handling io_uring: ensure open/openat2 name is cleaned on cancelation io_uring: fix error path cleanup in io_sqe_files_register() io_uring: make ctx cancel on exit targeted to actual ctx io_uring: fix SQPOLL IORING_OP_CLOSE cancelation state io_uring: ignore double poll add on the same waitqueue head io_uring: clean up io_kill_linked_timeout() locking io_uring: add missing REQ_F_COMP_LOCKED for nested requests io_uring: provide generic io_req_complete() helper io_uring: add 'io_comp_state' to struct io_submit_state io_uring: pass down completion state on the issue side io_uring: pass in completion state to appropriate issue side handlers io_uring: enable READ/WRITE to use deferred completions io_uring: use task_work for links if possible io_uring: abstract out task work running io_uring: use new io_req_task_work_add() helper throughout io_uring: only call kfree() for a non-zero pointer io_uring: get rid of __req_need_defer() io_uring: enable lookup of links holding inflight files io_uring: fix recursive completion locking on oveflow flush io_uring: always plug for any number of IOs io_uring: find and cancel head link async work on files exit io_uring: don't use poll handler if file can't be nonblocking read/written io_uring: don't recurse on tsk->sighand->siglock with signalfd io_uring: defer file table grabbing request cleanup for locked requests
Jiufei Xue (5): io_uring: check file O_NONBLOCK state for accept io_uring: change the poll type to be 32-bits io_uring: use EPOLLEXCLUSIVE flag to aoid thundering herd type behavior io_uring: fix removing the wrong file in __io_sqe_files_update() io_uring: set table->files[i] to NULL when io_sqe_file_register failed
Joseph Qi (1): io_uring: fix shift-out-of-bounds when round up cq size
LimingWu (1): io_uring: fix a typo in a comment
Lukas Bulwahn (1): io_uring: make spdxcheck.py happy
Marcelo Diop-Gonzalez (1): io_uring: flush timeouts that should already have expired
Mark Rutland (3): io_uring: fix SQPOLL cpu validation io_uring: free allocated io_memory once io_uring: avoid page allocation warnings
Nathan Chancellor (1): io_uring: Ensure mask is initialized in io_arm_poll_handler
Oleg Nesterov (6): signal: remove the wrong signal_pending() check in restore_user_sigmask() signal: simplify set_user_sigmask/restore_user_sigmask select: change do_poll() to return -ERESTARTNOHAND rather than -EINTR select: shift restore_saved_sigmask_unless() into poll_select_copy_remaining() task_work_run: don't take ->pi_lock unconditionally task_work: teach task_work_add() to do signal_wake_up()
Pavel Begunkov (249): io_uring: Fix __io_uring_register() false success io_uring: fix reversed nonblock flag for link submission io_uring: remove wait loop spurious wakeups io_uring: Fix corrupted user_data io_uring: Fix broken links with offloading io_uring: Fix race for sqes with userspace io_uring: Fix leaked shadow_req io_uring: remove index from sqe_submit io_uring: Fix mm_fault with READ/WRITE_FIXED io_uring: Merge io_submit_sqes and io_ring_submit io_uring: io_queue_link*() right after submit io_uring: allocate io_kiocb upfront io_uring: Use submit info inlined into req io_uring: use inlined struct sqe_submit io_uring: Fix getting file for timeout io_uring: Fix getting file for non-fd opcodes io_uring: break links for failed defer io_uring: remove redundant check io_uring: Fix leaking linked timeouts io_uring: Always REQ_F_FREE_SQE for allocated sqe io_uring: drain next sqe instead of shadowing io_uring: rename __io_submit_sqe() io_uring: add likely/unlikely in io_get_sqring() io_uring: remove io_free_req_find_next() io_uring: pass only !null to io_req_find_next() io_uring: simplify io_req_link_next() io_uring: only !null ptr to io_issue_sqe() io_uring: fix dead-hung for non-iter fixed rw io_uring: store timeout's sqe->off in proper place io_uring: inline struct sqe_submit io_uring: cleanup io_import_fixed() io_uring: fix error handling in io_queue_link_head io_uring: hook all linked requests via link_list io_uring: make HARDLINK imply LINK io_uring: don't wait when under-submitting io_uring: rename prev to head io_uring: move *queue_link_head() from common path pcpu_ref: add percpu_ref_tryget_many() io_uring: batch getting pcpu references io_uring: clamp to_submit in io_submit_sqes() io_uring: optimise head checks in io_get_sqring() io_uring: optimise commit_sqring() for common case io_uring: remove extra io_wq_current_is_worker() io_uring: optimise use of ctx->drain_next io_uring: remove extra check in __io_commit_cqring io_uring: hide uring_fd in ctx io_uring: remove REQ_F_IO_DRAINED io_uring: optimise sqe-to-req flags translation io_uring: use labeled array init in io_op_defs io_uring: prep req when do IOSQE_ASYNC io_uring: honor IOSQE_ASYNC for linked reqs io_uring: add comment for drain_next io_uring: fix refcounting with batched allocations at OOM io-wq: allow grabbing existing io-wq io_uring: add io-wq workqueue sharing io_uring: remove extra ->file check io_uring: iterate req cache backwards io_uring: put the flag changing code in the same spot io_uring: get rid of delayed mm check io_uring: fix deferred req iovec leak io_uring: remove unused struct io_async_open io_uring: fix iovec leaks io_uring: add cleanup for openat()/statx() io_uring: fix async close() with f_op->flush() io_uring: fix double prep iovec leak io_uring: fix openat/statx's filename leak io_uring: add missing io_req_cancelled() io_uring: fix use-after-free by io_cleanup_req() io-wq: fix IO_WQ_WORK_NO_CANCEL cancellation io-wq: remove io_wq_flush and IO_WQ_WORK_INTERNAL io_uring: fix lockup with timeouts io_uring: NULL-deref for IOSQE_{ASYNC,DRAIN} io_uring: don't call work.func from sync ctx io_uring: don't do full *prep_worker() from io-wq io_uring: remove req->in_async splice: make do_splice public io_uring: add interface for getting files io_uring: add splice(2) support io_uring: clean io_poll_complete io_uring: extract kmsg copy helper io-wq: remove unused IO_WQ_WORK_HAS_MM io_uring: remove IO_WQ_WORK_CB io-wq: use BIT for ulong hash io_uring: remove extra nxt check after punt io_uring: remove io_prep_next_work() io_uring: clean up io_close io_uring: make submission ref putting consistent io_uring: remove @nxt from handlers io_uring: get next work with submission ref drop io-wq: shuffle io_worker_handle_work() code io-wq: optimise locking in io_worker_handle_work() io-wq: optimise out *next_work() double lock io_uring/io-wq: forward submission ref to async io-wq: remove duplicated cancel code io-wq: don't resched if there is no work io-wq: split hashing and enqueueing io-wq: hash dependent work io-wq: close cancel gap for hashed linked work io_uring: Fix ->data corruption on re-enqueue io-wq: handle hashed writes in chains io_uring: fix ctx refcounting in io_submit_sqes() io_uring: simplify io_get_sqring io_uring: alloc req only after getting sqe io_uring: remove req init from io_get_req() io_uring: don't read user-shared sqe flags twice io_uring: fix fs cleanup on cqe overflow io_uring: remove obsolete @mm_fault io_uring: track mm through current->mm io_uring: early submission req fail code io_uring: keep all sqe->flags in req->flags io_uring: move all request init code in one place io_uring: fix cached_sq_head in io_timeout() io_uring: kill already cached timeout.seq_offset io_uring: don't count rqs failed after current one io_uring: fix extra put in sync_file_range() io_uring: check non-sync defer_list carefully io_uring: punt splice async because of inode mutex splice: move f_mode checks to do_{splice,tee}() io_uring: fix zero len do_splice() io_uring: don't prepare DRAIN reqs twice io_uring: fix FORCE_ASYNC req preparation io_uring: remove req->needs_fixed_files io_uring: rename io_file_put() io_uring: don't repeat valid flag list splice: export do_tee() io_uring: add tee(2) support io_uring: fix flush req->refs underflow io_uring: simplify io_timeout locking io_uring: don't re-read sqe->off in timeout_prep() io_uring: separate DRAIN flushing into a cold path io_uring: get rid of manual punting in io_close io_uring: move timeouts flushing to a helper io_uring: off timeouts based only on completions io_uring: fix overflowed reqs cancellation io_uring: fix {SQ,IO}POLL with unsupported opcodes io_uring: move send/recv IOPOLL check into prep io_uring: don't derive close state from ->func io_uring: remove custom ->func handlers io_uring: don't arm a timeout through work.func io_wq: add per-wq work handler instead of per work io_uring: fix lazy work init io-wq: reorder cancellation pending -> running io-wq: add an option to cancel all matched reqs io_uring: cancel all task's requests on exit io_uring: batch cancel in io_uring_cancel_files() io_uring: lazy get task io_uring: cancel by ->task not pid io-wq: compact io-wq flags numbers io-wq: return next work from ->do_work() directly io_uring: fix hanging iopoll in case of -EAGAIN io_uring: fix current->mm NULL dereference on exit io_uring: fix missing msg_name assignment io_uring: fix not initialised work->flags io_uring: fix recvmsg memory leak with buffer selection io_uring: missed req_init_async() for IOSQE_ASYNC io_uring: fix ->work corruption with poll_add io_uring: fix lockup in io_fail_links() io_uring: rename sr->msg into umsg io_uring: use more specific type in rcv/snd msg cp io_uring: extract io_sendmsg_copy_hdr() io_uring: simplify io_req_map_rw() io_uring: add a helper for async rw iovec prep io_uring: fix potential use after free on fallback request free io_uring: fix stopping iopoll'ing too early io_uring: briefly loose locks while reaping events io_uring: partially inline io_iopoll_getevents() io_uring: fix racy overflow count reporting io-wq: fix hang after cancelling pending hashed work io_uring: clean file_data access in files_register io_uring: refactor *files_register()'s error paths io_uring: keep a pointer ref_node in file_data io_uring: fix double poll mask init io_uring: fix recvmsg setup with compat buf-select io_uring: fix NULL-mm for linked reqs io_uring: fix missing ->mm on exit io_uring: return locked and pinned page accounting io_uring: don't burn CPU for iopoll on exit io_uring: don't miscount pinned memory io_uring: fix provide_buffers sign extension io_uring: fix stalled deferred requests io_uring: kill REQ_F_LINK_NEXT io_uring: deduplicate freeing linked timeouts io_uring: fix refs underflow in io_iopoll_queue() io_uring: remove inflight batching in free_many() io_uring: dismantle req early and remove need_iter io_uring: batch-free linked requests as well io_uring: cosmetic changes for batch free io_uring: clean up req->result setting by rw io_uring: do task_work_run() during iopoll io_uring: fix NULL mm in io_poll_task_func() io_uring: simplify io_async_task_func() io_uring: fix req->work corruption io_uring: fix punting req w/o grabbed env io_uring: fix feeding io-wq with uninit reqs io_uring: don't mark link's head for_async io_uring: fix missing io_grab_files() io_uring: replace find_next() out param with ret io_uring: kill REQ_F_TIMEOUT io_uring: kill REQ_F_TIMEOUT_NOSEQ io_uring: optimise io_req_find_next() fast check io_uring: remove setting REQ_F_MUST_PUNT in rw io_uring: remove REQ_F_MUST_PUNT io_uring: set @poll->file after @poll init io_uring: don't pass def into io_req_work_grab_env io_uring: do init work in grab_env() io_uring: factor out grab_env() from defer_prep() io_uring: do grab_env() just before punting io_uring: fix mis-refcounting linked timeouts io_uring: keep queue_sqe()'s fail path separately io_uring: fix lost cqe->flags io_uring: don't delay iopoll'ed req completion io_uring: remove nr_events arg from iopoll_check() io_uring: share completion list w/ per-op space io_uring: rename ctx->poll into ctx->iopoll io_uring: use inflight_entry list for iopoll'ing io_uring: use completion list for CQ overflow io_uring: add req->timeout.list io_uring: remove init for unused list io_uring: use non-intrusive list for defer io_uring: remove sequence from io_kiocb io_uring: place cflags into completion data io_uring: fix cancel of deferred reqs with ->files io_uring: fix linked deferred ->files cancellation io_uring: fix racy IOPOLL completions io_uring: inline io_req_work_grab_env() io_uring: alloc ->io in io_req_defer_prep() io_uring/io-wq: move RLIMIT_FSIZE to io-wq io_uring: mark ->work uninitialised after cleanup io_uring: follow **iovec idiom in io_import_iovec io_uring: de-unionise io_kiocb io_uring: consolidate *_check_overflow accounting io_uring: get rid of atomic FAA for cq_timeouts io-wq: update hash bits io_uring: indent left {send,recv}[msg]() io_uring: remove extra checks in send/recv io_uring: don't forget cflags in io_recv() io_uring: free selected-bufs if error'ed io_uring: move BUFFER_SELECT check into *recv[msg] io_uring: simplify file ref tracking in submission state io_uring: extract io_put_kbuf() helper io_uring: don't open-code recv kbuf managment io_uring: don't do opcode prep twice io_uring: deduplicate io_grab_files() calls io_uring: fix missing io_queue_linked_timeout() tasks: add put_task_struct_many() io_uring: batch put_task_struct() io_uring: fix racy req->flags modification io_uring: get an active ref_node from files_data io_uring: order refnode recycling
Randy Dunlap (2): io_uring: fix 1-bit bitfields to be unsigned io_uring: fix function args for !CONFIG_NET
Roman Gushchin (1): percpu_ref: introduce PERCPU_REF_ALLOW_REINIT flag
Roman Penyaev (3): io_uring: offload write to async worker in case of -EAGAIN io_uring: fix infinite wait in khread_park() on io_finish_async() io_uring: add mapping support for NOMMU archs
Shenghui Wang (1): io_uring: use cpu_online() to check p->sq_thread_cpu instead of cpu_possible()
Stefan Bühler (13): io_uring: fix race condition reading SQ entries io_uring: fix race condition when sq threads goes sleeping io_uring: fix poll full SQ detection io_uring: fix handling SQEs requesting NOWAIT io_uring: fix notes on barriers io_uring: remove unnecessary barrier before wq_has_sleeper io_uring: remove unnecessary barrier before reading cq head io_uring: remove unnecessary barrier after updating SQ head io_uring: remove unnecessary barrier before reading SQ tail io_uring: remove unnecessary barrier after incrementing dropped counter io_uring: remove unnecessary barrier after unsetting IORING_SQ_NEED_WAKEUP req->error only used for iopoll io_uring: fix race condition reading SQE data
Stefan Metzmacher (1): io_uring: add BUILD_BUG_ON() to assert the layout of struct io_uring_sqe
Stefano Garzarella (4): io_uring: flush overflowed CQ events in the io_uring_poll() io_uring: prevent sq_thread from spinning when it should stop io_uring: add 'cq_flags' field for the CQ ring io_uring: add IORING_CQ_EVENTFD_DISABLED to the CQ ring flags
Steve French (1): cifs: fix rmmod regression in cifs.ko caused by force_sig changes
Thomas Gleixner (2): sched: Remove stale PF_MUTEX_TESTER bit sched/core, workqueues: Distangle worker accounting from rq lock
Tobias Klauser (1): io_uring: define and set show_fdinfo only if procfs is enabled
Xiaoguang Wang (24): io_uring: fix __io_iopoll_check deadlock in io_sq_thread io_uring: fix poll_list race for SETUP_IOPOLL|SETUP_SQPOLL io_uring: io_uring_enter(2) don't poll while SETUP_IOPOLL|SETUP_SQPOLL enabled io_uring: cleanup io_alloc_async_ctx() io_uring: refactor file register/unregister/update handling io_uring: initialize fixed_file_data lock io_uring: do not always copy iovec in io_req_map_rw() io_uring: restore req->work when canceling poll request io_uring: only restore req->work for req that needs do completion io_uring: use cond_resched() in io_ring_ctx_wait_and_kill() io_uring: fix mismatched finish_wait() calls in io_uring_cancel_files() io_uring: handle -EFAULT properly in io_uring_setup() io_uring: reset -EBUSY error when io sq thread is waken up io_uring: remove obsolete 'state' parameter io_uring: don't submit sqes when ctx->refs is dying io_uring: avoid whole io_wq_work copy for requests completed inline io_uring: avoid unnecessary io_wq_work copy for fast poll feature io_uring: fix io_kiocb.flags modification race in IOPOLL mode io_uring: don't fail links for EAGAIN error in IOPOLL mode io_uring: add memory barrier to synchronize io_kiocb's result and iopoll_completed io_uring: fix possible race condition against REQ_F_NEED_CLEANUP io_uring: export cq overflow status to userspace io_uring: clear IORING_SQ_NEED_WAKEUP after executing task works io_uring: always let io_iopoll_complete() complete polled io
Xiaoming Ni (1): io_uring: remove duplicate semicolon at the end of line
Xuan Zhuo (1): io_uring: fix io_sq_thread no schedule when busy
Yang Yingliang (2): io_uring: fix memleak in __io_sqe_files_update() io_uring: fix memleak in io_sqe_files_register()
YueHaibing (3): io-wq: use kfree_rcu() to simplify the code io_uring: Remove unnecessary null check io_uring: Fix unused function warnings
Zhengyuan Liu (4): io_uring: fix the sequence comparison in io_sequence_defer io_uring: fix counter inc/dec mismatch in async_list io_uring: add a memory barrier before atomic_read io_uring: track io length in async_list based on bytes
yangerkun (9): fs: fix kabi change since add iopoll io_uring: compare cached_cq_tail with cq.head in_io_uring_poll io_uring: consider the overflow of sequence for timeout req io_uring: fix logic error in io_timeout fs: introduce __close_fd_get_file to support IORING_OP_CLOSE for io_uring fs: make filename_lookup available externally x86: fix kabi with io_uring interface arm64: fix kabi with io_uring interface io_uring: add IORING_OP_OPENAT2 for compatablity
zhangyi (F) (2): io_uring : correct timeout req sequence when waiting timeout io_uring: correct timeout req sequence when inserting a new entry
Documentation/filesystems/vfs.txt | 3 + arch/arm64/include/asm/syscall_wrapper.h | 5 + arch/arm64/kernel/syscall.c | 9 +- arch/x86/entry/common.c | 7 + arch/x86/include/asm/syscall_wrapper.h | 3 + block/blk-core.c | 12 +- block/blk-merge.c | 7 +- drivers/block/drbd/drbd_main.c | 2 + fs/Kconfig | 3 + fs/Makefile | 2 + fs/aio.c | 157 +- fs/cifs/connect.c | 3 +- fs/eventpoll.c | 143 +- fs/file.c | 53 +- fs/file_table.c | 9 +- fs/internal.h | 9 + fs/io-wq.c | 1158 +++ fs/io-wq.h | 152 + fs/io_uring.c | 8822 ++++++++++++++++++++++ fs/namei.c | 4 +- fs/open.c | 2 +- fs/select.c | 376 +- fs/splice.c | 62 +- fs/stat.c | 65 +- fs/sync.c | 141 +- include/linux/compat.h | 19 + include/linux/eventpoll.h | 9 + include/linux/fdtable.h | 1 + include/linux/file.h | 3 + include/linux/fs.h | 22 +- include/linux/ioprio.h | 13 + include/linux/mm.h | 1 + include/linux/percpu-refcount.h | 36 +- include/linux/sched.h | 2 +- include/linux/sched/jobctl.h | 4 +- include/linux/sched/signal.h | 12 +- include/linux/sched/task.h | 6 + include/linux/sched/user.h | 2 +- include/linux/signal.h | 15 +- include/linux/socket.h | 25 + include/linux/splice.h | 6 + include/linux/syscalls.h | 28 +- include/linux/task_work.h | 5 +- include/linux/uio.h | 4 +- include/net/af_unix.h | 1 + include/net/compat.h | 3 + include/trace/events/io_uring.h | 495 ++ include/uapi/linux/aio_abi.h | 2 + include/uapi/linux/io_uring.h | 294 + init/Kconfig | 10 + kernel/sched/core.c | 96 +- kernel/signal.c | 63 +- kernel/sys_ni.c | 3 + kernel/task_work.c | 34 +- kernel/workqueue.c | 54 +- kernel/workqueue_internal.h | 5 +- lib/iov_iter.c | 15 +- lib/percpu-refcount.c | 28 +- mm/madvise.c | 7 +- net/Makefile | 2 +- net/compat.c | 31 +- net/socket.c | 298 +- net/unix/Kconfig | 5 + net/unix/Makefile | 2 + net/unix/af_unix.c | 63 +- net/unix/garbage.c | 68 +- net/unix/scm.c | 151 + net/unix/scm.h | 10 + tools/io_uring/Makefile | 18 + tools/io_uring/README | 29 + tools/io_uring/barrier.h | 16 + tools/io_uring/io_uring-bench.c | 592 ++ tools/io_uring/io_uring-cp.c | 260 + tools/io_uring/liburing.h | 187 + tools/io_uring/queue.c | 156 + tools/io_uring/setup.c | 107 + tools/io_uring/syscall.c | 52 + 77 files changed, 13750 insertions(+), 829 deletions(-) create mode 100644 fs/io-wq.c create mode 100644 fs/io-wq.h create mode 100644 fs/io_uring.c create mode 100644 include/trace/events/io_uring.h create mode 100644 include/uapi/linux/io_uring.h create mode 100644 net/unix/scm.c create mode 100644 net/unix/scm.h create mode 100644 tools/io_uring/Makefile create mode 100644 tools/io_uring/README create mode 100644 tools/io_uring/barrier.h create mode 100644 tools/io_uring/io_uring-bench.c create mode 100644 tools/io_uring/io_uring-cp.c create mode 100644 tools/io_uring/liburing.h create mode 100644 tools/io_uring/queue.c create mode 100644 tools/io_uring/setup.c create mode 100644 tools/io_uring/syscall.c