mirrors
/
linux-stable
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
1
0
Fork 0
Code Issues Releases Wiki Activity
linux-stable/io_uring
History
Pavel Begunkov 8836df0212 io_uring: adjust defer tw counting 
[ Upstream commit dc12d1799c ]

The UINT_MAX work item counting bias in io_req_local_work_add() in case
of !IOU_F_TWQ_LAZY_WAKE works in a sense that we will not miss a wake up,
however it's still eerie. In particular, if we add a lazy work item
after a non-lazy one, we'll increment it and get nr_tw==0, and
subsequent adds may try to unnecessarily wake up the task, which is
though not so likely to happen in real workloads.

Half the bias, it's still large enough to be larger than any valid
->cq_wait_nr, which is limited by IORING_MAX_CQ_ENTRIES, but further
have a good enough of space before it overflows.

Fixes: 8751d15426 ("io_uring: reduce scheduling due to tw")
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/108b971e958deaf7048342930c341ba90f75d806.1705438669.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Sasha Levin <sashal@kernel.org>
 		2024-01-25 15:36:00 -08:00
..
Makefile
…
advise.c io_uring: always go async for unsupported fadvise flags 2023-01-29 15:18:26 -07:00
advise.h
…
alloc_cache.h io_uring/rsrc: consolidate node caching 2023-04-12 12:09:41 -06:00
cancel.c io_uring: use fget/fput consistently 2024-01-20 11:51:38 +01:00
cancel.h io_uring/cancel: support opcode based lookup and cancelation 2023-07-17 10:05:48 -06:00
epoll.c io_uring: undeprecate epoll_ctl support 2023-05-26 20:22:41 -06:00
epoll.h
…
fdinfo.c io_uring/fdinfo: remove need for sqpoll lock for thread/pid retrieval 2023-11-28 17:19:52 +00:00
fdinfo.h
…
filetable.c io_uring: add helpers to decode the fixed file file_ptr 2023-06-20 09:36:22 -06:00
filetable.h io_uring: add helpers to decode the fixed file file_ptr 2023-06-20 09:36:22 -06:00
fs.c io_uring/fs: consider link->flags when getting path for LINKAT 2023-12-03 07:33:07 +01:00
fs.h
…
io-wq.c io-wq: fully initialize wqe before calling cpuhp_state_add_instance_nocalls() 2023-10-05 14:11:18 -06:00
io-wq.h io_uring: break out of iowq iopoll on teardown 2023-09-07 09:02:27 -06:00
io_uring.c io_uring: adjust defer tw counting 2024-01-25 15:36:00 -08:00
io_uring.h io_uring: enable io_mem_alloc/free to be used in other parts 2023-12-08 08:52:20 +01:00
kbuf.c io_uring/kbuf: check for buffer list readiness after NULL check 2023-12-13 18:45:17 +01:00
kbuf.h io_uring/kbuf: defer release of mapped buffer rings 2023-12-08 08:52:20 +01:00
msg_ring.c io_uring: use io_file_from_index in io_msg_grab_file 2023-06-20 09:36:22 -06:00
msg_ring.h io_uring: get rid of double locking 2022-12-07 06:47:13 -07:00
net.c io_uring/net: ensure socket is marked connected on connect retry 2023-11-20 11:59:38 +01:00
net.h io_uring: Add KASAN support for alloc_caches 2023-04-03 07:16:14 -06:00
nop.c
…
nop.h
…
notif.c io_uring/notif: add constant for ubuf_info flags 2023-04-15 14:21:04 -06:00
notif.h io_uring/notif: add constant for ubuf_info flags 2023-04-15 14:21:04 -06:00
opdef.c io_uring: Pass whole sqe to commands 2023-05-04 08:19:05 -06:00
opdef.h io_uring: Split io_issue_def struct 2023-01-29 15:17:41 -07:00
openclose.c io_uring: correct check for O_TMPFILE 2023-08-07 12:34:23 -06:00
openclose.h
…
poll.c io_uring: never overflow io_aux_cqe 2023-08-11 10:42:57 -06:00
poll.h io_uring: avoid indirect function calls for the hottest task_work 2023-06-02 08:55:37 -06:00
refs.h
…
rsrc.c io_uring: fix off-by one bvec index 2023-12-03 07:33:07 +01:00
rsrc.h io_uring/af_unix: disable sending io_uring over sockets 2023-12-13 18:45:20 +01:00
rw.c io_uring/rw: ensure io->bytes_done is always initialized 2024-01-25 15:35:45 -08:00
rw.h io_uring: avoid indirect function calls for the hottest task_work 2023-06-02 08:55:37 -06:00
slist.h io_uring: silence variable ‘prev’ set but not used warning 2023-03-09 10:10:58 -07:00
splice.c io_uring/splice: use fput() directly 2023-08-10 10:24:25 -06:00
splice.h
…
sqpoll.c io_uring/fdinfo: remove need for sqpoll lock for thread/pid retrieval 2023-11-28 17:19:52 +00:00
sqpoll.h io_uring/sqpoll: fix io-wq affinity when IORING_SETUP_SQPOLL is used 2023-08-16 13:40:28 -06:00
statx.c io_uring: for requests that require async, force it 2023-01-29 15:18:26 -07:00
statx.h
…
sync.c io_uring: for requests that require async, force it 2023-01-29 15:18:26 -07:00
sync.h
…
tctx.c io_uring: Add io_uring_setup flag to pre-register ring fd and never install it 2023-05-16 08:06:00 -06:00
tctx.h io_uring: simplify __io_uring_add_tctx_node 2022-10-07 12:25:30 -06:00
timeout.c io_uring: never overflow io_aux_cqe 2023-08-11 10:42:57 -06:00
timeout.h io_uring: remove unused return from io_disarm_next 2022-09-21 13:15:01 -06:00
uring_cmd.c io_uring/cmd: fix breakage in SOCKET_URING_OP_SIOC* implementation 2023-12-20 17:01:52 +01:00
uring_cmd.h io_uring: Remove unnecessary BUILD_BUG_ON 2023-05-04 08:19:05 -06:00
xattr.c io_uring: for requests that require async, force it 2023-01-29 15:18:26 -07:00
xattr.h
…