io_uring: include task_work run after scheduling in wait for events
authorJens Axboe <axboe@kernel.dk>
Sat, 17 Dec 2022 20:42:24 +0000 (13:42 -0700)
committerJens Axboe <axboe@kernel.dk>
Sun, 18 Dec 2022 03:35:54 +0000 (20:35 -0700)
commit35d90f95cfa773b7e3b1f57ba15ce06a470f354c
treeee753101bc7a1ef45e98c6bc4cf9ff1d3536f97c
parent6434ec0186b80c734aa7a2acf95f75f5c6dd943b
io_uring: include task_work run after scheduling in wait for events

It's quite possible that we got woken up because task_work was queued,
and we need to process this task_work to generate the events waited for.
If we return to the wait loop without running task_work, we'll end up
adding the task to the waitqueue again, only to call
io_cqring_wait_schedule() again which will run the task_work. This is
less efficient than it could be, as it requires adding to the cq_wait
queue again. It also triggers the wakeup path for completions as
cq_wait is now non-empty with the task itself, and it'll require another
lock grab and deletion to remove ourselves from the waitqueue.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
io_uring/io_uring.c