Revert "selftest: add some basic testing for the io_uring vfs module"

This reverts commit a0730dda352b2d09f7b9fd1e3a9791c1d61643cb.

python/samba/tests/blackbox/smbcacls_basic.py try to make it more robust against flakey tests

Revert "source3/smbd/globals.h _vector[2]"

This reverts commit 8dfad2585729aa483d8b0bf277b758f3ee06e3e0.

Revert "HACK only one recvmsg..."

This reverts commit f33815aa65691658f79b2531986fd22ac250ef0f.

Revert "fix again source3/smbd/smb2_server.c v2 part2 vector2"

This reverts commit 892d0d9989b8bfdaf3133a7c149158dad3df454b.

Revert "fix again source3/smbd/smb2_server.c v3 part3 vector2"

This reverts commit b459152dfa539b083233c30ba82a0bd2340f0f28.

fix again source3/smbd/smb2_server.c v3 part3 vector2

fix again source3/smbd/smb2_server.c v2 part2 vector2

HACK only one recvmsg...

source3/smbd/globals.h _vector[2]

sq lib/util/samba_io_uring.c

Revert "V2 P4.1: prepare smbd_smb2_flush_with_io_uring smb2srv:send_uring smb2srv:send_uring_zc smb2srv:send_uring_async"

This reverts commit 3c156bc8cb2941ef050dc534ce5a89607a9c8508.

Revert "IORING_NOTIF_USAGE_ZC_COPIED"

This reverts commit 1c5de5ce14cabe7e3880e2c6b831a8214e3093a2.

Revert "samba_io_uring_ev_hybrit_im_handler"

This reverts commit c3dbf9b168855c4bc9b523c5e5bc0fba1ae19e47.

Revert "DEBUG SENDMSG"

This reverts commit e81d9d5a53fb77df50e167036555e0d197108406.

Revert "no IORING_SETUP_DEFER_TASKRUN"

This reverts commit 10c0b6c04218a0ce69a5ab78aab8a5736549f738.

Revert "Revert "DEBUG SENDMSG""

This reverts commit f3b44642df4735daa3c2ef957eb8c5625181116c.

Revert "source3/modules/vfs_io_uring.c pread noop DIRECT"

This reverts commit f29466d41328b79c093211928248b7c4543967f6.

Revert "Revert "source3/modules/vfs_io_uring.c pread noop DIRECT""

This reverts commit 5db281e9ae5ba95b62377702cc31c72a62c2823e.

Revert "source3/modules/vfs_io_uring.c pread noop DIRECT"

This reverts commit f29466d41328b79c093211928248b7c4543967f6.

source3/modules/vfs_io_uring.c pread noop DIRECT

Revert "DEBUG SENDMSG"

This reverts commit e81d9d5a53fb77df50e167036555e0d197108406.

no IORING_SETUP_DEFER_TASKRUN

DEBUG SENDMSG

samba_io_uring_ev_hybrit_im_handler

IORING_NOTIF_USAGE_ZC_COPIED

HACK test_smb2_bench_path_contention_shared uncontented

lib/tevent/testsuite.c 10.000.000

Revert "source4/torture/smb2/bench.c tcon/tdis"

This reverts commit f2820e5f3d01fa536e6abd9a0b77783fc93117d3.

source4/torture/smb2/bench.c tcon/tdis

V2 P4.1: prepare smbd_smb2_flush_with_io_uring smb2srv:send_uring smb2srv:send_uring_zc smb2srv:send_uring_async

sq revert IORING_OP_SENDMSG_ZC and debug

smb2_server: teardown in smbd_smb2_flush_send_queue() already

Moving this from smbd_smb2_flush_with_sendmsg() to smbd_smb2_flush_send_queue()
means it will also be used for io_uring in future.

Signed-off-by: Stefan Metzmacher <metze@samba.org>

revert IORING_OP_SENDMSG_ZC and debug

sq io_uring.pending_zc => if (samba_io_uring_completion_pending(&e->io_uring.completion) == 0) {

sq P4.1: prepare smbd_smb2_flush_with_io_uring smb2srv:send_uring smb2srv:send_uring_zc smb2srv:send_uring_async

sq P4.1: prepare smbd_smb2_flush_with_io_uring smb2srv:send_uring smb2srv:send_uring_zc smb2srv:send_uring_async

sq P4.1: prepare smbd_smb2_flush_with_io_uring smb2srv:send_uring smb2srv:send_uring_zc smb2srv:send_uring_async

P4.1: prepare smbd_smb2_flush_with_io_uring smb2srv:send_uring smb2srv:send_uring_zc smb2srv:send_uring_async

sq P4.X: xconn->send_uring queue_entry->io_uring

P4.X: xconn->send_uring queue_entry->io_uring

source3/wscript_build smbd_base => samba-io-uring

P4.0: source3/smbd/globals.h lib/util/samba_io_uring.h

xconn->smb2.request_read_state->xconn = xconn

TODO xconn->smb2.request_read_state talloced TODO really needed ??? costs 0,1%

TODO-LATER?? P2.x: smbd_smb2_send_queue->xconn

vfs_io_uring_request_submitted,completed optimized inline

sq source3/modules/vfs_io_uring.c samba_io_uring_completion_bypass_epoll

vfs_io_uring: add "io_uring:discard_writes_to_dev_null" option for testing

We want to be able to test the overhead we have for SMB2 write calls,
which are typically bound to the filesystem/store capabilities.

P1.6: source3/modules/vfs_io_uring.c force_async.{read,write}_threshhold

P1.4: TODO FIXME io_uring:sqpoll

P1.3: vfs_io_uring: make use of samba_io_uring infrastructure

Signed-off-by: Stefan Metzmacher <metze@samba.org>

P1.0: BUG??? vfs_io_uring: handle EINTR like EAGAIN and EBUSY

With EINTR we should just retry as well...

Signed-off-by: Stefan Metzmacher <metze@samba.org>

samba_tevent_context_init try samba_io_uring_ev if available

source4/torture/local/local.c call samba_io_uring_ev_register()

samba_tevent_context_init() call samba_io_uring_ev_register()

samba_io_uring_completion_bypass_epoll

samba_io_uring_completion->wait_for_poll

lib/util/samba_io_uring.c if0 debugging code

lib/util/samba_io_uring.c syscalls as before

source4/torture/smb2/bench.c start requests after creation all connections

TODO lib/util/samba_io_uring.c SAMBA_IO_URING_COMPLETION_GENERATION_SHIFT

sq lib/util/samba_io_uring 6.2/6.3 features

sq lib/util/samba_io_uring.h

lib/util/samba_io_uring.c _Static_assert()

lib/util/samba_io_uring.c USING_SYSTEM_TEVENT no needed

lib/util/samba_io_uring.c no tevent_set_default_backend("samba_io_uring_ev_hybrit");

io_uring_register(5, IORING_REGISTER_PROBE) and close() take a lot of time on (at least 5.4) kernels...
It's fast on 5.15 similar to all other syscalls.

13693 23:51:10.295204 syscall_0x1a9(0x20, 0x56109f1a2c00, 0x20, 0x56109eee7010, 0, 0) = 0x5 <0.000026>
13693 23:51:10.295252 mmap(NULL, 1472, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_POPULATE, 5, 0) = 0x7ff40f147000 <0.000011>
13693 23:51:10.295276 mmap(NULL, 2048, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_POPULATE, 5, 0x10000000) = 0x7ff40f146000 <0.000008>
13693 23:51:10.295296 madvise(0x7ff40f146000, 2048, MADV_DONTFORK) = 0 <0.000004>
13693 23:51:10.295312 madvise(0x7ff40f147000, 1472, MADV_DONTFORK) = 0 <0.000004>
13693 23:51:10.295345 syscall_0x1ab(0x5, 0x8, 0x56109f236150, 0x100, 0x77, 0) = -1 (errno 22) <0.011981>
...

13693 23:51:10.337035 munmap(0x7ff41b88c000, 2048) = 0 <0.000009>
13693 23:51:10.337056 munmap(0x7ff41b88d000, 1472) = 0 <0.000007>
13693 23:51:10.337074 close(3)          = 0 <0.010263>

samba_io_uring_destructor ring->self_mem_ctx = NULL;

P1.2: lib/util: add samba_io_uring.* abstraction

This will allow us to use io_uring also outside of
vfs_io_uring.c, which will be converted to use this new code
in the following commits.

Signed-off-by: Stefan Metzmacher <metze@samba.org>

third_party: import liburing-2.4

This makes it much easier to cope with
pushd third_party/liburing/ && ./update.sh 2.4; popd

Signed-off-by: Stefan Metzmacher <metze@samba.org>

P1.1: wafsamba: improve CHECK_SIZEOF() to have lib= and fallback to CHECK_VALUEOF()

Signed-off-by: Stefan Metzmacher <metze@samba.org>

P0.X: s3:param: add lp_parm_bytes() it's similar to lpcfg_parm_bytes()

Signed-off-by: Stefan Metzmacher <metze@samba.org>

lib/util: support SO_SNDBUFFORCE/SO_RCVBUFFORCE in "socket options"

Signed-off-by: Stefan Metzmacher <metze@samba.org>

smb2_server: monitor connections with TEVENT_FD_ERROR

By asking for TEVENT_FD_ERROR we're able to fail early
when a connection to a client is broken.

In that case it does not make any sense to process
pending requests in the recv queue as it's not
possible to deliver the response to the client anyway.

Signed-off-by: Stefan Metzmacher <metze@samba.org>

s3:rpc_server: make use of tstream_bsd_fail_readv_first_error(true)

This avoids doing useless work in case the client connection
is already broken.

Signed-off-by: Stefan Metzmacher <metze@samba.org>

s4:rpc_server: make use of tstream_bsd_fail_readv_first_error(true)

This avoids doing useless work in case the client connection
is already broken.

Signed-off-by: Stefan Metzmacher <metze@samba.org>

s4:service_named_pipe: make use of tstream_bsd_fail_readv_first_error(true)

This avoids doing useless work in case the client connection
is already broken.

Signed-off-by: Stefan Metzmacher <metze@samba.org>

libcli/named_pipe_auth: let tstream_npa_existing_socket use tstream_bsd_fail_readv_first_error(true)

This avoids doing useless work in case the client connection
is already broken.

Signed-off-by: Stefan Metzmacher <metze@samba.org>

s4:wrepl_server: make use of tstream_bsd_fail_readv_first_error(true)

This avoids doing useless work in case the client connection
is already broken.

Signed-off-by: Stefan Metzmacher <metze@samba.org>

s4:libcli/wrepl: make use of tstream_bsd_fail_readv_first_error(false)

As a client we want recv pending responses even if the server
already closed the connection.

While tstream_bsd_fail_readv_first_error(false) is the default for
tstream_bsd, the wins replication protocol is special as it has
a way to switch server and client roles on an existing tcp connection.

Signed-off-by: Stefan Metzmacher <metze@samba.org>

s4:ntp_signd: make use of tstream_bsd_fail_readv_first_error(true)

This avoids doing useless work in case the client connection
is already broken.

Signed-off-by: Stefan Metzmacher <metze@samba.org>

s3:libsmb: the unexpected handler use tstream_bsd_fail_readv_first_error(true)

This avoids doing useless work in case the client connection
is already broken.

Signed-off-by: Stefan Metzmacher <metze@samba.org>

s4:dns_server: make use of tstream_bsd_fail_readv_first_error(true)

This avoids doing useless work in case the client connection
is already broken.

Signed-off-by: Stefan Metzmacher <metze@samba.org>

s4:ldap_server: make use of tstream_bsd_fail_readv_first_error(true)

This avoids doing useless work in case the client connection
is already broken.

Signed-off-by: Stefan Metzmacher <metze@samba.org>

s4:kdc: make use of tstream_bsd_fail_readv_first_error(true)

This avoids doing useless work in case the client connection
is already broken.

Signed-off-by: Stefan Metzmacher <metze@samba.org>

lib/tsocket: add tstream_bsd_fail_readv_first_error()

This gives the caller the option to fail immediately if
TEVENT_FD_ERROR appear even with pending bytes in the
recv queue.

Servers typically want to activate this in order to avoid
pointless work, while clients typically want to read
pending responses from the recv queue.

Signed-off-by: Stefan Metzmacher <metze@samba.org>

lib/tsocket: make use of TEVENT_FD_ERROR in tstream_bsd_fde_handler()

This makes the logic introduced to fix bug #15202 simpler.

While developing this I noticed that a lot of callers
rely on the fact that they can read the pending bytes out
of the recv queue before EOF is reported.

So I changed the code handle TEVENT_FD_ERROR together with
TEVENT_FD_READ in a way that keep the existing callers happy.

In the next step we'll add a way to let callers opt-in in order
to fail immediately if TEVENT_FD_ERROR appears (even if there
are pending bytes remaining in the recv queue).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15202

Signed-off-by: Stefan Metzmacher <metze@samba.org>

lib/tsocket: let tstream_bsd_connect_send() use TEVENT_FD_ERROR instead of TEVENT_FD_READ

This mostly cosmetic, but now that we have TEVENT_FD_ERROR we should use it.

Signed-off-by: Stefan Metzmacher <metze@samba.org>

lib/async_req: let writev_send/recv use TEVENT_FD_ERROR

Unless err_on_readability is true, we use TEVENT_FD_READ only
to detect errors. Now that we have TEVENT_FD_ERROR we should use it.

As a side effect it makes the code much simpler and clearer, as
we can directly map TEVENT_FD_ERROR to EPIPE.

In addition the err_on_readability=true case is now also
clearer, where we just map TEVENT_FD_READ to EPIPE.

Signed-off-by: Stefan Metzmacher <metze@samba.org>

lib/async_req: let async_connect_send use TEVENT_FD_ERROR instead of TEVENT_FD_READ

This mostly cosmetic, but now that we have TEVENT_FD_ERROR we should use it.

Signed-off-by: Stefan Metzmacher <metze@samba.org>

lib/tsocket: make use of samba_socket_sock_error()

This is nicer than calling getsockopt(state->fd, SOL_SOCKET, SO_ERROR)
directly.

Signed-off-by: Stefan Metzmacher <metze@samba.org>

lib/tsocket: make use of samba_socket_poll_or_sock_error()

This is just a copy of the existing code...

Signed-off-by: Stefan Metzmacher <metze@samba.org>

lib/util: add samba_socket_{poll,sock,poll_or_sock}_error()

These are copies of the static functions in lib/tsocket/tsocket_bsd.c,
which we will replace in the next commit.

Signed-off-by: Stefan Metzmacher <metze@samba.org>

s4/torture: fix exit status of raw.bench-lookup

Use correct value of 'result' when the test passes.

Signed-off-by: Oleg Kravtsov <oleg@tuxera.com>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Mon Oct 16 02:01:17 UTC 2023 on atb-devel-224

gpdupate: Implement Drive Maps Client Side Extension

Signed-off-by: David Mulder <dmulder@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

gpupdate: Test Drive Maps Client Side Extension

Signed-off-by: David Mulder <dmulder@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tevent: version 0.16.0

- the epoll backend is no longer limited to 2 event handlers
  per low level fd.
- finally add support for TEVENT_FD_ERROR

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Fri Oct 13 10:45:51 UTC 2023 on atb-devel-224

tevent: add support for TEVENT_FD_ERROR

After 12 years we finally got TEVENT_FD_ERROR support :-)

TEVENT_FD_WRITE event handlers never get errors reported
instead the event handler is silently disabled.
There are likely callers relying on that behavior, so
we are not able to chance it.

Now TEVENT_FD_WRITE can be used together with TEVENT_FD_ERROR
in order to get errors reported without waiting for TEVENT_FD_READ.

TEVENT_FD_ERROR can also be used alone in order to detect errors
on sockets in order to cleanup resources.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

tevent: add test_event_fd3

The tests the interaction of multiple event handlers on
the same low level fd.

It shows that poll and epoll backends behave in the
same fair way.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

tevent: add test_fd_speed3

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>