smb2_server: make sure we detect stale smbXsrv_connection pointers in smbXsrv_channel_global

Pointer values can be reused (yes, I hit that during my testing!).
Introduce a channel_id to identify connections and also add
some timestamps to make debugging easier.

This makes smbXsrv_session_find_channel() much more robust.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

smb2_server: update inline comment for max channels

All Windows versions have the limit of 32 channels.

Signed-off-by: Guenther Deschner <gd@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>

smbXsrv_client: make sure that we store a valid blob

This fixes a regression introduced by
14182350f8397d27d7642dae595dc52691f0acfe
("librpc ndr: ndr_pull_advance check for unsigned overflow.")
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14236

ndr_push_smbXsrv_client_global0() is happy with
pushing NULL pointers for r->{local_address,remote_address,remote_name},
while the IDL doesn't allow it.
In turn ndr_pull_smbXsrv_client_global0() no longer ignores the error.

This means multi-channel connections were broken,
and we paniced on a NULL pointer.
It's really sad that we still don't have automated tests for it.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

smbXsrv_client: fix debug message in smbXsrv_client_create()

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

s4/rpc_server/dnsserver: Allow parsing of dnsProperty to fail gracefully

On (eg) the

DC=_msdcs.X.Y,CN=MicrosoftDNS,DC=ForestDnsZones,DC=X,DC=Y

record, in domains that have had a Microsoft Windows DC an attribute:

dNSProperty:: AAAAAAAAAAAAAAAAAQAAAJIAAAAAAAAA

000000 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00  >................<
000010 92 00 00 00 00 00 00 00                          >........<
000018

We, until samba 4.12, would parse this as:

pull returned Success
    dnsp_DnsProperty: struct dnsp_DnsProperty
        wDataLength              : 0x00000000 (0)
        namelength               : 0x00000000 (0)
        flag                     : 0x00000000 (0)
        version                  : 0x00000001 (1)
        id                       : DSPROPERTY_ZONE_NS_SERVERS_DA (146)
        data                     : union dnsPropertyData(case 0)
        name                     : 0x00000000 (0)
dump OK

However, the wDataLength is 0.  There is not anything in
[MS-DNSP] 2.3.2.1 dnsProperty to describe any special behaviour
for when the id suggests that there is a value, but wDataLength is 0.

https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-dnsp/445c7843-e4a1-4222-8c0f-630c230a4c80

We now fail to parse it, because we expect an entry with id DSPROPERTY_ZONE_NS_SERVERS_DA
to therefore have a valid DNS_ADDR_ARRAY (section 2.2.3.2.3).

As context we changed it in our commit fee5c6a4247aeac71318186bbff7708d25de5912
because of bug https://bugzilla.samba.org/show_bug.cgi?id=14206
which was due to the artificial environment of the fuzzer.

Microsoft advises that Windows also fails to parse this, but
instead of failing the operation, the value is ignored.

Reported by Alex MacCuish.  Many thanks for your assistance in
tracking down the issue.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14310

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri May 15 07:29:17 UTC 2020 on sn-devel-184

selftest: Add test for handling of "short" dnsProperty records

These have been known to be given by Windows DCs that share the same domain
as while invalid, they are not format-checked inbound when set by the DNS
Manager MMC applet over the dnsserver pipe to Windows.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14310

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

librpc/idl: Add dnsp_DnsProperty_short

This will be used by a test and the DNS server code to parse short dnsProperty
records which come from Windows servers.

This example is from the value that caused Samba to fail as it
can not be parsed as a normal dnsp_DnsProperty

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14310

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

selftest: Avoid running the slowest of the "none" tests in samba-o3

This job is already quite long and these tests are unlikely
to vary between hosts or under the -O3 compile

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

smbd: Store share_entries in locking.tdb again

The "base on g_lock" patch is a slowdown because we do more tdb
operations. Getting share_entries.tdb back into locking.tdb tries to
speed things up again.

Now that we're based on g_lock we'll most likely use a different
method in order to spread the entries across multiple records.

It still maintains the sorted array of share modes within locking.tdb,
but not as part of the expensive ndr marshalling of the complete
array.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri May 15 02:27:49 UTC 2020 on sn-devel-184

smbd: Move share mode entry parsing up in the file

Will be used by parsing code soon

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Base locking.tdb on g_lock.c

This means we don't hold locking.tdb mutexes anymore while we do
filesystem-level operations. This has hurt ctdb quite a bit: A smbd hanging in
an unlink() syscall prevents ctdb from recovering locking.tdb.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lib: Add g_lock_dump_send/recv

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lib: Add g_lock_set_lock_order()

Optionally allow a database with g_lock format to participate in the dbwarp
lock order check. Will be used once locking.tdb is based upon g_lock.c

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lib: Add g_lock_writev_data

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lib: Allow additional dbufs in g_lock_store()

Preparation for g_lock_writev_data()

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lib: Add g_lock_seqnum()

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lib: Add g_lock_wake_watchers()

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

torture: Test g_lock_watch_data_send()/recv()

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lib: Add g_lock_watch_data_send/recv()

Same concept as dbwrap_watched_watch_send/recv: Get informed if the
underlying data of a record changes. This utilizes the watched
database that g_lock is based upon anyway. To avoid spurious wakeups
by pure g_lock operations this patch adds a sequence number for the
data that is stored in the g_lock data field.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lib: Allow DBWRAP_LOCK_ORDER_NONE in db_open()

locking.tdb will not have a LOCK_ORDER anymore, this will be done by
the code in g_lock.c. We need to allow opening a database with dbwrap
without having a lock order.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

dbwrap: Make dbwrap_lock_order_[un]lock() public

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

dbwrap: Simplify dbwrap_lock_order_[un]lock()

Directly pass the database name and lock order to the core functions,
avoid passing struct db_context.

In the next steps these functions will become public: locking.tdb will
be based on g_lock.c to avoid holding a tdb-level locking.tdb mutex
while doing complex file system operations like unlink() which can
take ages on FAT for example.

This means that g_lock.c will participate in the dbwrap lock order
protection and needs access to dbwrap_lock_order_[un]lock() without
providing a direct db_context.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

srvsvc: Move brl_get_locks() out of enum_file_fn()

With share_infos.tdb this is a locking order violation:
share_infos.tdb is level 4, brlock.tdb is level 2. Avoid this by first
walking the share_infos.tdb and then fetching all the brlock entries.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=14355

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Thu May 14 22:06:32 UTC 2020 on sn-devel-184

test: Show that netfileenum is broken

Bug: https://bugzilla.samba.org/show_bug.cgi?id=14355

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

rpcclient: Make netfileenum cmd print the path names

Needed for the next commit testing netfileenum

Bug: https://bugzilla.samba.org/show_bug.cgi?id=14355

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

rpcclient: Use struct initializers in cmd_srvsvc_net_file_enum()

Bug: https://bugzilla.samba.org/show_bug.cgi?id=14355

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

srvsvc: Collect file ids in enum_file_fn()

Will be used a few patches down

Bug: https://bugzilla.samba.org/show_bug.cgi?id=14355

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

srvsvc: Use a struct initializer in net_enum_files()

Bug: https://bugzilla.samba.org/show_bug.cgi?id=14355

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

srvsvc: Directly use "ctr3->count" instead of "i"

To me this was not very transparent, and now that we have "ctr3" a
single indirect looks okay

Bug: https://bugzilla.samba.org/show_bug.cgi?id=14355

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

srvsvc: Use a struct assignment in enum_file_fn()

Looks nicer than 5 complex array references...

Bug: https://bugzilla.samba.org/show_bug.cgi?id=14355

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

srvsvc: Introduce ctx3 helper var in enum_file_fn()

Bug: https://bugzilla.samba.org/show_bug.cgi?id=14355

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

vfs: remove root_dir_fid arg from SMB_VFS_CREATE_FILE()

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu May 14 19:43:27 UTC 2020 on sn-devel-184

smbd: move processing of root_dir_fid outside of SMB_VFS_CREATE_FILE()

The only two callers that pass a root_dir_fid != 0 are reply_ntcreate_and_X()
and call_nt_transact_create(). By moving root_dir_fid processing outside of
SMB_VFS_CREATE_FILE() we can avoid rewriting get_relative_fid_filename() to work
with *AT semantics and dirfsps.

Bonus points: the previous code in both functions passed fname to
filename_convert() which can't really have worked as it is relative to
root_dir_fid so filename_convert() will not be able to stat() the relative path
and it's components.

The only test I can find that uses a root dir_fid is raw.samba3rootdirfid and
that uses a handle on the share root which will work.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14380

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s4/torture: add a *real* root_dir_fid test

raw.samba3rootdirfid tests with the share root directory as root_dir_fid handle,
that doesn't cover the case where the relative name has more then one path
component. It only works because in unix_convert() we run into the creating file
optimasation.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14380

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

testprogs: Add 'net ads join' test for fips

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
Autobuild-User(master): Günther Deschner <gd@samba.org>
Autobuild-Date(master): Thu May 14 14:27:26 UTC 2020 on sn-devel-184

lib:param: Fix memory leak in lpcfg_load_internal()

Signed-off-by: awalker <awalker@ixsystems.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Wed May 13 17:34:43 UTC 2020 on sn-devel-184

ctdb: increase TasksMax limit, the systemd default is just 512

In 2015 systemd introduced a TasksMax which limits the number of processes in a
unit:

https://lists.freedesktop.org/archives/systemd-devel/2015-November/035006.html

The default of 512 may be too low in certain situations leading to vfork()
failing with errno=EAGAIN when trying to spawn lock-helper processes.

With the default for LockProcessesPerDB being 200 the increased TasksMax limit
should cover the problematic scenario.

Additional background: the failing vfork()s have been seen on production
clusters and were tracked down to being logged in the context of ctdb calling
tdb_repack().

Links:

https://github.com/systemd/systemd/commit/9ded9cd14cc03c67291b10a5c42ce5094ba0912f
https://www.suse.com/support/kb/doc/?id=000015901
https://success.docker.com/article/how-to-reserve-resource-temporarily-unavailable-errors-due-to-tasksmax-setting
https://www.percona.com/blog/2019/01/02/tasksmax-another-setting-that-can-cause-mysql-error-messages/

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Wed May 13 13:30:12 UTC 2020 on sn-devel-184

s3: libsmbclient: Finish unifing bad iconv behavior across CORE NT1 SMB2 protocols.

On bad name conversion, exit the directory listing with an error, but leave the
connection intact. We were already checking for finfo->name == NULL here,
but were ignoring it and not reporting an error.

Remove the knownfail.d/bad_iconv file as we now
behave the same across CORE/NT1/SMB2.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14374

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue May 12 21:32:44 UTC 2020 on sn-devel-184

s3: libsmb: In SMB2 return NT_STATUS_INVALID_NETWORK_RESPONSE if name conversion ended up with a NULL filename.

Can happen if namelen == 0.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14374

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

s3: libsmb: In SMB1 old protocol - return NT_STATUS_INVALID_NETWORK_RESPONSE if name conversion ended up with a NULL filename.

Can happen if namelen == 0.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14374

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

s3: selftest: Add test_smbclient_iconv.sh to check client behavior on bad name conversion.

SMB2 and NT1 fail this, CORE already returns NT_STATUS_INVALID_NETWORK_RESPONSE
on bad conversion.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14374

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

s3: selftest: Add share definition [bad_iconv] in fileserver.

Creates a utf8 valid filename within that is invalid in CP850.
Useful to test smbclient list directory character set conversions.

https://bugzilla.samba.org/show_bug.cgi?id=14374

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

vfs_io_uring: retry after a short writes in vfs_io_uring_pwrite_completion()

We need to be prepared for short writes from the kernel depending on
the state of the page cache.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs_io_uring: retry after a short read in vfs_io_uring_pread_completion()

We need to be prepared for short reads from the kernel depending on
the state of the page cache. Windows and Mac clients don't
expect short reads for files, so we need to retry ourself.

For the future we may be able to play with some io_uring flags
in order to avoid the retries in userspace, but for now we just fix
the data corruption bug...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs_io_uring: protect vfs_io_uring_fsync_completion() against invalid results

We should never get back a value > 0.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs_io_uring: protect vfs_io_uring_pwrite_completion() against invalid results

We should never get more acked than we asked for.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs_io_uring: protect vfs_io_uring_pread_completion() against invalid results

We should never get back more than we asked for.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs_io_uring: split out a vfs_io_uring_pwrite_submit() function

This can be reused when we add handling for short writes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs_io_uring: split out a vfs_io_uring_pread_submit() function

This can be reused when we add handling for short reads.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs_io_uring: split out a vfs_io_uring_request_submit() function

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs_io_uring: avoid stack recursion of vfs_io_uring_queue_run()

Instead we remember if recursion was triggered and jump to
the start of the function again from the end.

This should make it safe to be called from the completion_fn().

This is hideously complex stuff, so document the hell
out of it.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs_io_uring: make use of sys_valid_io_range() in vfs_io_uring_pwrite_send()

This makes the follow up commits easier as we don't have to care
about overflows.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs_io_uring: make use of sys_valid_io_range() in vfs_io_uring_pread_send()

This makes the follow up commits easier as we don't have to care
about overflows.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs_io_uring: move error handling out of vfs_io_uring_fsync_recv()

We should do that as early as possible and that's in
vfs_io_uring_fsync_completion().

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs_io_uring: move error handling out of vfs_io_uring_pwrite_recv()

We should do that as early as possible and that's in
vfs_io_uring_pwrite_completion().

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs_io_uring: move error handling out of vfs_io_uring_pread_recv()

We should do that as early as possible and that's in
vfs_io_uring_pread_completion().

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs_io_uring: introduce vfs_io_uring_request->completion_fn()

We'll need to add more logic than a simple _tevent_req_done()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs_io_uring: replace vfs_io_uring_request->state with _tevent_req_data()

We don't need a direct pointer to the state...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs_io_uring: fix the prefix for parametric options from 'vfs_io_uring' to 'io_uring'

This is what the manpage describes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3: VFS: default. Change pwrite() -> sys_pwrite_full() in SMB_VFS_PWRITE_SEND() to protect against short writes.

Note that as sys_pwrite_full() deals with the EINTR case
we can remove the do {} while loop here.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s3: VFS: default. Change pread() -> sys_pread_full() in SMB_VFS_PREAD_SEND() to protect against short reads.

Note that as sys_pread_full() deals with the EINTR case
we can remove the do {} while loop here.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s3: VFS: default. Change sys_pwrite() -> sys_pwrite_full() in SMB_VFS_PWRITE() to protect against short writes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s3: VFS: default. Change sys_pread() -> sys_pread_full() in SMB_VFS_PREAD() to protect against short reads.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s3: VFS: aio_fork: Change sys_pwrite() -> sys_pwrite_full() to protect against short writes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s3: VFS: aio_fork: Change sys_pread() -> sys_pread_full() to protect against short reads.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

smbd: add vfs_valid_{pread,pwrite}_range() checks where needed

I checked all callers of SMB_VFS_PWRITE[_SEND](),
all callers of SMB_VFS_PREAD[_SEND]() and also
places where we append to the file and allocate
more space.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:smbd: add vfs_valid_{pread,pwrite}_range() helper functions

These implement the SMB2 visible behavior of the [MS-FSA]
2.1.5.2 Server Requests a Read and 2.1.5.3 Server Requests a Write
constraints. Note that offset < 0 is not allowed over SMB.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:smbd: handle 0 length writes as no-op.

They should never touch the SMB_VFS layer
and they never trigger an DISK_FULL error.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smb2_server: fix smbd_smb2_request_verify_sizes() for SMB2_OP_WRITE

Writes with a length of 0 are allowed.

The readfile related check we had before was not really useful
as min_dyn_len can only every be 0 or 1 (and for SMB2_OP_WRITE it's
always 1). So we checked
  if (unread_bytes > 0) {
     if (unread_bytes < 1) {
       return error;
     }
  }

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lib: util: Add sys_pwrite_full().

A pwrite wrapper that will deal with EINTR and never return a short
write unless the file system returns an error. Copes with the
unspecified edge condition of pwrite returning zero by changing
the return to -1, errno = ENOSPC.

Thread-safe so may be used as a replacement for pwrite
inside pwrite_do() thread functions.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Jeremy Allison <jra@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>

lib: util: Add sys_pread_full().

A pread wrapper that will deal with EINTR and never return a short
read unless pread returns zero meaning EOF.

Thread-safe so may be used as a replacement for pread
inside pread_do() thread functions.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Jeremy Allison <jra@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>

lib: util: Add sys_valid_io_range()

This implements the contraints of
[MS-FSA] 2.1.5.2 Server Requests a Read.

The special handling of [MS-FSA] 2.1.5.3 Server Requests a Write
with offset < 0, should be handled by higher layers!
Which means the check can also be used for writes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s4:torture: add tests to test the SMB2 read/write offset/length boundaries

[MS-FSA] 2.1.5.2 Server Requests a Read and
2.1.5.3 Server Requests a Write define some contraints.

These tests demonstrate that ((int64_t)offset) < 0) is
not allowed for both reads and writes for SMB.
Also the special case for writes at offset -2 is not possible
nor the append mode with offset < 0.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14361

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3/locking: prime flags in a fresh sharemode data object

This is needed to prime the logic in share_mode_flags_restrict() for the
following scenario:

* (First) CREATE on a file with FILE_SHARE_NONE and
  access_mask=FILE_READ_ATTRIBUTES (a stat-open).

* share_mode_flags_restrict() gets called with

    share_mode_flags_restrict(flags=0,
                              access_mask=0x80,
                              share_mode=0,
                              lease_type=UINT32_MAX)

  and returns a value where none of the FILE_SHARE_* flags is set.

As a result share_mode_data.flags doesn't reflect the share-modes in effect.

This doesn't change any current visible behaviour outside of open_mode_check(),
but it avoids calling share_mode_forall_entries() in open_mode_check_fn().

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14375

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Tue May 12 19:52:48 UTC 2020 on sn-devel-184

smbd: fix for bug 14375

... with many thanks to an enthusiastic Samba user from Poland for helping to
track this down.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14375

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

smbd: make conflicting_access available to other functions

The next commit adds more users of conflicting_access.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14375

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

s4/torture: reproducer for bug 14375

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14375

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

selftest: split a knownfail entry

Lists the two existing subtests indidivually in preparation of adding a third
that is going to pass against ad_dc_ntvfs.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14375

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

winbind.service: drop quotes from $WINBINDOPTIONS variable

systemd parser splits arguments at whitespaces respecting quotes
when necessary.

Signed-off-by: Marcos Mello <marcosfrm@gmail.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Tue May 12 12:27:11 UTC 2020 on sn-devel-184

Drop NotifyAccess=all from systemd units

The implicit NotifyAccess=main is enough since Samba daemons do not
fork()/exit() anymore under systemd.

Signed-off-by: Marcos Mello <marcosfrm@gmail.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>

lib ldb: Limit depth of ldb_parse_tree

Limit the number of nested conditionals allowed by ldb_parse tree to
128, to avoid potential stack overflow issues.

Credit Oss-Fuzz

REF: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=19508

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Gary Lockyer <gary@samba.org>
Autobuild-Date(master): Sun May 10 23:21:08 UTC 2020 on sn-devel-184

lib util ASN.1: Panic on ASN.1 tag mismatch

If the ASN.1 depth is zero in asn1_end_tag, call smb_panic. Rather than
ignoring the condition.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4 cldap server tests: request size limit tests

Add tests for packet size limits.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4 ldap server tests: request size limit tests

Extra tests for ldap maximum request size limits.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Fix clang 9 for-loop-analysis warnings

Review-note: The for loop increment operation was changed and the
             trailing i++ was removed from the loop body.
             The resulting for statement is equivalent to the original

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Fri May  8 11:16:18 UTC 2020 on sn-devel-184

Fix clang 9 format-nonliteral warnings

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>

Fix clang 9 missing-field-initializer warnings

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>

Fix clang 9 unused-function warnings

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>

Fix clang 9 enum-conversion warnings

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>

Fix clang 9 constant-conversion warnings

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>

Fix clang 9 logical-not-parentheses warnings

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>

Fix clang 9 parentheses-equality warnings

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>

build: Allow developer builds with clang 9

Disable clang warnings to allow samba to be compiled with clang 9.

Subsequent commits will fix offending code and re-enable the warnings.

However fixing cast-align warnings has been left for later.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>

s3: VFS: Complete the replacement of SMB_VFS_GET_NT_ACL() -> SMB_VFS_GET_NT_ACL_AT().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu May  7 21:04:59 UTC 2020 on sn-devel-184

s3: VFS: full_audit. Remove get_nt_acl_fn().

No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: time_audit. Remove get_nt_acl_fn().

No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: unityed_media. Remove get_nt_acl_fn().

No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: zfsacl. Remove get_nt_acl_fn().

No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: snapper. Remove get_nt_acl_fn().

No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: VFS: shadow_copy2. Remove get_nt_acl_fn().

No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>