s4:libcli: allow smb2_composite_unlink* to truncate the file before close

This is needed to delete streams...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15656

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

s3:tests: make use of TMPDIR in test_symlink_traversal_*.sh

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

lib/torture: add torture_assert_nttime_equal_goto()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15656

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

librpc/ndr: remove unused NDR_PUSH_ALLOC_SIZE()

Pair-Programmed-With: Volker Lendecke <vl@samba.org>

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

librpc/ndr: avoid alignment allocation in ndr_push_DATA_BLOB()

Pair-Programmed-With: Volker Lendecke <vl@samba.org>

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

libndr: Streamline ndr_token_retrieve_cmp_fn

Rename the public function to ndr_token_peek_cmp_fn, the only user
does not remove the token. Factor out ndr_token_find to move the
token-removing logic to ndr_token_retrieve, the only caller that does
remove the token.

Keep libndr at 6.0.0, this has not been released yet.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jennifer Sutton <josutton@catalyst.net.nz>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Thu Aug 29 08:40:52 UTC 2024 on atb-devel-224

libndr: Simplify ndr_token_retrieve_cmp_fn()

Avoid an if-statement inside by passing a pointer-comparing function

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jennifer Sutton <josutton@catalyst.net.nz>

libcli/smb: Fix failure of Smb3UnixTests.test_create_context_reparse

On tumbleweed at least the definition in py_reparse_put of 'reserved' as
'unsigned' causes the tag value to be overwritten. Note: ParseTuple is
given a format of 'Kk' where
  K = unsigned long long (for tag)
  k = unsigned long      (for reserved)

The problem is 'reserved' is defined as 'unsigned' which on a
64 bit linux system has size 4. The size however of the 'unsigned long' type
on the same 64 bit system is 8. This causes 'tag' to be overwritten by
the value of 'reserved' because it's destination size is smaller than
expected.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15702
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Wed Aug 28 18:38:01 UTC 2024 on atb-devel-224

smb2_ioctl: fix truncated FSCTL_QUERY_ALLOCATED_RANGES responses

As per MS-FSA 2.1.5.10.22 FSCTL_QUERY_ALLOCATED_RANGES, if response
range entries exceed in_max_output, then we should respond with
STATUS_BUFFER_OVERFLOW and a truncated output buffer.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=15699

Reported-by: David Howells <dhowells@redhat.com>
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
Autobuild-User(master): David Disseldorp <ddiss@samba.org>
Autobuild-Date(master): Wed Aug 28 08:54:11 UTC 2024 on atb-devel-224

s4:torture/smb2: test FSCTL_QUERY_ALLOCATED_RANGES truncation

FSCTL_QUERY_ALLOCATED_RANGES responses with more than one range should
be truncated to account for a ioctl.smb2.in.max_output_response limit.
Add a test for this.

Flag the new test knownfail; fix in subsequent commit.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Noel Power <npower@samba.org>

dsdb:cracknames: free more on error (CID 240724)

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jennifer Sutton <josutton@catalyst.net.nz>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Wed Aug 28 05:39:36 UTC 2024 on atb-devel-224

dsdb:util: dsdb_module_dn initialises on failure

I think this may be a root cause of some Coverity false positives.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jennifer Sutton <josutton@catalyst.net.nz>

dsdb:mod:operational: initialise a pointer (CID 1499411)

A Coverity false positive (we check for error) but it is worth
doing per README.Coding

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jennifer Sutton <josutton@catalyst.net.nz>

ndr:dnsp: avoid theoretical int overflow (CID 1609418)

Coverity points out that if the string is longer than INT_MAX, the int
will overflow and the cast to uint8_t will discard bits.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jennifer Sutton <josutton@catalyst.net.nz>

ldb:kv_index: help static analysers to not worry (CID 1615192)

The point of this realloc is that we are not using this array, but
keeping it around to remain a node the talloc tree. We'd prefer to
reduce it to nothing.

Coverity rightly spotted that it was reallocing an array of `struct
ldb_val` to an array of `struct ldb_val *`, which has a different size
and all. But it doesn't matter in this case, because we will never use
it.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15590

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jennifer Sutton <josutton@catalyst.net.nz>

s4:drs:test:getncchanges skips some tests with reserved_usn = 0

These tests are not affected by the reserved_usn change, so there is
no need to run them twice.

The test_repl_get_tgt_multivalued_links fails with or without
reserved_usn set to zero, but it fails differently in either case.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15701

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jennifer Sutton <josutton@catalyst.net.nz>

s4:drs:test:getncchanges: remove timeout failure

We don't need a timeout failure any more, since replication should
always work. Leaving the timeout in might sometimes cause a flapping
test if replication is being slow for some reason.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15701

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jennifer Sutton <josutton@catalyst.net.nz>

s4:drsuapi:getncchanges: allow 0 reserved_usn reply

Azure AD will set reserved_usn to zero when we expect it to be
the number we gave them.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15701

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jennifer Sutton <josutton@catalyst.net.nz>

s4:drsuapi:getncchanges: use DBG_ERR() macro

The next commit will indent this more, so it's a bit squished up.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15701

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jennifer Sutton <josutton@catalyst.net.nz>

s4:drsuapi:getncchanges: fix whitespace

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15701

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jennifer Sutton <josutton@catalyst.net.nz>

s4:drs:tests: repeat getncchanges test with zero reserved_usn

This emulates the behaviour of Azure AD.

As this is quite slow we will later reduce the test load in this case,
but for now we want to run all the getncchanges tests this way.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15701

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jennifer Sutton <josutton@catalyst.net.nz>

s4:drs:tests: add hook for changing highwatermark

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15701

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jennifer Sutton <josutton@catalyst.net.nz>

s4:drs:test:getncchanges: add a timeout failure

In the next commit we are going to add tests in which the client
modifies the highwatermark in a way that resets replication (on Samba
only). After that we'll fix it.

If we leave the test in an eternal loop, the commit history will not
be bisectable, so we are temporarily going to turn long waits into
failures.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15701

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jennifer Sutton <josutton@catalyst.net.nz>

libcli:auth: Remove unreachable code (CID 1272968)

For us to reach the statement ‘if (0 < len1)’, ‘len1’ must be equal to
‘len2’, and they must not both be equal to zero. That cannot be the case
if ‘len1’ is equal to zero, and therefore the ‘else’ branch cannot be
reached.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

util:charset: Remove unreachable code (CID 1272948)

Suppose that ‘slen’ is equal to (size_t)-1. A few lines up, we had:

    if (lastp != 0) goto slow_path;

Therefore, ‘lastp’ must evaluate to false.

Now suppose that ‘slen’ is not equal to (size_t)-1. In that case, we
would have executed:

    if (slen != 0) goto slow_path;

Therefore, ‘slen’ must evaluate to false.

Consequently, this code can be seen to be unreachable.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

librpc: Speed up GUID_buf_string()

This is faster than calling snprintf().

Signed-off-by: Jo Sutton <josutton@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

lib/util: Speed up slow data-blob-to-hex functions

This is much faster than calling sprintf() for every byte of data, and
improves the performance of functions outputting binary DNs.

Signed-off-by: Jo Sutton <josutton@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

bootstrap: Migrate to Rocky8

Signed-off-by: David Mulder <dmulder@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Wed Aug 28 00:24:35 UTC 2024 on atb-devel-224

idl: dnsp.h references NTTIME

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jennifer Sutton <jsutton@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Tue Aug 27 08:36:11 UTC 2024 on atb-devel-224

idl: misc.idl references DATA_BLOB

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jennifer Sutton <jsutton@samba.org>

pidl: Add headerhelper idl property

Add includes to for example misc.h if needed, so that misc.h can be
directly included without prerequisites

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jennifer Sutton <jsutton@samba.org>

lib: Make a few functions static

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jennifer Sutton <jsutton@samba.org>

libsmb: Simplify pdb_sethexpwd with hex_byte()

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jennifer Sutton <jsutton@samba.org>

pidl:Samba4/Header: only include ntstatus.h if required

Pair-Programmed-With: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jennifer Sutton <jsutton@samba.org>

pidl: add scalarTypeUsed() helper to find if a type was actually used...

Pair-Programmed-With: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jennifer Sutton <jsutton@samba.org>

lib: Add libcli/util/ntstatus.h in a few headers where it's needed

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jennifer Sutton <jsutton@samba.org>

idl: Remove unused KRB5_EDATA_NTSTATUS

I think the last user went away with e2028837b95 in 2017

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jennifer Sutton <jsutton@samba.org>

smbd: Inline set_sticky_write_time_path into set_sticky_write_time_fsp

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jennifer Sutton <jsutton@samba.org>

smbd: Modernize a few DEBUGs

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jennifer Sutton <jsutton@samba.org>

smbd: Introduce "ace" helper variable

Use "psd->dacl->aces[0]" once instead of 3 times

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jennifer Sutton <jsutton@samba.org>

idl: security.idl compiles with only secace.h

It does not need all of security.h

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jennifer Sutton <jsutton@samba.org>

pidl: replace.h is sufficient in ndr_*.c

Avoid a few uses of includes.h

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jennifer Sutton <jsutton@samba.org>

lib: Add a few explicit includes

Required to trim down includes from gen_ndr/*security*

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jennifer Sutton <jsutton@samba.org>

libndr: Use _talloc_array() to benefit from overflow protection

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jennifer Sutton <jsutton@samba.org>

libndr: Remove 368 bytes from R/W data into R/O text segment

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jennifer Sutton <jsutton@samba.org>

smbd: Modernize a few DEBUGs

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jennifer Sutton <jsutton@samba.org>

ctdb: Add a NULL check to convert_node_map_to_list()

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jennifer Sutton <jsutton@samba.org>

source3/wscript: Introduce auto mode to build ceph vfs modules

Use 'auto' mode as the default for building ceph vfs modules so that an
explicit --enable-cephfs can reliably fail in the absence of required
dependencies.

ref: https://lists.samba.org/archive/samba/2024-August/249569.html

Signed-off-by: Anoop C S <anoopcs@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
Autobuild-User(master): Anoop C S <anoopcs@samba.org>
Autobuild-Date(master): Tue Aug 27 06:18:51 UTC 2024 on atb-devel-224

s3:smbd: fix NULL dereference in case of readlink failure

When VFS readlinkat hook returns with error the following sequence
yields NULL-pointer dereference (SIGSEGV):

  symlink_target_below_conn (source3/smbd/open.c)
    char *target = NULL;
    ...
    readlink_talloc (source3/smbd/files.c)
      SMB_VFS_READLINKAT
        smb_vfs_call_readlinkat (source3/smbd/vfs.c)
          handle->fns->readlinkat_fn --> returns error

  status = safe_symlink_target_path(.., target /* NULL */ ..)
    safe_symlink_target_path (source3/smbd/filename.c)
      if (target[0] == '/') { /* NULL pointer dereference */

A failure in VFS module's readlinkat hook may happen due to run-time
error (e.g., network failure which cases libcephfs to disconnect from
MDS).

Bug: https://bugzilla.samba.org/show_bug.cgi?id=15700
Signed-off-by: Shachar Sharon <ssharon@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Fri Aug 23 09:27:06 UTC 2024 on atb-devel-224

s3:tests: Fix spelling error

Signed-off-by: Martin Schwenke <mschwenke@ddn.com>
Reviewed-by: Anoop C S <anoopcs@samba.org>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Thu Aug 22 10:38:09 UTC 2024 on atb-devel-224

codespell: Ignore CTDB bin/

This might be hanging around if standalone CTDB builds have been done.

Signed-off-by: Martin Schwenke <mschwenke@ddn.com>
Reviewed-by: Anoop C S <anoopcs@samba.org>

vfs_ceph_new: add missing newline in debug-logging

Commit d00f20f3 ("vfs_ceph_new: debug-log upon libcephfs low-level
calls") introduced debug-logging before each call to libcephfs low-level
APIs. Unfortunately, one of the logging messages missed the terminating
newline ('\n') character.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
Reviewed-by: John Mulligan <jmulligan@samba.org>
Reviewed-by: Anoop C S <anoopcs@samba.org>
Autobuild-User(master): Anoop C S <anoopcs@samba.org>
Autobuild-Date(master): Wed Aug 21 14:18:07 UTC 2024 on atb-devel-224

libsmb: Remove cli_state->rap_error

I could not find any caller which uses rap_error outside of the
routine implementing the rap client call itself.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Wed Aug 21 12:49:45 UTC 2024 on atb-devel-224

libsmb: Remove unused cli_NetWkstaUserLogoff()

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>

libsmb: Remove unused cli_ns_check_server_type()

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>

WHATSNEW: add 'vfs mkdir use tmp name' option

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15693

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Wed Aug 21 09:18:00 UTC 2024 on atb-devel-224

s3:smbd: let mkdir_internal() try VFS_RENAME_HOW_NO_REPLACE first

With renameat2(RENAME_NOREPLACE) being available
it's even better, as we don't even have the short
window where the incomplete directory is visible
to others.

The flow will be this:

tmp_name = ".::TMPNAME:D:$PID:client_name"
mkdirat(tmp_name, mode=client_mode);
prepare_acls(tmp_name);
renameat2(tmp_name, client_name, NOREPLACE);
if (EEXIST) {
   unlinkat(tmp_name);
   return EEXIST;
}
if (EINVAL) {
   /* fallback if NOREPLACE is not supported */
   mkdirat(client_name, mode=0);
   if (EEXIST) {
      unlinkat(tmp_name);
      return EEXIST;
   }
   renameat(tmp_name, client_name);
}

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15693

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3:vfs_default: implement VFS_RENAME_HOW_NO_REPLACE in vfswrap_renameat()

We can use renameat2() without ifdef as libreplace always provides it.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15693

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3:vfs: add VFS_RENAME_HOW_NO_REPLACE

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15693

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3:vfs: add vfs_rename_how to SMB_VFS_RENAMEAT()

This will support renameat2-like operations in future.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15693

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

lib/replace: add renameat2() replacement

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15693

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3:smbd: let mkdir_internal() work more atomically using a temporary name

Currently we do this in mkdir_internal():

mkdirat(client_name);
if (EEXIST) {
   return EEXIST;
}
prepare_acls(client_name);

Note 'prepare_acls()' is a placeholder for the complex steps
it is doing to prepare the directory. During these steps
we have the problem that other clients already see
the directory and are able to create files or subdirectories
in it and these may not inherit the correct acls as
the their parent directory is not created completely.

I think I found a good strategie even without relying on
renameat2(RENAME_NOREPLACE).

We would do this instead:

tmp_name = ".::TMPNAME:D:$PID:client_name"
mkdirat(tmp_name, mode=client_mode);
prepare_acls(tmp_name);
mkdirat(client_name, mode=0);
if (EEXIST) {
   unlinkat(tmp_name);
   return EEXIST;
}
renameat(tmp_name, client_name);

So instead of having a long windows during prepare_acls,
we just have a short window between mkdirat(client_name, mode=0)
and renameat(tmp_name, client_name);
And in that short window the directory with the client_name
has a mode of 0, so it's not possible for other clients
to create files or subdirs in it.

As the mkdirat(client_name, mode=0) still catches
EEXIST the race where two clients try to create
the same client_name is closed as before,
so we don't need any other protection.

Following patches will make use of renameat2(RENAME_NOREPLACE),
but this already a very good improvement.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15693

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

docs-xml: add 'vfs mkdir use tmp name' option

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15693

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3:smbd: improve DEBUG messages in mkdir_internal()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15693

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3:smbd: let mkdir_internal() use a talloc_stackframe()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15693

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3:smbd: call apply_new_nt_acl() already in mkdir_internals()

We should have preparation steps as close as possible together.

We could move it for files as well, but that's a task for another
day...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15693

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3:smbd: split out apply_new_nt_acl() helper

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15693

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

lib/util: let server_id_str_buf_unique() use server_id_buf

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15693

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

lib/util: split out server_id_from_string_ex allow the unique delimiter to be passed in

In future it also allows to specifiy another delemiter than '/'
in order to use the resulting buffer as part of a file/directory
name.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15693

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

lib/util: add server_id_str_buf_unique_ex() and allow the delimiter to be passed in

This way the server_id_str_buf() can just be a tiny wrapper using
'\0' as delemiter.

In future it also allows to specifiy another delemiter than '/'
in order to use the resulting buffer as part of a file/directory
name.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15693

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

lib/util: let server_id_str_buf() use PRIu32 and PRIu64

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15693

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

lib/util: add a comment to struct server_id_buf

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15693

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s4:torture/smb2: add smb2.create.mkdir-visible

This reproduces a race where one client creates
a directory and other clients see it before
the directory is fully setup including the correct
permissions and similar things.

We have a DENY ACE for SEC_DIR_ADD_FILE, which means
that files can't be created. This is set on
a base directory 'mkdir_visible'.

Then we have a lot of async loops trying to create
a file called 'mkdir_visible\dir\file_NR'. These loop
as fast as possible expecting OBJECT_PATH_NOT_FOUND,
because 'mkdir_visible\dir' is not there.

Then we send a create for 'mkdir_visible\dir' and
expect that to work.

This should turn the 'mkdir_visible\dir\file_NR' loop
into getting ACCESS_DENIED, because the
DENY ACE for SEC_DIR_ADD_FILE should be inherited
before 'mkdir_visible\dir' is visible to other clients.

Because of the complex steps in mkdir_internal(),
smbd allows the creation 'mkdir_visible\dir\file_NR',
as 'mkdir_visible\dir' is already visible after the
mkdirat(), before the DENY ACE is inherited.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15693

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

lib/param: Don't treat a missing include file as an error in handle_include().

Same fix as in commit 09d7690

'samba-tool domain provision -d10' fails if the included file does not
exist:

lpcfg_load: refreshing parameters from /etc/samba/smb.conf
Processing section "[global]"
Can't find include file /etc/samba/usershares.conf
pm_process() returned No
ERROR: Unable to load default file
  File "/usr/lib64/python3.12/site-packages/samba/netcmd/domain/provision.py", line 183, in run
    lp = sambaopts.get_loadparm()
         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/site-packages/samba/getopt.py", line 282, in get_loadparm
    self._lp.load_default()

Signed-off-by: Pavel Filipenský <pfilipensky@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Wed Aug 21 00:04:19 UTC 2024 on atb-devel-224

ctdb-scripts: Avoid flapping NFS services at startup

If an NFS service check is set to, say, unhealthy_after=2 then it will
always switch from the (default startup) unhealthy state to healthy,
even if there is a fatal problem.  If all services/scripts appear OK
then the node will become healthy.  When the counter hits the limit it
will return to unhealthy.  This is misleading.

Instead, never use the counter at startup, until the service becomes
healthy.  This stops services flapping unhealthy-healthy-unhealthy.

A side-effect is that a service that starts in a broken state will
never be restarted to try to fix the problem.  This makes sense.  The
counting and restarting really exist to deal with problems that might
occur under load.  The first monitor events occur before public IPs
are hosted, so there can be no load.  If a service doesn't start
reliably the first time then the admin probably wants to know about
it.

nfs_iterate_test() is updated to run an initial monitor event to mark
the services as healthy.  This initialises the counter so it can be
used for the important part of the test.  Passing the -i option avoids
running the extra monitor event, so the first iteration will be the
initial monitor event.

Signed-off-by: Martin Schwenke <mschwenke@ddn.com>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-scripts: Make initial statistics output empty

This makes initial failure to retrieve statistics less likely to
result in a statistics change.  To help with this, statistics
retrieval stderr now goes to the log - only stdout goes to the file.

This means that the test code for checking statistics changes needs to
be redone to actually run the statistics command and check.  As with
rpcinfo output, this output needs to behave as deterministically in
the test code as it done in the event script.

Signed-off-by: Martin Schwenke <mschwenke@ddn.com>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-scripts: Only consider statistics on timeout

Checking statistics is only really relevant to timeouts.  That is, if
an rpcinfo times out it is worth checking if the service making
progress.  If the RPC service is not registered then the statistics
don't need to be checked because they shouldn't be changing.

The 2 previously added tests added to check statistics progress now
behave identically and fail on all iterations.  To support testing
with "timeouts", an optional TIMEOUT flag can now be added to the RPC
service passed to nfs_iterate_test().  2 new tests are added to
exercise the new behaviour.

The 2 new "if" statements in nfs_iterate_test() could be combined.
However, a subsequent commit would split them and would be more
difficult to read.

Signed-off-by: Martin Schwenke <mschwenke@ddn.com>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Make _rpc_service_up() and _rpc_services_down() internal

Signed-off-by: Martin Schwenke <mschwenke@ddn.com>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Make NFS RPC monitoring tests consistent

Update the remaining RPC monitoring tests to use nfs_iterate_test(),
depending on it to set results.  This makes all RPC monitoring tests
consistent, so they will all benefit from future improvements.

Signed-off-by: Martin Schwenke <mschwenke@ddn.com>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Drop unnecessarily "else"

Doing this in a previous commit would have made it more difficult to
read that commit.

Signed-off-by: Martin Schwenke <mschwenke@ddn.com>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Replace implicit healthy behaviour with early exits

The early exits from the sub-shell make the obvious successes much
more obvious, and slightly simplify the code that follows.

Signed-off-by: Martin Schwenke <mschwenke@ddn.com>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Simplify handling of statistics change

Handling this across two different functions led to insanity, so
simplify.

The handling of unhealthy_after when $_numfails = 0 implicitly causes
the node to be healthy.  This is how the "rpcinfo succeeds" case
works.  Doing it this way for statistics makes this patch easier to
read.  The implicit behaviour will go away in the next patch.

Signed-off-by: Martin Schwenke <mschwenke@ddn.com>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Move result check to rpc_set_service_failure_response()

The current structure here is wrong and repetitive.  Checking rpcinfo
result and determining output should be in the same place.

Failure counting is now contained in
rpc_set_service_failure_response(), but needs a file to survive the
sub-shell.

Don't attempt to combine and simplify code yet.  That would make this
commit harder to review.

Signed-off-by: Martin Schwenke <mschwenke@ddn.com>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Initialise return code file

The output file is initialised, so doesn't need to be created on
success.  Treat the return code file the same way.

Signed-off-by: Martin Schwenke <mschwenke@ddn.com>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Add function rpc_failure() to log failures and warnings

Improves readability, makes future changes easier.

Signed-off-by: Martin Schwenke <mschwenke@ddn.com>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Argument 3 to nfs_iterate_test() is up iteration

Nothing more complex is ever done, so we might as well simplify and
reduce coupling.

Signed-off-by: Martin Schwenke <mschwenke@ddn.com>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: nfs_iterate_test() marks RPC service down

If an RPC service is given, it is automatically marked down.  This
avoids repetition in test cases and loosens coupling.

Signed-off-by: Martin Schwenke <mschwenke@ddn.com>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tcp: Remove a use of ctdb_addr_to_str()

This one is in a rarely used error path, so call a function that
talloc()s the string instead.

Again, this will also print the port, which might be useful if we ever
add the ability to also specify ports in the nodes list.

Signed-off-by: Martin Schwenke <mschwenke@ddn.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Tue Aug 20 14:24:14 UTC 2024 on atb-devel-224

ctdb-tcp: Consolidate failure code

Same thing several times, so change to common failure code.

Signed-off-by: Martin Schwenke <mschwenke@ddn.com>
Reviewed-by: Volker Lendecke <vl@samba.org>

ctdb-tcp: Use already constructed node name

Node has been found, so use the pre-constructed name instead of
calling ctdb_addr_to_str().

This will also print the port, which might be useful if we ever add
the ability to also specify ports in the nodes list.

Signed-off-by: Martin Schwenke <mschwenke@ddn.com>
Reviewed-by: Volker Lendecke <vl@samba.org>

ctdb-tcp: Use path_rundir_append() to construct lock_path

The current constant value doesn't respect CTDB_TEST_MODE/CTDB_BASE.
Instead use the path module to allow automatic listening in test mode
with local daemons.

A single node can be tested with local daemons, using something like:

  $ tests/local_daemons.sh foo setup -n 1 -C "node address"
  $ grep "node address" foo/node.0/ctdb.conf
      # node address = 127.0.0.1
  $ tests/local_daemons.sh foo start all
  $ tests/local_daemons.sh foo print-log 0 | grep -i chose
  ... node.0 ctdbd[24546]: ctdb chose network address 127.0.0.1:4379

The trick is that commenting out the node address in ctdb.conf means
the chosen node address is the first one from the nodes file that
allows bind/listen.  In this case it is the only line.

The following ensures that automatic listening works for a node that
isn't the first:

  $ cat >mynodes
  192.168.1.1
  127.0.0.1
  $ tests/local_daemons.sh foo setup -n 2 -N mynodes -C "node address"
  $ grep "node address" foo/node.1/ctdb.conf
      # node address = 127.0.0.1
  $ tests/local_daemons.sh foo start 1
  $ tests/local_daemons.sh foo print-log 1 | grep -i chose
  [...] node.1 ctdbd[22787]: ctdb chose network address 127.0.0.1:4379

Note that the first address isn't local on this host, so will always
fail.

So, doing the above and starting both nodes yields...

  ...
  $ tests/local_daemons.sh foo start 1
  $ sleep 3; tests/local_daemons.sh foo start 0
  $ tests/local_daemons.sh foo print-log all | grep -i 'chose\|bind'
  [...] node.1 ctdbd[26351]: ctdb chose network address 127.0.0.1:4379
  [...] node.0 ctdbd[26438]: ctdb_tcp_listen_addr: Failed to bind() to socket - Address already in use (98)
  [...] node.0 ctdbd[26438]: Unable to bind to any node address - giving up

... as expected.

It would be nice to add tests for this, but we don't really have
infrastructure for that.  At least manual testing shows, for the
obvious cases, the previous commits didn't break anything.  :-)

Signed-off-by: Martin Schwenke <mschwenke@ddn.com>
Reviewed-by: Volker Lendecke <vl@samba.org>

ctdb-ib: Remove a use of ctdb_set_error()

Now the transport code is free of ctdb_set_error().

Signed-off-by: Martin Schwenke <mschwenke@ddn.com>
Reviewed-by: Volker Lendecke <vl@samba.org>

ctdb-tcp: Factor out listening code to avoid repetition

Modernise debug and comments while here.

Signed-off-by: Martin Schwenke <mschwenke@ddn.com>
Reviewed-by: Volker Lendecke <vl@samba.org>

ctdb-tcp: Use talloc_strdup() instead of repeating logic

The node name is already constructed when the nodes file is loaded, so
just copy the node name.

Signed-off-by: Martin Schwenke <mschwenke@ddn.com>
Reviewed-by: Volker Lendecke <vl@samba.org>

ctdb-daemon: Remove a use of ctdb_errstr()

Code to setup the transport is about to be cleaned up, including
removing uses of ctdb_set_error(), so avoid logging a NULL pointer or
some other old error.

Signed-off-by: Martin Schwenke <mschwenke@ddn.com>
Reviewed-by: Volker Lendecke <vl@samba.org>

libsmb: Remove cli_state->raw_status

All callers which looked at this are gone

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Tue Aug 20 10:57:12 UTC 2024 on atb-devel-224

libsmb: Remove unused cli_is_error()

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

libsmb: Remove a call to cli_is_error()

We are already in the !NT_STATUS_IS_OK if-branch, so this call was
redundant

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

libsmb: Remove unused cli_errno()

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

libsmb: Don't set errno in cli_NetServerEnum()

No caller looks at it, and if someone should ever want to return a
more informative error, we now have NTSTATUS

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>