smb2_server: split smbd_smb2_advance_incoming() out of smbd_smb2_io_handler()

The logic in smbd_smb2_advance_incoming() will be reused for io_uring.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smb2_server: remove state->hdr.done and always set state->vector first

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smb2_server: move struct msghdr to smbd_smb2_request_read_state

This makes the code a little bit faster, but the main reason
is the preparation of io_uring support, as it can't be on the
stack for async operations.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smb2_server: avoid ZERO_STRUCT*() in the core code

We should avoid calling memset_s() in the core smbd processing,
we can use struct initializers instead.

This reduces the overhead...

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

testprogs: Add net offlinejoin composeodj tests

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13577

Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Sep  5 22:11:46 UTC 2023 on atb-devel-224

testprogs: Cleanup machine account in net offlinejoin tests

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13577

Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s3:net: Allow to load ODJ blob from stdin

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13577

Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s3:net: Load ODJ blob from file only if "loadfile" parameter is present

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13577

Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s3:net: Add "net offlinejoin composeodj" command

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13577

Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s3:libnetapi: Implement NetComposeOfflineDomainJoin_l()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13577

Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s3:libnetapi: Add NetComposeOfflineDomainJoin() to API.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13577

Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s3:libnetapi: Add NetComposeOfflineDomainJoin() boilerplate

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13577

Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s3:libnetapi: Add NetComposeOfflineDomainJoin() to IDL

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13577

Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s3:libnetapi: Add some comments to document ODJ blob charset conversions

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13577

Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s3:libnetapi: Return error from RequestOfflineJoin

The error code must be returned to caller even if the error string is not set.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13577

Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pidl/tests: Add tests for hang with nested struct.

make sure hang test calls Parse::Pidl::Typelist::LoadIdl which triggers
part of the hang

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pidl/lib: Add recursion detection logic to prevent looping.

Under some circumstances 'can_contain_deferred' & 'align_type functions' can
loop.

This prevents a hang when processing sample idl like

interface hang
{
typedef [public] struct {
wsp_cbasestoragevariant variant[NUM_ENTRIES];
} vt_variant_wrap;

typedef [public,nodiscriminant,switch_type(uint16)] union {
[case(VT_I1)] int8 vt_i1;
[case(VT_VARIANT)] vt_variant_wrap vt_variant_wrap;
} variant_types;

typedef [public] struct {
[switch_is(vtype)] variant_types vvalue;
} wsp_cbasestoragevariant;
};

which will hang with the following command

   pidl --header --ndr-parser -- foo.idl

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

.gitlab-ci: Do builds under /builds as this is never an overlayfs

On the GitLab shared runners / is overlayfs, which /builds being ext,
so we want this real filesystem, which should be faster in any case.

This may allow us to use GitLab shared runners for more jobs.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Tue Sep  5 14:20:48 UTC 2023 on atb-devel-224

s3:torture: Remove masktest.c

The file is not built.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Tue Sep  5 07:30:12 UTC 2023 on atb-devel-224

docs:smbdotconf: Inform that changing 'winbind max domain connections' needs a restart

Signed-off-by: Pavel Filipenský <pfilipensky@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Pavel Filipensky <pfilipensky@samba.org>
Autobuild-Date(master): Wed Aug 30 13:37:37 UTC 2023 on atb-devel-224

s3:winbindd: Use a correct value for the length of domain children

We often loop over the array of domain children. However, the size of
the array is calculated as lp_winbind_max_domain_connections() which can
change (it is based on smb.conf). The fix is the talloc_array_length().

Reproducer:

winbind max domain connections = 100

smbcontrol all reload-config
smbcontrol all debug 10

/var/log/samba/log.winbindd shows many lines with random garbage pid:

[2023/08/25 10:03:49.898994, 10, pid=158296, effective(0, 0), real(0, 0), class=winbind] ../../source3/winbindd/winbindd_dual.c:885(winbind_msg_relay_fn)
  winbind_msg_relay_fn: sending message to pid 1037686087.
[2023/08/25 10:03:49.899010,  3, pid=158296, effective(0, 0), real(0, 0)] ../../source3/lib/util_procid.c:53(pid_to_procid)
  pid_to_procid: messaging_dgm_get_unique failed: No such file or directory

In this scenario we dereference only a garbage PID, but if we would
dereference some garbage pointer we would segfault.

Signed-off-by: Pavel Filipenský <pfilipensky@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

s3:winbindd: Avoid doing the same assignment twice

Done already in setup_child(): child->domain = domain

Signed-off-by: Pavel Filipenský <pfilipensky@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

mdssvc: better support for search with mdfind from Macs

When searching Samba via Spotlight from a Mac with mdfind, only 50 results are
returned.

It seems the changes for bug #15342 where one step in the right
direction. There, a status indicator meaning "search is still being processed"
was implemented, returning a special status indicator (0x23) in a response, when
the query was still running in the backend, eg Elasticsearch, and we haven't got
any result when when the Mac already comes along asking for results of a query.

Turns out, we should also return 0x23, ie "search is still being processed" when
we have some initial search results from the backend. Otherwise mdfind will stop
querying for more results. It works in Finder, as the Finder by default employs
a "live" search where it just keeps polling for more results even after the
server returned an empty result set. The Finder just keeps on querying in some
interval, typically 4 seconds, and a Mac server Spotlight server might return
new results if new files where created that match the query, hence "live"
search.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15463

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Wed Aug 30 10:51:40 UTC 2023 on atb-devel-224

wscript: Refer to correct ConfigSet variable

LIB_GSSAPI appears to be an error of copy-and-pasting.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Aug 30 03:15:05 UTC 2023 on atb-devel-224

wscript: Remove semicolons

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

wscript: Remove unused imports

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

wscript: Use ‘is’ to compare with singletons

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

wscript: Remove unused variable

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

third_party/heimdal_build: Use ‘is’ to compare with singletons

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

third_party/heimdal_build: Remove semicolons

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

third_party/heimdal_build: Remove unused imports

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

selftest: Don’t use invalid escape sequences

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

selftest: Remove star imports

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

selftest: Remove unused imports

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:dsdb:tests: Fix usage lines

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:dsdb:tests: Remove unused imports

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:dsdb:tests: Fix spelling

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests/krb5: Have modified_ticket() not modify its arguments

Inexplicable behaviour could result when the same arguments were passed
in across multiple calls.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests/krb5: Add KerberosCredentials.get_rid()

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests/krb5: Re-raise any LdbError other than ERR_ENTRY_ALREADY_EXISTS

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests/krb5: Correctly assert that we found a LOGON_INFO PAC buffer

Because ‘found_logon_info’ was invariably true, we would miss the case
in which set_pac_sids() failed to accomplish anything, having been
unable to find the LOGON_INFO PAC buffer.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests/krb5: Remove local variable

This seems a bit clearer to me.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests/krb5: Move KDC TGT tests to new file

We can now rely on having MIT Kerberos 1.20 available.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests/krb5: Remove test of pre-1.20 MIT Kerberos behaviour

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

gp: Use read_file() instead of readfp()

readfp() is deprecated and could be removed in a future version of
Python.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

gp: Fix resource leaks

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

gp: Use assertEqual() instead of assertEquals()

assertEquals() is deprecated and could be removed in a future version of
Python.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

gp: Check correct variables

We were intending to check the keys, not the certificates.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

gp: Prefer ‘x not in y’ to ‘not x in y’

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

gp: Don’t shadow imports

Rename a variable that shadows an import of ‘gp_ext’.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

gp: Add missing import

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pytest/dns_aging: Assert that the name of the node to search for is a string

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pytest/dns_aging: Correctly check that record is tombstoned

We were passing in as the name parameter to assert_tombstoned() an NDR
Python object, rather than a string. This meant that the LDAP search
filter would look something like this:

(&(objectClass=dnsNode)(name=<dnsp.DnssrvRpcRecord talloc based object at 0x29abee0>))

and we would fail to find any records.

These searches should have a better chance of working if we pass in the
name of the record instead.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pytest/dns_aging: Check value of ‘dtime’

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python:tests: Rename test method so as not to mask previously-defined method

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python:tests: Complete assertion messages

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python:tests: Fix usage line

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python: Fix reference to undefined name ‘samba’

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool: Remove useless return

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python:join: Fix references to undefined variables

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python: Fix invalid escape sequences

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python: Remove redundant backslashes

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python: Remove unused imports

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python: Fix spelling

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

ldb: Heed return code from Python testsuite

When we added a second run of the Python testsuite, the return code from
the first run began to go ignored.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

ldb: Remove unused import

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

lib:ldb:tests: Remove explicit comparison with False

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

buildtools: Properly set global variable

Since we didn’t declare ‘old_refill_task_list’ as global, we were just
assigning a value to a local variable that was destroyed after we
returned from this function. We clearly intended instead to assign to a
global variable of the same name.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

buildtools: Prefer ‘x not in y’ to ‘not x in y’

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

buildtools: Use ‘is’ to compare with singletons

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

buildtools: Fix comments and documentation

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pyldb: Fix leaks

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pyldb: Check return values of Python functions

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pyldb: Check return values of talloc functions

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pyldb: Check whether Python object is a list

If we’re going to call PyList_Size() on an object, we should be sure
that it is a list first.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

gp: Send list of keys instead of dict to remove

`cache_get_all_attribute_values` returns a dict whereas we need to pass
a list of keys to `remove`. These will be interpolated in the gpdb search.

Signed-off-by: Gabriel Nagy <gabriel.nagy@canonical.com>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: David Mulder <dmulder@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Mon Aug 28 03:01:22 UTC 2023 on atb-devel-224

gp: Test disabled enrollment unapplies policy

For this we need to stage a Registry.pol file with certificate
autoenrollment enabled, but with checkboxes unticked.

Signed-off-by: Gabriel Nagy <gabriel.nagy@canonical.com>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: David Mulder <dmulder@samba.org>

gp: Template changes should invalidate cache

If certificate templates are added or removed, the autoenroll extension
should react to this and reapply the policy. Previously this wasn't
taken into account.

Signed-off-by: Gabriel Nagy <gabriel.nagy@canonical.com>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: David Mulder <dmulder@samba.org>

gp: Test adding new cert templates enforces changes

Ensure that cepces-submit reporting additional templates and re-applying
will enforce the updated policy.

Signed-off-by: Gabriel Nagy <gabriel.nagy@canonical.com>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: David Mulder <dmulder@samba.org>

gp: Convert CA certificates to base64

I don't know whether this applies universally, but in our case the
contents of `es['cACertificate'][0]` are binary, so cleanly converting
to a string fails with the following:

'utf-8' codec can't decode byte 0x82 in position 1: invalid start byte

We found a fix to be encoding the certificate to base64 when
constructing the CA list.

Section 4.4.5.2 of MS-CAESO also suggests that the content of
`cACertificate` is binary (OCTET string).

Signed-off-by: Gabriel Nagy <gabriel.nagy@canonical.com>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: David Mulder <dmulder@samba.org>

gp: Test with binary content for certificate data

This fails all GPO-related tests that call `gpupdate --rsop`.

Signed-off-by: Gabriel Nagy <gabriel.nagy@canonical.com>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: David Mulder <dmulder@samba.org>

gp: Change root cert extension suffix

On Ubuntu, certificates must end in '.crt' in order to be considered by
the `update-ca-certificates` helper.

Signed-off-by: Gabriel Nagy <gabriel.nagy@canonical.com>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: David Mulder <dmulder@samba.org>

gp: Support update-ca-trust helper

This is used on RHEL/Fedora instead of update-ca-certificates. They
behave similarly so it's enough to change the command name.

Signed-off-by: Gabriel Nagy <gabriel.nagy@canonical.com>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: David Mulder <dmulder@samba.org>

gp: Support more global trust directories

In addition to the SUSE global trust directory, add support for RHEL and
Debian-based distributions (including Ubuntu).

To determine the correct directory to use, we iterate over the variants
and stop at the first which is a directory.

In case none is found, fallback to the first option which will produce a
warning as it did previously.

Signed-off-by: Gabriel Nagy <gabriel.nagy@canonical.com>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: David Mulder <dmulder@samba.org>

samba-tool: Allow LDB URL to be None

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15458

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

waf: Build nmbd with -Wno-error=stringop-overflow

We use strlcpy() which has been added to glibc recently. This means we
also get fortification for strlcpy() now:

source3/nmbd/nmbd_browsesync.c: In function ‘find_domain_master_name_query_success’:
source3/nmbd/nmbd_browsesync.c:337:9: warning: ‘strlcpy’ writing 257 bytes into a
region of size 16 overflows the destination [-Wstringop-overflow=]
  337 |         strlcpy(userdata->data, work->work_group, size - sizeof(*userdata));
      |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

We allocate memory for the userdata struct + fstring. However the data
pointer we use only is 16 bytes. Also nowadays you would use offsetof()
for the allocation calculation, but it only works correctly on newer
compilers like gcc > 7. We could make use of it in future after CentOS 7
is gone.

As we don't want to touch nmbd anymore, just silence the warnings.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

util: Avoid logging to multiple backends for stdout/stderr

Commit 83fe7a0316d3e5867a56cfdc51ec17f36ea03889 converted the
stdout/stderr logging  types to DEBUG_FILE to get a header when using
DEBUG_SYSLOG_FORMAT_ALWAYS.  However, this causes all configured
backends to be invoked.  When syslog is one of those backends then
this is almost certainly not what is intended.

Instead, call debug_file_log() directly in that special case and
revert the parts of the above commit that convert to file logging.

Most of the changes to debughdrclass() still seem necessary, since
they handle the change of debug_syslog_format from a bool to an enum.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15460

Signed-off-by: Martin Schwenke <mschwenke@ddn.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Mon Aug 28 01:21:07 UTC 2023 on atb-devel-224

selftest:ndrdump: adjust xattr_NTACL test for ACE coda

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Aug 24 03:47:08 UTC 2023 on atb-devel-224

libcli/security: rm unused sec_ace_copy()

Unused since 2014.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pytest: compare Samba vs Windows SDDL security descriptors

Can Samba understand Windows security descriptors? Does it parse SDDL
the same way?

Here we test on over 7000 SDDL/descriptor pairs and find the answer
is pleasing. In later commits we will add more tests using different
classes of ACE.

The test cases are derived from fuzz seeds, exported to Windows via
the script in the last commit, with the Windows descriptor bytes found
using libcli/security/tests/windows/windows-sddl-test.py.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

libcli/security: test helper script extracts fuzz SDDL

This allows us to try the fuzz seeds as SDDL on Windows, then test
that Samba matches Windows' security descriptors in the cases where
the SDDL compiles. This will find SDDL edge cases that might otherwise
be missed.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

libcli/security: move Windows test script to windows subdir

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

libcli/security: rewrite SDDL tests for running on Windows

This script never worked well because it had to shell out to
Powershell, which never worked well due to syntax conflicts and
Powershell's specialness. The attempted ctypes version did not work,
due to the difficulty in expressing things like "relative
PSECURITY_DESCRIPTOR" in ctypes.

It turns out that pywin32 is easy to install and use, and we can
extract the NDR bytes which is far more useful than just testing if
the SDDL parses.

On Windows:

1. install Python from python.org
2. run `pip install pywin32`
3. copy e.g. libcli/security/tests/data/conditional-aces.txt to Windows
4. run `python windows-sddl-tests.py conditional-aces.txt`
5. add `--help` to see how to export descriptor bytes.

The default output is a whole lot of multi-coloured text, indicating
what failed and what didn't.

With --export-json it writes a JSON file mapping SDDL strings to NDR
byte sequences, which can be used to compare with Samba's attempts. If
you are only interested in --export-json, you might also like --quiet.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/colour: add a colour diff helper

Sometimes colour can help show what is different between two strings.

This is roughly the same as

`git diff --no-index --color-words=. <a> <b>`.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

libcli/security: add some test more Windows SDDL test strings

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

libcli/security: script to turn token/descriptor pairs into sddl

We fuzz security descriptors in a couple of different ways, and this
maps seeds from one form into the other. The SDDL examples can also be
used in Windows tests.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

libcli/security: remove unused tmp_ctx in calculate_inherited_from_parent()

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

libcli/security: avoid leak in calculate_inherited_from_parent()

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

libcli/security: used sec_ace_object() in sddl encoding

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

librpc/security.idl: adjust size calculations for upcoming ace types

Soon we will get Conditional ACEs and Resource Attribute ACES, each of
which have trailing bytes at the end of the ACE. Here's a diagram:

              ____      The ACE size field may indicate a size bigger
  .type      /    |     than the known parts, even when you take
  .flags    /     |     rounding to a multiple of four into account.
  .size  --'      |     This extra data is meaningful in some ACEs.
  .access_mask    |
  .trustee (sid) _|  <- known data ends here.
                  :
   "coda"      ___:  <- the trailing part, Zero size unless the size
                        field points beyond the end of the known data.
Probably empty for ordinary ACE types.

Until now we have thrown away these extra bytes, because they have no
meaning in the ACE types we recognise. But with conditional and
resource attribute ACEs we need to catch and process these bytes, so
we add an extra field for that.

Thus we can drop the manually written ndr_pull_security_ace() that
discarded the trailing bytes, because we just allow it to be pulled
into an unused blob. In the very common case, the blob will be empty.

Microsoft does not use a common name across different ACE types to
describe this end-data -- "coda" is a Samba term.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>