s3:locking: just use g_lock_dump() for fsp_update_share_mode_flags()

We don't need to protect this with g_lock_lock/g_lock_unlock
as we just want the current flags, we're still protected by the
dbwrap layer lock.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: move fsp_update_share_mode_flags* related functions further down

It will soon need to use 'struct locking_tdb_data'

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: replace locking_tdb_data_store() with share_mode_data_ltdb_store()

This means we flush share_mode_data at the same time...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: let share_mode_forall_entries() call TALLOC_FREE(ltdb)

We should free ltdb as soon as possible...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: split out share_mode_data_ltdb_store()

This will allow us to use it in other places too
and we'll avoid to storing multiple times.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: introduce share_mode_data->not_stored

share_mode_data->fresh was very similar, but only set
and never used.

Now we remember 'not_stored' instead, the 'not_' is easier
as ndr_pull sets [skip] elements to 0.

We use this as indication to move the value to
memcache.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: change some debug messages to level unless we got NT_STATUS_NOT_FOUND

NT_STATUS_NOT_FOUND is not a real error in most cases so we should keep
it on level 10, but all other errors should never be without notice...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: log all share_mode_forall_entries() errors at level 0

These should never fail without notice...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: let share_mode_forall_leases() log all errors at level 0

These should never fail without notice...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: let set_delete_on_close_lck() log errors and panic

Most of the calls in set_delete_on_close_lck() are checked with
asserts, so do panic in all situation where things go wrong in an
unexpected way.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: log g_lock_dump() error in locking_tdb_data_fetch() at level 0

This should never fail without notice...

Note we already checked for NT_STATUS_NOT_FOUND before.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: log add locking_tdb_data_store() errors at level 0

These should never fail without notice...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: log g_lock_locks() error at level 0

These should never fail without notice...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: let fsp_update_share_mode_flags() log all errors at level 0

These should never fail without notice, share_mode_do_locked() should
never fail with NT_STATUS_NOT_FOUND for an existing fsp.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: log all g_lock_writev_data() errors at level 0

These should never fail without notice.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: log all locking_tdb_data_{get,fetch}() errors at level 0

These should never fail without notice.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:open_files.idl: add share_mode_entry_op_type

This makes it easier to read log files...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:smbd: let smbXsrv_{session,tcon,open}_global.tdb use TDB_VOLATILE

This avoids using fcntl() locks for dbwrap_delete()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:g_lock: fix error handling in g_lock_watch_data_send()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15167

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:smbd: let lease_match() call TALLOC_FREE(lck); on error

We ignore the error from share_mode_forall_leases(), but
we still need to cleanup the share_mode_lock we are holding...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: let reset_share_mode_entry() report errors to the caller

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15166

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: remove unused NO_LOCKING_COUNT

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:torture: fix strict aliasing warnings in cmd_vfs.c

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libsmb: Use find_snapshot_token() for clistr_is_previous_version_path()

Dedup that string parsing logic

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Mon Sep 19 18:21:56 UTC 2022 on sn-devel-184

lib: Add separator argument to find_snapshot_token()

We'll use the logic for \ based strings next

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lib: Move extract_snapshot_token() to util_path.c

Make it available to replace clistr_is_previous_version_path() in
libsmb/

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs: Simplify xattr_tdb_mkdirat()

We have the dirfsp and the relative name. And with fstatat we don't
need the full pathname anymore.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

streams_xattr: Avoid a talloc_strdup

We can print a short string with %.*s, no talloc_strdup()
and *stype='\0' required.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Fix a typo

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Use PATH_MAX as symlink target buffer

We use that instead of the arbitrary 4k in open.c as well

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

shadow_copy2: Don't implicitly return memory off talloc_tos()

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

shadow_copy2: Avoid a few ZERO_STRUCT()s

Give the compiler more hints what's going on

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

source3: A few whitespace fixes

review with git sh -w

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

registry3: Move registry_value_cmp() to its only user

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

registry3: Remove some unused functions

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs: Simplify vfswrap_parent_pathname()

We don't really need a talloc_stackframe() here

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs: Avoid a talloc in vfswrap_parent_pathname()

We copy smb_fname_in->base_name just to overwrite it again
immediately. Expand synthetic_smb_fname() here.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libsmb: Use tevent_req_nterror()'s retval

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

WHATSNEW: samba-tool: fewer tracebacks, more colour

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Mon Sep 19 07:14:31 UTC 2022 on sn-devel-184

docs/man/samba-tool explain --color

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

py/dbcheck: improve 'please --fix' message

The dbcheck module is used in places other than samba-tool (backup,
provision) where the old 'use --fix' message made no sense. Also,
now that we're not necessarily claiming to fix all errors, we say
how many we think we can.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

dbcheck: don't recommend --fix for errors we can't fix

and/or won't fix.

I think there are others that should be here.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

dbcheck: do not crash on empty DN

we had

$ bin/samba-tool dbcheck -H st/rpc_proxy/private/sam.ldb
Checking 202 objects
ERROR(<class 'ValueError'>): uncaught exception - unable to parse dn string
  File "/home/douglasb/src/samba/bin/python/samba/netcmd/__init__.py", line 230, in _run
    return self.run(*args, **kwargs)
  File "/home/douglasb/src/samba/bin/python/samba/netcmd/dbcheck.py", line 173, in run
    error_count = chk.check_database(DN=DN, scope=search_scope,
  File "/home/douglasb/src/samba/bin/python/samba/dbchecker.py", line 255, in check_database
    error_count += self.check_object(object.dn, requested_attrs=attrs)
  File "/home/douglasb/src/samba/bin/python/samba/dbchecker.py", line 2616, in check_object
    expected_dn = ldb.Dn(self.samdb, "RDN=RDN,%s" % (parent_dn))

Now we have:

$ bin/samba-tool dbcheck -H st/rpc_proxy/private/sam.ldb
Checking 202 objects
ERROR: could not handle parent DN '': skipping RDN checks
Please use --fix to fix these errors
Checked 202 objects (1 errors)

which is still not really right, since --fix won't help.

(same with st/s4member/private/sam.ldb).

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool dbcheck: use colour if wanted

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

py/dbchecker: dbcheck prints bits of colour if asked

Prefixes like ERROR, WARNING, and INFO are given interpretive colours.

This won't change anything until samba-tool decides to ask for colour,
which, who knows, might even be in the next commit.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

shadow_copy2: Remove an intermediate if-statement

Now we always pass in a dirfsp from our only caller

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Sep 17 05:15:04 UTC 2022 on sn-devel-184

smbd: Remove non_widelink_open() support code

process_symlink_open() and check_reduced_name() are no longer used,
non_widelink_open() was the only user of both.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Rewrite non_widelink_open()

The previous implementation relied on recursion into
non_widelink_open() via process_symlink_open(). The latter used
readlink() to just make sure that the opened file is actually a
symlink.

This implementation now relies on a fstat/fstatat on failure to open a
file, removing a little complexity deciphering error codes
correctly. It also relies on reading the symlink in user space,
turning the recursion into a loop.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Make readlink_talloc() public

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Slightly simplify non_widelink_open()

Avoid the "is_share_root" boolean: One special case less to take care
of further down and in callers: Sanitize the relative name so that it
can never contain a path separator

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

shadow_copy2: Use dirfsp if it's around

Not used yet, and the "if" around dirfsp!=NULL will go away in a later
patch.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

shadow_copy2: Use dirfsp for connectpath

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs: Add dirfsp to connectpath_fn()

So far we only call CONNECTPATH on full paths. In the future, we'll
have a call that will not have converted a relative path to absolute
just for efficiency reasons. To give shadow_copy2 the chance to still
find the snapshot directory, pass the dirfsp down to it.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:auth: Flush the GETPWSID in memory cache for NTLM auth

Example valgrind output:

==22502== 22,747,002 bytes in 21,049 blocks are possibly lost in loss record 1,075 of 1,075
==22502==    at 0x4C29F73: malloc (vg_replace_malloc.c:309)
==22502==    by 0x11D7089C: _talloc_pooled_object (in /usr/lib64/libtalloc.so.2.1.16)
==22502==    by 0x9027834: tcopy_passwd (in /usr/lib64/libsmbconf.so.0)
==22502==    by 0x6A1E1A3: pdb_copy_sam_account (in /usr/lib64/libsamba-passdb.so.0.27.2)
==22502==    by 0x6A28AB7: pdb_getsampwnam (in /usr/lib64/libsamba-passdb.so.0.27.2)
==22502==    by 0x65D0BC4: check_sam_security (in /usr/lib64/samba/libauth-samba4.so)
==22502==    by 0x65C70F0: ??? (in /usr/lib64/samba/libauth-samba4.so)
==22502==    by 0x65C781A: auth_check_ntlm_password (in /usr/lib64/samba/libauth-samba4.so)
==22502==    by 0x14E464: ??? (in /usr/sbin/winbindd)
==22502==    by 0x151CED: winbind_dual_SamLogon (in /usr/sbin/winbindd)
==22502==    by 0x152072: winbindd_dual_pam_auth_crap (in /usr/sbin/winbindd)
==22502==    by 0x167DE0: ??? (in /usr/sbin/winbindd)
==22502==    by 0x12F29B12: tevent_common_invoke_fd_handler (in /usr/lib64/libtevent.so.0.9.39)
==22502==    by 0x12F30086: ??? (in /usr/lib64/libtevent.so.0.9.39)
==22502==    by 0x12F2E056: ??? (in /usr/lib64/libtevent.so.0.9.39)
==22502==    by 0x12F2925C: _tevent_loop_once (in /usr/lib64/libtevent.so.0.9.39)
==22502==    by 0x16A243: ??? (in /usr/sbin/winbindd)
==22502==    by 0x16AA04: ??? (in /usr/sbin/winbindd)
==22502==    by 0x12F29F68: tevent_common_invoke_immediate_handler (in /usr/lib64/libtevent.so.0.9.39)
==22502==    by 0x12F29F8F: tevent_common_loop_immediate (in /usr/lib64/libtevent.so.0.9.39)
==22502==    by 0x12F2FE3C: ??? (in /usr/lib64/libtevent.so.0.9.39)
==22502==    by 0x12F2E056: ??? (in /usr/lib64/libtevent.so.0.9.39)
==22502==    by 0x12F2925C: _tevent_loop_once (in /usr/lib64/libtevent.so.0.9.39)
==22502==    by 0x12F4C7: main (in /usr/sbin/winbindd)

You can find one for each string in pdb_copy_sam_account(), in total
this already has 67 MB in total for this valgrind run.

pdb_getsampwnam() -> memcache_add_talloc(NULL, PDB_GETPWSID_CACHE, ...)

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15169

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Sep 16 20:30:31 UTC 2022 on sn-devel-184

pytests:s4/dsdb/passwords: avoid unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Sep 16 06:47:43 UTC 2022 on sn-devel-184

pytests:s4/drs/getnc_schema: avoid unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pytests:s4/drs/repl_move: avoid unused and star imports

Found the names using something like:

flake8 repl_move.py | \
  grep -oP "(?<=F405 ')[\w.]+" /tmp/repl_move | sort | uniq

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pytests:s4/drs/repl_rodc: avoid unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pytests:s4/drs/linked_attributes_drs: avoid unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pytests:s4/drs/ridalloc_exop: avoid unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pytests: remove backwards compat workaround for python 2.6

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pytest: samba-tool visualize: improve a message

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool: no stack trace on missing ldb tdb

Now, in a testenv, if you forget to use '-s st/ad_dc/etc/smb.conf',
you only see this:

$ bin/samba-tool user rename  dsadsa
ldb: Unable to open tdb '$HERE/st/client/private/secrets.ldb': No such file or directory
ldb: Failed to connect to '$HERE/st/client/private/secrets.ldb' with backend 'tdb': Unable to open tdb '$HERE/st/client/private/secrets.ldb': No such file or directory
Could not find machine account in secrets database: Failed to fetch machine account password from secrets.ldb: Could not open secrets.ldb and failed to open $HERE/st/client/private/secrets.tdb: NT_STATUS_CANT_ACCESS_DOMAIN_INFO
ltdb: tdb($HERE/st/client/private/sam.ldb): tdb_open_ex: could not open file $HERE/st/client/private/sam.ldb: No such file or directory

Unable to open tdb '$HERE/st/client/private/sam.ldb': No such file or directory
Failed to connect to 'tdb://$HERE/st/client/private/sam.ldb' with backend 'tdb': Unable to open tdb '$HERE/st/client/private/sam.ldb': No such file or directory
ERROR(ldb): uncaught exception - Unable to open tdb '$HERE/st/client/private/sam.ldb': No such file or directory

rather than all that AND a stack trace.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool: write ERROR in red if colour is wanted

Often we'll write something like

   ERROR: Unable to find user "potato"

which can get lost in the jumble of other output. With this patch, we
colour the word "ERROR" red but not the rest of the string, unless it is
determined that colour is not wanted (due to one of --color=never,
NO_COLOR=1, output is not a tty).

We choose to redden the word "ERROR" only to maintain legibility in the
actual message, while hopefully increasing the noticeability of the line.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool visualize: simplify --color-scheme calculations

If you ask for a --color-scheme, you are implicitly asking for --color.
That was documented in --help, but not followed here.

Now --color=no --color-scheme=ansi will use colour for the graph, but not
for other output. This might be useful when the graph is going to a
different place than everything else (`-o foo.txt > bar.txt`).

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool visualise: use global --color

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

py:colour: is_colour_wanted() can take filenames

We need this for `samba-tool visualize -o -` which means output to
stdout, and which has always had a tty test for colour. Rather than
continue to duplicate the full logic there, we can reuse this.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool: --color=auto looks at stderr and stdout

More often than not we are using colour in stderr, but are deciding
based on stdout's tty-ness. This patch changes to use both, and will
affect the following situation:

 samba-tool  2>/tmp/errors   # used to be colour, now not.

of course, if you want colour, you can always

 samba-tool --color=yes 2>/tmp/errors

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool drs showrepl: use global --color option

This changes the default from --color=no to --color=auto.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool: save --color choice for subcommands

In particular, visualize needs it to decide colour for an output
file that may or may not be stdout, so it needs to make its own
decision for that file.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool: make --color a general option

We don't put --color into options.SambaOptions because we can't handle
the 'auto' case in the options module without knowing whether or not
self.outf is a tty, and a) this might not be resolved and b) is fiddly
to pass through.

The .use_colour class flag allows samba-tool subcommands to avoid having
--color, and is *also* useful in the short term for visualise and drs
commands to avoid having this --color clobber their own bespoke versions
(temporarily, during the transition).

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

py:colour: colour_if_wanted() returns the result

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pytest: samba-tool visualize: fix filename

Overwriting the other file was harmless but misleading.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

libcli/auth/proto.h: remove unneeded path details.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

ldb: ldb_build_search_req() check for a talloc failure

The failure in question would have to be a `talloc_strdup(dn, "")` in
ldb_dn_from_ldb_val().

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/server: stop suggesting ntvfs in error message

I am not sure about the rpc proxy.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

libaddns: remove duplicate declaration

Also declared on line 257, exactly the same.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pytest/password_lockout: be less verbose by default

leaving the carefully constructed verbosity there for whoever choses
to switch it on.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool: simplify and clarify SuperCommand._run() a little

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

ctdb-tests: Reformat remaining test stubs with "shfmt -w -p -i 0 -fn"

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Fri Sep 16 04:35:09 UTC 2022 on sn-devel-184

ctdb-tests: Include eventscript stub commands in shellcheck test

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Avoid shellcheck warnings in remaining test stubs

A small amount of effort...

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-scripts: Simplify determination of real interface

This can now be made trivial.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Implement "ip -brief link show" in ip stub

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Avoid ShellCheck warnings

Although this is a test stub, it is complicated enough to encourage
ShellCheck cleanliness.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Reformat script with "shfmt -w -p -i 0 -fn"

As per current Samba convention.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-scripts: Drop assumption that there are VLANs with no '@'

VLAN configuration on Linux often uses a convention of naming a VLAN
on <iface> with VLAN ID <tag> as <iface>.<tag>.  To be able to monitor
the underlying interface, the original 10.interface code naively
simply stripped off the '.' and everything after (i.e. ".*", as a glob
pattern).

Some users do not use the above convention.  A VLAN can be named
without including the underlying interface, but still with a
tag (e.g. vlan<tag> - the word "vlan" following by the tag) or, more
generally, perhaps without a tag (e.g. <vlan> - an arbitrary name).
The ip(8) command lists a VLAN as <vlan>@<iface>.  The underlying
interface can be found by stripping everything up to and including an
'@' (i.e. "*@").

Commit bc71251433ce618c95c674d7cbe75b01a94adad9 added support for
stripping "*@".  However, on suspicion, it kept support for the case
where there is no '@', falling back to stripping ".*".  If ip(8) ever
did this then it was a long time ago - it has been printing a format
including '@' since at least 2004.

Stripping ".*" interferes with interesting administrative decisions,
like having '.' in interface names.

So, drop the fallback to stripping ".*" because it appears to be
unnecessary and can cause inconvenience.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

CVE-2020-25720 s4:dsdb/descriptor: explain lack of dSHeuristics check

It is strange that sDRightsEffective pays no attention to the
dSHeuristics flags.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14810

Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Sep 16 03:31:42 UTC 2022 on sn-devel-184

CVE-2020-25720 s4:dsdb/descriptor: Validate owner SIDs written to security descriptors

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14810

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

CVE-2020-25720 s4-acl: Omit sDRightsEffective for computers unless all rights are granted

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14810

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

CVE-2020-25720: s4-acl: Owner no longer has implicit Write DACL

The implicit right of an object's owner to modify its security
descriptor no longer exists, according to the new access rules. However,
we continue to grant this implicit right for fileserver access checks.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14810

Signed-off-by: Nadezhda Ivanova <nivanova@symas.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

CVE-2020-25720 s4:ntvfs: Use se_file_access_check() to check file access rights

se_access_check() will be changed in a following commit to remove the
implicit WRITE_DAC right that comes with being the owner of an object.
We want to keep this implicit right for file access, and by using
se_file_access_check() we can preserve the existing behaviour.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14810

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

CVE-2020-25720: s4-acl: Adjusted some tests to work with the new behavior

Test using non-priviledged accounts now need to make sure they have
WP access on the prvided attributes, or Write-DACL
Some test create organizational units with a specific SD, and those now
need the user to have WD or else they give errors

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14810

Signed-off-by: Nadezhda Ivanova <nivanova@symas.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

CVE-2020-25720: s4-acl: Change behavior of Create Children check

Up to now, the rights to modify an attribute were not checked during an LDAP
add operation. This means that even if a user has no right to modify
an attribute, they can still specify any value during object creation,
and the validated writes were not checked.
This patch changes this behavior. During an add operation,
a security descriptor is created that does not include the one provided by the
user, and is used to verify that the user has the right to modify the supplied attributes.
Exception is made for an object's mandatory attributes, and if the user has Write DACL right,
further checks are skipped.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14810

Pair-Programmed-With: Joseph Sutton <josephsutton@catalyst.net.nz>

Signed-off-by: Nadezhda Ivanova <nivanova@symas.com>
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

CVE-2020-25720: s4-acl: Move definition of acl_check_self_membership()

This allows us to make use of it in acl_add().

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14810

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

CVE-2020-25720 s4-acl: Test Create Child permission should not allow full write to all attributes

Up to now, the rights to modify an attribute were not checked during an LDAP
add operation. This means that even if a user has no right to modify
an attribute, they can still specify any value during object creation,
and the validated writes were not checked.
This patch includes tests for the proposed change of behavior.
test_add_c3 and c4 pass, because mandatory attributes can still be
set, and in the old behavior SD permissions were irrelevant

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14810

Pair-Programmed-With: Joseph Sutton <josephsutton@catalyst.net.nz>

Signed-off-by: Nadezhda Ivanova <nivanova@symas.com>
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

CVE-2020-25720 pydsdb: Add AD schema GUID constants

This helps reduce the profusion of magic constant values in Python
tests.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14810

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

CVE-2020-25720 pydsdb: Add dsHeuristics constant definitions

We want to be able to use these values in Python tests.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14810

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

CVE-2020-25720 s4/dsdb/util: Add functions for dsHeuristics 28, 29

These are the newly-added AttributeAuthorizationOnLDAPAdd and
BlockOwnerImplicitRights.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14810

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

CVE-2020-25720 python:tests: Ensure that access checks don't succeed

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14810

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>