s3:locking: let get_share_mode_write_time() use share_mode_lock_access_private_data()

We should avoid dereference 'struct share_mode_lock' as much as possible.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: let set_write_time() use share_mode_lock_access_private_data()

We should avoid dereference 'struct share_mode_lock' as much as possible.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: let set_sticky_write_time() use share_mode_lock_access_private_data()

We should avoid dereference 'struct share_mode_lock' as much as possible.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: let is_delete_on_close_set() use share_mode_lock_access_private_data()

We should avoid dereference 'struct share_mode_lock' as much as possible.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: let get_delete_on_close_token() use share_mode_lock_access_private_data()

We should avoid dereference 'struct share_mode_lock' as much as possible.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: let set_delete_on_close_lck() use share_mode_lock_access_private_data()

We should avoid dereference 'struct share_mode_lock' as much as possible.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: let reset_delete_on_close_lck() use share_mode_lock_access_private_data()

We should avoid dereference 'struct share_mode_lock' as much as possible.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: let remove_lease_if_stale() use share_mode_lock_file_id()

We should avoid dereference 'struct share_mode_lock' as much as possible.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: add share_mode_lock_file_id()

This will simplify some (mostly debug) code soon.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: make use of share_mode_lock_access_private_data() in rename_share_filename()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: let rename_share_filename_state maintain a struct share_mode_data pointer

We only need to access lck->data once...

This will simplify further changes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: add share_mode_lock_access_private_data()

In future we should avoid dereference 'struct share_mode_lock'
as much as possible.

This will also allow us to load struct share_mode_data
only if required in future.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: let share_mode_wakeup_waiters() use share_mode_do_locked_vfs_denied()

This allows us get rid of the otherwise unused share_mode_do_locked().

It means we only have one code path that handles the g_lock handling.

This looks like a performance degradation, but all callers of
share_mode_wakeup_waiters() already took the share_mode_lock,
so we only increment the refcount. Note the additional
talloc(mem_ctx, struct share_mode_lock) will be optimized away
in the next commits.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: make share_mode_do_locked() static

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: protect do_lock() with share_mode_do_locked_vfs_allowed()

share_mode_do_locked() will be make static soon.

Here we just want to avoid concurrent access to brlock.tdb
in order to maintain the lock order, we're not interested in the
locking.tdb content at all, expect that there's at least one
entry.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:smbd: protect smbd_do_unlocking() with share_mode_do_locked_vfs_allowed()

share_mode_do_locked() will be make static soon.

Here we just want to avoid concurrent access to brlock.tdb
in order to maintain the lock order, we're not interested in the
locking.tdb content at all, expect that there's at least one
entry and we want to wake potential watchers.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: add share_mode_do_locked_vfs_{denied,allowed}()

These function will add an abstraction to protect
a function that is not allowed to call vfs functions
or allow vfs functions to be called.

Currently these are implemented similar,
but we'll optimize them in the next commits.

The idea is that share_mode_do_locked_vfs_denied()
will be able to run fast enough in order to run
under a tdb chainlock (just a pthread mutex).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:smbd: add smb_vfs_assert_allowed() to kernel oplock code

Kernel oplocks can block in the same way vfs operations can do.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:smbd: add helpers to deny vfs calls in some sections

Code denying vfs calls can do:

{
   struct smb_vfs_deny_state vfs_deny = {};

   smb_vfs_deny_push(&vfs_deny);

   VFS calls are not allowed here...

   smb_vfs_deny_pop(&vfs_deny);
}

This will allow us to safely run some code under a
tdb chainlock later...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:smbd: move VFS_FIND() to smbd/vfs.c

It's only used there...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:smbd: move locking related vfs functions to smbd/vfs.c

This allows us to make VFS_FIND local to smbd/vfs.c in the
next step.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: just use g_lock_dump() for fsp_update_share_mode_flags()

We don't need to protect this with g_lock_lock/g_lock_unlock
as we just want the current flags, we're still protected by the
dbwrap layer lock.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: move fsp_update_share_mode_flags* related functions further down

It will soon need to use 'struct locking_tdb_data'

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: replace locking_tdb_data_store() with share_mode_data_ltdb_store()

This means we flush share_mode_data at the same time...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: let share_mode_forall_entries() call TALLOC_FREE(ltdb)

We should free ltdb as soon as possible...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: split out share_mode_data_ltdb_store()

This will allow us to use it in other places too
and we'll avoid to storing multiple times.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: introduce share_mode_data->not_stored

share_mode_data->fresh was very similar, but only set
and never used.

Now we remember 'not_stored' instead, the 'not_' is easier
as ndr_pull sets [skip] elements to 0.

We use this as indication to move the value to
memcache.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: change some debug messages to level unless we got NT_STATUS_NOT_FOUND

NT_STATUS_NOT_FOUND is not a real error in most cases so we should keep
it on level 10, but all other errors should never be without notice...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: log all share_mode_forall_entries() errors at level 0

These should never fail without notice...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: let share_mode_forall_leases() log all errors at level 0

These should never fail without notice...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: let set_delete_on_close_lck() log errors and panic

Most of the calls in set_delete_on_close_lck() are checked with
asserts, so do panic in all situation where things go wrong in an
unexpected way.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: log g_lock_dump() error in locking_tdb_data_fetch() at level 0

This should never fail without notice...

Note we already checked for NT_STATUS_NOT_FOUND before.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: log add locking_tdb_data_store() errors at level 0

These should never fail without notice...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: log g_lock_locks() error at level 0

These should never fail without notice...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: let fsp_update_share_mode_flags() log all errors at level 0

These should never fail without notice, share_mode_do_locked() should
never fail with NT_STATUS_NOT_FOUND for an existing fsp.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: log all g_lock_writev_data() errors at level 0

These should never fail without notice.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: log all locking_tdb_data_{get,fetch}() errors at level 0

These should never fail without notice.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:open_files.idl: add share_mode_entry_op_type

This makes it easier to read log files...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:smbd: let smbXsrv_{session,tcon,open}_global.tdb use TDB_VOLATILE

This avoids using fcntl() locks for dbwrap_delete()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:g_lock: fix error handling in g_lock_watch_data_send()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15167

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:smbd: let lease_match() call TALLOC_FREE(lck); on error

We ignore the error from share_mode_forall_leases(), but
we still need to cleanup the share_mode_lock we are holding...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: let reset_share_mode_entry() report errors to the caller

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15166

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:locking: remove unused NO_LOCKING_COUNT

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:torture: fix strict aliasing warnings in cmd_vfs.c

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libsmb: Use find_snapshot_token() for clistr_is_previous_version_path()

Dedup that string parsing logic

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Mon Sep 19 18:21:56 UTC 2022 on sn-devel-184

lib: Add separator argument to find_snapshot_token()

We'll use the logic for \ based strings next

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lib: Move extract_snapshot_token() to util_path.c

Make it available to replace clistr_is_previous_version_path() in
libsmb/

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs: Simplify xattr_tdb_mkdirat()

We have the dirfsp and the relative name. And with fstatat we don't
need the full pathname anymore.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

streams_xattr: Avoid a talloc_strdup

We can print a short string with %.*s, no talloc_strdup()
and *stype='\0' required.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Fix a typo

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Use PATH_MAX as symlink target buffer

We use that instead of the arbitrary 4k in open.c as well

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

shadow_copy2: Don't implicitly return memory off talloc_tos()

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

shadow_copy2: Avoid a few ZERO_STRUCT()s

Give the compiler more hints what's going on

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

source3: A few whitespace fixes

review with git sh -w

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

registry3: Move registry_value_cmp() to its only user

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

registry3: Remove some unused functions

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs: Simplify vfswrap_parent_pathname()

We don't really need a talloc_stackframe() here

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs: Avoid a talloc in vfswrap_parent_pathname()

We copy smb_fname_in->base_name just to overwrite it again
immediately. Expand synthetic_smb_fname() here.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libsmb: Use tevent_req_nterror()'s retval

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

WHATSNEW: samba-tool: fewer tracebacks, more colour

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Mon Sep 19 07:14:31 UTC 2022 on sn-devel-184

docs/man/samba-tool explain --color

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

py/dbcheck: improve 'please --fix' message

The dbcheck module is used in places other than samba-tool (backup,
provision) where the old 'use --fix' message made no sense. Also,
now that we're not necessarily claiming to fix all errors, we say
how many we think we can.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

dbcheck: don't recommend --fix for errors we can't fix

and/or won't fix.

I think there are others that should be here.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

dbcheck: do not crash on empty DN

we had

$ bin/samba-tool dbcheck -H st/rpc_proxy/private/sam.ldb
Checking 202 objects
ERROR(<class 'ValueError'>): uncaught exception - unable to parse dn string
  File "/home/douglasb/src/samba/bin/python/samba/netcmd/__init__.py", line 230, in _run
    return self.run(*args, **kwargs)
  File "/home/douglasb/src/samba/bin/python/samba/netcmd/dbcheck.py", line 173, in run
    error_count = chk.check_database(DN=DN, scope=search_scope,
  File "/home/douglasb/src/samba/bin/python/samba/dbchecker.py", line 255, in check_database
    error_count += self.check_object(object.dn, requested_attrs=attrs)
  File "/home/douglasb/src/samba/bin/python/samba/dbchecker.py", line 2616, in check_object
    expected_dn = ldb.Dn(self.samdb, "RDN=RDN,%s" % (parent_dn))

Now we have:

$ bin/samba-tool dbcheck -H st/rpc_proxy/private/sam.ldb
Checking 202 objects
ERROR: could not handle parent DN '': skipping RDN checks
Please use --fix to fix these errors
Checked 202 objects (1 errors)

which is still not really right, since --fix won't help.

(same with st/s4member/private/sam.ldb).

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool dbcheck: use colour if wanted

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

py/dbchecker: dbcheck prints bits of colour if asked

Prefixes like ERROR, WARNING, and INFO are given interpretive colours.

This won't change anything until samba-tool decides to ask for colour,
which, who knows, might even be in the next commit.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

shadow_copy2: Remove an intermediate if-statement

Now we always pass in a dirfsp from our only caller

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Sep 17 05:15:04 UTC 2022 on sn-devel-184

smbd: Remove non_widelink_open() support code

process_symlink_open() and check_reduced_name() are no longer used,
non_widelink_open() was the only user of both.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Rewrite non_widelink_open()

The previous implementation relied on recursion into
non_widelink_open() via process_symlink_open(). The latter used
readlink() to just make sure that the opened file is actually a
symlink.

This implementation now relies on a fstat/fstatat on failure to open a
file, removing a little complexity deciphering error codes
correctly. It also relies on reading the symlink in user space,
turning the recursion into a loop.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Make readlink_talloc() public

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Slightly simplify non_widelink_open()

Avoid the "is_share_root" boolean: One special case less to take care
of further down and in callers: Sanitize the relative name so that it
can never contain a path separator

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

shadow_copy2: Use dirfsp if it's around

Not used yet, and the "if" around dirfsp!=NULL will go away in a later
patch.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

shadow_copy2: Use dirfsp for connectpath

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs: Add dirfsp to connectpath_fn()

So far we only call CONNECTPATH on full paths. In the future, we'll
have a call that will not have converted a relative path to absolute
just for efficiency reasons. To give shadow_copy2 the chance to still
find the snapshot directory, pass the dirfsp down to it.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:auth: Flush the GETPWSID in memory cache for NTLM auth

Example valgrind output:

==22502== 22,747,002 bytes in 21,049 blocks are possibly lost in loss record 1,075 of 1,075
==22502==    at 0x4C29F73: malloc (vg_replace_malloc.c:309)
==22502==    by 0x11D7089C: _talloc_pooled_object (in /usr/lib64/libtalloc.so.2.1.16)
==22502==    by 0x9027834: tcopy_passwd (in /usr/lib64/libsmbconf.so.0)
==22502==    by 0x6A1E1A3: pdb_copy_sam_account (in /usr/lib64/libsamba-passdb.so.0.27.2)
==22502==    by 0x6A28AB7: pdb_getsampwnam (in /usr/lib64/libsamba-passdb.so.0.27.2)
==22502==    by 0x65D0BC4: check_sam_security (in /usr/lib64/samba/libauth-samba4.so)
==22502==    by 0x65C70F0: ??? (in /usr/lib64/samba/libauth-samba4.so)
==22502==    by 0x65C781A: auth_check_ntlm_password (in /usr/lib64/samba/libauth-samba4.so)
==22502==    by 0x14E464: ??? (in /usr/sbin/winbindd)
==22502==    by 0x151CED: winbind_dual_SamLogon (in /usr/sbin/winbindd)
==22502==    by 0x152072: winbindd_dual_pam_auth_crap (in /usr/sbin/winbindd)
==22502==    by 0x167DE0: ??? (in /usr/sbin/winbindd)
==22502==    by 0x12F29B12: tevent_common_invoke_fd_handler (in /usr/lib64/libtevent.so.0.9.39)
==22502==    by 0x12F30086: ??? (in /usr/lib64/libtevent.so.0.9.39)
==22502==    by 0x12F2E056: ??? (in /usr/lib64/libtevent.so.0.9.39)
==22502==    by 0x12F2925C: _tevent_loop_once (in /usr/lib64/libtevent.so.0.9.39)
==22502==    by 0x16A243: ??? (in /usr/sbin/winbindd)
==22502==    by 0x16AA04: ??? (in /usr/sbin/winbindd)
==22502==    by 0x12F29F68: tevent_common_invoke_immediate_handler (in /usr/lib64/libtevent.so.0.9.39)
==22502==    by 0x12F29F8F: tevent_common_loop_immediate (in /usr/lib64/libtevent.so.0.9.39)
==22502==    by 0x12F2FE3C: ??? (in /usr/lib64/libtevent.so.0.9.39)
==22502==    by 0x12F2E056: ??? (in /usr/lib64/libtevent.so.0.9.39)
==22502==    by 0x12F2925C: _tevent_loop_once (in /usr/lib64/libtevent.so.0.9.39)
==22502==    by 0x12F4C7: main (in /usr/sbin/winbindd)

You can find one for each string in pdb_copy_sam_account(), in total
this already has 67 MB in total for this valgrind run.

pdb_getsampwnam() -> memcache_add_talloc(NULL, PDB_GETPWSID_CACHE, ...)

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15169

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Sep 16 20:30:31 UTC 2022 on sn-devel-184

pytests:s4/dsdb/passwords: avoid unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Sep 16 06:47:43 UTC 2022 on sn-devel-184

pytests:s4/drs/getnc_schema: avoid unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pytests:s4/drs/repl_move: avoid unused and star imports

Found the names using something like:

flake8 repl_move.py | \
  grep -oP "(?<=F405 ')[\w.]+" /tmp/repl_move | sort | uniq

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pytests:s4/drs/repl_rodc: avoid unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pytests:s4/drs/linked_attributes_drs: avoid unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pytests:s4/drs/ridalloc_exop: avoid unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pytests: remove backwards compat workaround for python 2.6

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pytest: samba-tool visualize: improve a message

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool: no stack trace on missing ldb tdb

Now, in a testenv, if you forget to use '-s st/ad_dc/etc/smb.conf',
you only see this:

$ bin/samba-tool user rename  dsadsa
ldb: Unable to open tdb '$HERE/st/client/private/secrets.ldb': No such file or directory
ldb: Failed to connect to '$HERE/st/client/private/secrets.ldb' with backend 'tdb': Unable to open tdb '$HERE/st/client/private/secrets.ldb': No such file or directory
Could not find machine account in secrets database: Failed to fetch machine account password from secrets.ldb: Could not open secrets.ldb and failed to open $HERE/st/client/private/secrets.tdb: NT_STATUS_CANT_ACCESS_DOMAIN_INFO
ltdb: tdb($HERE/st/client/private/sam.ldb): tdb_open_ex: could not open file $HERE/st/client/private/sam.ldb: No such file or directory

Unable to open tdb '$HERE/st/client/private/sam.ldb': No such file or directory
Failed to connect to 'tdb://$HERE/st/client/private/sam.ldb' with backend 'tdb': Unable to open tdb '$HERE/st/client/private/sam.ldb': No such file or directory
ERROR(ldb): uncaught exception - Unable to open tdb '$HERE/st/client/private/sam.ldb': No such file or directory

rather than all that AND a stack trace.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool: write ERROR in red if colour is wanted

Often we'll write something like

   ERROR: Unable to find user "potato"

which can get lost in the jumble of other output. With this patch, we
colour the word "ERROR" red but not the rest of the string, unless it is
determined that colour is not wanted (due to one of --color=never,
NO_COLOR=1, output is not a tty).

We choose to redden the word "ERROR" only to maintain legibility in the
actual message, while hopefully increasing the noticeability of the line.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool visualize: simplify --color-scheme calculations

If you ask for a --color-scheme, you are implicitly asking for --color.
That was documented in --help, but not followed here.

Now --color=no --color-scheme=ansi will use colour for the graph, but not
for other output. This might be useful when the graph is going to a
different place than everything else (`-o foo.txt > bar.txt`).

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool visualise: use global --color

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

py:colour: is_colour_wanted() can take filenames

We need this for `samba-tool visualize -o -` which means output to
stdout, and which has always had a tty test for colour. Rather than
continue to duplicate the full logic there, we can reuse this.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool: --color=auto looks at stderr and stdout

More often than not we are using colour in stderr, but are deciding
based on stdout's tty-ness. This patch changes to use both, and will
affect the following situation:

 samba-tool  2>/tmp/errors   # used to be colour, now not.

of course, if you want colour, you can always

 samba-tool --color=yes 2>/tmp/errors

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool drs showrepl: use global --color option

This changes the default from --color=no to --color=auto.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool: save --color choice for subcommands

In particular, visualize needs it to decide colour for an output
file that may or may not be stdout, so it needs to make its own
decision for that file.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool: make --color a general option

We don't put --color into options.SambaOptions because we can't handle
the 'auto' case in the options module without knowing whether or not
self.outf is a tty, and a) this might not be resolved and b) is fiddly
to pass through.

The .use_colour class flag allows samba-tool subcommands to avoid having
--color, and is *also* useful in the short term for visualise and drs
commands to avoid having this --color clobber their own bespoke versions
(temporarily, during the transition).

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

py:colour: colour_if_wanted() returns the result

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pytest: samba-tool visualize: fix filename

Overwriting the other file was harmless but misleading.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

libcli/auth/proto.h: remove unneeded path details.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

ldb: ldb_build_search_req() check for a talloc failure

The failure in question would have to be a `talloc_strdup(dn, "")` in
ldb_dn_from_ldb_val().

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/server: stop suggesting ntvfs in error message

I am not sure about the rpc proxy.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

libaddns: remove duplicate declaration

Also declared on line 257, exactly the same.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pytest/password_lockout: be less verbose by default

leaving the carefully constructed verbosity there for whoever choses
to switch it on.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool: simplify and clarify SuperCommand._run() a little

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>