tevent: version 0.9.37

* simplify "poll" and "poll_mt" backends
* make tevent_abort() reachable for backends
* add tevent_common_invoke_*_handler() functions
* add tevent_context_same_loop() function
* add tevent_context_wrapper_create() infrastructure
* add tevent_req_profile infrastructure

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

tevent: Add tevent_req_profile

This allows detailed reporting where a tevent_req spends its time

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

tevent: add a test that frees wrapper_ev with pending events

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

tevent: add a simple wrapper test

This checks that for all supported event types the before and after
handlers are called.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

tevent: add tevent_context_wrapper_create() infrastructure

This allows to specify wrapper tevent_contexts, which adds the ability
to run functions before and after the event handler functions.

This can be used to implement impersonation hooks
or advanced debugging/profiling hooks.

We'll undo the 0.9.36 ABI change on the 0.9.37 release
at the end of this patchset.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

tevent: make use of #include "system/threads.h"

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

tevent: split out tevent_common_invoke_fd_handler()

We'll undo the 0.9.36 ABI change on the 0.9.37 release
at the end of this patchset.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

tevent: split out tevent_common_invoke_immediate_handler()

We'll undo the 0.9.36 ABI change on the 0.9.37 release
at the end of this patchset.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

tevent: split out tevent_common_invoke_timer_handler()

As side effect this avoids tricks with an extra
tevent_common_timed_deny_destructor().

We'll undo the 0.9.36 ABI change on the 0.9.37 release
at the end of this patchset.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

tevent: split out tevent_common_invoke_signal_handler()

As side effect this avoids tricks with tevent_se_exists_destructor() to
figure out if the event handler removed itself.

We'll undo the 0.9.36 ABI change on the 0.9.37 release
at the end of this patchset.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

tevent: simplify tevent_signal_destructor()

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

tevent: use talloc_zero() in tevent_signal.c

This might not be strictly required, but it might
avoid problems in future...

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

tevent: simplify tevent_cleanup_pending_signal_handlers()

Calling tevent_signal_destructor() does the same as se->event_ctx is already
NULL.

This also makes sure we correctly cleanup the SA_SIGINFO array.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

tevent: add tevent_common_check_double_free() helper function

This will be used to generically support TALLOC_FREE() on
event which are currently running.

It aborts on every explicit talloc_free(), but ignores implicit
cleanup when the talloc parent is about to go.

We'll undo the 0.9.36 ABI change on the 0.9.37 release
at the end of this patchset.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

tevent: add tevent_threaded_schedule_immediate_destructor that just aborts

This will be active while the event is part of the ev->scheduled_immediates
list.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

tevent: use _tevent_schedule_immediate() to move events from a thread to the main_ev

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

tevent: use struct initializers for tevent_immediate

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

tevent: use struct initializers for tevent_signal

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

tevent: use struct initializers for tevent_timer

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

tevent: use struct initializers for tevent_fd

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

tevent: make tevent_abort() available for backends

We'll undo the 0.9.36 ABI change on the 0.9.37 release
at the end of this patchset.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

tevent: allow tevent_abort() to cope with ev == NULL

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

tevent/testsuite: return after torture_fail()

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

tevent.h: improve tevent_req documentation

Document tevent_req naming conventions.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

tevent: rewrite/simplify tevent_poll and maintain ev->fd_events correctly

The following patches will rely on having all valid fd events in
ev->fd_events, even if they are temporary disabled with
tevent_set_fd_flags(fde, 0);

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

tevent: make use of tevent_common_wakeup() in the poll and poll_mt backends

This simplifies the "poll_mt" logic a lot.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

winbindd: Remove ads.h include from nss_info

nss_info does not use libads. Removing this include
fixes a compile error when trying to compile with a
system provided heimdal library:

[2188/3043] Compiling source3/winbindd/nss_info.c
In file included from ../source3/libads/kerberos_proto.h:33:0,
                 from ../source3/include/ads.h:154,
                 from ../source3/winbindd/nss_info.c:24:
../lib/replace/system/kerberos.h:33:10: fatal error: krb5.h: No such file or directory
 #include <krb5.h>
          ^~~~~~~~

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Christof Schmitt <cs@samba.org>
Autobuild-Date(master): Wed Jul 11 22:39:36 CEST 2018 on sn-devel-144

ctdb-scripts: Provide a gstack function if gstack is not available

gstack isn't widely available, so provide a simple function that does
the same thing if it gstack can't be found.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Wed Jul 11 14:47:21 CEST 2018 on sn-devel-144

ctdb-tests: Drop residual CTDB_MANAGED_<service> variables

These no longer do anything.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-scripts: Drop event script CTDB_MANAGED_<service> variables

Enable required event scripts to manage services.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Drop event script tests where CTDB_MANAGED_<service>=no

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Ensure some event scripts are enabled for cluster tests

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-packaging: Enable some standard event scripts if none are enabled

CTDB needs the legacy/00.ctdb event script to be able to function
properly.  If this script is not enabled then assume a first-time
install or an upgrade to a version that requires events scripts to be
enabled via symlinks.  In these cases enable this script and other
commonly used scripts.

Remove links during uninstall (but not during upgrade).

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-build: Enable some standard event scripts if none are enabled

CTDB needs the legacy/00.ctdb event script to be able to function
properly.  If this script is not enabled then assume a first-time
install or an upgrade to a version that requires events scripts to be
enabled via symlinks.  In these cases enable this script and other
commonly used scripts.

Only do this for a direct install.  If DESTDIR is being used then
assume a package is being built and let the packager handle this case.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-build: Install event scripts in CTDB_DATADIR

Signed-off-by: Martin Schwenke <martin@meltin.net>
Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Clean up define_test() for event scripts

Factor out a little bit of common code.  More coming.

Most of this is whitespace changes.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Drop an unused case

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: New install path CTDB_SCRIPT_DATA_DIR

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tools: All ctdb event commands to run without ctdbd

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-event: Allow tool to enable/disable scripts without daemon

Only open the client socket when it is needed.  Note that this only
works for enabling/disabling event scripts via symlinks.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-event: Update event tool to handle symbolic links

Supports the case when scripts are installed in the data directory and
are linked to when enabled.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-common: Add path support for datadir

Signed-off-by: Martin Schwenke <martin@meltin.net>
Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-build: Add CTDB_DATADIR

Signed-off-by: Martin Schwenke <martin@meltin.net>
Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-daemon: Drop the noiphost "node flags" bitmap

This is no longer needed because inactive/disabled nodes no longer
report any available public IP addresses.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-daemon: Stop inactive/disabled nodes from reporting available IPs

This can be done now that NoIPHostOnAllDisabled is gone and will allow
the public IP address failover logic to be simplified.

In the test code, still filter available IP addresses by node state.
This code can't currently read information about available IP
addresses but that will change in future

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-daemon: Drop plumbing for obsolete tunable NoIPHostOnAllDisabled

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-daemon: Mark NoIPHostOnAllDisabled tunable as obsolete

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-daemon: Change default for tunable NoIPHostOnAllDisabled to 1

Defaulting to host public IP addresses when all nodes are unhealthy
does not obey the principle of least surprise.  It has caused much
confusion over the years.  It often leads to problems when all nodes
are unhealthy due to something like a cluster filesystem being
unmounted.

Change the default value for this tunable as the first step of
completely removing this behaviour.

Remove tests that set NoIPHostOnAllDisabled=1 and update the expected
result for other tests where no nodes are healthy.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Setup public addresses in 60.nfs unit tests

Even the monitor event runs update_tickles(), which needs public IP
addresses and FAKE_CTDB_NUMNODES to be initialised.  Currently this
works by default but soon we'll need FAKE_CTDB_NUMNODES in another
context.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Replace hardcoded IP address in test results

Parameterise them with a variable instead.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

WHATSNEW: Add more text about work done by Catalyst developers

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Jul 11 11:47:09 CEST 2018 on sn-devel-144

doc: samba-tool drs showrepl --json and --summary

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

doc: samba-tool visualize uptodateness

and --xdot option.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

WHATSNEW: samba-tool drs showrepl and visualize changes

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

vfs_audit: Extend list of recognized syslog(3) facilities and wrap them into #ifdef's. That list should be comprehensive enough to cover most of the existing OSes.

Signed-off-by: Timur I. Bakeyev <timur@iXsystems.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Gary Lockyer <gary@samba.org>
Autobuild-Date(master): Wed Jul 11 08:44:10 CEST 2018 on sn-devel-144

wscript: Add --with-system-heimdalkrb5

Add the configure option --with-system-heimdalkrb5 to build Samba
explicitly with a system Heimdal kerberos library. This does the same as
the more complicated syntax

--bundled-libraries='!heimdal,!asn1,!com_err,!roken,!hx509,!wind,!gssapi,!hcrypto,!krb5,!heimbase,!asn1_compile,!compile_et,!kdc,!hdb,!heimntlm'

and it also enforces the conflicts with MIT Kerbros and the AD DC
build.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Jul 11 05:18:59 CEST 2018 on sn-devel-144

WHATSNEW: Added entries for PSOs, domain backup/restore, and rename

Added WHATSNEW blurbs for the following features:
- Password Settings Objects
- Domain backup and restore
- Domain rename tool

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

pass 'rdonly' or 'directory' flag to open a directory file.

Signed-off-by: Pooja Mahadik <pooja.mahadik@veritas.com>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Wed Jul 11 02:22:18 CEST 2018 on sn-devel-144

s3/rpc_server: Character Encode Spotlight Queries

Fix path escaping in Spotlight so paths with spaces or special
characters can be properly matched to tracker paths.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=12688

Based-on-a-patch-from: Mike M Pestorich <mmpestorich@gmail.com>
(similar to github.com/netatalk/netatalk/commit/90aa43d)

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Jul 10 23:17:20 CEST 2018 on sn-devel-144

s3:messages: make the loop in msg_dgm_ref_recv() more robust against stale pointers

The interaction between msg_dgm_ref_recv() and msg_dgm_ref_destructor()
doesn't allow two references from messaging_dgm_ref() to be free'd
during the loop in msg_dgm_ref_recv().

In addition to the global 'refs' list, we also need to
have a global 'next_ref' pointer, which can be adjusted in
msg_dgm_ref_destructor().

As AD DC we hit this when using irpc in auth_winbind,
which uses imessaging_client_init().
In addition to the main messaging_dgm_ref() in smbd,
source3/auth/auth_samba4.c: prepare_gensec() and
make_auth4_context_s4() also generate a temporary
imessaging_context for auth_context->msg_ctx from within
auth_generic_prepare().

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13514

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s4:messaging: add local.messaging.multi_ctx.multi_ctx test

This tests the usage of multiple imessaging_contexts in one process
and also freeing two of them during a message handler.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13514

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

python/tests: make the test_assoc_group_fail2() test more resilient against timing

On a busy system [e]poll() on the server will mark both the
old connection fd and also the listening fd as readable.

epoll() returns the events in order, so the server processes the
disconnect first.

With poll() we don't have an order of the events and the
server is likely to process the connect before the disconnect.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

ctdb: close the correct pipe fd in a test

This was discovered in an autobuild with a patched tevent that used the
"poll" backend by default. Test failure:

$ bin/sock_daemon_test /dev/shm/sock_daemon_test.pid /dev/shm/sock_daemon_test.sock 5
test5[28011]: daemon started, pid=28011
test5[28011]: listening on /dev/shm/sock_daemon_test.sock
sock_daemon_test: ../ctdb/tests/src/sock_daemon_test.c:980: test5: Assertion `ret == i+1' failed.
Abgebrochen (Speicherabzug geschrieben)
metze@SERNOX14:~/devel/samba/4.0/master4-test$ test5[28011]: PID 28010 gone away, exiting
test5[28011]: Shutting down
sock_daemon_test: ../ctdb/tests/src/sock_daemon_test.c:964: test5:
Assertion `ret == EINTR' failed.

After an epic debugging session we spotted the problem.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

WHATSNEW add entries audit logging and lmdb.

Add WHATSNEW entries for dsdb, password and group change audit logging,
as well as the ldb lmdb backend

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Jul 10 12:53:54 CEST 2018 on sn-devel-144

heimdal: Fix build with system provided heimdal library

Trying to compile with a system provided heimdal library
results in this compile error:

[ 876/3043] Compiling source4/auth/kerberos/srv_keytab.c
In file included from /usr/include/heimdal/krb5.h:949:0,
                 from ../lib/replace/system/kerberos.h:33,
                 from ../source4/auth/kerberos/srv_keytab.c:31:
/usr/include/heimdal/krb5-protos.h:3894:1: error: unknown type name ‘HEIMDAL_WARN_UNUSED_RESULT_ATTRIBUTE’; did you mean ‘_WARN_UNUSED_RESULT_’?
 HEIMDAL_WARN_UNUSED_RESULT_ATTRIBUTE KRB5_LIB_FUNCTION krb5_error_code KRB5_LIB_CALL
 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 _WARN_UNUSED_RESULT_
/usr/include/heimdal/krb5-protos.h:3895:1: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘krb5_generate_random’
 krb5_generate_random (

The problem is that Samba provides a minimal krb5-types.h file
for the internal build that gets used during the build with
the system provided heimdal library. As the minimal file
does not provide all definitions, the build fails.

Fix this by having the krb-types.h file simply include the
include file from the system library, if the build is done
using the system provided heimdal library.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Jul 10 07:30:26 CEST 2018 on sn-devel-144

netcmd: Add sanity-check for invalid domain rename args

We are suggesting to users that it's safe to run a renamed domain in
parallel with the old backed-up domain. However, this would not be the
case if the user (foolishly) "renames" their domain using the exact same
NetBIOS name or DNS realm.

Using the same DNS realm fails later on (updating the dnsRoot values),
but using the same NetBIOS name actually succeeds. While we can't make
samba tools completely idiot-proof, we can protect users from the most
basic of (potentially unintended) errors with some simple sanity-checks.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

selftest: Add a 'LABDC' testenv to mimic a preproduction test-bed

One of the use-cases for the domain rename tool is to produce a lab
domain that can be used for pre-production testing of Samba.
Basically this involves taking a backup rename with --no-secrets (which
scrubs any sensitive info), and then restoring it.

This patch adds a testenv that mimics how a user would go about creating
a lab-domain. We run the same tests that we run against the restore and
rename testenvs.

Note that the rpc.echo tests for the testallowed and testdenied users
fail, because we don't backup the secrets for these users. So these
tests failing proves that the lab-DC testenv is correct.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

netcmd: Add brief log file of what the backup actually contains

There are now several different permutations of backup file that can be
created (i.e. online, rename, with/without secrets). Hopefully the admin
users would organize their backup files sensibly, but it can't hurt to
keep track of what the backup-file actually contains in a simple
human-readable file within the backup tar. E.g. We really don't want
backups with secrets-included and secrets-excluded getting mixed up.

Recording the DC used to make the domain backup may be useful in the
event of a catastrophic failure of the domain, e.g. DC replication may
have been broken for some time prior to the failure.

Recording the samba-tool version string may also be useful if there are
ever any backwards-compatibility issues introduced to the backup files.
The intention is to say we only support restoring a backup with the same
version of samba-tool that actually created the backup, however, it'd be
polite to users to actually record that version somewhere.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

netcmd: Add no-secrets option to domain backups

By default we include all the domain's secrets in the backup file. This
patch adds an extra option to exclude these secrets. In particular, this
is for the use case of creating a lab domain (where you might not feel
comfortable with the secrets for all your users being present).

Mostly this just involves passing the correct option to the join/clone.
I've also made sure that a password is also set for the Admin user
(samba does seem to start up without one set, but this behaviour is
closer to what happens during a provision).

The tests have been extended to use the new option, and to assert that
secrets are/aren't included as expected for some of the builtin testenv
users.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

README.Coding disable include sorting in clang format

Update the clang format configuration to disable include sorting. This
is enabled by default and breaks samba code.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Gary Lockyer <gary@samba.org>
Autobuild-Date(master): Tue Jul 10 04:40:51 CEST 2018 on sn-devel-144

Revert "s3/service: convert lp_force_group() to const"

This reverts commit c53646bccd87ef3b3133d3f7526ef85591909528.
As mentioned by Andrew, we shouldn't break environments where
"force group" has been configured to use substituted variables.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Jul 10 00:12:19 CEST 2018 on sn-devel-144

Revert "s3/service: convert lp_force_user() to const"

This reverts commit c58194e3d296f4e14e7689bdf192c561635ae161.
As mentioned by Andrew, we shouldn't break environments where
"force user" has been configured to use substituted variables.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libsmbclient: Initialize written in cli_splice_fallback()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13511

Signed-off-by: Bailey Berro <baileyberro@chromium.org>
Reviewed-by: David Mulder <dmulder@suse.com>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: David Disseldorp <ddiss@samba.org>
Autobuild-User(master): David Disseldorp <ddiss@samba.org>
Autobuild-Date(master): Mon Jul  9 21:29:48 CEST 2018 on sn-devel-144

libsmbclient: Initialize written value before use.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13511

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: David Disseldorp <ddiss@samba.org>

winbind_krb5_localauth: Fix a compiler warning

This can't used uninitialized but some compiler complains about it.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Sat Jul  7 16:24:30 CEST 2018 on sn-devel-144

s3: smbd/durable: remove dev and inode check from vfs_default_durable_reconnect_check_stat()

On a cluster filesystem the device numbers may differ on the cluster
nodes. We already verify the file_id in vfs_default_durable_reconnect(),
so we can safely remove the dev/inode checks.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13318

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lib:charset: Fix error messages from charset conversion

When e.g. trying to access a filename through Samba that does not adhere
to the encoding configured in 'unix charset', the log will show the
encoding problem, followed by "strstr_m: src malloc fail". The problem
is that strstr_m assumes that any failure from push/pull_ucs2_talloc is
a memory allocation problem, which is not correct.

Address this by removing the misleading messages and add a missing
message in convert_string_talloc_handle.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3: smbd: fix a check in stat_cache_add()

As the comment above the if condition says:

        /*
         * If we are in case insentive mode, we don't need to
         * store names that need no translation - else, it
         * would be a waste.
         */

Ie if stat_cache_add() is called as

        stat_cache_add("foo/bar", "foo/bar", false)

There's no need to cache the path, as a simple stat() on the client
supplied name (full_orig_name) matches the name used in the
filesystem (passed to stat_cache_add() as translated_path).

So fix the if condition to match the comment.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

docs/vfs_ceph: add CTDB_SAMBA_SKIP_SHARE_CHECK=yes caveat

Mostly copied from the vfs_gluster manpage: the CephFS share path is not
locally mounted, which breaks the ctdb_check_directories_probe() check.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Jul  6 23:19:02 CEST 2018 on sn-devel-144

vfs_ceph: don't lie about flock support

Instead, match vfs_gluster behaviour and require that users explicitly
disable "kernel share modes".

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13506

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3/loadparm: fix a few talloc stackframe leaks

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3/service: convert lp_force_user() to const

Avoid set_conn_force_user_group() talloc stackframe leaks in doing so.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3/service: convert lp_force_group() to const

set_conn_force_user_group() and change_to_user_internal() leak onto
the callers' talloc stackframe. Drop the unnecessary heap allocations.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3/uid: use lp_const_servicename() where possible

The majority of these lp_servicename(talloc_tos(), ...) callers leak
onto the talloc stackframe. Drop the unnecessary heap allocations.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3/service: use lp_const_servicename() where possible

The majority of these lp_servicename(talloc_tos(), ...) callers leak
onto the talloc stackframe. Drop the unnecessary heap allocations.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

nsswitch: Use a swtich in the wbinfo test to lookup users

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13503

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Fri Jul  6 17:14:44 CEST 2018 on sn-devel-144

ctdb-tests: Avoid segfault by initializing logging

This is in addition to af697008531.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Thu Jul  5 15:22:16 CEST 2018 on sn-devel-144

ctdb-common: Fix CID 437606

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

selftest: Use a longer self.account_lockout_duration and self.lockout_observation_window

This matches the changes made in the PSO tests and slows down the
whole testsuite but may make it more reliable on slower build hosts.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Jul  5 12:29:31 CEST 2018 on sn-devel-144

selftest: Use self.account_lockout_duration in self.update_lockout_settings for password_lockout tests

This allows the account_lockout_duration and
lockout_observation_window to be updated with longer values to cope
with slower build servers.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

Document that vfs_full_audit defaults are "none" for the successful and failed operations.

Signed-off-by: Timur I. Bakeyev <timur@iXsystems.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Make "none" the default setting for the successful and failed operations in the vfs_full_audit, so you don't blow up your server by just adding this module to the configuration.

Signed-off-by: Timur I. Bakeyev <timur@iXsystems.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Make sure that vfs*audit modules recognize and accept all the syslog facilities.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13436

Signed-off-by: Timur I. Bakeyev <timur@iXsystems.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

ctdb-tests: Switch to using new event daemon

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Thu Jul  5 09:39:33 CEST 2018 on sn-devel-144

ctdb-daemon: Add client code to talk to new event daemon

This fixes the build and now new eventd is integrated completely in CTDB.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-protocol: Remove protocol for old event daemon

This breaks the build.  The new eventd protocol cannot be introduced without
removing the old eventd protocol.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-client: Remove client code for old event daemon

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-tools: Remove old event daemon tool

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-daemon: Remove old event daemon

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-tests: Remove tests for old event daemon

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>