krb5_plugin: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

libcli:smbreadline: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

libcli:smb: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

lib:util: Avoid name confusion with config.h

The HAVE_* is normally used for config.h definitions, so rename it to
USE_ASM_BYTEORDER.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

lib:util: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

tdb: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

replace: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

krb5_wrap: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

ctdb: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

autobuild: Add _FORTIFY_SOURCE=2 to the -O3 build

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

wafsamba: Do not always set _FORTIFY_SOURCE=2

This requires to be compiled with optimization (-O).

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

s3: Remove unsused MMAP_BLACKLIST ifdef checks

This doesn't get defined by anything.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

lib:replace: Check if HAVE_DECL_ENVIRON is defined first

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

krb5_wrap: Fix a typo

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Nov 28 21:15:31 CET 2018 on sn-devel-144

auth: Align integer types

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lib: Align integer types

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

krb5_wrap: Add a talloc_ctx to smb_krb5_principal_get_realm()

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

credentials: Remove an unnecessary talloc_steal()

ccc was already allocated off cred, this talloc_steal was a no-op.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

credentials: Fix set_ccache with empty creds cache

This is an extension of bb2f7e3aee7e9b8: Without this fix in the
"empty ccache" case we never set cred->ccache, so the whole call to
cli_credentials_set_ccache became pointless

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

credentials: Fix an error path memleak

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

credentials: Only do shallow copies of valid ccaches

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

tfork: add a README how to run test torture test under valgrind

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Wed Nov 28 15:57:43 CET 2018 on sn-devel-144

tfork: add a suppresssions file for drd

drd reports:

 initialized twice: cond 0x514f188
    at 0x4C3A399: pthread_cond_init_intercept (drd_pthread_intercepts.c:1022)
    by 0x4C3A399: pthread_cond_init@* (drd_pthread_intercepts.c:1030)
    by 0x50F3FF3: tfork_atfork_child (tfork.c:250)
    by 0x9A4B95D: fork (fork.c:204)
    by 0x50F4834: tfork_start_waiter_and_worker (tfork.c:581)
    by 0x50F4CDB: tfork_create (tfork.c:780)
    by 0x2F7469: tfork_thread (tfork.c:431)
    by 0x4C358F8: vgDrd_thread_wrapper (drd_pthread_intercepts.c:444)
    by 0x8D46593: start_thread (pthread_create.c:463)
    by 0x9A7EE6E: clone (clone.S:95)
 cond 0x514f188 was first observed at:
    at 0x4C3A399: pthread_cond_init_intercept (drd_pthread_intercepts.c:1022)
    by 0x4C3A399: pthread_cond_init@* (drd_pthread_intercepts.c:1030)
    by 0x50F413A: tfork_global_initialize (tfork.c:287)
    by 0x8D4DEA6: __pthread_once_slow (pthread_once.c:116)
    by 0x4C377FD: pthread_once_intercept (drd_pthread_intercepts.c:800)
    by 0x4C377FD: pthread_once (drd_pthread_intercepts.c:806)
    by 0x50F4C0E: tfork_create (tfork.c:743)
    by 0x2F7469: tfork_thread (tfork.c:431)
    by 0x4C358F8: vgDrd_thread_wrapper (drd_pthread_intercepts.c:444)
    by 0x8D46593: start_thread (pthread_create.c:463)
    by 0x9A7EE6E: clone (clone.S:95)

This is intentional, the reinit is in a child process. Cf the comment in
tfork.c.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

tfork: add a suppresssions file for helgrind

tfork_atexit_unknown[1|2]:

  No idea what triggers this, definitely not tfork itself.

tfork_pthread_get_specific:

 Helgrind reports:

 Possible data race during read of size 4 at 0x5141304 by thread #3
 Locks held: none
    at 0x50E602E: tfork_global_get (tfork.c:301)
    by 0x50E69B1: tfork_create (tfork.c:737)
    by 0x2F7419: tfork_thread (tfork.c:431)
    by 0x4C35AC5: mythread_wrapper (hg_intercepts.c:389)
    by 0x8D38593: start_thread (pthread_create.c:463)
    by 0x9A70E6E: clone (clone.S:95)

 This conflicts with a previous write of size 4 by thread #2
 Locks held: none
    at 0x8D3F7B7: pthread_key_create (pthread_key_create.c:41)
    by 0x50E5F79: tfork_global_initialize (tfork.c:280)
    by 0x8D3FEA6: __pthread_once_slow (pthread_once.c:116)
    by 0x50E6999: tfork_create (tfork.c:728)
    by 0x2F7419: tfork_thread (tfork.c:431)
    by 0x4C35AC5: mythread_wrapper (hg_intercepts.c:389)
    by 0x8D38593: start_thread (pthread_create.c:463)
    by 0x9A70E6E: clone (clone.S:95)
  Location 0x5141304 is 0 bytes inside global var "tfork_global_key"
  declared at tfork.c:122

  This is nonsense, tfork_global_get() calls pthread_getspecific, so
  we're looking at the pthread_key_create()/pthread_[g|s]etspecific()
  API here which works with threads by design.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

tfork: TFORK_ANNOTATE_BENIGN_RACE

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

tfork/test: ensure all threads start with SIGCHLD unblocked

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

CVE-2018-16857 dsdb/util: Add better default lockOutObservationWindow

Clearly the lockOutObservationWindow value is important, and using a
default value of zero doesn't work very well.

This patch adds a better default value (the domain default setting of 30
minutes).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13683

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(master): Wed Nov 28 11:31:14 CET 2018 on sn-devel-144

CVE-2018-16857 dsdb/util: Fix lockOutObservationWindow for PSOs

Fix a remaining place where we were trying to read the
msDS-LockoutObservationWindow as an int instead of an int64.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13683

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

CVE-2018-16857 dsdb/util: Correctly treat lockOutObservationWindow as 64-bit int

Commit 442a38c918ae1666b35 refactored some code into a new
get_lockout_observation_window() function. However, in moving the code,
an ldb_msg_find_attr_as_int64() inadvertently got converted to a
ldb_msg_find_attr_as_int().

ldb_msg_find_attr_as_int() will only work for values up to -2147483648
(about 3.5 minutes in MS timestamp form). Unfortunately, the automated
tests used a low enough timeout that they still worked, however,
password lockout would not work with the Samba default settings.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13683

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

CVE-2018-16857 tests: Sanity-check password lockout works with default values

Sanity-check that when we use the default lockOutObservationWindow that
user lockout actually works.

The easiest way to do this is to reuse the _test_login_lockout()
test-case, but stop at the point where we wait for the lockout duration
to expire (because we don't want the test to wait 30 mins).

This highlights a problem currently where the default values don't work.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13683

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

CVE-2018-16853: fix crash in expired passowrd case

When calling encode_krb5_padata_sequence() make sure to
pass a null terminated array as required.

Fixes expired passowrd case in samba4.blackbox.kinit test.

Signed-off-by: Isaac Boukris <iboukris@gmail.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

CVE-2018-16853: Do not segfault if client is not set

This can be triggered with FAST but we don't support this yet.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13571

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

CVE-2018-16853: Add a test to verify s4u2self doesn't crash

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13571

Signed-off-by: Isaac Boukris <iboukris@gmail.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

CVE-2018-16853: The ticket in check_policy_as can actually be a TGS

This happens when we are called from S4U2Self flow, and in that case
kdcreq->client is NULL.  Use the name from client entry instead.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13571

Signed-off-by: Isaac Boukris <iboukris@gmail.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

CVE-2018-16853: Fix kinit test on system lacking ldbsearch

By fixing bindir variable name.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13571

Signed-off-by: Isaac Boukris <iboukris@gmail.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

CVE-2018-16853 WHATSNEW: The Samba AD DC, when build with MIT Kerberos is experimental

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13678

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

CVE-2018-16853 build: The Samba AD DC, when build with MIT Kerberos is experimental

This matches https://wiki.samba.org/index.php/Running_a_Samba_AD_DC_with_MIT_Kerberos_KDC

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13678

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

CVE-2018-16852 dcerpc dnsserver: refactor common properties handling

dnsserver_common.c and dnsutils.c both share similar code to process
zone properties.  This patch extracts the common code and moves it to
dnsserver_common.c.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13669

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

CVE-2018-16852 dcerpc dnsserver: Ensure properties are handled correctly

Fixes for
Bug 13669 - (CVE-2018-16852) NULL
            pointer de-reference in Samba AD DC DNS management

The presence of the ZONE_MASTER_SERVERS property or the
ZONE_SCAVENGING_SERVERS property in a zone record causes the server to
follow a null pointer and terminate.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13669

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

CVE-2018-16852 dcerpc dnsserver: Verification tests

Tests to verify
Bug 13669 - (CVE-2018-16852) NULL
            pointer de-reference in Samba AD DC DNS management

The presence of the ZONE_MASTER_SERVERS property or the
ZONE_SCAVENGING_SERVERS property in a zone record causes the server to
follow a null pointer and terminate.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13669

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

CVE-2018-16851 ldap_server: Check ret before manipulating blob

In the case of hitting the talloc ~256MB limit, this causes a crash in
the server.

Note that you would actually need to load >256MB of data into the LDAP.
Although there is some generated/hidden data which would help you reach that
limit (descriptors and RMD blobs).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13674

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

CVE-2018-16841 selftest: Check for mismatching principal in certficate compared with principal in AS-REQ

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13628
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

CVE-2018-16841 heimdal: Fix segfault on PKINIT with mis-matching principal

In Heimdal KRB5_KDC_ERR_CLIENT_NAME_MISMATCH is an enum, so we tried to double-free
mem_ctx.

This was introduced in 9a0263a7c316112caf0265237bfb2cfb3a3d370d for the
MIT KDC effort.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13628

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

CVE-2018-14629 dns: CNAME loop prevention using counter

Count number of answers generated by internal DNS query routine and stop at
20 to match Microsoft's loop prevention mechanism.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13600

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

dns: prevent self-referencing CNAME

Stops the user from adding a self-referencing CNAME over RPC, which is an easy
mistake to make with samba-tool.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13600

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

notifyd: Improve a debug message

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Nov 27 21:42:24 CET 2018 on sn-devel-144

winbind: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Tue Nov 27 10:38:11 CET 2018 on sn-devel-144

libcli: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libcli: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

idmap_cache: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

winbind: Fix "wbint_Principals" definition

A signed integer does not make any sense for an IDL array length

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

winbind: Use dom_sid_str_buf

Also fix a DBG format string specifier

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

winbind: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

winbind: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libgpo: Align integer types

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libgpo: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libads: Give krb5_errs.c its own header

The protos were declared in lib/krb5_wrap but the functions are not
available there.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs_shadow_copy2: in fstat also convert fsp->fsp_name and fsp->base_fsp->fsp_name

Stacked VFS modules might use the file name, not the file
handle. Looking at you, vfs_fruit...

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13455

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:smbd: pass down twrp from SMB2_CREATE to filename_convert()

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13455

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:smbd: add twrp args to filename_convert()

All existing callers pass NULL, no change in behaviour.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13455

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:smbd: add twrp processing to filename_convert_internal()

Not used for now, existing callers pass NULL.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13455

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:smbd: prepare filename_convert_internal() for twrp

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13455

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:selftest: add a VSS test reading a stream

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13455

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs_shadow_copy2: nicely deal with attempts to open previous version for writing

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13688

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs_shadow_copy2: add shadow_copy2_strip_snapshot_converted

Can be used by callers to determine if a path is in fact pointing at a
file in a snapshot. Will be used in the next commit.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13688

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs_shadow_copy2: add _already_converted arg to shadow_copy2_strip_snapshot_internal()

Not used for now, all existing callers pass NULL.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13688

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:script/tests: add a test for VSS write behaviour

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13688

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s4:torture: add a test-suite for VSS

This test will not be run from the main torture test runner in selftest,
as there we don't pass the required arguments 'twrp_file' and
'twrp_snapshot'.

The test needs a carefully prepared environment with provisioned
snapshot data, so the test will be started from a blackbox test
script. That comes next.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13688

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs_error_inject: add EBADF error

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13688

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs_error_inject: add pwrite

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13688

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

tests: Rework backup test inheritance to make LP constraints clearer

The backup tests have a special constraint where we always want to use
check_output() over runcmd(). The reason is we need the samba-tool
backup/restore commands executed in a separate process. Otherwise the
global underlying LoadParm can accumulate settings from earlier test
case runs.

We can avoid someone in future inadvertently running runcmd() by
mistake, by simply changing the inheritance so we no longer inherit from
SambaToolCmdTest (so the runcmd functions are no longer present).

The comment explaining this has been moved to the top of the file.

Note that the TestCaseInTempDir inheritance was redundant.
BlackboxTestCase inherits from TestCaseInTempDir (and SambaToolCmdTest
was inheriting from BlackboxTestCase).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Tim Beale <timbeale@samba.org>
Autobuild-Date(master): Tue Nov 27 06:57:03 CET 2018 on sn-devel-144

tests: Work out DOMSID via samdb rather than environs

Not all testenvs have the DOMSID set as an environment variable.
However, it's easy enough to work out from querying the samdb.

This is a slight change in that we use a source4-generated loadparm
to connect to the DB (self.lp is source3-generated, presumably for
some SMB connection dependency).

This change is so we can run the ntacls_backup tests against a DC with
SMBv1 disabled (the restoredc). Note that currently the tests fail in
the smb.SMB() connection in the setUp(), so we can't run them as part
of autobuild just yet (because we can't known-fail test errors).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests: Run backup tests against restoredc (SMBv1 disabled)

Running the backup tests against the restoredc highlights that the
backup online/rename commands don't work if SMBv1 is disabled. Note that
the offline commands still work because they don't rely on an SMB
connection to the server.

(Note that running the backup tests against the restoredc is probably a
good idea anyway, to prove that there's no limit to the number of times
you can restore a domain from backup, i.e. we support more than just a
one-off restore).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/samba/test: PY3 port samba.tests.domain_backup

The restoredc already runs under python3, so before we can run the
domain_backup tests against the restoredc, we need to make sure they
work under python3.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Noel Power <noel.power@suse.com>
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests: Handle backup command exceptions as test failures, not errors

If the backup command fails (i.e. throws an exception), we want the test
to fail. This makes it easier to mark tests as 'knownfail' (because we
can't knownfail test errors).

In theory, this should just involve updating run_cmd() to catch any
exceptions from the command and then call self.fail().

However, if the backup command fails, it can leave behind files in the
targetdir. Partly this is intentional, as these files may provide clues
to users as to why the command failed. However, in selftest, it causes
the TestCaseInTempDir._remove_tempdir() assertion to fire. Because this
assert actually gets run as part of the teardown, the assertion gets
treated as an error rather than a failure (and so we can't knownfail the
backup tests). To get around this, we remove any files in the tempdir
prior to calling self.fail().

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests: Rework backup test_backup_invalid_args test-case

self.create_backup() uses self.run_cmd(), which is a wrapper around
self.check_output(). Rework the code to call the underlying
check_output() function directly instead.

The reason we're doing this is we want run_cmd() to catch exceptions and
fail the test (i.e. in the next patch). However, we can't do that because
this test case relies on receiving the exceptions.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

selftest: Designate one testenv as having SMBv1 disabled

We recommend users disable SMBv1 to avoid potential security holes.
However, none of the AD DC testenvs have SMBv1 disabled.

This patch disables SMBv1 on an arbitrarily-chosen testenv (restoredc).

I chose restoredc as we'll want to run the backup tool tests against
this target, and it might be useful to check we can backup a DC if it's
already been restored once.

Note that SMBv2 doesn't support POSIX extensions (only SMBv1 does),
which is why we haven't just disabled SMBv1 on *all* testenvs.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests: Use s3 smbclient for testenv smoketests

smbclient4 doesn't support SMBv2 connections, i.e. it won't work against
a DC with SMBv1 disabled. The smoke-test here is that the DC accepts
the connection, so we don't really care what SMB tool we use to connect.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

selftest: Make testenv NTVFS usage match --use-ntvfs

Regardless of whether the testenv uses --use-ntvfs as part of its
provision options, the s4 testenvs all default to using the NTVFS file
server.

It's not particularly obvious that this is happening. The new restore
DCS (restoredc, renamedc, labdc) were all using NTVFS unintentionally.

The problem is the s4 testenvs default to using services '-s3fs +smb".
provision_ad_dc() explicitly overrides this to use s3fs again
(technically it ends up with both in its smb.conf and just uses whatever
comes last).

This patch changes the testenv setup to check for the presence of the
'--use-ntvfs' option and to set the 'server services' config option
appropriately. This way, the provision command and the smb.conf options
should always line up, with respect to NTVFS.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

selftest: Make chgdcpass's NTVFS usage more obvious

The chgdcpass testenv was not passing --use-ntvfs to the provision
command, but it was still actually using NTVFS.

The reason is the smb.conf generated by provision_raw_step1() would
always try to use the s4/NTVFS file server. Because the smb.conf already
existed, this trumped what was passed to the provision command.

This patch doesn't change the chgdcpass file server. It just makes it
more obvious that chgdcpass is using NTVFS.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

selftest: Be explicit about which testenvs use ntvfs

If a testenv didn't specify any other provision arguments, then it
defaulted to using the NTVFS file server.

This patch makes it explicit, so we just pass through "--use-ntvfs" as
extra provision args in the cases we want.

(Whether all these testenvs really need to use NTVFS or not is another
question, but at least now it's easy to see which testenvs use it).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

libads: Align integer types

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Fri Nov 23 20:23:57 CET 2018 on sn-devel-144

libads: Use dom_sid_string_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

source4 smbd prefork: Add code comments

Add some comments to the prefork code explaining what's going on.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Nov 23 11:40:49 CET 2018 on sn-devel-144

WHATSNEW: prefork restart

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool processes: display pre-fork masters and workers

Tag prefork work processes with "(worker 0)", and sort the process list
on server name to get a consistent order.

 Service:                          PID
 --------------------------------------
 cldap_server                     15588
 ...
 ldap_server                      15584
 ldap_server(worker 0)            15627
 ldap_server(worker 1)            15630
 ldap_server(worker 2)            15632
 ldap_server(worker 3)            15634
 nbt_server                       15576
 notify-daemon                    15638
 ...
 samba                                0
 ...
 wrepl_server                     15580

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

source4 dcerpc_server: remove irpc_add_name

Remove the irpc_add_name from dcesrv_sock_accept, as it results in two
identical names being registered for a process.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

source4 smbd prefork: Cleanup messaging on restart

Clean up names registered in messaging for a terminated process.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

source4 messaging: clean up terminated processes

Now that the smbd pre-fork process model restarts failed processes rather than
terminating, we end up with names registered to defunct processes.
This patch adds a function to clean up all the names registered to a process.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

source4 smbd prefork: Add backoff to process restart

Add new smbd.conf variables 'prefork backoff increment' and
'prefork maximum backoff' to control the rate at which failed pre-forked
processes are restarted.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

source4 smbd prefork: restart on non zero exit code

Restart any pre-fork master or worker process that exits with a non
zero exit code.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

source4 smbd process: pass the fatal flag to terminate

Pass the fatal flag supplied to task_server_terminate to the process
task_terminate method.  It will be used by the task_terminate methods to
set an appropriate exit code.  The process_prefork model will use a non
zero exit code to indicate that the process should be restarted.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

source4 smbd prefork: Restart failed processes

Restart any pre-forked master or worker process that terminated with SIGABRT,
SIGBUS, SIGFPE, SIGILL or SIGSYS

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

source4 smbd prefork: Pass restart information

Pass information about the pre-fork master and worker processes that
will allow them to be restarted.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

source4 smbd test: prefork process restart

Add tests for the restarting of failed/terminated process, by the
pre-fork process model.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

replmd: remove unnecessary indent

The previous refactor now means we return early if we don't need to
re-apply isDeleted to the object. The 'else' is redundant and we can
remove it to avoid unnecessary indent.

This patch is basically just a whitespace change. It should not alter
functionality.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Nov 23 08:10:41 CET 2018 on sn-devel-144

replmd: Move logic into new replmd_should_apply_isDeleted() function

It's easier to follow the logic involved here when it's split out into a
separate function.

This patch should not alter the existing logic/functionality.

Note the 'else' case is somewhat redundant, but it avoids excessive
whitespace changes to the function. It'll be tidied up in the next
patch.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

replmd: Avoid passing msg to replmd_process_linked_attribute()

We can prevent anyone from inadvertently adding/removing msg->elements[]
in replmd_process_linked_attribute() by just not passing msg into the
function. Currently we only actually need the source DN and a memory
context for reallocating old_el->values.

The warning comment has been moved to a more appropriate place.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests: Fix flappiness in DRS tests due to RID Set changing

The test_link_utdv_hwm test case in getnc_exop has started getting
slightly flappy (8 failures in the last 2 weeks). The problem is the
test case creates a new computer, which can occasionally result in a new
RID pool being allocated.

The problem can be reproduced by running the test case repeatedly (it
usually fails after ~250 times).

This patch updates the _check_ctr6() assertion to filter out the 'CN=RID
Set' object, if it happens to be present.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests: Add better error for DRS tests

We've got a flappy test hitting this assertion failure, but we can't
tell why it's failing intermittently (probably because we're bumping the
RID-Set, but there's no way to confirm this).

Add some extra debug info if the test assertion fails.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>