resolv_wrapper: Be strict when checking __attribute__ features

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13493

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Andreas Schneider <asn@samba.org>

pam_wrapper: Be strict when checking __attribute__ features

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13493

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Andreas Schneider <asn@samba.org>

nss_wrapper: Be strict when checking __attribute__ features

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13493

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Andreas Schneider <asn@samba.org>

replace: Be strict when checking __attribute__ features

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13493

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Andreas Schneider <asn@samba.org>

socket_wrapper: Be strict when checking __attribute__ features

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13493

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Andreas Schneider <asn@samba.org>

wafsamba: Be strict when checking __attribute__ features

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13493

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Andreas Schneider <asn@samba.org>

wafsamba: Add strict option to CHECK_CODE

Some compilers (e.g. xlc) ignores unsupported features, generates a
warning, but does not fail compilation.

This ensures that any compiler warnings are treated as errors and the
feature support is correctly identified.  This adds equivalent compiler
option to -Werror for xlc.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13493

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Andreas Schneider <asn@samba.org>

ctdb-daemon: Set environment variable if running in interactive mode

CTDB_INTERACTIVE will be used to tell the other daemons if the ctdb daemon
is started in interactive mode.  This is primarily used only for testing.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-daemon: Avoid closing stdin when running in interactive mode

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-tests: Avoid segfault by initializing logging

Setting DEBUGLEVEL before calling debug_init() causes segmentation
violation with gcc8.  DEBUGLEVEL_CLASS is statically initialized to
debug_class_list_initial which is defined as const.  Only after
debug_init() is called, DEBUGLEVEL_CLASS becomes a talloc'd array.

So before modifying DEBUGLEVEL, ensure debug_init() is called via
setup_logging().  (debug_init is a static function.)

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-common: Use correct return type for tevent_queue_add_entry

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13493

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

s3:winbind: Do not lookup local system accounts in AD

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13503

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Wed Jul  4 23:55:56 CEST 2018 on sn-devel-144

nsswitch: Add tests to lookup user via getpwnam

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13503

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

lib: smb_threads: fix access before init bug

talloc_stackframe_internal() calls SMB_THREAD_GET_TLS(global_ts)  which
calls smb_get_tls_pthread() in the POSIX pthread wrapper implementation.

If SMB_THREAD_SET_TLS() hasn't been called before, global_ts is NULL and
smb_get_tls_pthread dereferences it so it crashes.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13505

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

samba-gpupdate: Change machine option to target

On a Windows client, you designate machine/user
apply with a 'target' parameter. This change
makes gpupdate work more like that command.

Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Jul  4 13:23:09 CEST 2018 on sn-devel-144

samba_gpoupdate: Rename the command to samba-gpupdate

On a Windows client, this command is called 'gpupdate'

Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

MAN: Adding entry for net ads lookup

There is no man page description for net ads lookup.
This PR adds entry for the same.

Signed-off-by: Amit Kumar amitkuma@redhat.com
Reviewed-by: Ralph Böhme <slow@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Jul  4 07:38:04 CEST 2018 on sn-devel-144

samba-tool drs showrepl: correctly report failing repsFrom

Hopefully this fixes the flapping test.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Wed Jul  4 04:43:39 CEST 2018 on sn-devel-144

s3: lib/ctdbd_conn: include .h, not .c

Probably a copy/paste mistake. Detected by a failing autobuild on
sn-devel and a local make test:

Build failed: default/examples/libsmbclient/testbrowse2: Symbol
tevent_req_is_unix_error linked in multiple libraries
['samba-cluster-support', 'tevent-util']
UNEXPECTED(failure): wafsamba.duplicate_symbols.duplicate_symbols(none)

Wonder why this didn't fail before in autobuild.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Wed Jul  4 01:19:50 CEST 2018 on sn-devel-144

build: bundle and reduce huge number of EA function tests

It's sufficient to check for one basic function of an EA implementation and a
use a single ifdef for each group of EA functions. This makes more sense than
checking for each EA function on each platform.

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Jul  3 13:24:51 CEST 2018 on sn-devel-144

drs_utils: Add infrastructure to support 'clone with rename'

Our end goal is to create a backup clone of a DB, but rename the
domain/realm so we can startup the backup DC without interferring with
the existing Samba network. The basic strategy to do this is to leverage
DRS replication - by renaming the first object in the partition, all
subsequent objects will automatically be renamed.

This patch adds the infrastructure to do this. I've used object
inheritance to handle the special case of renaming the partition
objects. This means the domain-rename special case doesn't really
pollute the existing DRS replication code. All it needs is a small
refactor to create a new 'process_chunk()' function that the new
sub-class can then override.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

tests: Add a sub-set of tests to show the restored DC is sound

+ Add a new ldapcmp_restoredc.sh test that asserts that the original DC
backed up (backupfromdc) matches the new restored DC.
+ Add a new join_ldapcmp.sh test that asserts we can join a given DC,
and that the resulting DB matches the joined DC
+ Add a new login_basics.py test that sanity-checks Kerberos and NTLM
user login works. (This reuses the password_lockout base code, without
taking as long as the password_lockout tests do). Basic LDAP and SAMR
connections are also tested as a side-effect.
+ run the netlogonsvc test against the restored DC to prove we can
establish a netlogon connection.
+ run the same subset of rpc.echo tests that we do for RODC
+ run dbcheck over the new testenvs at the end of the test run

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

selftest: Add testenv for testing backup/restore

This adds a new testenv for testing that a DC created using the
samba-tool backup/restore can actually be started up. This actually
requires 2 new testenvs:

1. A 'backupfromdc' that solely exists to make a online backup of.
2. A 'restoredc' which takes the backup, and then uses the backup file
to do a restore, which we then start the DC based on.

The backupfromdc is just a plain vanilla AD DC. We use a separate test
env purely for this purpose, because the restoredc will use the same
domain (and so using an existing testenv would potentially interfere
with existing test cases).

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

tests: Add tests for the domain backup online/restore commands

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

netcmd: domain backup restore command

Add a command option that restores a backup file. This is only intended
for recovering from a catastrophic failure of the domain. The old domain
DCs are removed from the DB and a new DC is added.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

netcmd: domain backup online command

This adds a samba-tool command that can be run against a remote DC to
produce a backup-file for the current domain. The backup stores similar
info to what a new DC would get if it joined the network.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

join: Remove unnecessary clone_only flag

For the clone-only case, we have been avoiding a block of code in the
DCJoinContext's __init__(). The main reason we do this is because the
netbios_name is None for clones, and this block of code tries to derive
a bunch of values based on the netbios_name (otherwise, a few lines into
this block, it tries to do NoneType.lower(), which Python doesn't like
very much).

This code is not particularly clone-specific - it is just never going to
work if the netbios_name is None. So we can change the conditional
check, which allows us to get rid of the clone_only flag.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Gary Lockyer <gary@samba.org>
Autobuild-Date(master): Tue Jul  3 08:12:10 CEST 2018 on sn-devel-144

join: Refactor clone_only case to simplify code

Currently for DC clones, we create a regular DCJoinContext, se a
'clone_only' flag, and then make lots of special checks for this flag
throughout the code. Instead, we can use inheritance to create a
DCCloneContext sub-class, and put the specialization there.

This means we can remove all the 'clone_only' checks from the code. The
only 2 methods that really differ are do_join() and join_finalize(), and
these don't share much code at all. (To avoid duplication, I split the
first part of do_join() into a new build_nc_lists() function, but this
is a pretty trivial code move).

We still pass the clone_only flag into the __init__() as there's still
one case where we want to avoid doing work in the case of the clone.
For clarity, I'll refactor this in a subsequent patch.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

join: Rename dc_join() so it looks like an object

dc_join() is creating an object, but it currently looks like it's
just a function call. Rename it to look more object-like.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

join: Pipe through dns_backend option for clones

Allow join_clone() calls to specify a dns_backend parameter for the new
cloned DB.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

provision: Small refactor to host-IP logic

Split out the code that determines the host-IP of the new server into
separate functions. This will allow us to re-use the same logic in the
backup/restore case.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

dbchecker: Fixing up incorrect DNs wasn't working

dbcheck would fail to fix up attributes where the extended DN's GUID is
correct, but the DN itself is incorrect. The code failed attempting to
remove the old/incorrect DN, e.g.

 NOTE: old (due to rename or delete) DN string component for
 objectCategory in object CN=alice,CN=Users,DC=samba,DC=example,DC=com -
 <GUID=7bfdf9d8-62f9-420c-8a71-e3d3e931c91e>;
   CN=Person,CN=Schema,CN=Configuration,DC=samba,DC=bad,DC=com
 Change DN to <GUID=7bfdf9d8-62f9-420c-8a71-e3d3e931c91e>;
   CN=Person,CN=Schema,CN=Configuration,DC=samba,DC=example,DC=com?
 [y/N/all/none] y
 Failed to fix old DN string on attribute objectCategory : (16,
 "attribute 'objectCategory': no matching attribute value while deleting
 attribute on 'CN=alice,CN=Users,DC=samba,DC=example,DC=com'")

The problem was the LDB message specified the value to delete with its
full DN, including the GUID. The LDB code then helpfully corrected this
value on the way through, so that the DN got updated to reflect the
correct DN (i.e. 'DC=example,DC=com') of the object matching that GUID,
rather than the incorrect DN (i.e. 'DC=bad,DC=com') that we were trying
to remove. Because the requested value and the existing DB value didn't
match, the operation failed.

We can avoid this problem by passing down just the DN (not the extended
DN) of the value we want to delete. Without the GUID portion of the DN,
the LDB code will no longer try to correct it on the way through, and
the dbcheck operation will succeed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13495

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Pair-programmed-with: Andrew Bartlett <abartlet@samba.org>

dbcheck: Use symbolic control name for DSDB_CONTROL_DBCHECK_FIX_DUPLICATE_LINKS

While we do not wish to encourage use of this control, manually typed OIDs are
even more trouble, so pass out via pydsdb.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

ctdb-tests: Teach strace packet parser about non-octal escapes

strace output also encodes characters 7 to 13 as \a, \b, \t, \n, \v,
\f, \r.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Mon Jul  2 11:30:29 CEST 2018 on sn-devel-144

ctdb-daemon: Only consider client ID for local database attach

The comment immediately above this code says "don't allow local
clients to attach" and then looks up the client ID regardless of
whether the request is local or remote.

This means that an intentional remote attach from a client will not
work correctly.  No real client should ever do that since clients
attach so they an access databases locally.  Perhaps some sanity
checks should be added.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13500

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-docs: Fix the documentation for VNN map

It is incorrectly says that nodes not in the VNN map can not be
DMASTER.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13499

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-server: Rename CTDB_BROADCAST_VNNMAP -> CTDB_BROADCAST_ACTIVE

This broadcast is misnamed.  Both places where this type of broadcast
is used expect the broadcast to go to all active nodes.

Make the corresponding change to the semantics in the daemon by
sending to all active nodes.

There is a mismatch between the ideas of VNN map and active nodes.  A
node that is not in the VNN map but is active can still host database
records.  These were the same until the LMASTER capability was
introduced and then the logic was not updated.

The only place where the VNN map is relevant is when finding the
location master of a record in the migration code.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13499

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Add a simple test for database traverses

This tests that volatile databases traverse correctly, including the
case where a record was updated on a non-lmaster node.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13499

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Add check for non-lmaster node status in integration tests

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13499

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-client: Fix typo where CTDB_BROADCAST_ALL is repeated

Surely this is meant to be CTDB_BROADCAST_CONNECTED?

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13499

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-recoverd: Set the process name correctly

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-scripts: Drop 99.timeout event script

This is now implemented in local daemon testing.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Switch 90_debug_hung_script.sh to be a simple test

This test only runs against local daemons.  Configuration is done via
script.options, which simplifies things quite a bit.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Enable event script debugging in local daemon tests

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Support CTDB_RUN_TIMEOUT_MONITOR=yes in simple tests

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Clean up startup event in 00.test event script

Rewrite interface initialisation to avoid an error when there are no
interfaces configured.  Re-indent case label.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Add generic logging of event details in 00.test event script

No need for a separate case for each event just to log details.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Drop unnecessary code in 00.test event script

This script is only used with local daemons.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Clean up argument validation in 00.test event script

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Drop check for invalid event in 00.test event script

This isn't necessary and complicates the code.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

tdb: Fix build on AIX

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13493

Here is the build error on AIX 7.1.

../../lib/tdb/tools/tdbtool.c:39:12: error: 'disable_lock' redeclared as different kind of symbol
 static int disable_lock;
            ^~~~~~~~~~~~
In file included from /usr/include/sys/gfs.h:24:0,
                 from /usr/include/sys/vfs.h:27,
                 from ../../lib/replace/system/filesys.h:48,
                 from ../../lib/tdb/tools/tdbtool.c:26:
/usr/include/sys/lock_def.h:314:5: note: previous declaration of 'disable_lock' was here
 int disable_lock(int,simple_lock_t);
     ^~~~~~~~~~~~

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb: Fix build on AIX

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13493

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-tests: Switch fake_ctdbd to use ctdb_get_peer_pid()

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-common: Move capture_socket functions to ctdb_socket.[ch]

The system_<os>.c files contain a lot of duplication, making
maintenance difficult.  These functions are being merged into
system_socket.c and system.c.

Bring across ctdb_sys_open_capture_socket(),
ctdb_sys_close_capture_socket() and ctdb_sys_read_tcp_packet().

Remove empty system_<os>.c files.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-common: Move ctdb_sys_send_tcp() to ctdb_socket.[ch]

The system_<os>.c files contain a lot of duplication, making
maintenance difficult.  These functions are being merged into
system_socket.c and system.c.

Bring across tcp_checksum(), renamed to ip_checksum().
uint16_checksum() becomes static.

Use the BSD struct tcphdr field names for portability.  See the
comment in the code for more details about how we get this to compile
on older glibc versions.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-common: Move ctdb_sys_send_arp() to ctdb_socket.[ch]

The system_<os>.c files contain a lot of duplication, making
maintenance difficult.  These functions are being merged into
system_socket.c and system.c.

Bring a copy of tcp_checksum6(), renamed to ip6_checksum().

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-common: Move ctdb_get_peer_pid() to system.[ch]

The system_<os>.c files contain a lot of duplication, making
maintenance difficult.  These functions are being merged into
system_socket.c and system.c.

This function doesn't need ctdb_sock_addr so put it with general
system utilities.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-common: Move ctdb_system_check_iface_exists() to system.[ch]

The system_<os>.c files contain a lot of duplication, making
maintenance difficult.  These functions are being merged into
system_socket.[ch] and system.[ch].

This function doesn't need ctdb_sock_addr so put it with general
system utilities.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-common: Move parse_ip_mask() to system_socket.[ch]

This uses ctdb_sock_addr so belongs here.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-common: Rename system utility files

system_socket.[ch] will contain all the raw socket code and other
functions that use ctdb_sock_addr.  system.[ch] will contain other
platform dependent functions.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-build: Add ipv6 headers check for packet details

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-build: Add checks for raw pkt handling support

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-common: Use sin6_len only if the structure supports it

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13493

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

replace: Add test for sin6_len in sockaddr_in6 structure

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13493

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-common: Correctly handle conf->reload()

Configuration reload should reset the values of configuration options
missing from the config file to default.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Fri Jun 29 15:12:37 CEST 2018 on sn-devel-144

ctdb: Improve robust mutex test

This avoids some of the undefined behaviour, like initializing the same mutex
twice which happens when the low and high priority processes start (both
do the initialization and that's dangerous.) Instead now we start an
"init" process to start the shared memory segment, and then everything
else just uses it without truncation or unlinking (same mutex).

Signed-off-by: Carlos O'Donell <codonell@redhat.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Fri Jun 29 06:47:00 CEST 2018 on sn-devel-144

README.Coding: Fix link to Python coding style guide (PEP 8)

Signed-off-by: Björn Baumbach <bb@sernet.de>
Reviewed-by: David Mulder <dmulder@suse.com>
Autobuild-User(master): Björn Baumbach <bbaumbach@samba.org>
Autobuild-Date(master): Thu Jun 28 15:04:44 CEST 2018 on sn-devel-144

tests/ntacls: fix pep8 warnings

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Jun 28 12:14:59 CEST 2018 on sn-devel-144

tests/ntacls: use global vars to make code DRY

Move acl and dommain_sid to global vars so we don't repeat them in every
test.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

tests/ntacls_backup: register test

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

ntacls: add extra test file for ntacls backup

The ntacls backup tests have to run in ad_dc:local env, which is
different from existing ntacls tests. Add a separate file for backup
tests.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

ntacls: add functions to backup and restore ntacls

1. backup a share online from a smb connection with ntacls using pysmb API.
2. backup a share offline from service path with ntacls using pysmbd API.
3. restore from tarfile with pysmdb API.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

pysmbd: add py_smbd_create_file

Add create_file function to smbd API.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

pysmbd: extract init_files_struct function

Extract initialization code from set_nt_acl_conn for reuse.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

pysmbd: add py_smbd_mkdir

Add mkdir for smbd API.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

pysmb: add py_smb_unlink and test

Add unlink api to delete a file with a smb connection.
Test added.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

samba-tool showrepl tests: test all-good with --pull-summary

We test the all-good case with --pull-summary, which is the only one
we can be reasonably certain about.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Thu Jun 28 09:23:10 CEST 2018 on sn-devel-144

samba-tool drs showrepl: add --pull-summary and --notify-summary

These separate the two halves of --summary (which is still there),
allowing the repsto and repsfrom to be separately queried.

One motivation for this is testing: it is difficult to assert the
success of repsfrom (--notify-summary) in the test framework, because
we can't rely on the other end behaving properly and promptly.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

s4/torture/samba_tool_drs_showrepl: use assertRegexpMatches

rather than a local rewrite special to this file.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/torture/drs/python: don't double-call enable/disable replication

This is repeating work done in setup/teardown or doubling up in place (self._enable_all_repl includes self._enable_inbound_repl)

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool drs showrepl test: remove useless print

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool drs showrepl: Skip deleted DSAs when checking for success

The deleted DSAs are ignored by the server replication code, so ignore past failures
here also.

The repsFrom and repsTo entries will eventually be removed by the KCC.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

samba-tool drs show_repl: simplify the collection of DC lists

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool drs showrepl --summary for a quick local check

The default output ("classic") gives you a lot of very uninteresting
detail when everything is fine. --summary shuts up about things that
are fine but shouts a little bit when things are broken. It doesn't
provide any new information, just tries to present it in a more useful
format.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool drs showrepl: add a --color flag

Nothing is using it yet, but the next commit will

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

provision: set 'binddns dir' when making new smb.conf

When creating a new smb.conf from scratch during a join/clone/etc, the
'binddns dir' setting still uses the source smb.conf/default setting,
instead of the targetdir sub-directory.

I noticed this problem when trying to create a new testenv - the
provision() was trying to create /usr/local/samba/bind-dns directory,
which would fail if samba hadn't already been installed on the host
machine.

Now that this is fixed, we also need to fix tests that were explicitly
asserting that no unexpected directories were left behind after the test
completes.

This change also breaks the upgradeprovision script. The upgrade-
provision calls newprovision() to create a reference provision in a
temporary directory. However, previously this temporary provision was
creating the bind-dns directory in the actual upgrade directory as a
side-effect, e.g. it did a provision() with
targetdir=alpha13_upgrade_full/private/referenceprovisionLBKBh2 and this
ended up creating alpha13_upgrade_full/bind-dns as a side-effect.
The provision() now creates bind-dns in the specified targetdir, but
this means check_for_DNS() fails (it tries to create bind-dns sub-
directories, but the upgrade's bind-dns doesn't exist). I've avoided
this problem by making sure bind-dns exists as part of the
check_for_DNS() processing.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Jun 28 06:22:16 CEST 2018 on sn-devel-144

selftest: Update MAX_WRAPPED_INTERFACES comment to match code

Commit 19606e4dc657b0baf3ea84d updated the MAX_WRAPPED_INTERFACES define
in the C code from 40 to 64.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests: Add basic test for non-global LoadParm behaviour

Add a simple test to show that the new non-global LoadParm behaviour
works, i.e.
- by default all LoadParm objects are linked to the same underlying
  object
- using a non-global LoadParm creates a separate underlying object.
- using a non-global LoadParm with a bad filename fails.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

param: Add non-global smb.cfg option (support 2 different smb.confs)

The default behaviour is that there is only a single global underlying
LoadParm object. E.g. if you create 2 different LoadParm objects in
python, they both modify the same underlying object.

This patch adds a mechanism to override this and create a separate
non-global LoadParm object. The use-case is the backup tool, where we
want to manipulate 2 different smb.conf files (the one used to create
the backup, and the smb.conf in the backup itself).

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba: read backup date field on init and fail if present

This prevents a backup tar file, created with the new official
backup tools, from being extracted and replicated.

This is done here to ensure that samba-tool and ldbsearch can
still operate on the backup (eg for forensics) but starting
Samba as an AD DC will fail.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests: Add test that Samba cannot be started with a backup DB

We don't want users to take a backup file, and then simply untar it and
run Samba (Several modifications to the DB need to be made as part of
the restore process, so users should always run the 'backup restore'
command).

To enforce this, prime_ldb_databases() now refuses to start Samba if the
backupDate marker is present in the DB. This patch adds a test-case that
proves this basic behaviour works.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

docs: Add manpage for winbind_krb5_localauth.8

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13489

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Wed Jun 27 18:45:56 CEST 2018 on sn-devel-144

docs: Move winbind_krb5_locator manpage to volume 8

The vfs and idmap manpages are in volume 8 too.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13489

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>

krb5_plugin: Move krb5 locator plugin to krb5_plugin subdir

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13489

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>

krb5_plugin: Install plugins to krb5 modules dir

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13489

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>

Update .travis.yml to match package list used in docker container and Catalyst Cloud builds

The package list has some "strange" things in them, but more important is
using the same list everywhere.  We can hopefully harmonise the package lists
to a single file in Samba git soom, merging the docker and packer image creation.

Additionally, Travis CI will probably need to move to Docker once we change
beyond Ubuntu 14.04, so it will simple reference the gitlab.com image then.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Gary Lockyer <gary@samba.org>
Autobuild-Date(master): Wed Jun 27 07:51:14 CEST 2018 on sn-devel-144

remove_dc: Fix removal of an old Windows DC

Windows has 'CN=DNS Settings' child object underneath the Server object.
This was causing the removal of the server object in remove_dc() to
fail.

Noticed this problem while testing the backup/restore tool manually
against a Windows VM.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13484

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Gary Lockyer <gary@samba.org>
Autobuild-Date(master): Tue Jun 26 23:32:51 CEST 2018 on sn-devel-144

s3:tests: Add test for smbclient --quiet

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13485

Signed-off-by: Justin Stephenson <jstephen@redhat.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Björn Baumbach <bb@sernet.de>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Tue Jun 26 20:29:19 CEST 2018 on sn-devel-144

s3:client: Add --quiet option to smbclient

Add quiet command-line argument to allow suppressing the help log
message printed automatically after establishing a smbclient connection

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13485

Signed-off-by: Justin Stephenson <jstephen@redhat.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Björn Baumbach <bb@sernet.de>

dsdb audit: Fix timestamp tests

Fix flapping test:
  [242(3560)/242 at 25m3s] samba4.dsdb.samdb.ldb_modules.audit_log
UNEXPECTED(failure):
  samba4.dsdb.samdb.ldb_modules.audit_log.test_operation_json_empty(none)
REASON: Exception: Exception: difftime(after, actual) >= 0
../source4/dsdb/samdb/ldb_modules/tests/test_audit_log.c:74: error:

The tests truncate the microsecond portion of the time, so the
difference could be less than 0.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Jun 26 06:09:46 CEST 2018 on sn-devel-144