selftest: Use a longer self.account_lockout_duration and self.lockout_observation_window

This matches the changes made in the PSO tests and slows down the
whole testsuite but may make it more reliable on slower build hosts.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Jul  5 12:29:31 CEST 2018 on sn-devel-144

selftest: Use self.account_lockout_duration in self.update_lockout_settings for password_lockout tests

This allows the account_lockout_duration and
lockout_observation_window to be updated with longer values to cope
with slower build servers.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

Document that vfs_full_audit defaults are "none" for the successful and failed operations.

Signed-off-by: Timur I. Bakeyev <timur@iXsystems.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Make "none" the default setting for the successful and failed operations in the vfs_full_audit, so you don't blow up your server by just adding this module to the configuration.

Signed-off-by: Timur I. Bakeyev <timur@iXsystems.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Make sure that vfs*audit modules recognize and accept all the syslog facilities.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13436

Signed-off-by: Timur I. Bakeyev <timur@iXsystems.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

ctdb-tests: Switch to using new event daemon

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Thu Jul  5 09:39:33 CEST 2018 on sn-devel-144

ctdb-daemon: Add client code to talk to new event daemon

This fixes the build and now new eventd is integrated completely in CTDB.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-protocol: Remove protocol for old event daemon

This breaks the build.  The new eventd protocol cannot be introduced without
removing the old eventd protocol.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-client: Remove client code for old event daemon

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-tools: Remove old event daemon tool

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-daemon: Remove old event daemon

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-tests: Remove tests for old event daemon

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-tools: Switch to using new event daemon tool

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-daemon: Switch to starting new event daemon

From this patch onwards, CTDB daemon is broken till the client code for
new eventd is integrated.  This requires getting rid of the old eventd
protocol and client code and then switching to the new eventd protocol
and client code.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-event: Add tests for event daemon

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-tests: Rename eventd testsuite to ctdb_eventd

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-event: Add event daemon client tool

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-event: Add event daemon client code

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-event: Add event daemon implementation

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-event: Add event daemon protocol

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-common: Add client pid to connect callback in sock_daemon

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-tests: Separate testing code for basic data types

This will be used for testing other daemons' protocol code.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-build: Add ctdb prefix to build target

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-protocol: Separate protocol-basic subsystem

This includes marshalling code for basic data types.  This will be used
by other daemons.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

autobuild: Fix random-sleep.sh invocation in autobuild.py

The scripts were not running with the correct path and this causes sn-devel to hit
a very high load as many of the compile jobs start at once.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Thu Jul  5 06:51:26 CEST 2018 on sn-devel-144

samba_tool_showrepl_pull_summary_all_good is flapping

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool drs showrepl test: turn subprocess error into failure

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

netcmd: Use dbcheck to fix DB problems introduced by restore itself

As part of the restore process, we remove all the old DCs from the DB.
However, this introduces some dbcheck errors - there are some DN
attributes and one-way links that reference the deleted objects that
need fixing up. To resolve this, we can run dbcheck as part of the
restore process. This problem affects both renames and plain restores.

The dbcheck.sh test didn't spot this problem because it fixes this type
of DB error first, before it checks the DB.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests: Add new tests for backup-rename command

Extend the existing 'backup online' tests to also test the domain
rename case. This mostly involves some extra assertions that the
restored DB has been modified appropriatelt (i.e. domain NetBIOS
name is updated, etc).

I've also added an extra test case that creates a few objects and
links and specifically asserts that they get renamed appropriately.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

selftest: Add dedicated RENAMEDC testenv for 'backup rename'

Add a new testenv that's similar to the existing restoredc, except we
use 'backup rename' to rename the domain as we back it up.

Restoring this backup then proves that a valid DC can be started from a
renamed backup.

Run the same sub-set of RESTOREDC tests to prove that the new testenv is
sound.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

netcmd: Extend 'backup restore' command to handle renamed domains

When restoring a renamed domain backup, we need to register the new
realm's DNS zone. We do this in the restore step because we don't know
the new server's IP/hostname in the backup step.

Because we may have removed the old realm's DNS entries in the rename
step, the remove_dc() code may fail to find the expected DNS entries for
the DC's domain (the DCs' dnsHostname still maps to the old DNS realm).
We just needed to adjust remove_dns_references() as it was getting a
slightly different error code.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

netcmd: Add 'samba-tool domain backup rename' command

Add a new command that takes a clone of the domain's DB, and renames the
domain as well. (We rename the domain during the clone because it's
easier to implement - the DRS code handles most of the renaming for us,
as it applies the received replication chunks).

The new option is similar to an online backup, except we also do the
following:
- use the new DCCloneAndRenameContext code to clone the DB
- run dbcheck to fix up any residual old DNs (mostly objectCategory
  references)
- rename the domain's netBIOSName
- add dnsRoot objects for the new DNS realm
- by default, remove the old realm's DNS objects (optional)
- add an extra backupRename marker to the backed-up DB. In the restore
  code, if the backup was renamed, then we need to register the new
  domain's DNS zone at that point (we only know the new DC's host IP
  at the restore stage).

Note that the backup will contain the old DC entries that still use the
old dnsHostname, but these DC entries will all be removed during the
restore, and a new DC will be added with the correct dnsHostname.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests: Tweak the backup online tests so they're generic

Update backup-online tests to be more generic. We can then re-use the
common framework for other types of backups (offline, rename), and just
change what's specific to those particular cases.

This change includes asserting the restored backup's domain/realm are
correct, which we weren't doing previously but makes sense.

The new 'return samdb' is for convenience, so that child classes can
easily extend the checks we run over the restored DB.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

drs_utils: Always set the GET_TGT flag for clone renames

The DCCloneAndRenameContext replication was a little inefficient, in
that it would essentially replicate the entire DB twice. This was due to
resolving the link targets - it finds a target object it doesn't know
about, so retries the entire replication again with the GET_TGT flag set
this time.

Normally, the repl_meta_data code will use the target object's GUID,
however, it can't do this for cross-partition links (if it hasn't
replicated the target partition yet). The repl_md code can normally
detect that the link is a cross-parition link by checking the base-DN,
however, this doesn't work in the DCCloneAndRenameContext case because
we have renamed the base-DN.

This is not a big deal - it just means extra work. However, because the
domains being backed up could potentially be quite large, it probably
makes sense to just always set the GET_TGT in the rename case and skip
this extra work.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

join: Add more framework for renaming a domain

Add a DCCloneContext subclass which will rename the DB objects as they
get cloned. This uses the drs_ReplicateRenamer class added to drs_utils
in an earlier patch. Where the drs_Replicate object currently gets
created has been split out into a simple new function, which we can then
override in the rename case.

The other important difference is overriding the provision step, so that
we use the new domain-DN/realm when setting up the initial SAM DB (and
smb.conf, secrets.ldb, etc).

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

uid_wrapper: Be strict when checking __attribute__ features

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13493

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Thu Jul  5 03:01:33 CEST 2018 on sn-devel-144

resolv_wrapper: Be strict when checking __attribute__ features

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13493

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Andreas Schneider <asn@samba.org>

pam_wrapper: Be strict when checking __attribute__ features

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13493

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Andreas Schneider <asn@samba.org>

nss_wrapper: Be strict when checking __attribute__ features

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13493

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Andreas Schneider <asn@samba.org>

replace: Be strict when checking __attribute__ features

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13493

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Andreas Schneider <asn@samba.org>

socket_wrapper: Be strict when checking __attribute__ features

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13493

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Andreas Schneider <asn@samba.org>

wafsamba: Be strict when checking __attribute__ features

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13493

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Andreas Schneider <asn@samba.org>

wafsamba: Add strict option to CHECK_CODE

Some compilers (e.g. xlc) ignores unsupported features, generates a
warning, but does not fail compilation.

This ensures that any compiler warnings are treated as errors and the
feature support is correctly identified.  This adds equivalent compiler
option to -Werror for xlc.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13493

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Andreas Schneider <asn@samba.org>

ctdb-daemon: Set environment variable if running in interactive mode

CTDB_INTERACTIVE will be used to tell the other daemons if the ctdb daemon
is started in interactive mode.  This is primarily used only for testing.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-daemon: Avoid closing stdin when running in interactive mode

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-tests: Avoid segfault by initializing logging

Setting DEBUGLEVEL before calling debug_init() causes segmentation
violation with gcc8.  DEBUGLEVEL_CLASS is statically initialized to
debug_class_list_initial which is defined as const.  Only after
debug_init() is called, DEBUGLEVEL_CLASS becomes a talloc'd array.

So before modifying DEBUGLEVEL, ensure debug_init() is called via
setup_logging().  (debug_init is a static function.)

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-common: Use correct return type for tevent_queue_add_entry

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13493

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

s3:winbind: Do not lookup local system accounts in AD

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13503

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Wed Jul  4 23:55:56 CEST 2018 on sn-devel-144

nsswitch: Add tests to lookup user via getpwnam

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13503

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

lib: smb_threads: fix access before init bug

talloc_stackframe_internal() calls SMB_THREAD_GET_TLS(global_ts)  which
calls smb_get_tls_pthread() in the POSIX pthread wrapper implementation.

If SMB_THREAD_SET_TLS() hasn't been called before, global_ts is NULL and
smb_get_tls_pthread dereferences it so it crashes.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13505

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

samba-gpupdate: Change machine option to target

On a Windows client, you designate machine/user
apply with a 'target' parameter. This change
makes gpupdate work more like that command.

Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Jul  4 13:23:09 CEST 2018 on sn-devel-144

samba_gpoupdate: Rename the command to samba-gpupdate

On a Windows client, this command is called 'gpupdate'

Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

MAN: Adding entry for net ads lookup

There is no man page description for net ads lookup.
This PR adds entry for the same.

Signed-off-by: Amit Kumar amitkuma@redhat.com
Reviewed-by: Ralph Böhme <slow@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Jul  4 07:38:04 CEST 2018 on sn-devel-144

samba-tool drs showrepl: correctly report failing repsFrom

Hopefully this fixes the flapping test.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Wed Jul  4 04:43:39 CEST 2018 on sn-devel-144

s3: lib/ctdbd_conn: include .h, not .c

Probably a copy/paste mistake. Detected by a failing autobuild on
sn-devel and a local make test:

Build failed: default/examples/libsmbclient/testbrowse2: Symbol
tevent_req_is_unix_error linked in multiple libraries
['samba-cluster-support', 'tevent-util']
UNEXPECTED(failure): wafsamba.duplicate_symbols.duplicate_symbols(none)

Wonder why this didn't fail before in autobuild.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Wed Jul  4 01:19:50 CEST 2018 on sn-devel-144

build: bundle and reduce huge number of EA function tests

It's sufficient to check for one basic function of an EA implementation and a
use a single ifdef for each group of EA functions. This makes more sense than
checking for each EA function on each platform.

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Jul  3 13:24:51 CEST 2018 on sn-devel-144

drs_utils: Add infrastructure to support 'clone with rename'

Our end goal is to create a backup clone of a DB, but rename the
domain/realm so we can startup the backup DC without interferring with
the existing Samba network. The basic strategy to do this is to leverage
DRS replication - by renaming the first object in the partition, all
subsequent objects will automatically be renamed.

This patch adds the infrastructure to do this. I've used object
inheritance to handle the special case of renaming the partition
objects. This means the domain-rename special case doesn't really
pollute the existing DRS replication code. All it needs is a small
refactor to create a new 'process_chunk()' function that the new
sub-class can then override.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

tests: Add a sub-set of tests to show the restored DC is sound

+ Add a new ldapcmp_restoredc.sh test that asserts that the original DC
backed up (backupfromdc) matches the new restored DC.
+ Add a new join_ldapcmp.sh test that asserts we can join a given DC,
and that the resulting DB matches the joined DC
+ Add a new login_basics.py test that sanity-checks Kerberos and NTLM
user login works. (This reuses the password_lockout base code, without
taking as long as the password_lockout tests do). Basic LDAP and SAMR
connections are also tested as a side-effect.
+ run the netlogonsvc test against the restored DC to prove we can
establish a netlogon connection.
+ run the same subset of rpc.echo tests that we do for RODC
+ run dbcheck over the new testenvs at the end of the test run

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

selftest: Add testenv for testing backup/restore

This adds a new testenv for testing that a DC created using the
samba-tool backup/restore can actually be started up. This actually
requires 2 new testenvs:

1. A 'backupfromdc' that solely exists to make a online backup of.
2. A 'restoredc' which takes the backup, and then uses the backup file
to do a restore, which we then start the DC based on.

The backupfromdc is just a plain vanilla AD DC. We use a separate test
env purely for this purpose, because the restoredc will use the same
domain (and so using an existing testenv would potentially interfere
with existing test cases).

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

tests: Add tests for the domain backup online/restore commands

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

netcmd: domain backup restore command

Add a command option that restores a backup file. This is only intended
for recovering from a catastrophic failure of the domain. The old domain
DCs are removed from the DB and a new DC is added.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

netcmd: domain backup online command

This adds a samba-tool command that can be run against a remote DC to
produce a backup-file for the current domain. The backup stores similar
info to what a new DC would get if it joined the network.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

join: Remove unnecessary clone_only flag

For the clone-only case, we have been avoiding a block of code in the
DCJoinContext's __init__(). The main reason we do this is because the
netbios_name is None for clones, and this block of code tries to derive
a bunch of values based on the netbios_name (otherwise, a few lines into
this block, it tries to do NoneType.lower(), which Python doesn't like
very much).

This code is not particularly clone-specific - it is just never going to
work if the netbios_name is None. So we can change the conditional
check, which allows us to get rid of the clone_only flag.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Gary Lockyer <gary@samba.org>
Autobuild-Date(master): Tue Jul  3 08:12:10 CEST 2018 on sn-devel-144

join: Refactor clone_only case to simplify code

Currently for DC clones, we create a regular DCJoinContext, se a
'clone_only' flag, and then make lots of special checks for this flag
throughout the code. Instead, we can use inheritance to create a
DCCloneContext sub-class, and put the specialization there.

This means we can remove all the 'clone_only' checks from the code. The
only 2 methods that really differ are do_join() and join_finalize(), and
these don't share much code at all. (To avoid duplication, I split the
first part of do_join() into a new build_nc_lists() function, but this
is a pretty trivial code move).

We still pass the clone_only flag into the __init__() as there's still
one case where we want to avoid doing work in the case of the clone.
For clarity, I'll refactor this in a subsequent patch.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

join: Rename dc_join() so it looks like an object

dc_join() is creating an object, but it currently looks like it's
just a function call. Rename it to look more object-like.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

join: Pipe through dns_backend option for clones

Allow join_clone() calls to specify a dns_backend parameter for the new
cloned DB.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

provision: Small refactor to host-IP logic

Split out the code that determines the host-IP of the new server into
separate functions. This will allow us to re-use the same logic in the
backup/restore case.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

dbchecker: Fixing up incorrect DNs wasn't working

dbcheck would fail to fix up attributes where the extended DN's GUID is
correct, but the DN itself is incorrect. The code failed attempting to
remove the old/incorrect DN, e.g.

 NOTE: old (due to rename or delete) DN string component for
 objectCategory in object CN=alice,CN=Users,DC=samba,DC=example,DC=com -
 <GUID=7bfdf9d8-62f9-420c-8a71-e3d3e931c91e>;
   CN=Person,CN=Schema,CN=Configuration,DC=samba,DC=bad,DC=com
 Change DN to <GUID=7bfdf9d8-62f9-420c-8a71-e3d3e931c91e>;
   CN=Person,CN=Schema,CN=Configuration,DC=samba,DC=example,DC=com?
 [y/N/all/none] y
 Failed to fix old DN string on attribute objectCategory : (16,
 "attribute 'objectCategory': no matching attribute value while deleting
 attribute on 'CN=alice,CN=Users,DC=samba,DC=example,DC=com'")

The problem was the LDB message specified the value to delete with its
full DN, including the GUID. The LDB code then helpfully corrected this
value on the way through, so that the DN got updated to reflect the
correct DN (i.e. 'DC=example,DC=com') of the object matching that GUID,
rather than the incorrect DN (i.e. 'DC=bad,DC=com') that we were trying
to remove. Because the requested value and the existing DB value didn't
match, the operation failed.

We can avoid this problem by passing down just the DN (not the extended
DN) of the value we want to delete. Without the GUID portion of the DN,
the LDB code will no longer try to correct it on the way through, and
the dbcheck operation will succeed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13495

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Pair-programmed-with: Andrew Bartlett <abartlet@samba.org>

dbcheck: Use symbolic control name for DSDB_CONTROL_DBCHECK_FIX_DUPLICATE_LINKS

While we do not wish to encourage use of this control, manually typed OIDs are
even more trouble, so pass out via pydsdb.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

ctdb-tests: Teach strace packet parser about non-octal escapes

strace output also encodes characters 7 to 13 as \a, \b, \t, \n, \v,
\f, \r.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Mon Jul  2 11:30:29 CEST 2018 on sn-devel-144

ctdb-daemon: Only consider client ID for local database attach

The comment immediately above this code says "don't allow local
clients to attach" and then looks up the client ID regardless of
whether the request is local or remote.

This means that an intentional remote attach from a client will not
work correctly.  No real client should ever do that since clients
attach so they an access databases locally.  Perhaps some sanity
checks should be added.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13500

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-docs: Fix the documentation for VNN map

It is incorrectly says that nodes not in the VNN map can not be
DMASTER.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13499

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-server: Rename CTDB_BROADCAST_VNNMAP -> CTDB_BROADCAST_ACTIVE

This broadcast is misnamed.  Both places where this type of broadcast
is used expect the broadcast to go to all active nodes.

Make the corresponding change to the semantics in the daemon by
sending to all active nodes.

There is a mismatch between the ideas of VNN map and active nodes.  A
node that is not in the VNN map but is active can still host database
records.  These were the same until the LMASTER capability was
introduced and then the logic was not updated.

The only place where the VNN map is relevant is when finding the
location master of a record in the migration code.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13499

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Add a simple test for database traverses

This tests that volatile databases traverse correctly, including the
case where a record was updated on a non-lmaster node.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13499

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Add check for non-lmaster node status in integration tests

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13499

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-client: Fix typo where CTDB_BROADCAST_ALL is repeated

Surely this is meant to be CTDB_BROADCAST_CONNECTED?

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13499

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-recoverd: Set the process name correctly

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-scripts: Drop 99.timeout event script

This is now implemented in local daemon testing.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Switch 90_debug_hung_script.sh to be a simple test

This test only runs against local daemons.  Configuration is done via
script.options, which simplifies things quite a bit.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Enable event script debugging in local daemon tests

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Support CTDB_RUN_TIMEOUT_MONITOR=yes in simple tests

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Clean up startup event in 00.test event script

Rewrite interface initialisation to avoid an error when there are no
interfaces configured.  Re-indent case label.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Add generic logging of event details in 00.test event script

No need for a separate case for each event just to log details.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Drop unnecessary code in 00.test event script

This script is only used with local daemons.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Clean up argument validation in 00.test event script

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Drop check for invalid event in 00.test event script

This isn't necessary and complicates the code.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

tdb: Fix build on AIX

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13493

Here is the build error on AIX 7.1.

../../lib/tdb/tools/tdbtool.c:39:12: error: 'disable_lock' redeclared as different kind of symbol
 static int disable_lock;
            ^~~~~~~~~~~~
In file included from /usr/include/sys/gfs.h:24:0,
                 from /usr/include/sys/vfs.h:27,
                 from ../../lib/replace/system/filesys.h:48,
                 from ../../lib/tdb/tools/tdbtool.c:26:
/usr/include/sys/lock_def.h:314:5: note: previous declaration of 'disable_lock' was here
 int disable_lock(int,simple_lock_t);
     ^~~~~~~~~~~~

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb: Fix build on AIX

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13493

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-tests: Switch fake_ctdbd to use ctdb_get_peer_pid()

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-common: Move capture_socket functions to ctdb_socket.[ch]

The system_<os>.c files contain a lot of duplication, making
maintenance difficult.  These functions are being merged into
system_socket.c and system.c.

Bring across ctdb_sys_open_capture_socket(),
ctdb_sys_close_capture_socket() and ctdb_sys_read_tcp_packet().

Remove empty system_<os>.c files.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-common: Move ctdb_sys_send_tcp() to ctdb_socket.[ch]

The system_<os>.c files contain a lot of duplication, making
maintenance difficult.  These functions are being merged into
system_socket.c and system.c.

Bring across tcp_checksum(), renamed to ip_checksum().
uint16_checksum() becomes static.

Use the BSD struct tcphdr field names for portability.  See the
comment in the code for more details about how we get this to compile
on older glibc versions.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-common: Move ctdb_sys_send_arp() to ctdb_socket.[ch]

The system_<os>.c files contain a lot of duplication, making
maintenance difficult.  These functions are being merged into
system_socket.c and system.c.

Bring a copy of tcp_checksum6(), renamed to ip6_checksum().

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-common: Move ctdb_get_peer_pid() to system.[ch]

The system_<os>.c files contain a lot of duplication, making
maintenance difficult.  These functions are being merged into
system_socket.c and system.c.

This function doesn't need ctdb_sock_addr so put it with general
system utilities.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-common: Move ctdb_system_check_iface_exists() to system.[ch]

The system_<os>.c files contain a lot of duplication, making
maintenance difficult.  These functions are being merged into
system_socket.[ch] and system.[ch].

This function doesn't need ctdb_sock_addr so put it with general
system utilities.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-common: Move parse_ip_mask() to system_socket.[ch]

This uses ctdb_sock_addr so belongs here.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-common: Rename system utility files

system_socket.[ch] will contain all the raw socket code and other
functions that use ctdb_sock_addr.  system.[ch] will contain other
platform dependent functions.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-build: Add ipv6 headers check for packet details

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-build: Add checks for raw pkt handling support

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-common: Use sin6_len only if the structure supports it

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13493

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

replace: Add test for sin6_len in sockaddr_in6 structure

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13493

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>