talloc: Release talloc 2.2.0

 * Remove pytalloc_CObject_FromTallocPtr()
 * Remove --extra-python (a build time mode to produce Python2 and Python3
   bindings at the same time)
 * New minor version to allow Samba 4.10 to release a talloc if
   required from that branch

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

build: Standardise on calling conf.SAMBA_CHECK_PYTHON() in libraries

We do this by removing the confusing mandatory option to
conf.SAMBA_CHECK_PYTHON{,_HEADERS}(), instead just use the value of
--disable-python internally

This follows the default minimum of Python 3.4 and keeps things consistent
with the main Samba build where --disable-python is required to skip building
python bindings.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

build: Remove manual specification of minimum python version

We now used the default of 3.4 from conf.SAMBA_CHECK_PYTHON()

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

build: Set default minimum python version to 3.4.0

This is the current minimum, but this may change before the 4.11 release.

Python 2.x support is no longer available except to build using --disable-python.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

build: Do not make python mandatory to build

Clearly we have python or else we would not be running, so this is about if
we have a new enough version.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

ABI: Remove unused .py3*.sigs files

These are no longer used by the build system so avoid
confusion by removing them from the tree.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

build: Remove distinct .py3 ABI files

The only difference between the two built libraries is pytalloc_CObject_FromTallocPtr()
which is deprecated.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

selftest: Remove mention of --extra-python from comment

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

selftest: Remove obsolete py3_compatible=True markers

All our tests now run in python3.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

selftest: Remove support for running multiple tests against python versions in a single run

The extra_python support was added to aid the python3 transition

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

build: Remove bld.gen_python_environments()

This was part of --extra-python support.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

build: Remove --extra-python

This option is quite invasive in waf and was mainly for the python3 transition.

Testing with multiple python versions can be done by testing a full compile against
multiple versions, likewise multiple different binding versions can be created
the same way.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

pytalloc: Remove deprecated pytalloc_CObject_FromTallocPtr()

This function makes it harder to remove the --extra-python handlers and is only
provided for Python 2.x, support for which Samba is removing.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

libsmb: Make SMB1-only cli_nt_hardlink calls static

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Mar 20 23:32:31 UTC 2019 on sn-devel-144

utils: Use cli_hardlink instead of cli_nt_hardlink

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libsmb: Introduce protocol-agnostic cli_hardlink

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libsmb: Use cli_smb2_set_info_fnum()

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libsmb: Simplify cli_smb2_ftruncate

We don't need to talloc 8 bytes, they can live on the stack. When we
go async, this can go into the state struct.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libsmb: Simplify cli_smb2_delete_on_close

Use cli_smb2_set_info_fnum, remove "state" variables

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libsmb: Add cli_smb2_set_info_fnum

This wraps getting the file handle from the fnum

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s4:librpc: Fix installation of Samba

This breaks installation of Samba 4.10 on Fedora.

https://bugzilla.samba.org/show_bug.cgi?id=13847

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Wed Mar 20 13:11:28 UTC 2019 on sn-devel-144

py/graph: use 2.6 compatible check for set membership

It is better this way anyway.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Mar 20 06:36:05 UTC 2019 on sn-devel-144

regfio tests: Update comment style to match README.Coding

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13840

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

regfio: Update code near recent changes to match README.Coding

This file long predates our current code conventions.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13840

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

regfio: Improve handling of malformed registry hive files

* next_record: A malformed file can lead to an endless loop.
* regfio_rootkey: Supplying a malformed registry hive file to the
  registry hive I/O code can lead to out-of-bounds reads.

Test cases are included. Both issues resolved have been identified using
AddressSanitizer.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13840

Signed-off-by: Michael Hanselmann <public@hansmi.ch>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

regfio: Add trivial unit test

An upcoming commit will resolve two cases of insufficient handling of
mangled registry hive files and will include unit tests.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13840

Signed-off-by: Michael Hanselmann <public@hansmi.ch>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

regfio: Use correct function names in debug information

Signed-off-by: Michael Hanselmann <public@hansmi.ch>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Fix typos in "valid"

s/vald/valid/

Signed-off-by: Michael Hanselmann <public@hansmi.ch>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

build: Allow build when --disable-gnutls is set

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13844

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Wed Mar 20 05:25:48 UTC 2019 on sn-devel-144

s3: smbd: Clarify POSIX name handling in SMB2 get/setinfo calls.

We should never be looking at the info level here, the create
call will have set the correct POSIX flag on the smb_filename
struct.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Mar 19 22:55:11 UTC 2019 on sn-devel-144

smbd: Focus smb_file_link_information on correct posix'ness

In the SMB1 unix extensions, posix'ness is a property of the SMB
request, because it's tied to the connection. In the upcoming SMB3
extensions, this will become a property of the file name, it's being
set at SMB2_CREATE time. To correctly transfer posix'ness of the
filename to the destination name, we can't look at the
req->posix_pathnames here, the SMB2_SETINFO call has not set
that. This is still correct for SMB1, as the per-connection posix'ness
has spilled into the filename there too.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Mar 19 18:46:28 UTC 2019 on sn-devel-144

smbd: Enhance debugging for setting hard links

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

talloc: Fix alignment issues for casting pointers

warning: cast from 'char *' to 'struct talloc_chunk *' increases required
alignment from 1 to 8

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Tue Mar 19 12:38:50 UTC 2019 on sn-devel-144

vfs_nfs4acl_xattr: check status for NT_STATUS_ACCESS_DENIED in take ownership override

This was missing in commit 8b1b1cd8cc561378058b915e03996ff567355d81.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Mon Mar 18 20:19:30 UTC 2019 on sn-devel-144

selftest: test vfs_nfs4acl_xattr in nfs mode

The same tests as for the other modes xdr and ndr are marked as
knownfail.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs_nfs4acl_xattr: support for NFS 4.0 and 4.1 ACLs from NFS4 mount

This adds a new main switch "nfs" to "nfs4acl_xattr:encoding" which
enables to use NFS4 ACLs from an NFS4 mount on a Linux box. Tested with
a FreeBSD NFS4 server.

Supports both NFS 4.0 and 4.1 ACLs.

By default NFS4 servers send user and group identifiers in ACLs as
strings in the format "[USER|GROUP]@DNSDOMAIN". Some NFS4 servers
support sending identifiers as numeric strings. This module does support
this as well, the config knob "nfs4acl_xattr:nfs4_id_numeric = yes|no"
controls behaviour.

When "nfs4acl_xattr:encoding" is set to "nfs", the new option
"nfs4acl_xattr:validate_mode", which defauts to "yes" is set to "no" to
avoid checking and munging the mode on files.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

nfs4acl_xattr: reformat enum nfs4acl_encoding to one per line

In preperation of adding another enum value.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

nfs4acl_xattr: add RFC compliant NFS 4.0 and 4.1 ACL defintions

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

nfs4acl_xattr: move a define to a common header file

In preperation of a adding real NFS backend.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

nfs4acl_xattr: move some functions to a seperate file

These functions will be called from another translation unit in a
subsequent commit.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

nfs4acl_xattr: append 'i' to nfs4acl functions

This reflects the new struct names the functions use.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

nfs4acl_xattr: identifiers are XDR encoded as integers

The structs nfsacl41 and nfsace4 are not RFC 3530 compliant, let's
rename them adding a trailing "i" that indicates that the structs store
identifiers as integers.

  struct nfsace4  -> struct nfsace4i
  struct nfsacl41 -> struct nfsacl41i

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libsmb: Fix a "assuming signed overflow does not occur" error

Eventually we need to convert internal_resolve_name to "size_t" for
"return_count". But that's a patch for another day :-)

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

libsmb: Align integer types

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

heimdal: Fix "assuming signed overflow doesnt occur" error

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

libsmb: Use cli_smb2_query_info_fnum

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libsmb: Add cli_smb2_query_info_fnum

This wraps getting the file handle from the fnum

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libsmb: Change cli_posix_stat_send to take a pointer to sbuf

This moves the parsing work from the _recv function into the _done
function. This makes writing the SMB2 call easier later on: We can
have a smb2-specific done function doing the smb2-specific work and we
don't have to fork on protocol in the _recv function.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libsmb: Make cli_posix_[sym|hard]link proper tevent_req functions

Simplify adding SMB2 to those functions

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libsmb: Use tevent_req_simple_finish_ntstatus

Less lines of code

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Remove two unnecessary return; statements

return; is unnecessary at the end of a void function

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

torture3: Fix an error message

We tried to open "dname", not "fname"

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libsmb: Use a direct struct initializer

There's a few ways to initialize a DATA_BLOB to NULL. There's the
variable data_blob_null, there's ZERO_STRUCTP, and the explicit
initializer. And there's the data_blob() macro which calls into
data_blob_talloc_named. You have to look at that routine to see that
this is nothing more than a ZERO_STRUCT() and not calling into
talloc. Choose the more direct way.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libcli: Align integer types

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

idl: Remove a trailing empty line

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libcli: Add a comment explaining lack of overflow checks

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

idl: Fix a typo

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

rpc_server: Give mdssd its header file

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

rpc_server: Give fssd its header file

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

rpc_server: Give lsasd its header file

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

rpc_server: Give epmd its header file

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:client: Fix smbspool device uri handling

If we are executed as a CUPS backend, argv[0] is set to the device uri.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13832

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Bryan Mason <bmason@redhat.com>
Signed-off-by: Guenther Deschner <gd@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Mon Mar 18 16:47:55 UTC 2019 on sn-devel-144

s3:client: Make sure we work on a copy of the title

We can't be sure we can write to the input buffer.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13832

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Bryan Mason <bmason@redhat.com>
Signed-off-by: Guenther Deschner <gd@samba.org>

s3:client: Evaluate the AUTH_INFO_REQUIRED variable set by cups

This should not switch to username,password if cups has been configured
to use negotiate (Kerberos authentication).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13832

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Bryan Mason <bmason@redhat.com>
Signed-off-by: Guenther Deschner <gd@samba.org>

s3:client: Pass DEVICE_URI and AUTH_INFO_REQUIRED env to smbspool

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13832

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Bryan Mason <bmason@redhat.com>
Signed-off-by: Guenther Deschner <gd@samba.org>

s3:script: Fix jobid check in test_smbspool.sh

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13832

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Bryan Mason <bmason@redhat.com>
Signed-off-by: Guenther Deschner <gd@samba.org>

smbd: fix check_parent_access() talloc stackframe leaks

check_parent_access() currently leaks a number of allocations onto the
talloc_tos() context in both success and error paths.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): David Disseldorp <ddiss@samba.org>
Autobuild-Date(master): Fri Mar 15 11:32:04 UTC 2019 on sn-devel-144

ctdb-version: Simplify version string usage

There is no need to write SAMBA_VERSION_STRING as CTDB_VERSION_STRING.
Wherever required use SAMBA_VERSION_STRING directly.

Avoids the confusion with two version.h files.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13789

Signed-off-by: Amitay Isaacs <amitay@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Fri Mar 15 06:31:50 UTC 2019 on sn-devel-144

ctdb-build: Drop creation of .distversion in tarball

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13789

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-build: use a fixed ctdb_version.h using SAMBA_VERSION_STRING

This way we don't get constant rebuild as SAMBA_VERSION_STRING
is "4.7.0pre1.DEVELOPERBUILD" for the binaries under bin/
instead of "4.7.0pre1.GIT.59e51f6".

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13789

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-tests: Add a test for version consistency checking

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@samba.org>

ctdb-daemon: Add an environment variable to set version

This can be used to test the version checking logic.  Cache the
version to avoid re-checking the environment variable each time.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@samba.org>

ctdb-tests: Fix remaining common.sh ShellCheck hits

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@samba.org>

ctdb-tests: Shell cleanups in wait_until() function

This file is included by local_daemons.sh, which is not a bash script
and wait_until() uses the "local" keyword.  Prefixing variable names
with '_' to indicate that they are local changes a lot of lines in
this function.  So, fix indentation, potential quoting problems and
other ShellCheck hits while touching this function.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@samba.org>

ctdb-tests: export CTDB_SCRIPTS_TOOLS_BIN_DIR

This isn't used anywhere that requires it to be exported, but the lack
of consistency will cause problems and confusion at some later stage.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@samba.org>

ctdb-packaging: Test package requires tcpdump

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13838

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@samba.org>

ctdb-packaging: ctdb package should not own system library directory

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13838

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@samba.org>

ctdb-tests: Add some testing for IPv4-mapped IPv6 address parsing

ctdb_sock_addr values are hashed in some contexts.  This means that
all of the memory used for the ctdb_sock_addr should be consistent
regardless of how parsing is done.  The first 2 cases are just sanity
checks but the 3rd case involving an IPv4-mapped IPv6 address is the
real target of this test addition.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13839

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@samba.org>

ctdb: Initialize addr struct to zero before reparsing as IPV4

Failed to kill the tcp connection that using IPv4-mapped IPv6 address
(e.g. ctdb_killtcp eth0 ::ffff:192.168.200.44:2049
::ffff:192.168.200.45:863).

When the ctdb_killtcp is used to kill the tcp connection, the IPs and
ports in the connection will be parsed to conn.client and conn.server
(call stack: main->ctdb_sock_addr_from_string->ip_from_string). In
the ip_from_string, as we are using IPv4-mapped IPv6 addresses, the
ipv6_from_string will be used to parse ip to addr.ip6 first. The next
step the ipv4_from_string will be used to reparse ip to addr.ip.

As a result, the data that dump from conn.server is "2 0 8 1 192 168
200 44 0 0 0 0 0 0 0 0 0 0 255 255 192 168 200 44 0 0 0 0", the data
from conn.client is "2 0 3 95 192 168 200 45 0 0 0 0 0 0 0 0 0 0 255 255
192 168 200 45 0 0 0 0". The connection will be add to conn_list by
ctdb_connection_list_add. Then the reset_connections_send uses conn_list
as parameter to start to reset connections in the conn_list.

In the reset_connections_send, the database "connections" will be
created. The connections from conn_list will be written to the
database(call db_hash_add), and use the data that dump from conn_client
and conn_server as key.

In the reset_connections_capture_tcp_handler, the
ctdb_sys_read_tcp_packet will receive data on the raw socket. And
extract the IPs and ports from the tcp packet. when extracting IP and
port, the tcp4_extract OR tcp6_extract will be used. Then we got the
new conn.client and conn.server. the data that dump from the
conn.server is "2 0 8 1 192 168 200 44 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0", the data from conn.client is "2 0 3 95 192 168 200 45 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0". Finally, we use the data as key to check
if this connection is one being reset(call db_hash_delete). The
db_hash_delete will return ENOENT. Because the two key that being used
by db_hash_delete and db_hash_add are different.

So, the TCP RST will be NOT sent for the connection forever. We should
initialize addr struct to zero before reparsing as IPV4 in the
ip_from_string.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13839

Signed-off-by: Zhu Shangzhong <zhu.shangzhong@zte.com.cn>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@samba.org>

ctdb-tests: Build cluster mutex path manually

CTDB_CLUSTER_MUTEX_HELPER can't be depended on because it is only set
when the tests are not installed and setting it unconditionally for
this particular use would be wrong.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13800

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@samba.org>

dbcheck: don't check expired tombstone objects by default anymore

These will be removed anyway and any change on them risks to
be an originating update that causes replication problems.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Mar 14 03:12:27 UTC 2019 on sn-devel-144

blackbox/dbcheck-links.sh: prepare regression test for skipping expired tombstones

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

blackbox/dbcheck*.sh: pass --selftest-check-expired-tombstones to dbcheck

These tests operate on provision dumps created long ago, they still
want to run tests on deleted objects, when the next commits remove
processing expired tombstone objects in dbcheck.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

dbcheck: add --selftest-check-expired-tombstones cmdline option

This will be used by dbcheck tests which operate on static/old provision
dumps in the following commits.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/samba/netcmd: provide SUPPRESS_HELP via Option class

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

dbcheck: detect the change after deletion bug

Old versions of 'samba-tool dbcheck' could reanimate
deleted objects, when running at the same time as the
tombstone garbage collection.

When the (deleted) parent of a deleted object
(with the DISALLOW_MOVE_ON_DELETE bit in systemFlags),
is removed before the object itself, dbcheck moved
it in the LostAndFound[Config] subtree of the partition
as an originating change. That means that the object
will be in tombstone state again for 180 days on the local
DC. And other DCs fail to replicate the object as
it's already removed completely there and the replication
only gives the name and lastKnownParent attributes, because
all other attributes should already be known to the other DC.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

blackbox/dbcheck-links.sh: add regression test for lost deleted object repair

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

dbcheck: add find_repl_attid() helper function

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

dbcheck: don't remove dangling one-way links on already deleted objects

This would typically happen when the garbage collection
removed a parent object before a child object (both with
the DISALLOW_MOVE_ON_DELETE bit set in systemFlags),
while dbcheck is running at the same time as the garbage collection.
In this case the lastKnownParent attributes points a non existing
object.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

dbcheck: don't move already deleted objects to LostAndFound

This would typically happen when the garbage collection
removed a parent object before a child object (both with
the DISALLOW_MOVE_ON_DELETE bit set in systemFlags),
while dbcheck is running at the same time as the garbage collection.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

dbcheck: do isDeleted, systemFlags and replPropertyMetaData detection first

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

dbcheck: use DSDB_CONTROL_DBCHECK_FIX_LINK_DN_NAME when renaming deleted objects

We should never do originating updates on deleted objects.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

dsdb:repl_meta_data: allow CONTROL_DBCHECK_FIX_LINK_DN_NAME to by pass rename

We need a way to rename an object without updating the replication meta
data.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

blackbox/dbcheck-links.sh: reproduce lost deleted object problem

When a parent object is removed during the tombstone garbage collection
before a child object and samba-tool dbcheck runs at the same time, the
following can happen:

- If the object child had DISALLOW_MOVE_ON_DELETE in systemFlags,
  samba-tool dbcheck moves the object under the LostAndFound[Config]
  object (as an originating update!)
- The lastKnownParent attribute is removed (as an originating update!)

These originating updates cause the object to have an extended time
as tombstone. And these changes are replicated to other DCs,
which very likely already removed the object completely!

This means the destination DC of replication has no chance to handle
the object it gets from the source DC with just 2 attributes (name, lastKnownParent).

The destination logs something like:

  No objectClass found in replPropertyMetaData

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

blackbox/*.sh: pass -u to 'diff'

This is what we work with every day...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

selftest: Remove unnecessary dns_hub hashmap entries

These are only used within the function, and there's already a local
variable that stores the same info.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

selftest: Use consistent env variables for dns_hub

Setting up a testenv involves populating 2 different hashmaps - an
intermediary one (usually called 'ctx') and one that is used to populate
the testenv environment variables (usually called 'env_vars' or
'dcvars').
Because the dns_hub setup is very simple, it doesn't need two different
hashmaps. However, the variable names are still a mix of the two
hashmaps.

This patch updates dns_hub to use the second, more finalized hashmap
variable-names.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

selftest: Use new helper function for client's smb.conf IP

This has the side-effect of giving the client an IPv6 address, which it
hasn't had up until now. But it at least makes the client and server
interfaces settings consistent, and gets rid of a hard-coded IP address.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests: Make IPv4 assumption explicit

This test is asserting the expected number of *IPv4* addresses, not any
interface address (including IPv6). It works currently because the
selftest client doesn't have an IPv6 address in its smb.conf.

This patch makes the IPv4 assumption explicit by importing
interface_ips_v4() from the provision code. We need to tweak this to
pass through an 'all_interfaces' flag, otherwise it filters out the
loopback IP addresses that the testenv is using.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

selftest: Add helper function to get interfaces config

Add a helper function to return the IPv4/IPv6 addresses for the
smb.conf. This keeps the netmask assumptions in the same places as
the IP subnet assumptions.

This refactor means we no longer need to store $ctx->{interfaces}, as it
was only used in one place.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>