s3:tests: Set/return $failed in test_smbclient_auth.sh

Update the test so the shell script returns pass/fail as the exit code.

Note that subunit is just looking for 'failure:' in the test output for
whether the test passed or failed, so setting $failed isn't strictly
required.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

nssswitch: Log user access to kerberos

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Mon Feb 18 13:01:12 CET 2019 on sn-devel-144

.gitlab-ci.yml: merge .gitlab-ci-private.yml

`.gitlab-ci.yml` support conditional jobs with `only` and `except`.
And variables can be read from repo CI/CD settings as condition:

    build_samba:
      script: ...
      only:
        variables:
          - $SUPPORT_PRIVATE_TEST == 'yes'

Instead of having 2 copies of yml file, we can use this feature to
trigger private jobs only when a var like `SUPPORT_PRIVATE_TEST` is defined.

I've already added above var to our repos.
Once merged, we can remove custom CI config file in
gitlab repo settings, and remove .gitlab-ci-private.yml file from code.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Mon Feb 18 10:54:19 CET 2019 on sn-devel-144

wafsamba/symbols: change regex to match both rpath and runpath for different readelf output

In `wafsamba.dumplicate_symbols` test, it will use Popen to call:

    readelf --dynamic bin/default/source3/lib/netapi/examples/netlogon/netlogon_control2

then try to find rpath lib lines from output with regex:

    re_rpath     = re.compile(b'Library rpath: \[(.*)\]')

In ubuntu 14.04 docker image, which current CI is using, the actual output
from `readelf` is `runpath` instead of 'rpath':

    ...
    Library runpath: [/home/gitlab-runner/samba/bin/shared:/home/gitlab-runner/samba/bin/shared/private]\n'
    ...

So the regex never matched, and hide a bug.

In Ubuntu 1604 docker image, the output changes to `rpath` and matched the
regex, which expose the error in previous commit.

Improve the regex to match both `rpath` and `runpath`.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

wafsamba/symbols: always split Popen output by bytes

In py3, `wafsamba.duplicate_symbols` test may fail:

    ...
    elfpipe = subprocess.Popen(['readelf', '--dynamic', binname], stdout=subprocess.PIPE).stdout
    ...

    File "./buildtools/wafsamba/symbols.py", line 153, in get_libs
        rpath.extend(m.group(1).split(":"))
    TypeError: a bytes-like object is required, not 'str'

Because Popen will always return bytestr even in py3, and ":" is a
unicode str in py3.  Change ":" to b":" to fix.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

selftest: Initialize DC_SERVER/etc variables in one place

It's simpler to do setup DC_SERVER/etc in the same place we set
SERVER/etc. (Rather than initializing them for every single testenv,
like we were doing).

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Mon Feb 18 07:24:05 CET 2019 on sn-devel-144

selftest: Add helper function to avoid repeated {DC_SERVER}=x

For every 2-DC testenv we setup, we copy the $DC_SERVER value of the
dependent testenv (i.e. the PDC) into the env variables for the new
testenv. This means DC_SERVER always points to the PDC (or first DC).

This adds a helper function to avoid repeating this code for every 2-DC
environment we setup.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

selftest: Remove need for $RODC_DC_SERVER env variable

Same deal as earlier patch - we can use the $SERVER env variable instead
and avoid the need for this extra variable.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

selftest: Remove unused environment variables

We only really want generic environment variables. For 2+ DC
environments, we have the $SERVER and $DC_SERVER (aka PDC) variables.
However, lots of testenvs also export really specific environment
variables, e.g. VAMPIRE_2000_DC_SERVER_IP (despite that testenv being
only used for a single test case).

Previously the <testenv>_SERVER variable was used for DRS tests, but we
can avoid the need to do this now. The other variables are not used at
all.

The RODC and TRUST environment variables are still used by a few tests.
SUBDOM_DC_SERVER is only used within Samba4.pm and not exported.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests: Tweak DRS tests to avoid unnecessary env variables

Each DC used in a DRS test has its own '<testenv>_SERVER' environment
variable, e.g. VAMPIRE_DC_SERVER. These variables are only used by
test.py for DRS, but they're not actually needed.

The $SERVER environment variable holds the same information, so we can
just use this in test.py instead.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

selftest: Export DC conf path for special cases

In a few rare cases, a test needs to assert aspects both client-side and
server-side aspects. A typical example would be the audit logging, which
is exercising client-side behaviour, but also asserting the server-side
logging.

Usually this has involved a kludge in tests.py to either use
socket-wrapper explicitly, or hardcode in the server smb.conf path.

This patch exposes the existing SERVERCONFFILE env variable to the
tests. DC_SERVERCONFFILE has been added for 2 DC testenvs, where we need
the PDC's smb.conf.

The benefit of doing this way is the filepath/testenv-dependency logic
is all self-contained with the Perl code, and it doesn't bleed out into
tests.py as well.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

selftest: Add common function to return cmd environment variables

We prefix samba-tool commands with a bunch of WRAPPER/CONF environment
variables, in order for the command to work properly. These variables
are duplicated all over the place in the selftest code. This patch adds
a helper function to return the variables, so we can reduce the required
code down to a single line in a lot of places.

A couple of exceptions I've left alone:
- drs replicate, which omits the RESOLV_WRAPPER_CONF/_HOSTS variables
  (I'm not sure whether that's deliberate or not).
- create_backup(), which uses the backupfromdc's krb5.conf rather than
  the new testenv.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

third_party/nss_wrapper/wscript: check for libnsl and libsocket

this is needed as there are HAVE_LIBNSL and HAVE_LIBSOCKET in the code and
Samba fails to build in a terrible obscure way on Solaris if this is not
working inside nss_wrapper here.

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Björn Jacke <bjacke@samba.org>
Autobuild-Date(master): Sun Feb 17 15:05:20 CET 2019 on sn-devel-144

waf: add library dependency for sendfile on Solaris

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

third_party/nss_wrapper/wscript: fix check for gethostbyname

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

waf: fix setting of RPATH_ST variable

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

wafsamba: we should also remove stale symlinks here

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

waf: print the library name in which we search for a function

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

waf: use the correct WERROR_CFLAGS in CHECK_CODE

all the non gcc version were incorrectly set here till now

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

waf: remove redundant WERROR flag

CHECK_CFLAGS always uses WERROR flags

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

waf: remove duplicate WERROR cflags

WERROR flags are already added by the strict=True switch.

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

waf: fix compiler warnings in configure checks

the studio compiler issued here:

warning: statement not reached

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

waf: fix WERROR_CFLAGS check

if we found the right WERROR flags of the compiler then the compiler is right
to fail because we explicitly give it an empty file to compile. We
should not do that because that makes the almost successful test fail.
This fixed the studio compiler test.

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

waf: fix some missing newline compiler warnings

without a trailing newline the studio compiler issues:

warning: newline not last character in file

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

sysquotas_4B: raise log level of a sometimes too noisy message

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

waf/quotas: fail configure when quotas were requested but not found

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

quotas: remove legacy AIX quota code that is covered by sysquotas now

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

tests/sysquotas.c: include jfs/quota.h for 4B test on AIX

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

add sysquotas_jfs2.c

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

sysquotas_4B: enable for jfs/quota.h on AIX

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

sysquotas_4B: make quota block calculation adopt to platform quota block size

the correct QUOTABLOCK_SIZE for platform is taken from sysquotas.h

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

waf: check for jfs/quota.h

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

sysquotas.h: collect more platform quotablock sizes

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

tests/quota: tidy up includes of sysquotas 4B

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

sysquotas_nfs: also honor reported inode/file limits

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

sysquotas_xfs: fix inode limit setting, which is not depeding on blocksize

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

sysquotas_linux: fix inode limit setting, which is not depeding on blocksize

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

ntquotas: do not set inode limits when setting space quota

we are supposed to set a space quota limit, we should not calculate an
additional artifical inode limit out of that.

Signed-off-by: Bjoern Jacke <bjacke@samba.org>

replace: remove needless vxfs header file check

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

sambaundoguididx: use the right escaped oder unescaped sam ldb files

the correct filename is taken from the partition database before, we should not
unescape that because this can result in a new unescaped ldb file being created
and the script not to work at all.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13759

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

.gitignore: add ignore rules for a few dev tools

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Feb 15 06:07:07 CET 2019 on sn-devel-144

autobuild: Drop 'py2' flag

This isn't used any more. It was only being set, never referenced.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

autobuild: Replace samba-buildpy2-only with samba-nopython-py2

For Samba 4.11, the minimum python2 functionality we will support (for
now, at least - we may change our minds) is for the --disable-python
target, i.e. if you're excluding all the python functionality from
samba, then WAF should still support being built with python2.

The use case here is old unix platforms that want to use smbd, but don't
have python3 support.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

autobuild: Remove ${EXTRA_PYTHON} variable

We no longer build the python2 bindings, only python3. So we can get rid
of this variable now.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

autobuild: Tidy up unnecessary line-breaks in 'TESTS='

Now that we've dropped the {PY3_ONLY} variable, there's no need for
line-breaks in some of the 'TESTS=' values. We can tidy this up a bit.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

autobuild: Remove the PY3_ONLY variable

This variable is no longer needed as all the tests run using python3
now.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

autobuild: Update variable name to make more sense

When we switched from python2 being the default to python3, we didn't
update this variable name. It's now handling the python2 case, but it's
a boolean flag named 'py3', which is rather confusing.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

autobuild: Drop py2 autobuild jobs

Samba v4.11 will no longer support python2, so let's drop the autobuild
jobs. This will save some gitlab/sn-devel time and money, as it's less
work for CI to do.

Note that this highlights some previous inconsistencies:
- samba-none-env-py2 was being built for gitlab but not sn-devel.
- samba-nt4-py2 was being built for sn-devel but not gitlab

I've left samba-buildpy2-only for now, which will be addressed in a
subsequent patch.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

gitlab-ci: Rename DOCKER to CONTAINER

This is just a runc container, not tied to docker. See podman.io

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Björn Jacke <bj@sernet.de>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu Feb 14 17:32:38 CET 2019 on sn-devel-144

waf: Do not install internal header

We should not install header files without an public API:

- memory.h
- safe_strings.h
- talloc_stack.h

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13778

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

lib:util: Move discard_const(_p) to own header for libndr.h

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13778

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

pidl: Use NDR_ZERO_STRUCT(P) macros

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13778

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

librpc:ndr: Add NDR_ZERO_STRUCT(P) macros

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13778

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

librpc:ndr: Implement ndr_zero_memory()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13778

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

lib:util: Add GPL header to tftw.c

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13778

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

lib:util: Add missing "replace.h" header in tftw

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13778

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s4 dsdb util: samdb_client_site_name clean up

* Initialise pointers to NULL
* replace talloc_free with TALLOC_FREE
* add goto exit to ensure memory deallocated correctly

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Feb 14 06:53:14 CET 2019 on sn-devel-144

s4 dsdb util: remove samdb_search_count

All the uses have been replaced with calls to dsdb_domain_count, so it
is no longer needed.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4 dsdb util: samdb_client_site_name use dsdb_domain_count

Replace the call to samdb_search_count with dsdb_domain_count. As this
is the only remaining caller of samdb_search_count, replacing it will
allow the removal of samdb_search_count.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4 rpc_server_samr: DomGeneralInformation use dsdb_domain_count

Use dsdb_domain_count instead of samdb_search_count to determine the
number of users, groups and aliases.  This gives a performance gain of
around 10%, reduces the total memory allocated and fixes the incorrect
count returned for aliases.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4 dsdb util: add dsdb_domain_count

This counts the number of objects that are in the domain,
provided a domain SID was supplied (otherwise it just
counts all the objects).

This routine avoids allocating memory for the full
result set by using a callback.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s2 decrpc samr: Add tests for QueryDomainInfo

Add tests for the number of domain users, groups and aliases returned by
QueryDomainInfo.

These tests revealed that the existing code was not checking the
returned elements to ensure they were part of the domain.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

.gitlab-ci.yml: Make docker image name more explicit

The 'image' YAML tag implies a docker image, but for people who find
gitlab mysterious, let's make it blatantly obvious what we're doing
here.

+ added a comment
+ added 'DOCKER' to the variable names
+ removed 'BUILD', as we've now dropped this from all the job-names
+ tried to make the variable names consistent, both within the file and
WRT docker terminology

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Feb 14 03:51:21 CET 2019 on sn-devel-144

.gitlab-ci.yml: Include the actual command used by gitlab

Someone who finds gitlab mysterious will have no idea what $CI_JOB_NAME
should be, if they wanted to reproduce the autobuild job manually. It
should be trivial to include the actual command being run in the logs.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

.gitlab-ci.yml: Re-indent comments on test times

(some of these need to be updated, but for now improve the formatting)

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Tim Beale <timbeale@catalyst.net.nz>

.gitlab-ci.yml: Use .extends to avoid duplication of autobuild command

This should make our .gitlab-ci.yml file much less overwealming.

The downside is that $CI_JOB_NAME is printed rather than the job name
in the log, but the upside is that the names must now strictly match.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Tim Beale <timbeale@catalyst.net.nz>

.gitlab-ci.yml: Fix the registry as "registry.gitlab.com"

This is important as our team docker images are only in this registry, but
the .gitlab-ci.yml file is also run on other private GitLab hosts.

This partially undoes 8989916b5af6fed9c4c63035d4488583396b8c5a

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Tim Beale <timbeale@catalyst.net.nz>

.gitlab-ci.yml: add retries on runner_system_failure/stuck_or_timeout_failure

This hopefully avoids pipeline failures due too docker/runner
errors.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

.gitlab-ci*.yml: use 'extends: ' instead of YAML Anchors

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

.gitlab-ci*.yml: remove build_ prefixes

It's useless to see 'builf_samba_ad...' 7 times in the gitlab pipeline summary.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

.gitlab-ci.yml: print out more information in the before_script section

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

script/autobuild.py: add 'lsb_release -a' and 'mount' to system-info.txt

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

selftest:Samba4: wait for DNS names being registered

We can't reliable start tests without registered dns names.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

selftest:Samba4: report when samba is started and ready

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba_dnsupdate: make rodc_dns_update() more robust against timing problems

Without this we had an interesting race!

The messaging_dgm code caches connected datagram sockets based on the
destination pid for 1 second.

The fact that samba_dnsupdate constantly recreates its messaging
context (and the underlying datagram socket) means that we the winbindd
messaging context may get a stale connection. As a result sending any
message from winbindd back to samba_dnsupdate will result in
ECONNREFUSED.

That means the IRPC response from winbindd never reaches
samba_dnsupdate, which will then hit a timeout.

In turn samba_dnsupdate on the RODC times out.

This was a workaround for the problem, by having just one global
IRPC handle and thus just one messaging_dgm context.
The actual problem is solved a few commits before
("messages_dgm: Properly handle receiver re-initialization").
But we keep this as an performance optimization, which hopefully
means that the overall samba_dnsupdate is less likely to
timeout after the hardcoded 20 seconds.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba_dnsupdate: make it clear that opts.use_file is active and we're not using nsupdate

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

winbindd_irpc: remove unused winbind_DsrUpdateReadOnlyServerDnsRecords from wb_irpc_forward_state

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:setup: register ${NTDSGUID}._msdcs.${DNSFOREST} first in dns_update_list

After the A and AAAA records for the ${HOSTNAME} this is the most
important name.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

messages_dgm: Properly handle receiver re-initialization

This only properly covers the small-message nonblocking case. Covering
the large-message and the blocking case is a much larger effort assuming
we want to re-send the failed message if parts of the message has gone
through properly. Don't do that for now.

This was found by sanba_dnsupdate constantly recreating its irpc handle to
winbindd in the RODC case.

The messaging_dgm code cached connected datagram sockets based on the
destination pid for 1 second. Which means the IRPC responses from
winbindd are never delivered to samba_dnsupdate,
which will then hit a timeout.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13786

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

torture3: Extend read3 for the "messaging target re-inits" failure

Do ping_pong a hundred times, re-initializing the msg_ctx every time.

https://bugzilla.samba.org/show_bug.cgi?id=13786

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

messages_dgm: Use saved errno value

In this case this is just a cleanup, the value has just been set by
messaging_dgm_sendmsg. But as that already saves errno into a local
variable, use that.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13786

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

waf: Check for libnscd

The check was in the old autoconf, but not in waf. As the code is still
in source3/lib/util_nscd.c, add the check for libnscd to allow building
and using the code.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13787

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
Autobuild-User(master): Günther Deschner <gd@samba.org>
Autobuild-Date(master): Wed Feb 13 17:58:33 CET 2019 on sn-devel-144

Make sure results from GetAttrString are decref'ed where needed

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett abartlet@samba.org
Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Wed Feb 13 14:51:12 CET 2019 on sn-devel-144

Fix instances of PyDict_SetItem to decref the value

Although it would be better to use the BuildValue approach to
create the dictionares here, unfortunately the dictionaries created
here have key/values that are created dynamically (based on input params).

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett abartlet@samba.org

Fix mem leak with PyBytes_FromStringAndSize
Reviewed-by: Andrew Bartlett abartlet@samba.org

selftest: Only set clockskew to 5 seconds for MIT Kerberos

This was added in ac5427c6eba09134411f76a5e6f7e2643fa74eed as part of the MIT KDC
effort, but makes some tests much less reliable under high load.

As the Heimdal build does not need this, only specify for the MIT build.

Tested with an MIT AD DC build with:
 make test TESTS="samba3.raw.session samba3.smb2.session"

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Feb 13 05:49:43 CET 2019 on sn-devel-144

wafsamba/samba_utils.py: override symlink to allow force link

if bin is not empty and I have been sharing the samba tree into
a Vagrant environment and we run make, we get annoying linking error like this:

     File "~/samba/lib/tevent/wscript", line 130, in build
        installdir='python')
      File "./buildtools/wafsamba/wafsamba.py", line 745, in SAMBA_SCRIPT
        os.symlink(link_src, link_dst)
    FileExistsError: [Errno 17] File exists: '~/samba/lib/tevent/tevent.py' -> '~/samba/bin/default/../python/tevent.py'
    Makefile:7: recipe for target 'all' failed

Override the symlink method to allow force linking.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

selftest: Change backup/restore testenvs to use 1 prefork child

Recently the gitlab CI jobs were hitting memory resource limits and
using swap, which then caused test failures. The process model used in
the testenvs seemed to be contributing to this problem.

We can reduce the memory overhead of the restore/backup testenvs by
using 1 prefork child process instead of the default of 4 (kudos to
Garming for the idea). The tests run against these testenvs are basic
sanity-checks, rather than heavy-duty stress tests, so the number of
prefork workers shouldn't matter.

This is a bit of a tradeoff between testing the defaults that will
actually be used in production vs using limited resources efficiently on
shared CI runner machines.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

selftest: Use default 'prefork children' smb.conf setting

The default setting should be 4, so there should be no need to specify
this in the testenv smb.conf.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

ldb_dn: don't free a known NULL pointer

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

ldb_dn: remove unreachable code in dn_explode

Every time I look at this file, I spend a few minutes wondering how
these bits of code are ever run. Never again.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

replmd: move a if (ret) closer to ret source

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

dsdb: check NULL guid strings in la_fix_links

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

dsdb: linked attrs: check a talloc_new()

Also we can defer it past a thing that doesn't need or check for it.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

dsdb: make get_parsed_dns_trusted() a common helper function

We are already using it in two places, and are about to add a third.

The version in repl_meta_data.c did more work in the case that the
parsed_dns can't really be trusted to conform to the expected format;
this is now a wrapper called get_parsed_dns_trusted_fallback().

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

spelling of associated

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s3:utils: Add missing NULL check in rpc_fetch_domain_aliases()

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Feb 13 00:52:25 CET 2019 on sn-devel-144

s3:locking: Add missing NULL check

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

CI: split out "samba-ad-dc-ntvfs[-py2]" test targets

Many AD tests currently use the "samba" target. Split out a new target
"samba-ad-dc-ntvfs" and have all tests that use the "ad_dc_ntvfs" env
use the new target. This should greatly speed up the runtime for the "samba"
target and avoid swapping.

This reduces the total CI time by ~ 55%, I got an autobuild and a gitlab
pipeline finished in just ~ 100 mins!

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Ralph Boehme <slow@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Mon Feb 11 14:10:12 CET 2019 on sn-devel-144

dsdb/tests/vlv: use only one toplevel dn that is correctly cleaned up

Before "OU=vlvtestou2,%s" % (self.base_dn) was left behind after the
test.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>