bbaumbach/samba-autobuild/.git
16 months agonotifyd: Improve a debug message
Volker Lendecke [Mon, 26 Nov 2018 15:21:16 +0000 (16:21 +0100)]
notifyd: Improve a debug message

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Nov 27 21:42:24 CET 2018 on sn-devel-144

16 months agowinbind: Use dom_sid_str_buf
Volker Lendecke [Sat, 24 Nov 2018 12:25:25 +0000 (13:25 +0100)]
winbind: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Tue Nov 27 10:38:11 CET 2018 on sn-devel-144

16 months agolibcli: Use dom_sid_str_buf
Volker Lendecke [Sat, 24 Nov 2018 12:16:56 +0000 (13:16 +0100)]
libcli: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
16 months agolibcli: Use dom_sid_str_buf
Volker Lendecke [Sat, 24 Nov 2018 12:16:56 +0000 (13:16 +0100)]
libcli: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
16 months agoidmap_cache: Use dom_sid_str_buf
Volker Lendecke [Sat, 24 Nov 2018 12:14:23 +0000 (13:14 +0100)]
idmap_cache: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
16 months agowinbind: Fix "wbint_Principals" definition
Volker Lendecke [Fri, 23 Nov 2018 08:03:13 +0000 (09:03 +0100)]
winbind: Fix "wbint_Principals" definition

A signed integer does not make any sense for an IDL array length

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
16 months agowinbind: Use dom_sid_str_buf
Volker Lendecke [Fri, 23 Nov 2018 07:58:59 +0000 (08:58 +0100)]
winbind: Use dom_sid_str_buf

Also fix a DBG format string specifier

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
16 months agowinbind: Use dom_sid_str_buf
Volker Lendecke [Fri, 23 Nov 2018 07:55:13 +0000 (08:55 +0100)]
winbind: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
16 months agowinbind: Use dom_sid_str_buf
Volker Lendecke [Fri, 23 Nov 2018 07:53:45 +0000 (08:53 +0100)]
winbind: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
16 months agolibgpo: Align integer types
Volker Lendecke [Fri, 23 Nov 2018 07:50:47 +0000 (08:50 +0100)]
libgpo: Align integer types

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
16 months agolibgpo: Use dom_sid_str_buf
Volker Lendecke [Fri, 23 Nov 2018 07:49:44 +0000 (08:49 +0100)]
libgpo: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
16 months agolibads: Give krb5_errs.c its own header
Volker Lendecke [Fri, 23 Nov 2018 11:34:50 +0000 (12:34 +0100)]
libads: Give krb5_errs.c its own header

The protos were declared in lib/krb5_wrap but the functions are not
available there.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
16 months agovfs_shadow_copy2: in fstat also convert fsp->fsp_name and fsp->base_fsp->fsp_name
Ralph Boehme [Wed, 21 Nov 2018 16:20:30 +0000 (17:20 +0100)]
vfs_shadow_copy2: in fstat also convert fsp->fsp_name and fsp->base_fsp->fsp_name

Stacked VFS modules might use the file name, not the file
handle. Looking at you, vfs_fruit...

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13455

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
16 months agos3:smbd: pass down twrp from SMB2_CREATE to filename_convert()
Ralph Boehme [Sat, 24 Nov 2018 09:54:06 +0000 (10:54 +0100)]
s3:smbd: pass down twrp from SMB2_CREATE to filename_convert()

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13455

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
16 months agos3:smbd: add twrp args to filename_convert()
Ralph Boehme [Sat, 24 Nov 2018 09:45:49 +0000 (10:45 +0100)]
s3:smbd: add twrp args to filename_convert()

All existing callers pass NULL, no change in behaviour.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13455

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
16 months agos3:smbd: add twrp processing to filename_convert_internal()
Ralph Boehme [Sat, 24 Nov 2018 08:05:37 +0000 (09:05 +0100)]
s3:smbd: add twrp processing to filename_convert_internal()

Not used for now, existing callers pass NULL.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13455

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
16 months agos3:smbd: prepare filename_convert_internal() for twrp
Ralph Boehme [Sat, 24 Nov 2018 07:56:49 +0000 (08:56 +0100)]
s3:smbd: prepare filename_convert_internal() for twrp

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13455

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
16 months agos3:selftest: add a VSS test reading a stream
Ralph Boehme [Fri, 23 Nov 2018 13:36:56 +0000 (14:36 +0100)]
s3:selftest: add a VSS test reading a stream

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13455

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
16 months agovfs_shadow_copy2: nicely deal with attempts to open previous version for writing
Ralph Boehme [Fri, 23 Nov 2018 13:08:15 +0000 (14:08 +0100)]
vfs_shadow_copy2: nicely deal with attempts to open previous version for writing

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13688

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
16 months agovfs_shadow_copy2: add shadow_copy2_strip_snapshot_converted
Ralph Boehme [Thu, 22 Nov 2018 10:04:54 +0000 (11:04 +0100)]
vfs_shadow_copy2: add shadow_copy2_strip_snapshot_converted

Can be used by callers to determine if a path is in fact pointing at a
file in a snapshot. Will be used in the next commit.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13688

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
16 months agovfs_shadow_copy2: add _already_converted arg to shadow_copy2_strip_snapshot_internal()
Ralph Boehme [Thu, 22 Nov 2018 10:02:24 +0000 (11:02 +0100)]
vfs_shadow_copy2: add _already_converted arg to shadow_copy2_strip_snapshot_internal()

Not used for now, all existing callers pass NULL.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13688

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
16 months agos3:script/tests: add a test for VSS write behaviour
Ralph Boehme [Fri, 23 Nov 2018 09:18:44 +0000 (10:18 +0100)]
s3:script/tests: add a test for VSS write behaviour

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13688

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
16 months agos4:torture: add a test-suite for VSS
Ralph Boehme [Wed, 14 Nov 2018 12:45:11 +0000 (13:45 +0100)]
s4:torture: add a test-suite for VSS

This test will not be run from the main torture test runner in selftest,
as there we don't pass the required arguments 'twrp_file' and
'twrp_snapshot'.

The test needs a carefully prepared environment with provisioned
snapshot data, so the test will be started from a blackbox test
script. That comes next.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13688

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
16 months agovfs_error_inject: add EBADF error
Ralph Boehme [Fri, 23 Nov 2018 09:18:10 +0000 (10:18 +0100)]
vfs_error_inject: add EBADF error

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13688

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
16 months agovfs_error_inject: add pwrite
Ralph Boehme [Fri, 23 Nov 2018 09:07:29 +0000 (10:07 +0100)]
vfs_error_inject: add pwrite

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13688

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
16 months agotests: Rework backup test inheritance to make LP constraints clearer
Tim Beale [Thu, 22 Nov 2018 03:56:22 +0000 (16:56 +1300)]
tests: Rework backup test inheritance to make LP constraints clearer

The backup tests have a special constraint where we always want to use
check_output() over runcmd(). The reason is we need the samba-tool
backup/restore commands executed in a separate process. Otherwise the
global underlying LoadParm can accumulate settings from earlier test
case runs.

We can avoid someone in future inadvertently running runcmd() by
mistake, by simply changing the inheritance so we no longer inherit from
SambaToolCmdTest (so the runcmd functions are no longer present).

The comment explaining this has been moved to the top of the file.

Note that the TestCaseInTempDir inheritance was redundant.
BlackboxTestCase inherits from TestCaseInTempDir (and SambaToolCmdTest
was inheriting from BlackboxTestCase).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Tim Beale <timbeale@samba.org>
Autobuild-Date(master): Tue Nov 27 06:57:03 CET 2018 on sn-devel-144

16 months agotests: Work out DOMSID via samdb rather than environs
Tim Beale [Thu, 22 Nov 2018 20:46:38 +0000 (09:46 +1300)]
tests: Work out DOMSID via samdb rather than environs

Not all testenvs have the DOMSID set as an environment variable.
However, it's easy enough to work out from querying the samdb.

This is a slight change in that we use a source4-generated loadparm
to connect to the DB (self.lp is source3-generated, presumably for
some SMB connection dependency).

This change is so we can run the ntacls_backup tests against a DC with
SMBv1 disabled (the restoredc). Note that currently the tests fail in
the smb.SMB() connection in the setUp(), so we can't run them as part
of autobuild just yet (because we can't known-fail test errors).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agotests: Run backup tests against restoredc (SMBv1 disabled)
Tim Beale [Thu, 22 Nov 2018 01:05:01 +0000 (14:05 +1300)]
tests: Run backup tests against restoredc (SMBv1 disabled)

Running the backup tests against the restoredc highlights that the
backup online/rename commands don't work if SMBv1 is disabled. Note that
the offline commands still work because they don't rely on an SMB
connection to the server.

(Note that running the backup tests against the restoredc is probably a
good idea anyway, to prove that there's no limit to the number of times
you can restore a domain from backup, i.e. we support more than just a
one-off restore).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agopython/samba/test: PY3 port samba.tests.domain_backup
Noel Power [Mon, 5 Nov 2018 19:00:20 +0000 (19:00 +0000)]
python/samba/test: PY3 port samba.tests.domain_backup

The restoredc already runs under python3, so before we can run the
domain_backup tests against the restoredc, we need to make sure they
work under python3.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Noel Power <noel.power@suse.com>
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agotests: Handle backup command exceptions as test failures, not errors
Tim Beale [Thu, 22 Nov 2018 01:35:58 +0000 (14:35 +1300)]
tests: Handle backup command exceptions as test failures, not errors

If the backup command fails (i.e. throws an exception), we want the test
to fail. This makes it easier to mark tests as 'knownfail' (because we
can't knownfail test errors).

In theory, this should just involve updating run_cmd() to catch any
exceptions from the command and then call self.fail().

However, if the backup command fails, it can leave behind files in the
targetdir. Partly this is intentional, as these files may provide clues
to users as to why the command failed. However, in selftest, it causes
the TestCaseInTempDir._remove_tempdir() assertion to fire. Because this
assert actually gets run as part of the teardown, the assertion gets
treated as an error rather than a failure (and so we can't knownfail the
backup tests). To get around this, we remove any files in the tempdir
prior to calling self.fail().

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agotests: Rework backup test_backup_invalid_args test-case
Tim Beale [Thu, 22 Nov 2018 01:35:58 +0000 (14:35 +1300)]
tests: Rework backup test_backup_invalid_args test-case

self.create_backup() uses self.run_cmd(), which is a wrapper around
self.check_output(). Rework the code to call the underlying
check_output() function directly instead.

The reason we're doing this is we want run_cmd() to catch exceptions and
fail the test (i.e. in the next patch). However, we can't do that because
this test case relies on receiving the exceptions.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agoselftest: Designate one testenv as having SMBv1 disabled
Tim Beale [Thu, 22 Nov 2018 00:22:19 +0000 (13:22 +1300)]
selftest: Designate one testenv as having SMBv1 disabled

We recommend users disable SMBv1 to avoid potential security holes.
However, none of the AD DC testenvs have SMBv1 disabled.

This patch disables SMBv1 on an arbitrarily-chosen testenv (restoredc).

I chose restoredc as we'll want to run the backup tool tests against
this target, and it might be useful to check we can backup a DC if it's
already been restored once.

Note that SMBv2 doesn't support POSIX extensions (only SMBv1 does),
which is why we haven't just disabled SMBv1 on *all* testenvs.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agotests: Use s3 smbclient for testenv smoketests
Tim Beale [Sun, 25 Nov 2018 22:58:31 +0000 (11:58 +1300)]
tests: Use s3 smbclient for testenv smoketests

smbclient4 doesn't support SMBv2 connections, i.e. it won't work against
a DC with SMBv1 disabled. The smoke-test here is that the DC accepts
the connection, so we don't really care what SMB tool we use to connect.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agoselftest: Make testenv NTVFS usage match --use-ntvfs
Tim Beale [Mon, 26 Nov 2018 01:28:59 +0000 (14:28 +1300)]
selftest: Make testenv NTVFS usage match --use-ntvfs

Regardless of whether the testenv uses --use-ntvfs as part of its
provision options, the s4 testenvs all default to using the NTVFS file
server.

It's not particularly obvious that this is happening. The new restore
DCS (restoredc, renamedc, labdc) were all using NTVFS unintentionally.

The problem is the s4 testenvs default to using services '-s3fs +smb".
provision_ad_dc() explicitly overrides this to use s3fs again
(technically it ends up with both in its smb.conf and just uses whatever
comes last).

This patch changes the testenv setup to check for the presence of the
'--use-ntvfs' option and to set the 'server services' config option
appropriately. This way, the provision command and the smb.conf options
should always line up, with respect to NTVFS.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agoselftest: Make chgdcpass's NTVFS usage more obvious
Tim Beale [Mon, 26 Nov 2018 01:59:06 +0000 (14:59 +1300)]
selftest: Make chgdcpass's NTVFS usage more obvious

The chgdcpass testenv was not passing --use-ntvfs to the provision
command, but it was still actually using NTVFS.

The reason is the smb.conf generated by provision_raw_step1() would
always try to use the s4/NTVFS file server. Because the smb.conf already
existed, this trumped what was passed to the provision command.

This patch doesn't change the chgdcpass file server. It just makes it
more obvious that chgdcpass is using NTVFS.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agoselftest: Be explicit about which testenvs use ntvfs
Tim Beale [Mon, 26 Nov 2018 00:32:03 +0000 (13:32 +1300)]
selftest: Be explicit about which testenvs use ntvfs

If a testenv didn't specify any other provision arguments, then it
defaulted to using the NTVFS file server.

This patch makes it explicit, so we just pass through "--use-ntvfs" as
extra provision args in the cases we want.

(Whether all these testenvs really need to use NTVFS or not is another
question, but at least now it's easy to see which testenvs use it).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agolibads: Align integer types
Volker Lendecke [Fri, 23 Nov 2018 07:40:57 +0000 (08:40 +0100)]
libads: Align integer types

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Fri Nov 23 20:23:57 CET 2018 on sn-devel-144

16 months agolibads: Use dom_sid_string_buf
Volker Lendecke [Fri, 23 Nov 2018 07:39:02 +0000 (08:39 +0100)]
libads: Use dom_sid_string_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
16 months agosource4 smbd prefork: Add code comments
Tim Beale [Wed, 21 Nov 2018 21:35:01 +0000 (10:35 +1300)]
source4 smbd prefork: Add code comments

Add some comments to the prefork code explaining what's going on.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Nov 23 11:40:49 CET 2018 on sn-devel-144

16 months agoWHATSNEW: prefork restart
Gary Lockyer [Wed, 19 Sep 2018 03:25:02 +0000 (15:25 +1200)]
WHATSNEW: prefork restart

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agosamba-tool processes: display pre-fork masters and workers
Gary Lockyer [Thu, 13 Sep 2018 21:38:56 +0000 (09:38 +1200)]
samba-tool processes: display pre-fork masters and workers

Tag prefork work processes with "(worker 0)", and sort the process list
on server name to get a consistent order.

 Service:                          PID
 --------------------------------------
 cldap_server                     15588
 ...
 ldap_server                      15584
 ldap_server(worker 0)            15627
 ldap_server(worker 1)            15630
 ldap_server(worker 2)            15632
 ldap_server(worker 3)            15634
 nbt_server                       15576
 notify-daemon                    15638
 ...
 samba                                0
 ...
 wrepl_server                     15580

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agosource4 dcerpc_server: remove irpc_add_name
Gary Lockyer [Mon, 10 Sep 2018 19:38:06 +0000 (07:38 +1200)]
source4 dcerpc_server: remove irpc_add_name

Remove the irpc_add_name from dcesrv_sock_accept, as it results in two
identical names being registered for a process.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agosource4 smbd prefork: Cleanup messaging on restart
Gary Lockyer [Thu, 13 Sep 2018 21:45:38 +0000 (09:45 +1200)]
source4 smbd prefork: Cleanup messaging on restart

Clean up names registered in messaging for a terminated process.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agosource4 messaging: clean up terminated processes
Gary Lockyer [Thu, 13 Sep 2018 21:43:59 +0000 (09:43 +1200)]
source4 messaging: clean up terminated processes

Now that the smbd pre-fork process model restarts failed processes rather than
terminating, we end up with names registered to defunct processes.
This patch adds a function to clean up all the names registered to a process.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agosource4 smbd prefork: Add backoff to process restart
Gary Lockyer [Tue, 4 Sep 2018 19:31:22 +0000 (07:31 +1200)]
source4 smbd prefork: Add backoff to process restart

Add new smbd.conf variables 'prefork backoff increment' and
'prefork maximum backoff' to control the rate at which failed pre-forked
processes are restarted.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agosource4 smbd prefork: restart on non zero exit code
Gary Lockyer [Tue, 4 Sep 2018 00:12:49 +0000 (12:12 +1200)]
source4 smbd prefork: restart on non zero exit code

Restart any pre-fork master or worker process that exits with a non
zero exit code.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agosource4 smbd process: pass the fatal flag to terminate
Gary Lockyer [Mon, 3 Sep 2018 22:09:38 +0000 (10:09 +1200)]
source4 smbd process: pass the fatal flag to terminate

Pass the fatal flag supplied to task_server_terminate to the process
task_terminate method.  It will be used by the task_terminate methods to
set an appropriate exit code.  The process_prefork model will use a non
zero exit code to indicate that the process should be restarted.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agosource4 smbd prefork: Restart failed processes
Gary Lockyer [Sun, 2 Sep 2018 21:34:17 +0000 (09:34 +1200)]
source4 smbd prefork: Restart failed processes

Restart any pre-forked master or worker process that terminated with SIGABRT,
SIGBUS, SIGFPE, SIGILL or SIGSYS

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agosource4 smbd prefork: Pass restart information
Gary Lockyer [Thu, 30 Aug 2018 23:40:18 +0000 (11:40 +1200)]
source4 smbd prefork: Pass restart information

Pass information about the pre-fork master and worker processes that
will allow them to be restarted.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agosource4 smbd test: prefork process restart
Gary Lockyer [Mon, 17 Sep 2018 20:37:02 +0000 (08:37 +1200)]
source4 smbd test: prefork process restart

Add tests for the restarting of failed/terminated process, by the
pre-fork process model.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agoreplmd: remove unnecessary indent
Tim Beale [Tue, 20 Nov 2018 03:02:05 +0000 (16:02 +1300)]
replmd: remove unnecessary indent

The previous refactor now means we return early if we don't need to
re-apply isDeleted to the object. The 'else' is redundant and we can
remove it to avoid unnecessary indent.

This patch is basically just a whitespace change. It should not alter
functionality.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Nov 23 08:10:41 CET 2018 on sn-devel-144

16 months agoreplmd: Move logic into new replmd_should_apply_isDeleted() function
Tim Beale [Tue, 20 Nov 2018 02:54:31 +0000 (15:54 +1300)]
replmd: Move logic into new replmd_should_apply_isDeleted() function

It's easier to follow the logic involved here when it's split out into a
separate function.

This patch should not alter the existing logic/functionality.

Note the 'else' case is somewhat redundant, but it avoids excessive
whitespace changes to the function. It'll be tidied up in the next
patch.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agoreplmd: Avoid passing msg to replmd_process_linked_attribute()
Tim Beale [Mon, 19 Nov 2018 22:45:07 +0000 (11:45 +1300)]
replmd: Avoid passing msg to replmd_process_linked_attribute()

We can prevent anyone from inadvertently adding/removing msg->elements[]
in replmd_process_linked_attribute() by just not passing msg into the
function. Currently we only actually need the source DN and a memory
context for reallocating old_el->values.

The warning comment has been moved to a more appropriate place.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agotests: Fix flappiness in DRS tests due to RID Set changing
Tim Beale [Tue, 20 Nov 2018 04:30:37 +0000 (17:30 +1300)]
tests: Fix flappiness in DRS tests due to RID Set changing

The test_link_utdv_hwm test case in getnc_exop has started getting
slightly flappy (8 failures in the last 2 weeks). The problem is the
test case creates a new computer, which can occasionally result in a new
RID pool being allocated.

The problem can be reproduced by running the test case repeatedly (it
usually fails after ~250 times).

This patch updates the _check_ctr6() assertion to filter out the 'CN=RID
Set' object, if it happens to be present.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agotests: Add better error for DRS tests
Tim Beale [Tue, 20 Nov 2018 04:15:41 +0000 (17:15 +1300)]
tests: Add better error for DRS tests

We've got a flappy test hitting this assertion failure, but we can't
tell why it's failing intermittently (probably because we're bumping the
RID-Set, but there's no way to confirm this).

Add some extra debug info if the test assertion fails.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agolibrpc:ndr: Fix undefined behavior in ndr.c
Andreas Schneider [Thu, 22 Nov 2018 14:01:44 +0000 (15:01 +0100)]
librpc:ndr: Fix undefined behavior in ndr.c

librpc/ndr/ndr.c:1430 runtime error: left shift of 1 by 31 places cannot
be represented in type 'int'

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Gary Lockyer <gary@samba.org>
Autobuild-Date(master): Fri Nov 23 01:23:09 CET 2018 on sn-devel-144

16 months agolib:compression: Fix undefined behavior in lzxpress
Andreas Schneider [Thu, 22 Nov 2018 14:11:09 +0000 (15:11 +0100)]
lib:compression: Fix undefined behavior in lzxpress

lib/compression/lzxpress.c:228 runtime error: store to misaligned
address 0x5631d53ca9fe for type 'uint32_t', which requires 4 byte
alignment

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
16 months agolib:util: Fix undefined behavior in bitmap.c
Andreas Schneider [Thu, 22 Nov 2018 14:06:42 +0000 (15:06 +0100)]
lib:util: Fix undefined behavior in bitmap.c

lib/util/bitmap.c:77: runtime error: left shift of 1 by 31 places cannot
be represented in type 'int'

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
16 months agolib:util: Fix undefined behavior in idtree
Andreas Schneider [Thu, 22 Nov 2018 13:53:21 +0000 (14:53 +0100)]
lib:util: Fix undefined behavior in idtree

lib/util/idtree.c:84 runtime error: left shift of 1 by 31 places cannot
be represented in type 'int'

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
16 months agolib:util: Fix undefined behavior in asn1 parser
Andreas Schneider [Thu, 22 Nov 2018 13:45:20 +0000 (14:45 +0100)]
lib:util: Fix undefined behavior in asn1 parser

lib/util/asn1.c:969 runtime error: left shift of negative value -1

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
16 months agos3:lib: Fix undefined behavior in messages_dgm
Andreas Schneider [Thu, 22 Nov 2018 12:57:18 +0000 (13:57 +0100)]
s3:lib: Fix undefined behavior in messages_dgm

source3/lib/messages_dgm.c:1290:7: runtime error: variable length array
bound evaluates to non-positive value 0

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
16 months agolib:crypto: Fix undefined behavior in md4
Andreas Schneider [Thu, 22 Nov 2018 08:22:38 +0000 (09:22 +0100)]
lib:crypto: Fix undefined behavior in md4

runtime error: left shift of 145 by 24 places cannot be represented in type 'int'

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
16 months agos3:lib: Do not redefine bool and use stdbool.h
Andreas Schneider [Tue, 20 Nov 2018 09:39:28 +0000 (10:39 +0100)]
s3:lib: Do not redefine bool and use stdbool.h

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
16 months agolib:util: Always include unistd.h for setgroups
Andreas Schneider [Tue, 20 Nov 2018 09:37:01 +0000 (10:37 +0100)]
lib:util: Always include unistd.h for setgroups

This is needed to pass configure checks

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
16 months agos3:tests: Initialize pointers with NULL in netdisplay test
Andreas Schneider [Fri, 16 Nov 2018 19:02:26 +0000 (20:02 +0100)]
s3:tests: Initialize pointers with NULL in netdisplay test

Found by Undefined Sanitizer

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
16 months agos4:torture: Initialize pointers in libnetapi user test
Andreas Schneider [Fri, 16 Nov 2018 17:25:58 +0000 (18:25 +0100)]
s4:torture: Initialize pointers in libnetapi user test

Found by Undefined Sanitizer.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
16 months agos4:bind_dlz: Only copy the dn once
Andreas Schneider [Fri, 16 Nov 2018 15:07:42 +0000 (16:07 +0100)]
s4:bind_dlz: Only copy the dn once

../source4/dns_server/dlz_bind9.c:1132:4: error: 'dn' may be used uninitialized in this function [-Werror=maybe-uninitialized]
    state->log(ISC_LOG_INFO, "failed to find dnsRecord for %s",
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        ldb_dn_get_linearized(dn));
        ~~~~~~~~~~~~~~~~~~~~~~~~~~

Found by Undefined Sanitizer.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
16 months agoutils: Free host realm after processing
Swen Schillig [Wed, 21 Nov 2018 12:30:08 +0000 (13:30 +0100)]
utils: Free host realm after processing

In case of error and at the end of processing the list
of realms have to be free'd again.

Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Thu Nov 22 11:18:41 CET 2018 on sn-devel-144

16 months agoutils: Free krb5 context for net_lookup_kdc
Swen Schillig [Wed, 21 Nov 2018 12:17:38 +0000 (13:17 +0100)]
utils: Free krb5 context for net_lookup_kdc

In case of an error while processing and at the end of processing
the krb5 conext must be free'd.

Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
16 months agolibrpc: Free krb5 context on error
Swen Schillig [Wed, 21 Nov 2018 11:53:30 +0000 (12:53 +0100)]
librpc: Free krb5 context on error

If the call to krb5_cc_resolve() fails and processing is aborted,
the krb5 conext must be free'd before return.

Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
16 months agos3: Free principal if smb_krb5_principal_get_realm() fails
Swen Schillig [Wed, 21 Nov 2018 11:05:24 +0000 (12:05 +0100)]
s3: Free principal if smb_krb5_principal_get_realm() fails

If smb_krb5_principal_get_realm() fails, procesing is aborted and
resources have to be free'd. In this context free'ing the principal
was missing.

Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
16 months agoREADME.Coding: Approve DBG statements using dom_sid_str_buf
Volker Lendecke [Wed, 21 Nov 2018 14:58:21 +0000 (15:58 +0100)]
README.Coding: Approve DBG statements using dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
16 months agosmbspool: Free kerberos context on error
Swen Schillig [Wed, 21 Nov 2018 09:21:46 +0000 (10:21 +0100)]
smbspool: Free kerberos context on error

If processing cannot continue and the function returns
the kerberos context must be free'd.

Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
16 months agolibads: set proper ads_keytab_flush() return code on error
Swen Schillig [Wed, 21 Nov 2018 09:59:31 +0000 (10:59 +0100)]
libads: set proper ads_keytab_flush() return code on error

The return code was left on success when the calls to
ads_get_machine_kvno() or ads_clear_service_principal_names()
failed and the processing had to be aborted.

Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
16 months agolibsmb: Make cli_notify_send cancellable
Volker Lendecke [Fri, 16 Nov 2018 20:11:57 +0000 (21:11 +0100)]
libsmb: Make cli_notify_send cancellable

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Wed Nov 21 20:50:13 CET 2018 on sn-devel-144

16 months agolibsmb: Make cli_notify_send/recv smb2-aware
Volker Lendecke [Fri, 16 Nov 2018 12:53:01 +0000 (13:53 +0100)]
libsmb: Make cli_notify_send/recv smb2-aware

smb2-awareness only existed in the sync wrapper

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
16 months agolibsmb: Introduce cli_smb2_notify_send/recv
Volker Lendecke [Fri, 16 Nov 2018 12:47:00 +0000 (13:47 +0100)]
libsmb: Introduce cli_smb2_notify_send/recv

This only existed as a sync wrapper where the underlying function could do
send/recv already.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
16 months agolibcli: Make smb2cli_notify_send cancellable
Volker Lendecke [Fri, 16 Nov 2018 18:01:57 +0000 (19:01 +0100)]
libcli: Make smb2cli_notify_send cancellable

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
16 months agolibsmb: Make cli_trans_send cancellable
Volker Lendecke [Fri, 16 Nov 2018 20:03:45 +0000 (21:03 +0100)]
libsmb: Make cli_trans_send cancellable

Required to cancel SMB1 notify

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
16 months agolibsmb: Make cli_trans a normal subreq function
Volker Lendecke [Fri, 16 Nov 2018 20:01:19 +0000 (21:01 +0100)]
libsmb: Make cli_trans a normal subreq function

A lot of "state" variables, but still I don't think the additional complexity
is justified by saving a few lines. Make this a completely regular tevent_req
"req/subreq" function.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
16 months agologger: use color automatically for a tty
Joe Guo [Wed, 24 Oct 2018 03:35:50 +0000 (16:35 +1300)]
logger: use color automatically for a tty

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Nov 21 10:46:20 CET 2018 on sn-devel-144

16 months agosamba-tool ldapcmp: print DNs on predictable order
Douglas Bagnall [Wed, 21 Nov 2018 02:33:57 +0000 (15:33 +1300)]
samba-tool ldapcmp: print DNs on predictable order

Rather than unstable hash order. Ideally we'd do them in proper DN order.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agonetcmd/ldapcmp: make code pythonic
Joe Guo [Tue, 6 Nov 2018 09:10:55 +0000 (22:10 +1300)]
netcmd/ldapcmp: make code pythonic

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agonetcmd/ldapcmp: promote re object to global
Joe Guo [Tue, 6 Nov 2018 08:37:47 +0000 (21:37 +1300)]
netcmd/ldapcmp: promote re object to global

Then we can reuse the re obj.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agonetcmd/ldapcmp: use set instead of list to compare attrs
Joe Guo [Tue, 6 Nov 2018 05:16:34 +0000 (18:16 +1300)]
netcmd/ldapcmp: use set instead of list to compare attrs

This will simplify the logic and improve performance.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agonetcmd/ldapcmp: fix wrong way for string copy
Joe Guo [Tue, 6 Nov 2018 04:41:12 +0000 (17:41 +1300)]
netcmd/ldapcmp: fix wrong way for string copy

Two mistakes here:

- res[:-1] will copy but lost the last char
- string is immutable in python, there is no need to copy it explicitly

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agonetcmd/ldapcmp: pass --skip-missing-dn to LDAPBase
Joe Guo [Mon, 29 Oct 2018 04:45:28 +0000 (17:45 +1300)]
netcmd/ldapcmp: pass --skip-missing-dn to LDAPBase

This option has default value False, and was actually not passed down from cli
to LDAPBase. However, LDAPBase.__init__ has default value True for it.

After the change, a few tests using ldapcmp are affected.
Add --skip-missing-dn explicitly to keep the behavior consistent,
otherwise test will fail.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agonetcmd/ldapcmp: avoid modifying data while looping on dict
Joe Guo [Mon, 29 Oct 2018 04:28:56 +0000 (17:28 +1300)]
netcmd/ldapcmp: avoid modifying data while looping on dict

Just define another dict for return value, seems no need to modify
original dict.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agonetcmd/ldapcmp: rm unused global var summary
Joe Guo [Mon, 29 Oct 2018 03:44:20 +0000 (16:44 +1300)]
netcmd/ldapcmp: rm unused global var summary

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agonetcmd/ldapcmp: use set instead of list to find missing DNs
Joe Guo [Mon, 29 Oct 2018 02:00:15 +0000 (15:00 +1300)]
netcmd/ldapcmp: use set instead of list to find missing DNs

This simplify the logic and improve performance a lot.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agonetcmd/ldapcmp: avoid list comprehension in for loop
Joe Guo [Sun, 28 Oct 2018 21:16:02 +0000 (10:16 +1300)]
netcmd/ldapcmp: avoid list comprehension in for loop

The list comprehension will repeat for each item.
For large database, this make the command freeze.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agonetcmd/ldapcmp: add choices arg to --view option
Joe Guo [Sun, 28 Oct 2018 23:12:38 +0000 (12:12 +1300)]
netcmd/ldapcmp: add choices arg to --view option

So we don't need to validate ourselves.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agonetcmd/ldapcmp: add choices arg to --scope option
Joe Guo [Sun, 28 Oct 2018 22:54:57 +0000 (11:54 +1300)]
netcmd/ldapcmp: add choices arg to --scope option

So we don't need to validate ourselves.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agonetcmd/ldapcmp: rename __eq__ to diff
Joe Guo [Mon, 29 Oct 2018 01:49:28 +0000 (14:49 +1300)]
netcmd/ldapcmp: rename __eq__ to diff

This method actually changed both objects and print info.
__eq__ is not a proper name and is not designed for this case.
Rename to diff.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agonetcmd/ldapcmp: fix typo for Bundle
Joe Guo [Sun, 28 Oct 2018 23:29:58 +0000 (12:29 +1300)]
netcmd/ldapcmp: fix typo for Bundle

Bundel -> Bundle

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agoreplmd: Make replmd_process_linked_attribute() mem dependencies clearer
Tim Beale [Mon, 19 Nov 2018 21:59:40 +0000 (10:59 +1300)]
replmd: Make replmd_process_linked_attribute() mem dependencies clearer

This patch should not alter functionality - it is just making memory
assumptions used in replmd_process_linked_attribute() clearer.

When adding/removing msg->elements we have to take care, as this will
invalidate things like the parsed-DN array or old ldb_message_element
pointers. This has always been the case (i.e. f6bc4c08b19f5615a49),
however, now we need to take even more care, as the msg being modified
is re-used and split across 2 different functions.

Add more code comments to highlight this. We can also free
pdn_list/old_el to prevent them being incorrectly used after realloc.
It seems appropriate to also add a sanity-check that the tmp_ctx alloc
succeeds (which all the other memory hangs off).

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Tim Beale <timbeale@samba.org>
Autobuild-Date(master): Wed Nov 21 05:31:10 CET 2018 on sn-devel-144

16 months agoreplmd: Avoid redundant dsdb_get_deleted_objects_dn() checks
Tim Beale [Mon, 12 Nov 2018 02:49:28 +0000 (15:49 +1300)]
replmd: Avoid redundant dsdb_get_deleted_objects_dn() checks

Quite a bit of time was spent in dsdb_get_deleted_objects_dn()
processing during either a join (~9%) or a full-sync (~28%).

The problem is we're *always* doing the dsdb_get_deleted_objects_dn()
call for each object, regardless of whether it's actually deleted or
not. i.e. we were doing an expensive query and a lot of the time just
ignoring the query result.

If it's not a deleted object we're dealing with, we can just return
early and skip the unnecessary processing.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agoreplmd: Minimize get_parsed_dns_trusted() calls during replication
Tim Beale [Sun, 11 Nov 2018 23:11:38 +0000 (12:11 +1300)]
replmd: Minimize get_parsed_dns_trusted() calls during replication

When a group has 10,000+ links, get_parsed_dns_trusted() can be costly
(simply the talloc calls alone are expensive). Instead of re-generating
the pdn_list for every single link attribute, we can change to only
re-generate it when we really need to.

When we add a new link, it reallocates old_el->values, and so we need to
recreate the pdn_list because all the memory pointers will have changed.
However, in the other cases, where we're simply updating the existing
link value (or ignoring the update, if it's already applied), we can
continue using the same pdn_list (rather than re-parsing it again).

This would generally only save time with a full-sync - it won't really
help with the join case (because every link processed results in a
realloc).

On a DB with 5000 users, this makes a full-sync about ~13% faster.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agoreplmd: Pass old_el into replmd_process_linked_attribute()
Tim Beale [Sun, 11 Nov 2018 23:00:47 +0000 (12:00 +1300)]
replmd: Pass old_el into replmd_process_linked_attribute()

We should only need to lookup the msg attribute once per source object.
The old_el->values may change due to link-processing, but old_el itself
should not.

This is not aimed at improving performance, but we need to change how
old_el is used before we can change pdn_list (which is more costly
processing-wise).

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
16 months agoreplmd: Remove some redundant code
Tim Beale [Sun, 11 Nov 2018 22:21:36 +0000 (11:21 +1300)]
replmd: Remove some redundant code

At first glance, this code seemed completely unnecessary. However, it
was added (by commit f6bc4c08b19f5615) for a valid reason: adding the
whenChanged/uSNChanged attributes to the message can cause msg->elements
to be reallocated, which means the old_el pointer (which points to
msg->elements memory) can be out of date.

whenChanged/uSNChanged now get added to the msg last, just before the DB
modify operation. So old_el can no longer become out of date within
replmd_process_link_attribute(), so re-fetching it is now redundant.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>