vlendec/samba-autobuild/.git
2 years agoctdb-common: Move PID file creation to sock_daemon_run_send()
Amitay Isaacs [Mon, 28 Aug 2017 08:39:40 +0000 (18:39 +1000)]
ctdb-common: Move PID file creation to sock_daemon_run_send()

Only create PID file when actually starting the daemon, rather than
when setting up the context.  This will facilitate future changes.

Tweak test to confirm that PID file is no longer created during setup.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2 years agoctdb-tests: Add improved PID file check to test2
Martin Schwenke [Mon, 28 Aug 2017 06:00:53 +0000 (16:00 +1000)]
ctdb-tests: Add improved PID file check to test2

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2 years agoctdb-tests: Add comments describing sock daemon tests
Martin Schwenke [Wed, 16 Aug 2017 07:37:16 +0000 (17:37 +1000)]
ctdb-tests: Add comments describing sock daemon tests

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2 years agoctdb-daemon: Use become_daemon() instead of custom code
Martin Schwenke [Tue, 15 Aug 2017 02:53:02 +0000 (12:53 +1000)]
ctdb-daemon: Use become_daemon() instead of custom code

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2 years agoctdb-daemon: Narrow a #include
Martin Schwenke [Tue, 15 Aug 2017 02:51:59 +0000 (12:51 +1000)]
ctdb-daemon: Narrow a #include

Only time.h is needed here, not all of samba_util.h.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2 years agoldb_tdb: Rework ltdb_modify_internal() to use ltdb_search_dn1() internally
Andrew Bartlett [Wed, 16 Aug 2017 00:51:09 +0000 (12:51 +1200)]
ldb_tdb: Rework ltdb_modify_internal() to use ltdb_search_dn1() internally

This avoids duplicate code and allows us to use the allocation-avoiding
LDB_UNPACK_DATA_FLAG_NO_DATA_ALLOC flag.

We can not use LDB_UNPACK_DATA_FLAG_NO_VALUES_ALLOC as el2->values
is talloc_realloc()ed in the routine.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Aug 29 11:13:50 CEST 2017 on sn-devel-144

2 years agoldb: Add LDB_UNPACK_DATA_FLAG_NO_ATTRS
Andrew Bartlett [Thu, 10 Aug 2017 23:31:05 +0000 (11:31 +1200)]
ldb: Add LDB_UNPACK_DATA_FLAG_NO_ATTRS

This will allow us to avoid a full unpack in situations where we just want to confirm
if the DN exists

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2 years agoselftest: Use a unique(ish) OU for every run of getnc_unpriv
Tim Beale [Thu, 17 Aug 2017 00:30:30 +0000 (12:30 +1200)]
selftest: Use a unique(ish) OU for every run of getnc_unpriv

An intermittent problem I noticed with tests in the past is that the
setup can fail to create the base OU because it already exists.
I believe this is because the previous testenv DC has replicated out the
test object, but not its deletion at the point that the next testenv DC
starts running the test.

This only seemed to happen very occassionally (I haven't seen it
happen with getnc_unpriv yet, but I also haven't run it through the
autobuild yet).

Using same randomness in the test OU should help avoid this sort of
problem, and it matches what some other replication tests do.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2 years agos4-drsuapi/selftest: Add extra tests for invalid DNs
Tim Beale [Wed, 16 Aug 2017 23:36:24 +0000 (11:36 +1200)]
s4-drsuapi/selftest: Add extra tests for invalid DNs

Add some test cases to check for requests for invalid/non-existent DNs.
This exercises the first return case added in commit:
  s4-drsuapi: Refuse to replicate an NC is that not actually an NC

I've also updated the error code returned here to match Windows.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2 years agoselftest: Update getnc_unpriv tests to pass against Samba
Tim Beale [Wed, 16 Aug 2017 03:00:31 +0000 (15:00 +1200)]
selftest: Update getnc_unpriv tests to pass against Samba

In general Windows seems to return BAD_DN rather than ACCESS_DENIED for
an unprivileged user. In the the long-term, it's unrealistic to think
that Samba and Windows will agree exactly on every error code returned.
So for the tests to be maintainable and pass against Windows and Samba,
they need to handle differences in expected errors. To get around this
problem, I've changed the expected_error to be a set, so that multiple
error codes (one for Microsoft, one for Samba) can be specified for each
test case. This approach also highlights the cases where Microsoft and
Samba currently differ.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2 years agos4-drsuapi: Set getnc_state *after* we've checked request is valid
Tim Beale [Wed, 16 Aug 2017 04:20:37 +0000 (16:20 +1200)]
s4-drsuapi: Set getnc_state *after* we've checked request is valid

We were creating the getnc_state (and storing it on the connection)
before we had done some basic checks that the request was valid. If the
request was not valid and we returned early with an error, then the
partially-initialized getnc_state was left hanging on the connection.
The next request that got sent on the connection would try to use this,
rather than creating a new getnc_state from scratch.

The main side-effect of this was if you sent an invalid GetNCChanges
request twice, then it could be rejected the first time and accepted the
second time.

Note that although an invalid request was accepted, it would typically
not return any objects, so it would not actually leak any secure
information.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2 years agoselftest: GetNCChanges can 'accept' a repeated bad request
Tim Beale [Mon, 14 Aug 2017 03:31:08 +0000 (15:31 +1200)]
selftest: GetNCChanges can 'accept' a repeated bad request

In theory, if we send the exact same rejected request again, we should
get the same response back from the DC. However, we don't - the request
is accepted if we send it a second time.

This patch updates the repl_rodc test to demonstrate the problem (which
now causes the test to fail).

Note that although the bad GetNCChanges request is not rejected outright,
the response that gets sent back is empty - it has no objects in it, so
it's not an actual security hole. It is annoying problem for writing
self-tests though.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2 years agos4-drsuapi: Change REPL_SECRET error code to match Windows
Tim Beale [Wed, 16 Aug 2017 03:09:55 +0000 (15:09 +1200)]
s4-drsuapi: Change REPL_SECRET error code to match Windows

The existing SOURCE_DISABLED error code doesn't seem to make a lot of
sense. Window sends back an ACCESS_DENIED error in the same situation,
which seems more appropriate.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2 years agoselftest: Extend further getnc_unpriv tests to pass against windows 2012R2
Tim Beale [Wed, 16 Aug 2017 04:57:15 +0000 (16:57 +1200)]
selftest: Extend further getnc_unpriv tests to pass against windows 2012R2

An important change in this patch is changing the ACE type from
 A (Allow)
to
 AO (Object Allow)

as that will then respect the supplied GUID, which we also make use
the constant from the security.idl.

This reworks the tests to check replication with users with the
following rights:
- only GET_CHANGES
- only GET_ALL_CHANGES
- both GET_CHANGES and GET_ALL_CHANGES
- no rights

We basically want to test various different GetNCChanges requests
against each type of user rights, and the only difference is the
error/success value we get back. I've structured the tests this way, so
that we have 4 test_repl_xyz_userpriv() functions (to cover each of the
above user rights cases), and each test sends the same series of
GetNCChanges requests of varying validity.

Currently all these tests fail against Samba because Samba sends
different error codes to Windows.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2 years agoselftest: Confirm privileged replication of an OU is not permitted
Andrew Bartlett [Tue, 8 Aug 2017 04:52:04 +0000 (16:52 +1200)]
selftest: Confirm privileged replication of an OU is not permitted

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2 years agoselftest: Move get_partial_attribute_set() to DrsBaseTestCase
Andrew Bartlett [Tue, 8 Aug 2017 04:50:11 +0000 (16:50 +1200)]
selftest: Move get_partial_attribute_set() to DrsBaseTestCase

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2 years agoselftest: encrypt the LDAP connection in drs_base.py
Andrew Bartlett [Tue, 8 Aug 2017 04:49:24 +0000 (16:49 +1200)]
selftest: encrypt the LDAP connection in drs_base.py

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2 years agos4-drsuapi: Refuse to replicate an NC is that not actually an NC
Andrew Bartlett [Tue, 8 Aug 2017 02:34:14 +0000 (14:34 +1200)]
s4-drsuapi: Refuse to replicate an NC is that not actually an NC

This prevents replication of an OU, you must replicate a whole NC per Windows 2012R2

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2 years agoselftest: Make dirsync test use symobolic name and OA not A
Andrew Bartlett [Wed, 9 Aug 2017 01:56:07 +0000 (13:56 +1200)]
selftest: Make dirsync test use symobolic name and OA not A

A is for Allow, OA is for Object Allow, which means check the GUID.

The previous ACE allowed all access, which was not the intention.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2 years agodsdb: Use samba.generate_random_password() in dirsync test
Andrew Bartlett [Fri, 4 Aug 2017 03:21:31 +0000 (15:21 +1200)]
dsdb: Use samba.generate_random_password() in dirsync test

We do not like fixed passwords

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12946

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2 years agos4-drsuapi: Use sam_ctx consistently in dcesrv_drsuapi_DsGetNCChanges()
Tim Beale [Sun, 13 Aug 2017 23:02:05 +0000 (11:02 +1200)]
s4-drsuapi: Use sam_ctx consistently in dcesrv_drsuapi_DsGetNCChanges()

Trying to use bstate->sam_ctx_system by mistake can cause crashes if
non-admin users replicate. To avoid this problem we use the sam_ctx
variable, however it wasn't used consistently everywhere. Replace the
remaining references to b_state->sam_ctx to avoid potential confusion.

This change was made based on review feedback from Metze.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2 years agos4-drsuapi: Avoid segfault when replicating as a non-admin with GUID_DRS_GET_CHANGES
Andrew Bartlett [Thu, 3 Aug 2017 23:44:19 +0000 (11:44 +1200)]
s4-drsuapi: Avoid segfault when replicating as a non-admin with GUID_DRS_GET_CHANGES

Users who are not administrator do not get b_state->sam_ctx_system filled in.

We should probably use the 'sam_ctx' variable in all cases (instead of
b_state->sam_ctx*), but I'll make this change in a separate patch, so
that the bug fix remains independent from other tidy-ups.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12946

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2 years agotests/fake_snap: sanitize paths
David Disseldorp via samba-technical [Sat, 26 Aug 2017 21:40:37 +0000 (23:40 +0200)]
tests/fake_snap: sanitize paths

Ensure fake_snap.pl can be run in taint mode (-T), by sanitizing paths
and the PATH env. This fixes the following samba3.rpc.fsrvp selftest
failures:
Insecure dependency in mkdir while running setgid at (eval 2) line 4.
snap create failed: NT_STATUS_UNSUCCESSFUL
snap create failed for shadow copy of /home/ddiss/isms/samba/st/nt4_dc/share

Bug: https://bugzilla.samba.org/show_bug.cgi?id=12988

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Aug 29 04:54:51 CEST 2017 on sn-devel-144

2 years agos3/mdssvc: missing assignment in sl_pack_float
Ralph Boehme [Sun, 27 Aug 2017 17:22:38 +0000 (19:22 +0200)]
s3/mdssvc: missing assignment in sl_pack_float

Spotted by -Werror=maybe-uninitialized:

../source3/rpc_server/mdssvc/marshalling.c: In function ‘sl_pack_float’:
../source3/rpc_server/mdssvc/marshalling.c:171:11: error:
‘ieee_fp_union.w’ may be used uninitialized in this function
[-Werror=maybe-uninitialized]
  offset = sl_push_uint64_val(buf, offset, bufsize, ieee_fp_union.w);

Bug: https://bugzilla.samba.org/show_bug.cgi?id=12991

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2 years agoHEIMDAL: don't bother seeing q if not sent
Love Hornquist Astrand [Mon, 29 Apr 2013 18:42:46 +0000 (11:42 -0700)]
HEIMDAL: don't bother seeing q if not sent

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12986

Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
(cherry picked from heimdal commit 19f9fdbcea11013cf13ac72c416f161ee55dee2b)

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Mon Aug 28 15:10:54 CEST 2017 on sn-devel-144

2 years agoHEIMDAL: allow optional q in DH DomainParameters
Love Hornquist Astrand [Mon, 29 Apr 2013 18:37:39 +0000 (11:37 -0700)]
HEIMDAL: allow optional q in DH DomainParameters

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12986

Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
(cherry picked from heimdal commit e8317b955f5a390c4f296871ba6987ad05478c95)

2 years agos4/torture: add a test for rename change notification with inotify enabled
Ralph Boehme [Thu, 13 Jul 2017 14:05:49 +0000 (16:05 +0200)]
s4/torture: add a test for rename change notification with inotify enabled

This is already fixed in master by
5eccc2fd0072409f166c63e6876266f926411423~10..5eccc2fd0072409f166c63e6876266f926411423.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=12903

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Aug 26 05:05:08 CEST 2017 on sn-devel-144

2 years agoselftest: run smb2.notify-inotify testsuite against fileserver
Ralph Boehme [Thu, 13 Jul 2017 14:04:50 +0000 (16:04 +0200)]
selftest: run smb2.notify-inotify testsuite against fileserver

Next commit adds the suite and a test.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=12903

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2 years agoselftest: enable kernel change notifications in the fileserver environment
Ralph Boehme [Thu, 13 Jul 2017 14:01:53 +0000 (16:01 +0200)]
selftest: enable kernel change notifications in the fileserver environment

This environment is currently not used for any test in the smb2
testsuite, so this change doesn't affect any existing test.

A subsequent commit will add a test for change notifications with kernel
change notify enabled. It verifies a bug (this one) that only crops up
in such a setup and causes rename events to get lost.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=12903

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2 years agos3:vfs: Do not overrun array ad->ad_eid
Andreas Schneider [Fri, 25 Aug 2017 12:45:29 +0000 (14:45 +0200)]
s3:vfs: Do not overrun array ad->ad_eid

The array is defined as:
    struct ad_entry ad_eid[ADEID_MAX]

Found by Coverity.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Fri Aug 25 20:05:32 CEST 2017 on sn-devel-144

2 years agos4:torture: Do not overrun arrays in test_displayshares()
Andreas Schneider [Fri, 25 Aug 2017 12:24:59 +0000 (14:24 +0200)]
s4:torture: Do not overrun arrays in test_displayshares()

If we do not 'break', we overrun the array access size.

Found by Coverity.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2 years agos4:torture: Fix talloc_array in test_EnumValue()
Andreas Schneider [Fri, 25 Aug 2017 12:17:54 +0000 (14:17 +0200)]
s4:torture: Fix talloc_array in test_EnumValue()

Found by Coverity.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2 years agos3:modules: Avoid setting the sign bit to 1.
Andreas Schneider [Fri, 25 Aug 2017 12:11:02 +0000 (14:11 +0200)]
s3:modules: Avoid setting the sign bit to 1.

Found by Coverity.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2 years agoctdb-client: Fix ctdb_attach() to use database flags
Amitay Isaacs [Fri, 18 Aug 2017 04:00:47 +0000 (14:00 +1000)]
ctdb-client: Fix ctdb_attach() to use database flags

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12978

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Fri Aug 25 13:32:58 CEST 2017 on sn-devel-144

2 years agoctdb-client: Optionally return database id from ctdb_ctrl_createdb()
Amitay Isaacs [Wed, 23 Aug 2017 02:09:22 +0000 (12:09 +1000)]
ctdb-client: Optionally return database id from ctdb_ctrl_createdb()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12978

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2 years agoctdb-client: Fix ctdb_ctrl_createdb() to use database flags
Amitay Isaacs [Fri, 18 Aug 2017 03:50:39 +0000 (13:50 +1000)]
ctdb-client: Fix ctdb_ctrl_createdb() to use database flags

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12978

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2 years agoctdb-tests: Add a test to check databases are attached with correct flags
Amitay Isaacs [Fri, 18 Aug 2017 04:27:10 +0000 (14:27 +1000)]
ctdb-tests: Add a test to check databases are attached with correct flags

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12978

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2 years agoctdb-tests: Add functions to start/stop/restart a single local daemon
Amitay Isaacs [Fri, 18 Aug 2017 04:45:30 +0000 (14:45 +1000)]
ctdb-tests: Add functions to start/stop/restart a single local daemon

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12978

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2 years agoctdb-tests: Add functions to start/stop/restart ctdb on single node
Amitay Isaacs [Tue, 22 Aug 2017 02:53:43 +0000 (12:53 +1000)]
ctdb-tests: Add functions to start/stop/restart ctdb on single node

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12978

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2 years agoschannel.idl: Fix a typo
Volker Lendecke [Thu, 24 Aug 2017 15:03:30 +0000 (17:03 +0200)]
schannel.idl: Fix a typo

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Aug 25 04:10:25 CEST 2017 on sn-devel-144

2 years agoctdb: Fix a typo
Volker Lendecke [Wed, 23 Aug 2017 16:11:32 +0000 (18:11 +0200)]
ctdb: Fix a typo

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2 years agotdb: version 1.3.15 tdb-1.3.15
Stefan Metzmacher [Thu, 24 Aug 2017 11:29:17 +0000 (13:29 +0200)]
tdb: version 1.3.15

* Add protection against EINTR.
* Truncate the file after expand failure, ENOSPC
* Use posix_fallocate() to expand the file
* Fix GCC compiler warnings

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Thu Aug 24 21:17:48 CEST 2017 on sn-devel-144

2 years agos3:libsmb: Print the kinit failed message with DBGLVL_NOTICE
Andreas Schneider [Thu, 24 Aug 2017 10:51:35 +0000 (12:51 +0200)]
s3:libsmb: Print the kinit failed message with DBGLVL_NOTICE

The default debug level of smbclient is set to 'log level = 1'. So we
need to use at least NOTICE to not get the message when we do not force
kerberos.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12704

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu Aug 24 17:22:18 CEST 2017 on sn-devel-144

2 years agos4:torture: The teardown function should just return
Andreas Schneider [Tue, 8 Aug 2017 10:05:24 +0000 (12:05 +0200)]
s4:torture: The teardown function should just return

The teardown functions should not return on error but finish cleaning
up!

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12984

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu Aug 24 13:23:22 CEST 2017 on sn-devel-144

2 years agos4:torture: Delete printer before we remove the driver
Andreas Schneider [Tue, 8 Aug 2017 08:40:19 +0000 (10:40 +0200)]
s4:torture: Delete printer before we remove the driver

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12984

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
2 years agos4:torture: Use a different driver name for add_driver tests
Andreas Schneider [Tue, 8 Aug 2017 09:25:48 +0000 (11:25 +0200)]
s4:torture: Use a different driver name for add_driver tests

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12984

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
2 years agos3:script: Untaint user supplied data in modprinter.pl
Andreas Schneider [Tue, 8 Aug 2017 06:40:34 +0000 (08:40 +0200)]
s3:script: Untaint user supplied data in modprinter.pl

spoolss_SetPrinter fails because of the error produced by modprinter.pl.

Perl error:
Insecure dependency in open while running setgid at modprinter.pl line 76.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12950

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
2 years agos3:spoolss: Set timeout values to the one which Windows uses by default
Andreas Schneider [Fri, 18 Nov 2016 14:06:22 +0000 (15:06 +0100)]
s3:spoolss: Set timeout values to the one which Windows uses by default

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
2 years agodynconfig: Use INSTALL_DIR to create directories
Andreas Schneider [Thu, 10 Aug 2017 09:42:46 +0000 (11:42 +0200)]
dynconfig: Use INSTALL_DIR to create directories

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12957

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Aug 24 09:29:05 CEST 2017 on sn-devel-144

2 years agowafsamba: Call INSTALL_DIR in INSTALL_DIRS
Andreas Schneider [Thu, 10 Aug 2017 09:40:06 +0000 (11:40 +0200)]
wafsamba: Call INSTALL_DIR in INSTALL_DIRS

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12957

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2 years agowafsamba: Add INSTALL_DIR function
Andreas Schneider [Thu, 10 Aug 2017 09:36:52 +0000 (11:36 +0200)]
wafsamba: Add INSTALL_DIR function

The install_dir function in waf has been deprecated and it doesn't
support setting directory permissions. So we need to implement our own
function anyway.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12957

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2 years agotdb: Use posix_fallocate
Volker Lendecke [Wed, 23 Aug 2017 11:02:57 +0000 (13:02 +0200)]
tdb: Use posix_fallocate

This should be significantly faster than pwriting.

openbsd doesn't have posix_fallocate, so we do need the fallback. Also, it
might have weird failure modes, so we keep the old code in place except for
posix_fallocate returning success or ENOSPC.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Aug 24 05:38:49 CEST 2017 on sn-devel-144

2 years agotdb: Add an intermediate variable
Volker Lendecke [Wed, 23 Aug 2017 10:59:19 +0000 (12:59 +0200)]
tdb: Add an intermediate variable

More README.Coding, but I need "ret" in the next commit as well :-)

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2 years agotdb: Truncate the file after expand failure
Volker Lendecke [Wed, 23 Aug 2017 10:48:03 +0000 (12:48 +0200)]
tdb: Truncate the file after expand failure

Without this it's very easy to create virtually huge files: ftruncate expands a
file, the pwrites fail with ENOSPC, thus the write fails. The next writer runs
into the same situation, and ftruncate-expands the file even further. tdb_check
will then spend ages reading the 4GB of zeros byte by byte.

Here we hold the freelist lock or are inside a transaction, so it is safe to
cut the file again. Nobody can have used the space that we have tried to
allocate, so we can't have any stray pointers corrupting the database.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2 years agotdb: Protect against EINTR
Volker Lendecke [Wed, 23 Aug 2017 10:00:00 +0000 (12:00 +0200)]
tdb: Protect against EINTR

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2 years agoconfigure: Centralize check for posix_fallocate
Volker Lendecke [Tue, 22 Aug 2017 15:09:01 +0000 (17:09 +0200)]
configure: Centralize check for posix_fallocate

This checks for posix_fallocate unless we are sitting on an ancient glibc.
With this we don't need HAVE_BROKEN_POSIX_FALLOCATE anymore,
HAVE_POSIX_FALLOCATE will only be defined if we have a valid [g]libc.

./configure tested on Debian, FreeBSD (which does have posix_fallocate) and
OpenBSD (which does not have posix_fallocate). Also tested with changing the
not have an old-enough glibc around. All did the right thing.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2 years agotdb: Fix a typo
Volker Lendecke [Tue, 22 Aug 2017 12:51:18 +0000 (14:51 +0200)]
tdb: Fix a typo

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2 years agovfs_default: Fix passing of errno from async calls
Christof Schmitt [Wed, 23 Aug 2017 21:37:28 +0000 (14:37 -0700)]
vfs_default: Fix passing of errno from async calls

Current code assigns errno from async pthreadpool calls to the
vfs_default internal vfswrap_*_state.  The callers of the vfs_*_recv
functions expect the value from errno in vfs_aio_state.error.

Correctly assign errno to vfs_aio_state.error and remove the unused
internal err variable.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12983

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2 years agoacl_common: Avoid "#include vfs_acl_common.c"
Volker Lendecke [Fri, 18 Aug 2017 12:41:57 +0000 (14:41 +0200)]
acl_common: Avoid "#include vfs_acl_common.c"

This makes vfs_acl_common.c a subsystem of its own that acl_xattr and acl_tdb
now link against, not #include it.

This patch is a bit on the large and clumsy side, but splitting it up would
(I believe) involve a separate intermediate copy of acl_common.c.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2 years agoacl_common: Pass store_acl_blob_fsp through store_v3_blob
Volker Lendecke [Fri, 18 Aug 2017 11:52:31 +0000 (13:52 +0200)]
acl_common: Pass store_acl_blob_fsp through store_v3_blob

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2 years agoacl_common: Pass get_acl_blob_fn as a pointer to get_nt_acl_internal
Volker Lendecke [Fri, 18 Aug 2017 11:46:14 +0000 (13:46 +0200)]
acl_common: Pass get_acl_blob_fn as a pointer to get_nt_acl_internal

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2 years agos3:tests: Add test for changing the local user password with smbpasswd
Andreas Schneider [Mon, 21 Aug 2017 10:23:56 +0000 (12:23 +0200)]
s3:tests: Add test for changing the local user password with smbpasswd

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12975

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Wed Aug 23 17:05:48 CEST 2017 on sn-devel-144

2 years agos3:utils: Remove pointless if-clause for remote_machine
Andreas Schneider [Tue, 22 Aug 2017 13:46:07 +0000 (15:46 +0200)]
s3:utils: Remove pointless if-clause for remote_machine

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12975

Review with: git show -U20

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
2 years agos3:utils: Make sure we authenticate against our SAM name in smbpasswd
Andreas Schneider [Fri, 18 Aug 2017 14:17:08 +0000 (16:17 +0200)]
s3:utils: Make sure we authenticate against our SAM name in smbpasswd

If a local user wants to change his password using smbpasswd and the
machine is a domain member, we need to make sure we authenticate against
our SAM and not ask winbind.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12975

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
2 years agos3:utils: Pass domain to password_change() in smbpasswd
Andreas Schneider [Fri, 18 Aug 2017 14:14:57 +0000 (16:14 +0200)]
s3:utils: Pass domain to password_change() in smbpasswd

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12975

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
2 years agos3:utils: Make strings const passed to password_change() in smbpasswd
Andreas Schneider [Fri, 18 Aug 2017 14:13:15 +0000 (16:13 +0200)]
s3:utils: Make strings const passed to password_change() in smbpasswd

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12975

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
2 years agos3:libsmb: Move prototye of remote_password_change()
Andreas Schneider [Fri, 18 Aug 2017 14:10:06 +0000 (16:10 +0200)]
s3:libsmb: Move prototye of remote_password_change()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12975

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
2 years agos3:libsmb: Pass domain to remote_password_change()
Andreas Schneider [Fri, 18 Aug 2017 14:08:46 +0000 (16:08 +0200)]
s3:libsmb: Pass domain to remote_password_change()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12975

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
2 years agos3:utils: Do not report an invalid range for AD DC role
Andreas Schneider [Fri, 18 Aug 2017 08:35:55 +0000 (10:35 +0200)]
s3:utils: Do not report an invalid range for AD DC role

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12629

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Aug 23 03:23:55 CEST 2017 on sn-devel-144

2 years agotdb: fix tbdtool list freelist output
Ralph Boehme [Sun, 9 Jul 2017 16:39:27 +0000 (18:39 +0200)]
tdb: fix tbdtool list freelist output

Due to the non-fixable bug in the BUCKET macro tdbtool list printed some
other hash chainlist, not the freelist.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=12888

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2 years agotdb: document hashtable bucket offset calculation madness
Ralph Boehme [Sun, 2 Jul 2017 09:41:43 +0000 (11:41 +0200)]
tdb: document hashtable bucket offset calculation madness

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2 years agotbd: BUCKET(-1) returns wrong offset because tdb->hash_size is an unsigned int
Ralph Boehme [Sat, 1 Jul 2017 20:21:26 +0000 (22:21 +0200)]
tbd: BUCKET(-1) returns wrong offset because tdb->hash_size is an unsigned int

The following C program demonstrates the issue:

  #include <stdio.h>
  #include <stdlib.h>
  #include <stdarg.h>
  #include <stdbool.h>
  #include <string.h>
  #include <unistd.h>
  #include <errno.h>
  #include <dirent.h>
  #include <sys/types.h>

  int main(int argc, char **argv)
  {
      int hash = -1;
      int tsize_signed = 10;
      unsigned int tsize_unsigned = 10;
      int bucket;

  #define BUCKET(hash, tsize) ((hash) % (tsize))

      bucket = BUCKET(hash, tsize_unsigned);
      printf("hash [%d] tsize [%d] bucket [%d]\n", hash, tsize_unsigned, bucket);

      bucket = BUCKET(hash, tsize_signed);
      printf("hash [%d] tsize [%d] bucket [%d]\n", hash, tsize_signed, bucket);

      return 0;
  }

Output:

$ ./tmp
hash [-1] tsize [10] bucket [5]
hash [-1] tsize [10] bucket [-1]

The first version is what the current BUCKET() macro does. As a result
we lock the hashtable chain in bucket 5, NOT the freelist.

-1 is sign converted to an unsigned int 4294967295 and

  4294967295 % 10 = 5.

As all callers will lock the same wrong list consistently locking is
still consistent.

Stumpled across this when looking at the output of `tdbtool DB list`
which always printed some other hashchain and not the freelist.

The freelist bucket offset computation doesn't use the BUCKET macro in
freelist.c (directly or indirectly) when working on the freelist, it
just directly uses the FREELIST_TOP define, so this problem only affects
tdbtool list.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2 years agotdb: rename struct tdb_traverse_lock hash member to list
Ralph Boehme [Sun, 2 Jul 2017 05:46:17 +0000 (07:46 +0200)]
tdb: rename struct tdb_traverse_lock hash member to list

The variable stores the hashtable bucket, not the hash. No change in
behaviour.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2 years agopython: Make generated modules samba.ntstatus and samba.werror Python 3 compatible.
Lumir Balhar [Tue, 8 Aug 2017 09:50:30 +0000 (11:50 +0200)]
python: Make generated modules samba.ntstatus and samba.werror Python 3 compatible.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Tue Aug 22 17:38:17 CEST 2017 on sn-devel-144

2 years agopython: scripting: Port ntstatus and werror generators to Python 3 compatible form.
Lumir Balhar [Tue, 8 Aug 2017 08:56:17 +0000 (10:56 +0200)]
python: scripting: Port ntstatus and werror generators to Python 3 compatible form.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
2 years agos3:printing: Add NULL check for state_path()
Andreas Schneider [Mon, 31 Jul 2017 08:09:52 +0000 (10:09 +0200)]
s3:printing: Add NULL check for state_path()

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Aug 19 05:33:41 CEST 2017 on sn-devel-144

2 years agos3:libsmb: let do_connect() debug the negotiation result similar to "session request ok"
Stefan Metzmacher [Wed, 16 Aug 2017 10:42:48 +0000 (12:42 +0200)]
s3:libsmb: let do_connect() debug the negotiation result similar to "session request ok"

Also modify non-specified max_protocol to be PROTOCOL_LATEST
(currently PROTOCOL_SMB3_11).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12881

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2 years agos3:libsmb: don't call cli_NetServerEnum() on SMB2/3 connections in SMBC_opendir_ctx()
Stefan Metzmacher [Wed, 16 Aug 2017 10:38:30 +0000 (12:38 +0200)]
s3:libsmb: don't call cli_NetServerEnum() on SMB2/3 connections in SMBC_opendir_ctx()

This is all we can do with when using we allow SMB2/3 and the server supports
it, 'smb://' can't work unless we implement LLMNR and maybe WSD.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12876

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2 years agos3:smbclient: don't try any workgroup listing with "client min protocol = SMB2"
Stefan Metzmacher [Wed, 16 Aug 2017 06:56:39 +0000 (08:56 +0200)]
s3:smbclient: don't try any workgroup listing with "client min protocol = SMB2"

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12863

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2 years agos3:smbclient: improve the error messages for smbclient -L
Stefan Metzmacher [Wed, 16 Aug 2017 06:55:43 +0000 (08:55 +0200)]
s3:smbclient: improve the error messages for smbclient -L

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12863

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2 years agos3:libsmb: let get_ipc_connect() use CLI_FULL_CONNECTION_FORCE_SMB1
Stefan Metzmacher [Fri, 7 Jul 2017 22:57:59 +0000 (00:57 +0200)]
s3:libsmb: let get_ipc_connect() use CLI_FULL_CONNECTION_FORCE_SMB1

get_ipc_connect() is only used in code paths that require cli_NetServerEnum()
to work, so it must already require SMB1 only.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12876

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2 years agoxattr.id: Fix a typo
Volker Lendecke [Fri, 18 Aug 2017 12:59:58 +0000 (14:59 +0200)]
xattr.id: Fix a typo

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Fri Aug 18 20:49:42 CEST 2017 on sn-devel-144

2 years agos3:utils: Allow to run smbpasswd as user
Andreas Schneider [Fri, 18 Aug 2017 07:41:36 +0000 (09:41 +0200)]
s3:utils: Allow to run smbpasswd as user

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12974

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Fri Aug 18 14:01:27 CEST 2017 on sn-devel-144

2 years agos3:gse_krb5: make use of precalculated krb5 keys in fill_mem_keytab_from_secrets()
Stefan Metzmacher [Thu, 17 Aug 2017 15:45:21 +0000 (17:45 +0200)]
s3:gse_krb5: make use of precalculated krb5 keys in fill_mem_keytab_from_secrets()

This avoids a lot of cpu cycles, which were wasted for each single smb
connection, even if the client didn't use kerberos.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12973

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Aug 18 10:04:57 CEST 2017 on sn-devel-144

2 years agos3:secrets: allow secrets_fetch_or_upgrade_domain_info() on an AD DC
Stefan Metzmacher [Thu, 17 Aug 2017 19:42:34 +0000 (21:42 +0200)]
s3:secrets: allow secrets_fetch_or_upgrade_domain_info() on an AD DC

The reason for the check is for write access as secrets.ldb is the
master database.

But secrets_fetch_or_upgrade_domain_info() just syncs the values
we got from if they got overwritten by secrets_store_machine_pw_sync().

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12973

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2 years agogetncchanges.py: Add test for GET_ANC and linked attributes
Tim Beale [Tue, 11 Jul 2017 22:16:00 +0000 (10:16 +1200)]
getncchanges.py: Add test for GET_ANC and linked attributes

Add a basic test that when we use GET_ANC and the parents have linked
attributes, then we receive all the expected links and all the expected
objects by the end of the test.

This extends the test code to track what linked attributes get received
and check whether they match what's present on the DC.

Also made some minor cleanups to store the received objects/links each
time we successfully receive a GETNCChanges response (this saves the
test case having to repeat this code every time).

Note that although this test involves linked attributes, it shouldn't
exercise the GET_TGT case at all.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972

2 years agogetncchanges.py: Add GET_ANC replication test case
Tim Beale [Tue, 6 Jun 2017 06:21:40 +0000 (18:21 +1200)]
getncchanges.py: Add GET_ANC replication test case

This test:
- creates blocks of parent/child objects
- modifies the parents, so the child gets received first in the
  replication (which means the client has to use GET_ANC)
- checks that we always receive the parent before the child (if not, it
  either retries with GET_ANC, or asserts if GET_ANC is already set)
- modifies the parent objects to change their USN while the
  replication is in progress
- checks that all expected objects are received by the end of the
  test

I've added a repl_get_next() function to help simulate a client's
behaviour - if it encounters an object it doesn't know the parent of,
then it retries with GET_ANC.

Also added some debug to drs_base.py that developers can turn on to make
it easier to see what objects we're actually receiving in the
responses.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972

2 years agogetncchanges.py: Add a new test for replication
Tim Beale [Tue, 6 Jun 2017 06:06:22 +0000 (18:06 +1200)]
getncchanges.py: Add a new test for replication

This adds a new test to check that if objects are modified during a
replication, then those objects don't wind up missing from the
replication data.

Note that when this scenario occurs, samba returns the objects in a
different order to Windows. This test doesn't care what order the
replicated objects get returned in, so long as they all have been
received by the end of the test.

As part of this, I've refactored _check_replication() in drs_base.py so
it can be reused in new tests. In these cases, the objects are split up
over multiple different chunks. So asserting that the objects are returned
in a specific order makes it difficult to run the same test on both Samba
and Windows.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972

2 years agoreplmd: Try to add forward-link for unknown cross-partition links
Tim Beale [Mon, 24 Jul 2017 04:20:58 +0000 (16:20 +1200)]
replmd: Try to add forward-link for unknown cross-partition links

Previously Samba would just drop cross-partition links where the link
target object is unknown. Instead, what we want to do is try to add the
forward link for the GUID specified. We can't add the backlink because
we don't know the target, however, dbcheck should be able to fix any
missing backlinks.

The new behaviour should now mean dbcheck will detect the problem and be
able to fix it. It's still not ideal, but it's better than dropping the
link completely.

I've updated the log so that it has higher severity and tells the user
what they need to do to fix it.

These changes now mean that the selftests now detect an error - instead
of completely dropping the serverReference, we now have a missing
backlink. I've updated the selftests to fix up any missing
serverReference backlinks before running dbcheck.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972

2 years agoreplmd: Don't fail cycle if we get link for deleted object with GET_TGT
Tim Beale [Wed, 19 Jul 2017 23:14:27 +0000 (11:14 +1200)]
replmd: Don't fail cycle if we get link for deleted object with GET_TGT

We are going to end up supporting 2 different server schemes:
A. the old/default behaviour of sending all the linked attributes last,
   at the end of the replication cycle.
B. the new/Microsoft way of sending the linked attributes interleaved
   with the source/target objects.

Normally if we're talking to a server using the old scheme-A, we won't
ever use the GET_TGT flag. However, there are a couple of cases where
it can happen:
- A link to a new object was added during the replication cycle.
- An object was deleted while the replication was in progress (and
the linked attribute got queued before the object was deleted).

Talking to an Samba DC running the old scheme will just cause it to
start the replication cycle from scratch again, which is fairly
harmless. However, there is a chance that the same thing can happen
again, in which case the replication cycle will fail (because GET_TGT
was already set).

Even if we're using the new scheme (B), we could still potentially hit
this case, as we can still queue up linked attributes between requests
(group memberships can be larger than what can fit into a single
replication chunk).

If GET_TGT is set in the GetNcChanges request, then the local copy of
the target object should always be up-to-date when we process the linked
attribute. So if we still think the target object is deleted/recycled at
this point, then it's safe to ignore the linked attribute (because we
know our local copy is up-to-date). This logic matches the MS spec logic
in ProcessLinkValue().

Not failing the replication cycle may be beneficial if we're trying to
do a full-sync of a large database. Otherwise it might be time-consuming
and frustrating to repeat the sync unnecessarily.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972

2 years agoreplmd: Avoid dropping links if link target is deleted
Tim Beale [Wed, 19 Jul 2017 04:18:20 +0000 (16:18 +1200)]
replmd: Avoid dropping links if link target is deleted

The server-side can potentially send the linked attribute before the
target-object. This happens on Microsoft, and will happen on Samba once
server-side GET_TGT support is added. In these cases there is a hole
where the Samba client can silently drop the linked attribute.

If the old copy of the target object was deleted/recycled, then the
client can receive the new linked attribute before it realizes the target
has now been reincarnated. It silently ignores the linked attribute,
thinking its receiving out of date information, when really it's the
client's copy of the target object that's out of date.

In this case we want to retry with the GET_TGT flag set, which will
force the updated version of the target object to be sent along with the
linked attribute. This deleted/recycled target case is the main reason
that Windows added the GET_TGT flag.

If the server sends all the links at the end, instead of along with the
source object, then this case can still be hit. If so, it will cause the
server to restart the replication from the beginning again. This is
probably preferential to silently dropping links.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972

2 years agoreplmd: Move where we store linked attributes
Tim Beale [Wed, 21 Jun 2017 22:43:31 +0000 (10:43 +1200)]
replmd: Move where we store linked attributes

There was a bug in my previous patch where the code would verify
*all* links in the list, rather than just the ones that are new. And it
would do this for every replication chunk it received, regardless of
whether there were actually any links in that chunk.

The problem is by the time we want to verify the attributes, we don't
actually know which attributes are new. We can fix this by moving where
we store the linked attributes from the start of processing the
replication chunk to the end of processing the chunk. We can then verify
the new linked attributes at the same time we store them.

Longer-term we may want to try to apply the linked attribute at this
point. This would save looking up the source/target objects twice, but
it makes things a bit more complicated (attributes will usually apply at
this point *most* of the time, but not *all* the time).

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972

2 years agodrs_utils: Add GET_TGT support to 'samba-tool drs replicate --local'
Tim Beale [Fri, 16 Jun 2017 00:54:32 +0000 (12:54 +1200)]
drs_utils: Add GET_TGT support to 'samba-tool drs replicate --local'

Update drs_Replicate.replicate() so it handles being passed the GET_TGT
flag (more_flags). To do this, we need to always use a v10 GetNCChanges
request (v8 and v10 are essentially the same except for the more_flags).

If the replicate_chunk() call into the C bindings throws an error, check
to see whether the error could be fixed by setting the GET_TGT flag, and
re-send the request if so.

Unfortunately because WERR_DS_DRA_RECYCLED_TARGET isn't documented with
the other AD error codes, I've left it hardcoded for now (Microsoft
should be fixing up their Docs).

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972

2 years agoreplmd: Set GET_ANC if Windows sends a link with unknown source object
Tim Beale [Thu, 15 Jun 2017 21:49:16 +0000 (09:49 +1200)]
replmd: Set GET_ANC if Windows sends a link with unknown source object

Windows replication can send the linked attribute before it sends the
source object. The MS-DRSR spec says that in this case the client should
resend the GetNCChanges request with the GET_ANC flag set. In my testing
this resolves the problem - Windows will include the source object for the
linked attribute in the same replication chunk.

This problem doesn't happen with Samba-to-Samba replication, because the
source object for the linked attribute is guaranteed to have already been
sent.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972

2 years agodrepl: Support GET_TGT on periodic replication client
Tim Beale [Thu, 15 Jun 2017 04:52:33 +0000 (16:52 +1200)]
drepl: Support GET_TGT on periodic replication client

- Update IDL comments to include Microsoft reference doc
- Add support for sending v10 GetNCChanges request (needed for the
  GET_TGT flag, which is in the new 'more_flags' field)
- Update to also set the GET_TGT flag in the same place we were setting
  GET_ANC (I split this logic out into a separate function).
- The state struct now needs to hold a 'more_flags' field as well (this
  flag is different to the GET_ANC replica flag)

Note that using the GET_TGT when replicating from a Windows DC could be
highly inefficient. Because Samba keeps the GET_TGT flag set throughout
the replication cycle, it will basically receive a repeated object from
Windows for every single linked attribute that it receives.

I believe Windows behaviour only expects the client to set the GET_TGT
flag when it actually needs to (i.e. when it receives a target object it
doesn't know about), rather than throughout the replication cycle.
However, this approach won't work with Samba-to-Samba replication,
because when the server receives the GET_TGT flag it restarts the
replication cycle from scratch. So if we only set the GET_TGT flag when
the client encountered an unknown target then Samba-to-Samba could
potentially get into an endless replication loop.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972

2 years agodrs: Check target object is known after applying objects
Tim Beale [Thu, 15 Jun 2017 02:09:27 +0000 (14:09 +1200)]
drs: Check target object is known after applying objects

Currently we only check that the target object is known at the end of
the transaction (i.e. the .prepare_commit hook). It's too late at this
point to resend the request with GET_TGT. Move this processing earlier
on, after we've applied all the objects (i.e. off the .extended hook).

In reality, we need to perform the checks at both points. I've
split the common code that gets the source/target details out of the
la_entry into a helper function. It's not the greatest function ever,
but seemed to make more sense than duplicating the code.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972

2 years agodrs: Fail replication transaction instead of dropping links
Tim Beale [Tue, 13 Jun 2017 23:35:36 +0000 (11:35 +1200)]
drs: Fail replication transaction instead of dropping links

If the DRS client received a linked attribute that it couldn't resolve
the target for, then it would just ignore that link and keep going. That
link would then be lost forever (although a full-sync would resolve
this). Instead of silently ignoring the link, fail the transaction.

This *can* happen on Samba, but it is unusual. The target object and
linked-attribute would need to be added while a replication is still in
progress. It can also happen fairly easily when talking to a Windows DC.

There are two import exceptions to this:

1). Linked attributes that span partitions. We can never guarantee that
we will have received the target object, because it may be in a partition
we haven't replicated yet. Samba doesn't have a great way of handling
this currently, but we shouldn't fail the replication (because that breaks
basic join tests). Just skip that linked attribute and hope that a
subsequent full-sync will fix it.
(I queried Microsoft and they said resolving cross-partition linked
attributes is a implementation-specific problem to solve. GET_TGT won't
resolve it)

2). When the replication involves a subset of objects, e.g.
critical-only. In these cases, we don't increase the highwater-mark, so
it is probably not such a dire problem if we don't add the link. In the
case of critical-only, we will do a subsequent full sync which will then
add the links.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972

2 years agoreplmd: Split checking link attr target into new function
Tim Beale [Tue, 13 Jun 2017 22:51:59 +0000 (10:51 +1200)]
replmd: Split checking link attr target into new function

We want to re-use this code to check that the linked attribute's target
object exists *before* we try to commit the transaction. This will allow
us to re-request the block with the GET_TGT flag set.

This splits checking the target object exists into a separate function.

Minor changes of note:
- the 'parent' argument was passed to replmd_process_linked_attribute()
  as NULL, so I've just replaced where it was used in the refactored code
  with NULL.
- I've tweaked the "Failed to find GUID" error message slightly to display
  the attribute ID rather than the attribute name (saves repeating
  lookups and/or passing extra arguments).
- Tweaked the replmd_deletion_state() logic - it only made sense to call
  it in the code block where we actually found the target

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972

2 years agowerror: Add WERR_DS_DRA_RECYCLED_TARGET
Tim Beale [Tue, 30 May 2017 03:08:44 +0000 (15:08 +1200)]
werror: Add WERR_DS_DRA_RECYCLED_TARGET

When the DRS client encounters a linked attribute with an unknown target
object, it should return a RECYCLED_TARGET error, which should result in
the client resending the GETNCChanges request with the GET_TGT flag set.

This error code is currently documented by Microsoft under System Error
Codes (8200-8999). I contacted them and they will also add it to the
MS-ERREF doc in future.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972

2 years agolibcli/smb: debug an error if smb1cli_req_writev_submit() is called for SMB2/3
Stefan Metzmacher [Wed, 16 Aug 2017 20:27:15 +0000 (22:27 +0200)]
libcli/smb: debug an error if smb1cli_req_writev_submit() is called for SMB2/3

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12968

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Aug 18 04:45:03 CEST 2017 on sn-devel-144