Ralph Boehme [Mon, 15 Oct 2018 13:17:08 +0000 (15:17 +0200)]
s4:torture/vfs/fruit: update test "creating rsrc with read-only access" for newer macOS versions
While this operation failed against older macOS versions, it passes
against versions 10.12 and newer. Update the test accordingly, a
subsequent commit will then update our implementation.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Wed, 10 Oct 2018 10:47:07 +0000 (12:47 +0200)]
s4:torture/vfs/fruit: expand existing vfs_test "null afpinfo"
This adds a check that a read on a seperate handle also sees the
previously created AFP_AfpInfo stream.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Tue, 9 Oct 2018 16:48:08 +0000 (18:48 +0200)]
s4:torture/vfs/fruit: expand existing test "setinfo delete-on-close AFP_AfpInfo" a little bit
Add a check that verifies a create on a stream gets
NT_STATUS_DELETE_PENDING after delete-on-close has been set.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Mon, 15 Oct 2018 14:24:19 +0000 (16:24 +0200)]
s4:torture/vfs/fruit: update test "read open rsrc after rename" to work with macOS
macOS SMB server seems to return NT_STATUS_SHARING_VIOLATION in this
case while Windows 2016 returns NT_STATUS_ACCESS_DENIED.
Lets stick with the Windows error code for now in the Samba fileserver,
but let the test pass against macOS.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Wed, 31 Oct 2018 14:49:45 +0000 (15:49 +0100)]
s4:torture/vfs/fruit: ensure a directory handle is closed in all code paths
Otherwise we get a sharing violation when running against Samba and
opening the directory a second time.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Mon, 15 Oct 2018 13:39:12 +0000 (15:39 +0200)]
s4:torture/vfs/fruit: update test "stream names" to work with macOS
o create the basefile before trying to create a stream on it, otherwise
this fails on macOS
o write something to the stream, otherwise the stream is not listed as
macOS hides 0-byte sized streams
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Mon, 15 Oct 2018 13:31:21 +0000 (15:31 +0200)]
s4:torture/vfs/fruit: update test "SMB2/CREATE context AAPL" to work against macOS
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Thu, 11 Oct 2018 15:13:52 +0000 (17:13 +0200)]
s4:torture/vfs/fruit: set share_access to NTCREATEX_SHARE_ACCESS_MASK in check_stream_list
Avoid sharing conflicts with other opens on the basefile.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Wed, 17 Oct 2018 08:51:45 +0000 (10:51 +0200)]
s4:torture/vfs/fruit: fix a few error checks in "delete AFP_AfpInfo by writing all 0"
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Fri, 19 Oct 2018 20:21:10 +0000 (22:21 +0200)]
s4:torture/vfs/fruit: skip a few tests when running against a macOS SMB server
These tests are designed to test specific vfs_fruit functionality.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Thu, 23 Aug 2018 10:07:20 +0000 (12:07 +0200)]
vfs_streams_xattr: fix open implementation
Since a long time the modules's open function happily returned success
when opening a non existent stream without O_CREAT.
This change fixes it to return -1 and errno=ENOATTR if
o get_ea_value() returns NT_STATUS_NOT_FOUND (eg mapped from
getxattr() = -1, errno=ENOATTR) and
o flags doesn't contain O_CREAT
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Tue, 9 Oct 2018 12:54:31 +0000 (14:54 +0200)]
vfs_fruit: optionally delete AppleDouble files without Resourcefork data
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13642
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Wed, 3 Oct 2018 10:01:00 +0000 (12:01 +0200)]
vfs_fruit: add option "delete_empty_adfiles"
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13642
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Tue, 2 Oct 2018 14:05:28 +0000 (16:05 +0200)]
vfs_fruit: detect empty resource forks in ad_convert()
For some reason the macOS client often writes AppleDouble files with a
non-zero sized resource fork, but the resource fork data is just
boilerplate data with the following string close to the start
This resource fork intentionally left blank
A dump with apple_dump looks like this:
Entry ID :
00000002 : Resource Fork
Offset :
00000052 : 82
Length :
0000011E : 286
-RAW DUMP--: 0 1 2 3 4 5 6 7 8 9 A B C D E F : (ASCII)
00000000 : 00 00 01 00 00 00 01 00 00 00 00 00 00 00 00 1E : ................
00000010 : 54 68 69 73 20 72 65 73 6F 75 72 63 65 20 66 6F : This resource fo
00000020 : 72 6B 20 69 6E 74 65 6E 74 69 6F 6E 61 6C 6C 79 : rk intentionally
00000030 : 20 6C 65 66 74 20 62 6C 61 6E 6B 20 20 20 00 00 : left blank ..
00000040 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
00000050 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
00000060 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
00000070 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
00000080 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
00000090 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
000000A0 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
000000B0 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
000000C0 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
000000D0 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
000000E0 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
000000F0 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
00000100 : 00 00 01 00 00 00 01 00 00 00 00 00 00 00 00 1E : ................
00000110 : 00 00 00 00 00 00 00 00 00 1C 00 1E FF FF : ..............
We can safely discard this Resource Fork data.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13642
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Wed, 3 Oct 2018 10:01:00 +0000 (12:01 +0200)]
vfs_fruit: add option "wipe_intentionally_left_blank_rfork"
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13642
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Thu, 4 Oct 2018 12:28:15 +0000 (14:28 +0200)]
s4:torture: add test for AppleDouble ResourceFork conversion
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13642
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Thu, 4 Oct 2018 11:47:20 +0000 (13:47 +0200)]
s3:selftest: list vfs testssuites one per line
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13642
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Thu, 4 Oct 2018 16:22:31 +0000 (18:22 +0200)]
docs:vfs_fruit: add "delete_empty_adfiles" option
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13642
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Tue, 2 Oct 2018 14:31:15 +0000 (16:31 +0200)]
docs:vfs_fruit: add "wipe_intentionally_left_blank_rfork" option
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13642
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Wed, 31 Oct 2018 14:16:18 +0000 (15:16 +0100)]
s3:smbd: remove "0x" string prefix from dev/ino
We used %llu as conversion specifier which results in a decimal number
being printed, so remove the misleading "0x" prefix.
While at it, I'll change %llu to the terse %ju.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Andreas Schneider [Wed, 31 Oct 2018 07:44:08 +0000 (08:44 +0100)]
third_party: Update nss_wrapper to version 1.1.5
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Tim Beale [Thu, 18 Oct 2018 04:08:32 +0000 (17:08 +1300)]
netcmd: Add 'samba-tool group stats' command
With large domains it's hard to get an idea of how many groups there
are, and how many users are in each group, on average. However, this
could have a big impact on whether a problem can be reproduced or not.
This patch dumps out some summary information so that you can get a
quick idea of how big the groups are.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Wed Oct 31 03:40:41 CET 2018 on sn-devel-144
Tim Beale [Thu, 18 Oct 2018 03:59:24 +0000 (16:59 +1300)]
netcmd: Include num-members in 'samba-tool group list --verbose'
This adds an easy way for users to see (via samba-tool) how many members
are in various groups, without querying the members for each individual
group.
For example, you could pipe this output to grep to check for groups with
zero or one members (i.e. historic groups that may no longer make
sense).
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Tim Beale [Fri, 26 Oct 2018 00:21:48 +0000 (13:21 +1300)]
selftest: Specify different DB backends for restored testenvs
Vary the DB backend that we use for the renamed DCs. The labdc and
renamedc are fairly similar, so let's have each of them use a different
backend.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Tim Beale [Wed, 24 Oct 2018 20:03:53 +0000 (09:03 +1300)]
netcmd: Add backend-store option to domain backup/rename cmds
Currently the online/rename backup files always use the default backend
(TDB) and there is no way to change this.
This patch adds the backend-store option to the backup commands so that
you can create a backup with an MDB backend, if needed.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Tim Beale [Mon, 22 Oct 2018 21:19:38 +0000 (10:19 +1300)]
traffic_replay: logger was ignoring smb.conf log-level
We were trying to access the debug-level (in python C bindings) before
the smb.conf had been loaded and actually set the debug-level. So it
would default to zero, regardless of what was in the smb.conf.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Tim Beale [Tue, 16 Oct 2018 22:26:28 +0000 (11:26 +1300)]
traffic_replay: Change print() to use logger()
This reduces noise, so the messages only come out if you specify
--debug.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Tim Beale [Thu, 11 Oct 2018 01:47:28 +0000 (14:47 +1300)]
traffic_replay: Generate users faster by writing to local DB
We can create user accounts much faster if the LDB connection talks
directly to the local sam.ldb file rather than going via LDAP. This
patch allows the 'host' argument to the tool to be a .ldb file (e.g.
"/usr/local/samba/private/sam.ldb") instead of a server name/IP.
In most cases, the traffic_replay tool wants to run on a remote device
(because the point of it is to send traffic to the DC). However, the
--generate-users-only is one case where the tool can be run locally,
directly on the test DC. (The traffic_replay user generation is handy
for standalone testing, because it also handles assigning group
memberships to the generated user accounts).
Note that you also need to use '--option="ldb:nosync = true"' to get
the improvement in performance.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Gary Lockyer [Wed, 24 Oct 2018 21:52:55 +0000 (10:52 +1300)]
dsdb group audit tests: log_membership_changes extra tests
Add extra tests to ensure better test coverage of log_membership_changes
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Tue Oct 30 20:20:26 CET 2018 on sn-devel-144
Gary Lockyer [Thu, 25 Oct 2018 01:38:31 +0000 (14:38 +1300)]
dsdb group audit tests: check_version improve diagnostics
Change check_version to display the expected, actual along with the
line and name of the failing test, rather than the line in check_version
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Gary Lockyer [Thu, 25 Oct 2018 00:28:09 +0000 (13:28 +1300)]
dsdb group audit tests: check_timestamp improve diagnostics
Change check_timestamp to display the expected, actual along with the
line and name of the failing test, rather than the line in
check_timestamp.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Gary Lockyer [Wed, 24 Oct 2018 21:52:27 +0000 (10:52 +1300)]
dsdb group audit: align dn_compare with memcmp
Rename the parameter names and adjust the return codes from dn_compare
so that:
dn_compare(a, b) =>
LESS_THAN means a is less than b.
GREATER_THAN means a is greater than b.
Thanks to metze for suggesting the correct semantics for dn_compare
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13664
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Gary Lockyer [Tue, 23 Oct 2018 04:14:34 +0000 (17:14 +1300)]
dsdb group_audit: Test to replicate BUG 13664
The group audit code incorrectly logs member additions and deletions.
Thanks to metze for the debugging that isolated the issue, and for
suggesting the fix to dn_compare.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13664
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Andrew Bartlett [Tue, 30 Oct 2018 02:56:43 +0000 (15:56 +1300)]
dsdb: Add comments explaining the limitations of our current backlink behaviour
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Tim Beale <timbeale@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Oct 30 10:32:51 CET 2018 on sn-devel-144
Stefan Metzmacher [Fri, 24 Aug 2018 13:33:49 +0000 (15:33 +0200)]
s4:samldb: internally use extended dns while changing the primaryGroupID field
This is important, otherwise we'll loose the <SID=> component of the
linked attribute.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Stefan Metzmacher [Fri, 12 Oct 2018 13:56:18 +0000 (15:56 +0200)]
s4:repl_meta_data: add support for DSDB_CONTROL_DBCHECK_FIX_LINK_DN_SID
This will be used by dbcheck in the next commits.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Stefan Metzmacher [Fri, 12 Oct 2018 16:43:25 +0000 (18:43 +0200)]
s4:repl_meta_data: pass down struct replmd_replicated_request to replmd_modify_la_replace()
This will simplify further changes.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Stefan Metzmacher [Fri, 12 Oct 2018 16:43:25 +0000 (18:43 +0200)]
s4:repl_meta_data: pass down struct replmd_replicated_request to replmd_modify_la_delete()
This will simplify further changes.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Stefan Metzmacher [Fri, 12 Oct 2018 17:34:08 +0000 (19:34 +0200)]
s4:repl_meta_data: add missing \n to a DEBUG message in replmd_modify_la_add()
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Stefan Metzmacher [Fri, 12 Oct 2018 16:43:25 +0000 (18:43 +0200)]
s4:repl_meta_data: pass down struct replmd_replicated_request to replmd_modify_la_add()
This will simplify further changes.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Stefan Metzmacher [Fri, 12 Oct 2018 16:43:25 +0000 (18:43 +0200)]
s4:repl_meta_data: pass down struct replmd_replicated_request to replmd_modify_handle_linked_attribs()
This will simplify further changes.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Stefan Metzmacher [Fri, 12 Oct 2018 13:56:18 +0000 (15:56 +0200)]
blackbox/dbcheck-links: Test broken links with missing <SID=...> on linked attributes
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Stefan Metzmacher [Fri, 12 Oct 2018 13:56:18 +0000 (15:56 +0200)]
dbchecker: Fix missing <SID=...> on linked attributes
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Stefan Metzmacher [Mon, 8 Oct 2018 15:14:28 +0000 (17:14 +0200)]
dbchecker: improve verbose output of do_modify()
This makes it easier to debug dbcheck problems.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Stefan Metzmacher [Mon, 8 Oct 2018 15:13:52 +0000 (17:13 +0200)]
s4:dsdb: add DSDB_CONTROL_DBCHECK_FIX_LINK_DN_SID oid
This will be used to fix missing <SID=> components in future.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Stefan Metzmacher [Tue, 16 Oct 2018 13:16:18 +0000 (15:16 +0200)]
testprogs/blackbox: add samba4.blackbox.test_primary_group test
This demonstrates the bug, that happens when the primaryGroupID
of a user is changed.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Stefan Metzmacher [Mon, 8 Oct 2018 15:13:13 +0000 (17:13 +0200)]
s4:dsdb: fix comment on DSDB_CONTROL_DBCHECK_FIX_LINK_DN_NAME
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Stefan Metzmacher [Mon, 8 Oct 2018 13:35:52 +0000 (15:35 +0200)]
schema_samba4.ldif: add allocation of DSDB_CONTROL_DBCHECK_FIX_LINK_DN_NAME
This was already allocated in source4/dsdb/samdb/samdb.h with
commit
22208f52e6096fbe9413b8ff339d9446851e0874.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Volker Lendecke [Wed, 24 Oct 2018 12:31:34 +0000 (14:31 +0200)]
tdb: Make record deletion circular-chain safe
Before this patch we had 3 loops walking a hash chain to delete
records:
tdb_do_delete() to find the predecessor of the record that was to be
deleted. tdb_count_dead(), the name says it all and tdb_purge_dead()
to give back all dead records from a chain to the freelist.
This patch introduces tdb_trim_dead that walks a hash chain just
once. While it does so it counts the number of dead records, and all
records beyond tdb->max_dead_records are moved to the freelist.
Normal record deletion now works by always marking a record as dead in
step 1 and then calling tdb_trim_dead. This is made safe against
circular chains by doing the slow chain walk only in the case when we
did not delete a dead record during our walk.
It changes our dynamics a bit:
When deleting a record with non-zero max_dead_records, now we always
leave that number of records around when deleting, doing a blocking
lock on the freelist when we found too many dead records.
Previously when exceeding max_dead_records we wiped all dead records
to start accumulating them from scratch, assuming we could lock the
freelist in a nonblocking fashion.
The net effect for an uncontended freelist is the same: In
tdb_allocate() we still completely hand over all dead records to the
freelist when we could lock it, it just happens later than without
this patch.
This means for a lightly loaded system we will potentially leave more
dead records around in databases like locking.tdb. However, on a
heavily loaded system we become more predictable: If the freelist is
so heavily contended that across many deletes we can't get hold of it,
previously we accumulated more dead records than max_dead_records
would allow. This is a really lowlevel tradeoff that is likely hard to
measure, but to me becoming more deterministic without sacrificing too
much parallelism (we keep more dead records around) is worth trying.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Oct 30 02:48:38 CET 2018 on sn-devel-144
Volker Lendecke [Thu, 25 Oct 2018 13:59:48 +0000 (15:59 +0200)]
tdb: Do early RDONLY error check for tdb_delete
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Volker Lendecke [Thu, 25 Oct 2018 13:55:29 +0000 (15:55 +0200)]
tdb: Purge dead records whenever we block the freelist
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Volker Lendecke [Tue, 23 Oct 2018 11:40:34 +0000 (13:40 +0200)]
tdb: Don't delete dead records in traverse
The next commit will change the handling of dead records, removing the
"tdb_do_delete" function. As traverses should not happen in normal
operations, dead records from them should be rare, and relying on
traverses to remove them is a very bad idea IMHO.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Volker Lendecke [Wed, 24 Oct 2018 05:26:49 +0000 (07:26 +0200)]
tdb: Align an integer type
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Volker Lendecke [Thu, 25 Oct 2018 18:53:52 +0000 (20:53 +0200)]
tdb: Fix a typo
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Fri, 19 Oct 2018 10:15:42 +0000 (12:15 +0200)]
vfs_fruit: remove check for number of xattrs from ad_convert_xattr
Turns out that there exist AppleDouble files with an extended FinderInfo
entry that includes the xattr marshall buffer, but the count of xattrs
in the buffer is just zero.
We do want to discard this extended FinderInfo entry and convert it to a
simple fixed size FinderInfo entry, so remove the check.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Douglas Bagnall [Fri, 26 Oct 2018 07:25:59 +0000 (20:25 +1300)]
python: do not use "is" for string equality
This is not always going to work, and is not guaranteed to be
consistent even between minor versions.
Here is a simple counterexample:
>>> a = 'hello'
>>> a is 'hello'
True
>>> a is 'hello'.lower()
False
>>> a == a.lower()
True
Possibly it always works for the empty string, but we cannot rely
on that.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Mon Oct 29 23:13:36 CET 2018 on sn-devel-144
Douglas Bagnall [Thu, 25 Oct 2018 09:09:59 +0000 (22:09 +1300)]
python/samdb: properly use property()
Python's property() function works like this:
property([getter[, setter[, delete[, doc]]]])
but we have been forgetting the delete function, or rather setting it
to be a string. A string is not callable and is unlikely to succeed at
deleting the property.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Andreas Schneider [Mon, 29 Oct 2018 18:45:58 +0000 (19:45 +0100)]
s3:winbind: Check return code of initialize_password_db()
See https://retrace.fedoraproject.org/faf/reports/
1577174/
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13668
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
David Mulder [Wed, 24 Oct 2018 16:55:02 +0000 (10:55 -0600)]
lib:socket: If returning early, set ifaces
Prevents a segfault in load_interfaces() when total interfaces == 1.
Fixes regression caused by
da68a1b2f417ec82ea4ed3e7a4d867cef8ca8f93.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13665
Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Ralph Böhme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Sun Oct 28 00:35:35 CEST 2018 on sn-devel-144
Volker Lendecke [Tue, 25 Sep 2018 15:59:20 +0000 (08:59 -0700)]
lib: Remove unused tdb_trans_* functions
The transactions have all moved to dbwrap
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Andreas Schneider [Fri, 19 Oct 2018 13:24:14 +0000 (15:24 +0200)]
s3:torture: Rename the test file and remove it if it exists
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Andreas Schneider [Fri, 19 Oct 2018 12:38:34 +0000 (14:38 +0200)]
s3:torture: Don't use the same testdir twice
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Andreas Schneider [Thu, 18 Oct 2018 12:00:13 +0000 (14:00 +0200)]
s3:selftest: Fix test names of smbtorture_s3.plain
The env name will be appended. There is no need to have it twice. Can't
we remove the tests againa ad_dc_ntvfs completely?
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Christian Ambach [Tue, 23 Oct 2018 18:05:04 +0000 (20:05 +0200)]
s3:utils/smbget fix recursive download with empty source directories
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13199
Signed-off-by: Christian Ambach <ambi@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Fri Oct 26 09:58:07 CEST 2018 on sn-devel-144
Christian Ambach [Mon, 22 Oct 2018 14:28:21 +0000 (16:28 +0200)]
s3:utils/smbget add error handling for mkdir() calls
Signed-off-by: Christian Ambach <ambi@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Christian Ambach [Mon, 22 Oct 2018 14:22:00 +0000 (16:22 +0200)]
s3:script/tests reduce code duplication
Signed-off-by: Christian Ambach <ambi@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Douglas Bagnall [Thu, 11 Oct 2018 22:52:49 +0000 (11:52 +1300)]
pytests/samba3sam: unshadow and fix a search_non_mapped test
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Fri Oct 26 00:50:37 CEST 2018 on sn-devel-144
Douglas Bagnall [Thu, 11 Oct 2018 01:45:58 +0000 (14:45 +1300)]
python/tests/ou: unshadow a test
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Wed, 24 Oct 2018 22:40:53 +0000 (11:40 +1300)]
samba-tool tests: fix bytes/str issue in masked test
This test will be revealed to the world in the next commit.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Thu, 11 Oct 2018 01:42:40 +0000 (14:42 +1300)]
python/tests/raw_protocol: reveal shadowed test via disambiguation
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Thu, 11 Oct 2018 04:00:46 +0000 (17:00 +1300)]
s4/dsdb/pytest/ldap_schema: remove duplicate test
The tests differ in two lines, thus:
def test_subClassOf(self):
- """ Testing usage of custom child schamaClass
+ """ Testing usage of custom child classSchema
[...]
-governsId: 1.3.6.1.4.1.7165.4.6.2.6.3.""" + str(random.randint(1, 100000)) + """
+governsId: 1.3.6.1.4.1.7165.4.6.2.6.7.""" + str(random.randint(1, 100000)) + """
with the governsId OID changed because it was noticed they were colliding.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Thu, 11 Oct 2018 03:47:44 +0000 (16:47 +1300)]
ldb/tests/py/api: reveal shadowed casefold test
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Thu, 11 Oct 2018 03:15:43 +0000 (16:15 +1300)]
python/tests/kcc_utils: disambiguate/unshadow a test
Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Thu, 11 Oct 2018 00:50:06 +0000 (13:50 +1300)]
python/tests/lsa_string: remove duplicate method
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Thu, 25 Oct 2018 09:34:31 +0000 (22:34 +1300)]
python/gp_parse/gp_inf: remove unused variables
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Thu, 11 Oct 2018 00:23:01 +0000 (13:23 +1300)]
python/gp_parse/gp_inf: remove shadowed method
The 'from_xml()' definition is replaced a few lines down
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Thu, 11 Oct 2018 05:33:48 +0000 (18:33 +1300)]
s4/scripting/demodirsync: fix syntax error
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Thu, 11 Oct 2018 05:32:52 +0000 (18:32 +1300)]
s4/scripting: py3 style 0o123 octal, not 0123
this works with py2.6+ too.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Thu, 11 Oct 2018 04:23:54 +0000 (17:23 +1300)]
s4/dsdb/pytest/sort: use compat.cmp_fn instead of cmp
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Thu, 11 Oct 2018 04:06:32 +0000 (17:06 +1300)]
s4/dsdb/pytest/ad_dc_medley: do not use xrange
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Thu, 11 Oct 2018 03:52:14 +0000 (16:52 +1300)]
s4/dsdb/pytests: Py3 compatitble except clauses
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Thu, 11 Oct 2018 03:04:38 +0000 (16:04 +1300)]
script/show_test_time: approach python 3 compatibility
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Thu, 11 Oct 2018 00:37:28 +0000 (13:37 +1300)]
python/kcc: use compat.cmp_fn (PY3)
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Thu, 11 Oct 2018 00:07:30 +0000 (13:07 +1300)]
python/remove_dc: use a local variable in offline_remove_server
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Wed, 10 Oct 2018 23:52:09 +0000 (12:52 +1300)]
python/samba/common: py3 compat raw_input
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Wed, 10 Oct 2018 23:40:09 +0000 (12:40 +1300)]
samba-tool ldapcmp: use ValueError, not obsolete StandardError
The error is in the value, and StandardError is not in Python 3
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Wed, 24 Oct 2018 02:56:20 +0000 (15:56 +1300)]
python tests: always use Python's unicodedata
We had our own special version with very few entries, but only
used it in one place.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Thu, 11 Oct 2018 05:31:09 +0000 (18:31 +1300)]
pytest/dcerpc.integer: force py2 long int without incompatible syntax
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Thu, 11 Oct 2018 03:52:28 +0000 (16:52 +1300)]
s4/selftest/tests: don't use import *
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Thu, 11 Oct 2018 03:51:23 +0000 (16:51 +1300)]
s3/selftest/tests: don't use import *
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Thu, 11 Oct 2018 03:05:23 +0000 (16:05 +1300)]
selftest/tests.py: avoid import *
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Thu, 11 Oct 2018 00:40:59 +0000 (13:40 +1300)]
python/tests/common: avoid import *
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Thu, 11 Oct 2018 03:05:11 +0000 (16:05 +1300)]
selftest/perftests: avoid import *
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Thu, 11 Oct 2018 01:51:55 +0000 (14:51 +1300)]
python/tests/kcc_*: avoid * imports
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Thu, 11 Oct 2018 01:44:02 +0000 (14:44 +1300)]
python/tests/kcc.graph: avoid import *
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Thu, 11 Oct 2018 05:33:23 +0000 (18:33 +1300)]
s4/scripting/pfm_verify: remove duplicate import
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Wed, 10 Oct 2018 23:34:53 +0000 (12:34 +1300)]
samba-tool: samba.getopt is not used here
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Thu, 11 Oct 2018 05:25:53 +0000 (18:25 +1300)]
auth/cred/tests/bind: remove unused import
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Thu, 11 Oct 2018 04:24:22 +0000 (17:24 +1300)]
s4/dsdb/pytest/: unused imports
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
Douglas Bagnall [Thu, 11 Oct 2018 04:07:22 +0000 (17:07 +1300)]
s4/dsdb/pytest/dsdb_schema_info: do not double import
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>