Andrew Bartlett [Thu, 10 Aug 2017 05:00:48 +0000 (17:00 +1200)]
ldb_tdb: replace strange dn_list_cmp() in index code
This replaces dn_list_cmp() with functions that do not attempt to
to care about string termination. All index values are case sensitive
and correctly length-bound already, even for a DN index
so just use a length check and memcmp()
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Andrew Bartlett [Thu, 17 Aug 2017 00:42:25 +0000 (12:42 +1200)]
ldb_tdb: Do not allow a modification of the GUID_index_attribute (objectGUID)
This would totally break our index scheme if this could be modified.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Andrew Bartlett [Thu, 10 Aug 2017 05:13:48 +0000 (17:13 +1200)]
ldb_tdb: Add GUID_index_attribute to ltdb_private
This will be used to determine if we are in GUID index mode
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Andrew Bartlett [Thu, 10 Aug 2017 04:23:33 +0000 (16:23 +1200)]
ldb_tdb: Provide struct ltdb_private to index routines
This will make it easier to switch the GUID index mode on and off
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Andrew Bartlett [Wed, 16 Aug 2017 00:46:57 +0000 (12:46 +1200)]
ldb_tdb: Use a more complete error mapping in ltdb_search_key()
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Andrew Bartlett [Thu, 10 Aug 2017 04:06:08 +0000 (16:06 +1200)]
ldb_tdb: Add ltdb_search_key()
This allows us to slowly split out the tdb key in the DB from being the DN
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Andrew Bartlett [Thu, 10 Aug 2017 04:01:12 +0000 (16:01 +1200)]
ldb_tdb: Use ltdb_key_msg() in re_index()
This will allow changing to a GUID tdb key in the future
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Andrew Bartlett [Thu, 10 Aug 2017 02:31:18 +0000 (14:31 +1200)]
ldb_tdb: provide ldb_key_dn() and ldb_key_msg()
This will in time allow us to generate a TDB key from
the msg, eg from an objectGUID.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Andrew Bartlett [Tue, 15 Aug 2017 05:21:28 +0000 (17:21 +1200)]
ldb_tdb: Add helper function ltdb_search_and_return_base()
This avoids an extra DB lookup for the base, when that is the only
record we will return, and avoids going into the index code for
a base search, as that won't work for special DNs once the GUID
index mode is enabled.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Andrew Bartlett [Mon, 28 Aug 2017 02:55:00 +0000 (14:55 +1200)]
ldb_tdb: Make ldb_match_message() available to ldb_tdb
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Amitay Isaacs [Fri, 22 Sep 2017 04:17:59 +0000 (14:17 +1000)]
ctdb-tests: Add tests for client with multiple connections
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13042
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Fri Sep 22 20:58:46 CEST 2017 on sn-devel-144
Amitay Isaacs [Fri, 22 Sep 2017 04:14:00 +0000 (14:14 +1000)]
ctdb-tests: Add support for multiple ctdb connections in dummy_client
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13042
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Amitay Isaacs [Fri, 22 Sep 2017 04:04:50 +0000 (14:04 +1000)]
ctdb-tests: Check all connections from a process in CHECK_PID_SRVID control
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13042
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Amitay Isaacs [Fri, 22 Sep 2017 03:52:09 +0000 (13:52 +1000)]
ctdb-daemon: Check all connections from a process in CHECK_PID_SRVID control
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13042
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Christof Schmitt [Wed, 20 Sep 2017 23:13:38 +0000 (16:13 -0700)]
selftest: Also run smbtorture smb2.compound with aio enabled
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13047
Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Sep 22 09:49:30 CEST 2017 on sn-devel-144
Christof Schmitt [Wed, 20 Sep 2017 23:07:50 +0000 (16:07 -0700)]
torture: Add testcase for compound CREATE-WRITE-CLOSE request
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13047
Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Christof Schmitt [Thu, 21 Sep 2017 19:08:01 +0000 (12:08 -0700)]
smbd/aio: Do not go async for SMB2 compound requests
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13047
Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Christof Schmitt [Fri, 22 Sep 2017 00:41:25 +0000 (17:41 -0700)]
smbd: Move check for SMB2 compound request to new function
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13047
Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Richard Sharpe [Tue, 19 Sep 2017 19:31:55 +0000 (12:31 -0700)]
s3: Fix a small spelling mistake in smbcacls.
Signed-off-by: Richard Sharpe <realrichardsharpe@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Fri Sep 22 05:44:09 CEST 2017 on sn-devel-144
Volker Lendecke [Wed, 20 Sep 2017 16:18:33 +0000 (09:18 -0700)]
auth3: Remove auth_domain
If you're a domain member, use winbind. Auth_domain is from times when we did
not have winbind. It has served its purpose, but we should move on.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Sep 22 00:02:29 CEST 2017 on sn-devel-144
Amitay Isaacs [Wed, 30 Aug 2017 06:35:49 +0000 (16:35 +1000)]
ctdb-tests: Add tests for PID with srvid
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13042
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Thu Sep 21 12:56:30 CEST 2017 on sn-devel-144
Amitay Isaacs [Wed, 30 Aug 2017 07:04:44 +0000 (17:04 +1000)]
ctdb-tests: Implement control CHECK_PID_SRVID in fake daemon
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13042
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Amitay Isaacs [Wed, 30 Aug 2017 06:59:16 +0000 (16:59 +1000)]
ctdb-tests: Reimplement message handlers using srvid abstraction
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13042
This is required for CHECK_PID_SRVID control implementation.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Amitay Isaacs [Wed, 30 Aug 2017 06:29:01 +0000 (16:29 +1000)]
ctdb-tool: Update process-exists command to pass optional srvid
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13042
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Amitay Isaacs [Thu, 21 Sep 2017 05:52:14 +0000 (15:52 +1000)]
ctdb-client: Add client code for control CHECK_PID_SRVID
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13042
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Amitay Isaacs [Wed, 30 Aug 2017 06:18:02 +0000 (16:18 +1000)]
ctdb-daemon: Add implementation of control CHECK_PID_SRVID
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13042
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Amitay Isaacs [Wed, 30 Aug 2017 05:13:53 +0000 (15:13 +1000)]
ctdb-protocol: Add marshalling for control CHECK_PID_SRVID
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13042
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Amitay Isaacs [Wed, 30 Aug 2017 05:13:12 +0000 (15:13 +1000)]
ctdb-protocol: Add marshalling for struct ctdb_pid_srvid
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13042
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Amitay Isaacs [Wed, 30 Aug 2017 05:10:56 +0000 (15:10 +1000)]
ctdb-protocol: Add new control CTDB_CONTROL_CHECK_PID_SRVID
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13042
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Amitay Isaacs [Wed, 30 Aug 2017 03:27:12 +0000 (13:27 +1000)]
ctdb-common: Extend srvid_exists() check to support optional private_data
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13042
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Volker Lendecke [Wed, 20 Sep 2017 00:12:33 +0000 (00:12 +0000)]
testsuite: Fix the 32-bit test build
samba_init_module returns 32-bit. For some reason on my
32-bit lxc "return 0" was converted to something but
NT_STATUS_OK, making initialization fail.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Sep 21 02:49:32 CEST 2017 on sn-devel-144
Volker Lendecke [Tue, 19 Sep 2017 22:26:55 +0000 (15:26 -0700)]
WHATSNEW: Mention code removal from "net" and "rpcclient"
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Volker Lendecke [Tue, 19 Sep 2017 22:17:38 +0000 (15:17 -0700)]
libnet: Remove libnet_samsync
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Volker Lendecke [Tue, 19 Sep 2017 22:14:32 +0000 (15:14 -0700)]
net: Don't depend on libnet_samsync anymore
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Volker Lendecke [Tue, 19 Sep 2017 22:09:05 +0000 (15:09 -0700)]
net: Remove NT4-based vampire keytab
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Volker Lendecke [Tue, 19 Sep 2017 22:06:11 +0000 (15:06 -0700)]
net: Remove NT4-based rpc vampire ldif
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Volker Lendecke [Tue, 19 Sep 2017 22:03:43 +0000 (15:03 -0700)]
net: Remove rpc vampire from NT4 domains
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Volker Lendecke [Tue, 19 Sep 2017 22:02:09 +0000 (15:02 -0700)]
net: Remove rpc samdump
This uses the NT4 replication commands. Samba does not have a server
for this, no tests, and whoever needs to migrate a native domain can
use an old Samba version
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Volker Lendecke [Tue, 19 Sep 2017 21:33:07 +0000 (14:33 -0700)]
rpcclient: Remove sam_sync related commands
These three commands don't use the netlogon credential chain
correctly. They are missing the netlogon_creds_store after the dcerpc
call, so they destroy the correct use of the netlogon creds.
The only valid server for these calls that I know of would be NT4, and
that should be gone long ago.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Richard Sharpe [Sun, 10 Sep 2017 19:50:57 +0000 (12:50 -0700)]
Make sure smbtorture tests can run if someone has set their min protocol above NT1.
This code is SMB1 only, and already modifies
maxprotocol, so this change is appropriate.
Signed-off-by: Richard Sharpe <richard.sharpe@primarydata.com>
Reviewed-by: Jeremy Allison <jra@samba.org>
Jeremy Allison [Fri, 8 Sep 2017 17:13:14 +0000 (10:13 -0700)]
CVE-2017-12163: s3:smbd: Prevent client short SMB1 write from writing server memory to file.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13020
Signed-off-by: Jeremy Allison <jra@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(master): Wed Sep 20 17:06:23 CEST 2017 on sn-devel-144
Stefan Metzmacher [Mon, 12 Dec 2016 05:07:56 +0000 (06:07 +0100)]
CVE-2017-12150: s3:libsmb: only fallback to anonymous if authentication was not requested
With forced encryption or required signing we should also don't fallback.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12997
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Stefan Metzmacher [Tue, 29 Aug 2017 13:35:49 +0000 (15:35 +0200)]
CVE-2017-12150: libcli/smb: add smbXcli_conn_signing_mandatory()
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12997
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Stefan Metzmacher [Tue, 29 Aug 2017 13:24:14 +0000 (15:24 +0200)]
CVE-2017-12150: auth/credentials: cli_credentials_authentication_requested() should check for NTLM_CCACHE/SIGN/SEAL
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12997
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Stefan Metzmacher [Mon, 12 Dec 2016 04:49:46 +0000 (05:49 +0100)]
CVE-2017-12150: libgpo: make use of SMB_SIGNING_REQUIRED in gpo_connect_server()
It's important that we use a signed connection to get the GPOs!
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12997
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Stefan Metzmacher [Fri, 9 Dec 2016 08:26:32 +0000 (09:26 +0100)]
CVE-2017-12150: s3:pylibsmb: make use of SMB_SIGNING_DEFAULT for 'samba.samba3.libsmb_samba_internal'
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12997
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Stefan Metzmacher [Thu, 3 Nov 2016 16:16:43 +0000 (17:16 +0100)]
CVE-2017-12150: s3:lib: get_cmdline_auth_info_signing_state smb_encrypt SMB_SIGNING_REQUIRED
This is an addition to the fixes for CVE-2015-5296.
It applies to smb2mount -e, smbcacls -e and smbcquotas -e.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12997
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Stefan Metzmacher [Tue, 29 Aug 2017 15:06:21 +0000 (17:06 +0200)]
CVE-2017-12150: s3:popt_common: don't turn a guessed username into a specified one
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12997
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Stefan Metzmacher [Sat, 17 Dec 2016 09:36:49 +0000 (10:36 +0100)]
CVE-2017-12151: s3:libsmb: make use of cli_state_is_encryption_on()
This will keep enforced encryption across dfs referrals.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12996
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Stefan Metzmacher [Mon, 14 Aug 2017 10:13:18 +0000 (12:13 +0200)]
CVE-2017-12151: s3:libsmb: add cli_state_is_encryption_on() helper function
This allows to check if the current cli_state uses encryption
(either via unix extentions or via SMB3).
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12996
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Andrew Bartlett [Mon, 11 Sep 2017 01:53:19 +0000 (13:53 +1200)]
dsdb: Only trigger a re-index once per @INDEXLIST modification
A modify of both @INDEXLIST and @ATTRIBUTES will still trigger two re-index passes
but that is a task for later.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=9527
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Sep 20 12:29:49 CEST 2017 on sn-devel-144
Andrew Bartlett [Fri, 25 Aug 2017 05:37:05 +0000 (17:37 +1200)]
selftest: sort dbcheck output to avoid sort order impacting results
The GUID index code will change the returned results order
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Andrew Bartlett [Tue, 29 Aug 2017 02:19:22 +0000 (14:19 +1200)]
s4-dnsserver: Check for too many DNS results
If we had this check in when the wildcard DNS tests were written, we would have
noticed that the name needed to be escaped (see previous commit).
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12994
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Andrew Bartlett [Mon, 28 Aug 2017 23:48:46 +0000 (11:48 +1200)]
s4-dnsserver: Always encode user-supplied names when looking up DNS records
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12994
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Garming Sam [Sun, 17 Sep 2017 21:55:21 +0000 (09:55 +1200)]
kcc: Remove unused, untested KCC code
This code tries to implement the full KCC algorithm, but never
actually worked correctly.
Removing this doesn't affect the full-mesh KCC. This code only
attempted to calculate a graph using the "proper" algorithm, though it
neglected to write its results back into the database. The full-mesh
calculation occurs elsewhere.
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Wed Sep 20 06:28:07 CEST 2017 on sn-devel-144
Andrew Bartlett [Thu, 14 Sep 2017 03:07:10 +0000 (15:07 +1200)]
ldap_server: Plumb ldb error string from a failed connect to ldapsrv_terminate_connection()
However, do not plumb it to the client-seen error string, as it could contain server paths.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Andrew Bartlett [Thu, 14 Sep 2017 03:02:36 +0000 (15:02 +1200)]
samdb: Rework samdb_connect_url() to return LDB error code and an error string
This allows debugging of why the LDB failed to start up.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Andrew Bartlett [Thu, 14 Sep 2017 02:44:39 +0000 (14:44 +1200)]
ldb: Release 1.2.3
* Bug #13033 LDB open with LDB_FLG_RDONLY can cause the database
to fail to open
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13033
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Gary Lockyer [Wed, 13 Sep 2017 23:37:41 +0000 (11:37 +1200)]
ldb: Add tests for read only behaviour
As the kernel is no longer enforcing the read-only DB
add some tests.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13033
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Andrew Bartlett [Sun, 17 Sep 2017 22:43:32 +0000 (10:43 +1200)]
ldb_tdb: Change ltdb_connect() NOT to request a kernel-level read only TDB
We support opening and LDB multiple times in a process, but do not support this in tdb.
As we can open the ldb with different flags, we must ensure a later read-write
open is possible.
Additionally, a read-only TDB will refuse the all-record lock, preventing
the ldb from even loading.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13033
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Andrew Bartlett [Thu, 14 Sep 2017 03:01:39 +0000 (15:01 +1200)]
ldb_tdb: Give a debug message as well as setting the error string if prepare_commit() fails
This is a serious condition, and should be logged.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13033
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Andrew Bartlett [Thu, 14 Sep 2017 02:04:51 +0000 (14:04 +1200)]
ldb_tdb: Map TDB error codes into LDB error codes in ltdb_lock_read()
The ltdb_lock_read() routine did not return an LDB error code, but -1.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13033
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Martin Schwenke [Wed, 6 Sep 2017 08:11:41 +0000 (18:11 +1000)]
ctdb-tools: Fix a typo for a talloc context
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Tue Sep 19 17:31:18 CEST 2017 on sn-devel-144
Martin Schwenke [Tue, 4 Jul 2017 02:11:20 +0000 (12:11 +1000)]
ctdb-tools: Add debug to ctdb_killtcp
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Tue, 4 Jul 2017 04:02:14 +0000 (14:02 +1000)]
ctdb-tools: Move special case of 0 connections into computation
This avoids other potential users from unnecessarily setting up file
descriptors and such.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Fri, 30 Jun 2017 09:50:43 +0000 (19:50 +1000)]
ctdb-tools: Rework killtcp logic into a tevent_req-based computation
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Thu, 29 Jun 2017 06:35:06 +0000 (16:35 +1000)]
ctdb-tools: New function ctdb_kill_tcp_init()
This replaces ctdb_killtcp(), which did the initialisation inside a
loop. The new logic is inverted, making it more natural.
The variable containing all the state is called "state" in
anticipation of the next commit that will convert this to a tevent_req
computation. This will mean less churn.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Thu, 14 Sep 2017 05:19:43 +0000 (15:19 +1000)]
ctdb-tools: Improve error handling
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Fri, 30 Jun 2017 07:12:48 +0000 (17:12 +1000)]
ctdb-tools: Drop global variable prog
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Thu, 29 Jun 2017 05:57:19 +0000 (15:57 +1000)]
ctdb-tools: Use db_hash in ctdb_killtcp
One less use of trbt_tree_t. The code is easier to read and is
significantly smaller.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Thu, 29 Jun 2017 04:46:31 +0000 (14:46 +1000)]
ctdb-tools: Use ctdb_connection and ctdb_connection_list structs
Also use new connection and sock addr utilities.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Mon, 4 Sep 2017 07:01:05 +0000 (17:01 +1000)]
ctdb-protocol: Add ctdb_connection_list utilities
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Tue, 5 Sep 2017 00:52:58 +0000 (10:52 +1000)]
ctdb-protocol: Add marshalling for ctdb_connection_list
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Mon, 4 Sep 2017 06:48:40 +0000 (16:48 +1000)]
ctdb-protocol: Add new data structure ctdb_connection_list
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Mon, 4 Sep 2017 06:41:30 +0000 (16:41 +1000)]
ctdb-protocol: Add ctdb_connection utilities
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Wed, 13 Sep 2017 06:18:29 +0000 (16:18 +1000)]
ctdb-protocol: Factor out static function ctdb_sock_addr_cmp_family()
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Mon, 11 Sep 2017 06:39:38 +0000 (16:39 +1000)]
ctdb-protocol: Add ctdb_sock_addr_from_string()
This and the supporting functions duplicate functionality (parse_ip()
and parse_ip_port()) from common/system_util.c. The old functions
will be removed at a later time.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Mon, 11 Sep 2017 05:00:10 +0000 (15:00 +1000)]
ctdb-protocol: Optionally print port for address printing functions
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Fri, 30 Jun 2017 05:11:04 +0000 (15:11 +1000)]
ctdb-protocol: Add utility function ctdb_sock_addr_to_buf()
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Fri, 2 Jun 2017 06:17:22 +0000 (16:17 +1000)]
ctdb-protocol: Add ctdb_sock_addr_port() and sock_addr_set_port()
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Mon, 4 Sep 2017 06:20:55 +0000 (16:20 +1000)]
ctdb-protocol: Add server and client aliases in ctdb_connection
The current code is ambiguous in its use of src and dst. This allows
new code to use server and client for clarity.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Martin Schwenke [Tue, 19 Sep 2017 04:54:26 +0000 (14:54 +1000)]
ctdb-common: Initialise socket addresses before reading into them
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Mon, 4 Sep 2017 06:00:48 +0000 (16:00 +1000)]
ctdb-build: Split protocol-util as a separate subsystem
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Mon, 4 Sep 2017 06:00:27 +0000 (16:00 +1000)]
ctdb-build: Fix dependency for ctdbd
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Günther Deschner [Thu, 29 Sep 2016 06:41:01 +0000 (08:41 +0200)]
s4-torture: move lease break handler outside the lease testsuite.
Guenther
Signed-off-by: Guenther Deschner <gd@samba.org>
Pair-Programmed-With: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Tue Sep 19 09:36:40 CEST 2017 on sn-devel-144
Martin Schwenke [Mon, 18 Sep 2017 22:33:11 +0000 (08:33 +1000)]
selftest: Avoid a build started just before midnight failing
It looks like commit
070f24bc9d2 was incomplete because it doesn't
match on the trailing fullstop and newline.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Tue Sep 19 05:32:22 CEST 2017 on sn-devel-144
Martin Schwenke [Mon, 18 Sep 2017 06:36:32 +0000 (16:36 +1000)]
Revert "ctdb-daemon: Don't explicitly stop monitoring during shutdown"
This reverts commit
19318d28354190a9cfafa1696e41c90351f85af9.
With this commit, a shutdown that occurs while the startup event is
running can cause an abort because the startup callback will try to
decrease the run state from SHUTDOWN to RUNNING.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Mon, 18 Sep 2017 06:36:08 +0000 (16:36 +1000)]
Revert "ctdb-daemon: Remove unused function ctdb_stop_monitoring()"
This reverts commit
b119104267635d96b1f99a2e98d35438841585bf.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Tim Beale [Wed, 23 Aug 2017 00:45:09 +0000 (12:45 +1200)]
selftest: Add some tests for linked attribute conflicts
Currently we have tests that check we can resolve object conflicts, but
these don't test anything related to conflicting linked attributes.
This patch adds some basic tests that checks that Samba can resolve
conflicting linked attributes.
This highlights some problems with Samba, as the following tests
currently fail:
- test_conflict_single_valued_link: Samba currently can't resolve a
conflicting targets for a single-valued linked attribute - the
replication exits with an error.
- test_link_deletion_conflict: If 2 DCs add the same linked attribute,
currently when they resolve this conflict the RMD_VERSION for the
linked attribute incorrectly gets incremented. This means the version
numbers get out of step and subsequent changes to the linked attribute
can be dropped/ignored.
- test_full_sync_link_conflict: fails for the same reason as above.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Autobuild-User(master): Garming Sam <garming@samba.org>
Autobuild-Date(master): Mon Sep 18 09:56:41 CEST 2017 on sn-devel-144
Tim Beale [Tue, 15 Aug 2017 04:15:14 +0000 (16:15 +1200)]
getncchanges.c: max_links calculation didn't work well in some cases
The max_links calculation didn't work particularly well if max_links was
set to a value lower than max_objects.
As soon as repl_chunk->object_count exceeded repl_chunk->max_links, the
chunk would be deemed full, even if there was only one link to send (or
even worse, no links to send). For example, if max_objects=100 and
max_links=10, then it would send back chunks of 10 objects (or 9 objects
and 1 link).
I believe the historic reason this logic exists is to avoid overfilling
the response message. It's hard to tell what the appropriate limit would
be because the total message size would depend on how many attributes
each object has.
I couldn't think of logic that would be suitable for all cases. I toyed
with the idea of working out a percentage of how full the message is.
However, adjusting the max_links doesn't really make sense when the
settings are small enough, e.g. max_objects=100 and max_links=100 is
never going to overfill the message, so there's no reason to alter the
values.
In the end I went with:
- If the user is using non-default values, just use those.
- In the default value case, just use the historic calculation
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Tim Beale [Fri, 11 Aug 2017 03:39:35 +0000 (15:39 +1200)]
replmd: Avoid duplicated debug/warnings
We display warnings if a target object is missing but it's still OK to
continue the replication. Currently we need to check the target twice -
once to verify it when we first receive it, and once when we actually
commit it (we can't skip the 2nd check altogether because in the join
case, they could occur quite far apart).
One annoying side-effect is we get the same warning message coming out
twice in these special cases.
In the cases where we're checking the dsdb_repl_flags, we can actually
just bypass the verification checks for the target object (if it doesn't
exist we still continue anyway). This may save us a tiny bit of
unnecessary work.
For cross-partition links, we can limit logging these warnings to when
the objects are actually being committed. This avoids spurious warnings
in the join case (i.e. we receive the link before we receive the target
object's partition, but we have received all partitions by the time we
actually commit the objects).
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Tim Beale [Fri, 11 Aug 2017 01:53:31 +0000 (13:53 +1200)]
replmd: Allow missing targets if GET_TGT has already been set
While running the selftests, I noticed a case where DC replication
unexpectedly sends a linked attribute for a deleted object (created in
the drs.ridalloc_exop tests). The problem is due to the
msDS-NC-Replica-Locations attribute, which is a (known) one-way link.
Because it is a one-way link, when the test demotes the DC and deletes
the link target, there is no backlink to delete the link from the source
object.
After much debate and head-scratching, we decided that there wasn't an
ideal way to resolve this problem. Any automated intervention could
potentially do the wrong thing, especially if the link spans partitions.
Running dbcheck will find this problem and is able to fix it (providing
the deleted object is still a tombstone). So the recommendation is to
run dbcheck on your DCs every 6 months (or more frequently if using a
lower tombstone lifetime setting).
However, it does highlight a problem with the current GET_TGT
implementation. If the tombstone object had been expunged and you
upgraded to 4.8, then you would be stuck - replication would fail
because the target object can't be resolved, even with GET_TGT, and
dbcheck would not be able to fix the hanging link. The solution is to
not fail the replication for an unknown target if GET_TGT has already
been set (i.e. the dsdb_repl_flags contains
DSDB_REPL_FLAG_TARGETS_UPTODATE).
It's debatable whether we should add a hanging link in this case or
ignore/drop the link. Some cases to consider:
- If you're talking to a DC that still sends all the links last, you
could still get object deletion between processing the source object's
links and sending the target (GET_TGT just restarts the replication
cycle from scratch). Adding a hanging link in this case would be
incorrect and would add spurious information to the DB.
- Suppose there's a bug in Samba that incorrectly results in an object
disappearing. If other DCs then remove any links that pointed to that
object, it makes recovering from the problem harder. However, simply
ignoring the link shouldn't result in data loss, i.e. replication won't
remove the existing link information from other DCs. Data loss in this
case would only occur if a new DC were brought online, or if it were a
new link that was affected.
Based on this, I think ignoring the link does the least harm.
This problem also highlights that we should really be using the same
logic in both the unknown target and the deleted target cases.
Combining the logic and moving it into a common
replmd_allow_missing_target() function fixes the problem. (This also has
the side-effect of fixing another logic flaw - in the deleted object
case we would unnecessarily retry with GET_TGT if the target object was
in another partition. This is pointless work, because GET_TGT won't
resolve the target).
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Tim Beale [Fri, 11 Aug 2017 04:08:15 +0000 (16:08 +1200)]
getncchanges.c: Support GET_TGT better with large numbers of links
A source object can potentially link to thousands of target objects.
We have to be careful not to overfill the GetNCChanges response message
with more data than it's possible to send. We also don't want the client
to timeout while we're busy checking the linked attributes. The GET_TGT
support added so far is fairly dumb - this patch extends it to better
handle larger numbers of links.
To do so, this extends the repl_chunk usage so that it also works out if
the current chunk is full of links. Now as soon as the chunk is full of
either links or objects, we stop and send it back.
These changes now mean that we need to also check:
- that all the links for the last source object in the previous chunk
have been sent, before we move on and send the next object. This only
takes effect when immediate_link_sync is configured. It also means
that a chunk in the middle of the replication cycle can now contain
only links, and no objects.
- when GET_TGT is used, we only send back the links that we've verified
the target object for. i.e. if we stop checking links because we timed
out, we only send back the links whose targets were checked.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Garming Sam [Tue, 8 Aug 2017 04:27:18 +0000 (16:27 +1200)]
getncchanges.c: Refactor to track more state using repl_chunk
To prepare GET_TGT to deal with a large number of links better, there
is now a 'repl_chunk' struct to help keep track of all the factors
relating to the current chunk of replication data (i.e. how many
objects/links we can send and how many we've already processed). This
means we can have a consistent way of working out whether the current
chunk is full (whether that be due to objects, links, or just too much
time taken).
This patch should not alter functionality. This is just a refactor to
add the basic framework, which will be used in the next patch.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Tim Beale [Tue, 15 Aug 2017 00:18:02 +0000 (12:18 +1200)]
getncchanges.py: Add a multi-valued linked attribute test
Add a test where a source object links to multiple different targets.
First we do the replication without GET_TGT and check that the server
can handle sending a chunk containing only links (in the middle of the
replication). Then we repeat the replication forcing GET_TGT to be used.
To avoid having to create 1500 objects/links, I've lowered the 'max
link sync' setting on the vampire_dc testenv to 250.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Tim Beale [Mon, 24 Jul 2017 02:43:54 +0000 (14:43 +1200)]
getncchanges.py: Add a test for dropped cross-partition links
Samba would drop linked attributes that span partitions if it didn't
know about the target object. This patch adds a test that exposes the
problem.
I've re-used the code from the previous re-animation test to do this.
I've also added a very basic DcConnection helper class that basically
stores the connection state information the drs_base.py uses for
replication. This allows us to switch the DC we want to replicate from
easily. This approach could potentially be retro-fitted to some of the
existing test cases, as it allows us to test both the DRS client code
and server code at the same time.
Note this test case relates to the code change for commit
fae5df891c11f642cb.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12972
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Tim Beale [Thu, 20 Jul 2017 05:06:14 +0000 (17:06 +1200)]
getncchanges.py: Add test for replicating reanimated objects
Reading between the lines, this scenario seems to be the main reason
that Microsoft added the GET_TGT flag. MS AD can handle getting links
for unknown targets OK, but if it receives links for a deleted/recycled
target then it would tend to drop the received links. Samba client also
used to drop the links if talking to a Microsoft DC (or a Samba server
with GET_TGT support).
The specific scenario is the client side already knows about a deleted
object. That object is then re-animated and used as the target for a
linked attribute. *Then* the target object gets updated again so it gets
sent in a later replication chunk to the linked attribute, i.e. the
client receives the link before it learns that the target object has
been re-animated.
In this test we're interested in particular at how the client behaves
when it receives a linked attribute for a deleted object. (It *should*
retry with GET_TGT to make sure the target is up-to-date. However, it
was just dropping the linked attribute).
To exercise the client-side, we disable replication, setup the
links/objects on one DC the way we want them, then force a replication
to the second DC. We then check that when we query each DC, they both
tell us about the links/objects we're expecting (i.e. no links got
lost).
Note that this wasn't a problem with older versions of Samba-to-Samba
because sending the links last guaranteed that the target objects were
always up-to-date.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Tim Beale [Tue, 22 Aug 2017 22:23:10 +0000 (10:23 +1200)]
drs: Add basic GET_TGT support
This adds basic DRS_GET_TGT support. If the GET_TGT flag is specified
then the server will use the object cache to store the objects it sends
back. If the target object for a linked attribute is not in the cache
(i.e. it has not been sent already), then it is added to the response
message.
Note that large numbers of linked attributes will not be handled well
yet - the server could potentially try to send more than will fit in a
single repsonse message.
Also note that the client can sometimes set the GET_TGT flag even if the
server is still sending the links last. In this case, we know the client
supports GET_TGT so it's safe to send the links interleaved with the
source objects (the alternative of fetching the target objects but not
sending the links until last doesn't really make any sense).
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Tim Beale [Tue, 18 Jul 2017 23:38:55 +0000 (11:38 +1200)]
getncchanges.py: Add tests for object deletion during replication
Add tests that delete the source and target objects for linked
attributes in the middle of a replication cycle.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Tim Beale [Mon, 17 Jul 2017 02:04:38 +0000 (14:04 +1200)]
getnc_exop.py: Extend EXOP_REPL_OBJ test case to use GET_TGT
We already check that when we use GET_ANC that we still only receive a
single object when EXOP_REPL_OBJ is used. This extends the test to also
check that only a single object is returned when GET_TGT is used.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>