samba.git
6 years agoldb: Add an unused objectGUID to each record in SearchTests
Andrew Bartlett [Fri, 25 Aug 2017 00:06:48 +0000 (12:06 +1200)]
ldb: Add an unused objectGUID to each record in SearchTests

This will then be used by the GUID index tests.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Duplicate values are no longer permitted in the index
Andrew Bartlett [Mon, 21 Aug 2017 23:55:01 +0000 (11:55 +1200)]
ldb_tdb: Duplicate values are no longer permitted in the index

By removing the qsort() we avoid work.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Reduce memory consumption in list_intersect()
Andrew Bartlett [Mon, 21 Aug 2017 23:17:56 +0000 (11:17 +1200)]
ldb_tdb: Reduce memory consumption in list_intersect()

We will never have more results than is in either list or list2.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Use the binary search more efficiently in list_intersect()
Andrew Bartlett [Mon, 21 Aug 2017 23:16:56 +0000 (11:16 +1200)]
ldb_tdb: Use the binary search more efficiently in list_intersect()

This change ensures we walk the short list and look up into the longer of the two lists.

ltdb_dn_list_find_val() will do a binary search for the GUID case.

Before GUID indexes this was O(n*m), now it is O(n*log(m)).

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Use a binary search to speed up ltdb_dn_list_find_val()
Andrew Bartlett [Mon, 21 Aug 2017 03:51:19 +0000 (15:51 +1200)]
ldb_tdb: Use a binary search to speed up ltdb_dn_list_find_val()

This only works if we have the GUID index format, as otherwise these are unsorted.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Rework list_union to not return duplicates, and keep sort order
Andrew Bartlett [Mon, 21 Aug 2017 23:07:45 +0000 (11:07 +1200)]
ldb_tdb: Rework list_union to not return duplicates, and keep sort order

This allows the binary search to still operate on the list, even after
a or operator in the search expression

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Sort inputs to list_union()
Andrew Bartlett [Mon, 21 Aug 2017 23:51:30 +0000 (11:51 +1200)]
ldb_tdb: Sort inputs to list_union()

This allows us to merge the lists finding common values.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: sort GUID index list at add time by inserting in sorted order
Andrew Bartlett [Mon, 21 Aug 2017 03:35:32 +0000 (15:35 +1200)]
ldb_tdb: sort GUID index list at add time by inserting in sorted order

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agobinsearch.h: Re-licence under LGPLv3 per agreement of the copyright holders
Andrew Bartlett [Thu, 21 Sep 2017 00:35:45 +0000 (12:35 +1200)]
binsearch.h: Re-licence under LGPLv3 per agreement of the copyright holders

Documented in mails to contributing@samba.org.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
6 years agoldb_tdb: Change error code on unique index violation
Andrew Bartlett [Mon, 21 Aug 2017 02:25:46 +0000 (14:25 +1200)]
ldb_tdb: Change error code on unique index violation

The LDB_ERR_ENTRY_ALREADY_EXISTS error code is detected in repl_meta_data as indicating
that the DN exists, and that a conflict record should be created.

This is really a constraint violation, not a duplicate record.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Re-add of both existing DN and GUID must gives LDB_ERR_ENTRY_ALREADY_EXISTS
Andrew Bartlett [Wed, 13 Sep 2017 00:37:51 +0000 (12:37 +1200)]
ldb_tdb: Re-add of both existing DN and GUID must gives LDB_ERR_ENTRY_ALREADY_EXISTS

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Add unique index checking for @IDXDN
Andrew Bartlett [Mon, 21 Aug 2017 02:24:44 +0000 (14:24 +1200)]
ldb_tdb: Add unique index checking for @IDXDN

This will give us errors earlier if the index code becomes broken

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Improve debugging in ltdb_modify_index_dn() on casefold failure
Andrew Bartlett [Mon, 21 Aug 2017 00:59:50 +0000 (12:59 +1200)]
ldb_tdb: Improve debugging in ltdb_modify_index_dn() on casefold failure

This is unlikely, but when it happens it will be really painful to debug.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Add improved error strings on ltdb_key_dn_from_idx() failure
Andrew Bartlett [Sun, 20 Aug 2017 23:17:59 +0000 (11:17 +1200)]
ldb_tdb: Add improved error strings on ltdb_key_dn_from_idx() failure

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Read from @INDEXLIST or an override if we are using a GUID index
Andrew Bartlett [Tue, 15 Aug 2017 03:57:57 +0000 (15:57 +1200)]
ldb_tdb: Read from @INDEXLIST or an override if we are using a GUID index

This allows all the previous patches to be enabled.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Optionally use GUID index values a direct TDB keys
Andrew Bartlett [Thu, 10 Aug 2017 05:05:37 +0000 (17:05 +1200)]
ldb_tdb: Optionally use GUID index values a direct TDB keys

This connects the GUID based index records to GUID based TDB keys.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Trust the BASE and ONELEVEL index
Andrew Bartlett [Mon, 28 Aug 2017 21:59:54 +0000 (09:59 +1200)]
ldb_tdb: Trust the BASE and ONELEVEL index

This avoids re-checking the fetched DN against the scope

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Add ltdb_idx_to_key() and use it in ltdb_index_filter()
Andrew Bartlett [Wed, 16 Aug 2017 19:15:50 +0000 (07:15 +1200)]
ldb_tdb: Add ltdb_idx_to_key() and use it in ltdb_index_filter()

This will allow a common point to parse index records into a TDB key,
allowing them to be a GUID or DN in the future

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Do not add an index for GUID_index_attribute
Andrew Bartlett [Thu, 17 Aug 2017 00:44:34 +0000 (12:44 +1200)]
ldb_tdb: Do not add an index for GUID_index_attribute

This would be pointless and we no longer query for it.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Do not query an index on the GUID_index_attribute
Andrew Bartlett [Thu, 21 Sep 2017 10:21:36 +0000 (22:21 +1200)]
ldb_tdb: Do not query an index on the GUID_index_attribute

The objectGUID (or similar) is already the record key, there is
no need to index it to itself.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Optionally use GUID index in ltdb_search_dn1()
Andrew Bartlett [Thu, 10 Aug 2017 05:12:30 +0000 (17:12 +1200)]
ldb_tdb: Optionally use GUID index in ltdb_search_dn1()

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Use the objectGUID (or similar) as the TDB key in ltdb_key_msg()
Andrew Bartlett [Thu, 10 Aug 2017 05:11:13 +0000 (17:11 +1200)]
ldb_tdb: Use the objectGUID (or similar) as the TDB key in ltdb_key_msg()

When we have the full ldb_message we can read the objectGUID as the TDB key

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Use ltdb_key_msg() in ltdb_delete_noindex()
Andrew Bartlett [Thu, 17 Aug 2017 00:53:34 +0000 (12:53 +1200)]
ldb_tdb: Use ltdb_key_msg() in ltdb_delete_noindex()

This allows the optional use of GUID based TDB key.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Add mem_ctx to ltdb_key_dn() and ltdb_key_msg()
Andrew Bartlett [Mon, 14 Aug 2017 03:47:15 +0000 (15:47 +1200)]
ldb_tdb: Add mem_ctx to ltdb_key_dn() and ltdb_key_msg()

This follows modern Samba coding style where memory
returned is allocated on a supplied memory context.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Check version number on index
Andrew Bartlett [Thu, 17 Aug 2017 05:27:23 +0000 (17:27 +1200)]
ldb_tdb: Check version number on index

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Add an index shortcut for a <GUID= DN
Andrew Bartlett [Mon, 14 Aug 2017 04:13:42 +0000 (16:13 +1200)]
ldb_tdb: Add an index shortcut for a <GUID= DN

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Add a function to get the GUID key for a DN
Andrew Bartlett [Fri, 11 Aug 2017 06:09:01 +0000 (18:09 +1200)]
ldb_tdb: Add a function to get the GUID key for a DN

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Add a function to take a GUID and make the TDB_DATA key
Andrew Bartlett [Tue, 15 Aug 2017 22:44:34 +0000 (10:44 +1200)]
ldb_tdb: Add a function to take a GUID and make the TDB_DATA key

This allows us to format the TDB key as DN=GUID=f7c953ee-cf9c-433f-b423-21ce04d09591
and so be compatible with an un-indexed search and a re-index with an old ldb.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Load the syntax of the GUID index attr during ltdb_cache_load()
Andrew Bartlett [Tue, 15 Aug 2017 22:42:40 +0000 (10:42 +1200)]
ldb_tdb: Load the syntax of the GUID index attr during ltdb_cache_load()

This allows us to use the ldif_write function later to create a string GUID for the TDB key.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: add control points for the new GUID index mode
Andrew Bartlett [Tue, 15 Aug 2017 03:33:24 +0000 (15:33 +1200)]
ldb_tdb: add control points for the new GUID index mode

The @IDXGUID attribute in the @INDEXLIST will be objectGUID
in Samba.

The @IDX_DN_GUID attribute in the @INDEXLIST will be GUID
in Samba.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Do not directly assign DN into the index result list
Andrew Bartlett [Fri, 11 Aug 2017 05:51:40 +0000 (17:51 +1200)]
ldb_tdb: Do not directly assign DN into the index result list

With the GUID index option, the values in the index result list may
not be a DN but the objectGUID.  We look up the @IDXDN index with the
case-folded DN to get that if required.

We re-use the code from the SCOPE_BASE search do avoid duplication
and for reliablity.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
6 years agoldb_tdb: Pass ltdb_private to ltdb_dn_list_load()
Andrew Bartlett [Fri, 11 Aug 2017 01:26:30 +0000 (13:26 +1200)]
ldb_tdb: Pass ltdb_private to ltdb_dn_list_load()

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Give LDB_ERR_CONSTRAINT_VIOLATION when a duplicate GUID index is detected
Andrew Bartlett [Fri, 25 Aug 2017 10:21:26 +0000 (22:21 +1200)]
ldb_tdb: Give LDB_ERR_CONSTRAINT_VIOLATION when a duplicate GUID index is detected

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Give LDB_ERR_ENTRY_ALREADY_EXISTS when a duplicate DN index is detected
Andrew Bartlett [Fri, 25 Aug 2017 10:21:00 +0000 (22:21 +1200)]
ldb_tdb: Give LDB_ERR_ENTRY_ALREADY_EXISTS when a duplicate DN index is detected

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Add/remove a GUID index of the DN during ltdb_index_add_all()/ltdb_index_del...
Andrew Bartlett [Fri, 11 Aug 2017 00:25:42 +0000 (12:25 +1200)]
ldb_tdb: Add/remove a GUID index of the DN during ltdb_index_add_all()/ltdb_index_delete()

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Split ltdb_index_onelevel() into a helper function
Andrew Bartlett [Mon, 21 Aug 2017 00:58:58 +0000 (12:58 +1200)]
ldb_tdb: Split ltdb_index_onelevel() into a helper function

This will allow the code to be re-used for storing the DN->GUID index

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Prepare to handle rename with GUID index by using ltdb_search_base()
Andrew Bartlett [Thu, 10 Aug 2017 23:33:17 +0000 (11:33 +1200)]
ldb_tdb: Prepare to handle rename with GUID index by using ltdb_search_base()

This will allow use of a GUID TDB key in the future.  While ltdb_search_base()
might be marginally slower than tdb_exists(), no allocation is done for the
attributes or DN, and renmaes are not a very common operation.

This allows a check if the target DN exists even when the direct DN -> key
link is broken.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Implement ltdb_search_base() for a GUID index
Andrew Bartlett [Tue, 15 Aug 2017 02:39:08 +0000 (14:39 +1200)]
ldb_tdb: Implement ltdb_search_base() for a GUID index

The GUID index case can not directly use ltdb_key_dn() and tdb_exists() to
show that a records exists.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Optionally store a GUID as the index record
Andrew Bartlett [Tue, 15 Aug 2017 02:18:19 +0000 (14:18 +1200)]
ldb_tdb: Optionally store a GUID as the index record

This allows, when enabled, the index record to contain (say) the objectGUID, not the DN
of the record.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Optionally use a GUID index key in ltdb_dn_list_find_msg()
Andrew Bartlett [Tue, 15 Aug 2017 02:13:32 +0000 (14:13 +1200)]
ldb_tdb: Optionally use a GUID index key in ltdb_dn_list_find_msg()

This function is used to find an existing index value and this
change allows it to find the value by GUID rather than by DN once
the GUID index is enabled.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Pass the full ldb_message to ldb index funtions
Andrew Bartlett [Tue, 15 Aug 2017 01:08:42 +0000 (13:08 +1200)]
ldb_tdb: Pass the full ldb_message to ldb index funtions

This allows the objectGUID, rather than the DN, to be the index key

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Delete a successful tdb_store on index add fail in ltdb_add_internal()
Andrew Bartlett [Fri, 25 Aug 2017 10:33:15 +0000 (22:33 +1200)]
ldb_tdb: Delete a successful tdb_store on index add fail in ltdb_add_internal()

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: modify ltdb_delete_noindex() to take a struct ldb_message
Andrew Bartlett [Wed, 16 Aug 2017 21:08:34 +0000 (09:08 +1200)]
ldb_tdb: modify ltdb_delete_noindex() to take a struct ldb_message

This will make it easier to delete records with the GUID TDB key

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Write GUID index values as version 3
Andrew Bartlett [Thu, 10 Aug 2017 05:04:23 +0000 (17:04 +1200)]
ldb_tdb: Write GUID index values as version 3

Nothing reads these currently, but we should refuse to load a mixed up index
in the future

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Refuse to load a GUID index that is not a multiple of 16 bytes
Andrew Bartlett [Fri, 25 Aug 2017 03:33:14 +0000 (15:33 +1200)]
ldb_tdb: Refuse to load a GUID index that is not a multiple of 16 bytes

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Read GUID index values as one packed LDB attribute
Andrew Bartlett [Thu, 17 Aug 2017 05:26:35 +0000 (17:26 +1200)]
ldb_tdb: Read GUID index values as one packed LDB attribute

This packing should be more efficient to read than the ldb_pack format.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Store GUID index values in one packed ldb attribute
Andrew Bartlett [Thu, 10 Aug 2017 05:02:14 +0000 (17:02 +1200)]
ldb_tdb: Store GUID index values in one packed ldb attribute

This should make them more memory efficient

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Move constants into ldb_tdb.h
Andrew Bartlett [Fri, 1 Sep 2017 08:04:43 +0000 (20:04 +1200)]
ldb_tdb: Move constants into ldb_tdb.h

This helps ensure we keep these all in sync.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: replace strange dn_list_cmp() in index code
Andrew Bartlett [Thu, 10 Aug 2017 05:00:48 +0000 (17:00 +1200)]
ldb_tdb: replace strange dn_list_cmp() in index code

This replaces dn_list_cmp() with functions that do not attempt to
to care about string termination.  All index values are case sensitive
and correctly length-bound already, even for a DN index
so just use a length check and memcmp()

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Do not allow a modification of the GUID_index_attribute (objectGUID)
Andrew Bartlett [Thu, 17 Aug 2017 00:42:25 +0000 (12:42 +1200)]
ldb_tdb: Do not allow a modification of the GUID_index_attribute (objectGUID)

This would totally break our index scheme if this could be modified.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Add GUID_index_attribute to ltdb_private
Andrew Bartlett [Thu, 10 Aug 2017 05:13:48 +0000 (17:13 +1200)]
ldb_tdb: Add GUID_index_attribute to ltdb_private

This will be used to determine if we are in GUID index mode

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Provide struct ltdb_private to index routines
Andrew Bartlett [Thu, 10 Aug 2017 04:23:33 +0000 (16:23 +1200)]
ldb_tdb: Provide struct ltdb_private to index routines

This will make it easier to switch the GUID index mode on and off

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Use a more complete error mapping in ltdb_search_key()
Andrew Bartlett [Wed, 16 Aug 2017 00:46:57 +0000 (12:46 +1200)]
ldb_tdb: Use a more complete error mapping in ltdb_search_key()

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Add ltdb_search_key()
Andrew Bartlett [Thu, 10 Aug 2017 04:06:08 +0000 (16:06 +1200)]
ldb_tdb: Add ltdb_search_key()

This allows us to slowly split out the tdb key in the DB from being the DN

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Use ltdb_key_msg() in re_index()
Andrew Bartlett [Thu, 10 Aug 2017 04:01:12 +0000 (16:01 +1200)]
ldb_tdb: Use ltdb_key_msg() in re_index()

This will allow changing to a GUID tdb key in the future

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: provide ldb_key_dn() and ldb_key_msg()
Andrew Bartlett [Thu, 10 Aug 2017 02:31:18 +0000 (14:31 +1200)]
ldb_tdb: provide ldb_key_dn() and ldb_key_msg()

This will in time allow us to generate a TDB key from
the msg, eg from an objectGUID.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Add helper function ltdb_search_and_return_base()
Andrew Bartlett [Tue, 15 Aug 2017 05:21:28 +0000 (17:21 +1200)]
ldb_tdb: Add helper function ltdb_search_and_return_base()

This avoids an extra DB lookup for the base, when that is the only
record we will return, and avoids going into the index code for
a base search, as that won't work for special DNs once the GUID
index mode is enabled.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoldb_tdb: Make ldb_match_message() available to ldb_tdb
Andrew Bartlett [Mon, 28 Aug 2017 02:55:00 +0000 (14:55 +1200)]
ldb_tdb: Make ldb_match_message() available to ldb_tdb

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
6 years agoctdb-tests: Add tests for client with multiple connections
Amitay Isaacs [Fri, 22 Sep 2017 04:17:59 +0000 (14:17 +1000)]
ctdb-tests: Add tests for client with multiple connections

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13042

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Fri Sep 22 20:58:46 CEST 2017 on sn-devel-144

6 years agoctdb-tests: Add support for multiple ctdb connections in dummy_client
Amitay Isaacs [Fri, 22 Sep 2017 04:14:00 +0000 (14:14 +1000)]
ctdb-tests: Add support for multiple ctdb connections in dummy_client

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13042

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
6 years agoctdb-tests: Check all connections from a process in CHECK_PID_SRVID control
Amitay Isaacs [Fri, 22 Sep 2017 04:04:50 +0000 (14:04 +1000)]
ctdb-tests: Check all connections from a process in CHECK_PID_SRVID control

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13042

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
6 years agoctdb-daemon: Check all connections from a process in CHECK_PID_SRVID control
Amitay Isaacs [Fri, 22 Sep 2017 03:52:09 +0000 (13:52 +1000)]
ctdb-daemon: Check all connections from a process in CHECK_PID_SRVID control

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13042

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
6 years agoselftest: Also run smbtorture smb2.compound with aio enabled
Christof Schmitt [Wed, 20 Sep 2017 23:13:38 +0000 (16:13 -0700)]
selftest: Also run smbtorture smb2.compound with aio enabled

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13047

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Sep 22 09:49:30 CEST 2017 on sn-devel-144

6 years agotorture: Add testcase for compound CREATE-WRITE-CLOSE request
Christof Schmitt [Wed, 20 Sep 2017 23:07:50 +0000 (16:07 -0700)]
torture: Add testcase for compound CREATE-WRITE-CLOSE request

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13047

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
6 years agosmbd/aio: Do not go async for SMB2 compound requests
Christof Schmitt [Thu, 21 Sep 2017 19:08:01 +0000 (12:08 -0700)]
smbd/aio: Do not go async for SMB2 compound requests

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13047

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
6 years agosmbd: Move check for SMB2 compound request to new function
Christof Schmitt [Fri, 22 Sep 2017 00:41:25 +0000 (17:41 -0700)]
smbd: Move check for SMB2 compound request to new function

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13047

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
6 years agos3: Fix a small spelling mistake in smbcacls.
Richard Sharpe [Tue, 19 Sep 2017 19:31:55 +0000 (12:31 -0700)]
s3: Fix a small spelling mistake in smbcacls.

Signed-off-by: Richard Sharpe <realrichardsharpe@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Fri Sep 22 05:44:09 CEST 2017 on sn-devel-144

6 years agoauth3: Remove auth_domain
Volker Lendecke [Wed, 20 Sep 2017 16:18:33 +0000 (09:18 -0700)]
auth3: Remove auth_domain

If you're a domain member, use winbind. Auth_domain is from times when we did
not have winbind. It has served its purpose, but we should move on.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Sep 22 00:02:29 CEST 2017 on sn-devel-144

6 years agoctdb-tests: Add tests for PID with srvid
Amitay Isaacs [Wed, 30 Aug 2017 06:35:49 +0000 (16:35 +1000)]
ctdb-tests: Add tests for PID with srvid

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13042

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Thu Sep 21 12:56:30 CEST 2017 on sn-devel-144

6 years agoctdb-tests: Implement control CHECK_PID_SRVID in fake daemon
Amitay Isaacs [Wed, 30 Aug 2017 07:04:44 +0000 (17:04 +1000)]
ctdb-tests: Implement control CHECK_PID_SRVID in fake daemon

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13042

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
6 years agoctdb-tests: Reimplement message handlers using srvid abstraction
Amitay Isaacs [Wed, 30 Aug 2017 06:59:16 +0000 (16:59 +1000)]
ctdb-tests: Reimplement message handlers using srvid abstraction

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13042

This is required for CHECK_PID_SRVID control implementation.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
6 years agoctdb-tool: Update process-exists command to pass optional srvid
Amitay Isaacs [Wed, 30 Aug 2017 06:29:01 +0000 (16:29 +1000)]
ctdb-tool: Update process-exists command to pass optional srvid

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13042

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
6 years agoctdb-client: Add client code for control CHECK_PID_SRVID
Amitay Isaacs [Thu, 21 Sep 2017 05:52:14 +0000 (15:52 +1000)]
ctdb-client: Add client code for control CHECK_PID_SRVID

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13042

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
6 years agoctdb-daemon: Add implementation of control CHECK_PID_SRVID
Amitay Isaacs [Wed, 30 Aug 2017 06:18:02 +0000 (16:18 +1000)]
ctdb-daemon: Add implementation of control CHECK_PID_SRVID

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13042

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
6 years agoctdb-protocol: Add marshalling for control CHECK_PID_SRVID
Amitay Isaacs [Wed, 30 Aug 2017 05:13:53 +0000 (15:13 +1000)]
ctdb-protocol: Add marshalling for control CHECK_PID_SRVID

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13042

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
6 years agoctdb-protocol: Add marshalling for struct ctdb_pid_srvid
Amitay Isaacs [Wed, 30 Aug 2017 05:13:12 +0000 (15:13 +1000)]
ctdb-protocol: Add marshalling for struct ctdb_pid_srvid

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13042

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
6 years agoctdb-protocol: Add new control CTDB_CONTROL_CHECK_PID_SRVID
Amitay Isaacs [Wed, 30 Aug 2017 05:10:56 +0000 (15:10 +1000)]
ctdb-protocol: Add new control CTDB_CONTROL_CHECK_PID_SRVID

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13042

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
6 years agoctdb-common: Extend srvid_exists() check to support optional private_data
Amitay Isaacs [Wed, 30 Aug 2017 03:27:12 +0000 (13:27 +1000)]
ctdb-common: Extend srvid_exists() check to support optional private_data

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13042

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
6 years agotestsuite: Fix the 32-bit test build
Volker Lendecke [Wed, 20 Sep 2017 00:12:33 +0000 (00:12 +0000)]
testsuite: Fix the 32-bit test build

samba_init_module returns 32-bit. For some reason on my
32-bit lxc "return 0" was converted to something but
NT_STATUS_OK, making initialization fail.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Sep 21 02:49:32 CEST 2017 on sn-devel-144

6 years agoWHATSNEW: Mention code removal from "net" and "rpcclient"
Volker Lendecke [Tue, 19 Sep 2017 22:26:55 +0000 (15:26 -0700)]
WHATSNEW: Mention code removal from "net" and "rpcclient"

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
6 years agolibnet: Remove libnet_samsync
Volker Lendecke [Tue, 19 Sep 2017 22:17:38 +0000 (15:17 -0700)]
libnet: Remove libnet_samsync

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
6 years agonet: Don't depend on libnet_samsync anymore
Volker Lendecke [Tue, 19 Sep 2017 22:14:32 +0000 (15:14 -0700)]
net: Don't depend on libnet_samsync anymore

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
6 years agonet: Remove NT4-based vampire keytab
Volker Lendecke [Tue, 19 Sep 2017 22:09:05 +0000 (15:09 -0700)]
net: Remove NT4-based vampire keytab

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
6 years agonet: Remove NT4-based rpc vampire ldif
Volker Lendecke [Tue, 19 Sep 2017 22:06:11 +0000 (15:06 -0700)]
net: Remove NT4-based rpc vampire ldif

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
6 years agonet: Remove rpc vampire from NT4 domains
Volker Lendecke [Tue, 19 Sep 2017 22:03:43 +0000 (15:03 -0700)]
net: Remove rpc vampire from NT4 domains

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
6 years agonet: Remove rpc samdump
Volker Lendecke [Tue, 19 Sep 2017 22:02:09 +0000 (15:02 -0700)]
net: Remove rpc samdump

This uses the NT4 replication commands. Samba does not have a server
for this, no tests, and whoever needs to migrate a native domain can
use an old Samba version

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
6 years agorpcclient: Remove sam_sync related commands
Volker Lendecke [Tue, 19 Sep 2017 21:33:07 +0000 (14:33 -0700)]
rpcclient: Remove sam_sync related commands

These three commands don't use the netlogon credential chain
correctly. They are missing the netlogon_creds_store after the dcerpc
call, so they destroy the correct use of the netlogon creds.

The only valid server for these calls that I know of would be NT4, and
that should be gone long ago.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
6 years agoMake sure smbtorture tests can run if someone has set their min protocol above NT1.
Richard Sharpe [Sun, 10 Sep 2017 19:50:57 +0000 (12:50 -0700)]
Make sure smbtorture tests can run if someone has set their min protocol above NT1.

This code is SMB1 only, and already modifies
maxprotocol, so this change is appropriate.

Signed-off-by: Richard Sharpe <richard.sharpe@primarydata.com>
Reviewed-by: Jeremy Allison <jra@samba.org>
6 years agoCVE-2017-12163: s3:smbd: Prevent client short SMB1 write from writing server memory...
Jeremy Allison [Fri, 8 Sep 2017 17:13:14 +0000 (10:13 -0700)]
CVE-2017-12163: s3:smbd: Prevent client short SMB1 write from writing server memory to file.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13020

Signed-off-by: Jeremy Allison <jra@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(master): Wed Sep 20 17:06:23 CEST 2017 on sn-devel-144

6 years agoCVE-2017-12150: s3:libsmb: only fallback to anonymous if authentication was not requested
Stefan Metzmacher [Mon, 12 Dec 2016 05:07:56 +0000 (06:07 +0100)]
CVE-2017-12150: s3:libsmb: only fallback to anonymous if authentication was not requested

With forced encryption or required signing we should also don't fallback.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12997

Signed-off-by: Stefan Metzmacher <metze@samba.org>
6 years agoCVE-2017-12150: libcli/smb: add smbXcli_conn_signing_mandatory()
Stefan Metzmacher [Tue, 29 Aug 2017 13:35:49 +0000 (15:35 +0200)]
CVE-2017-12150: libcli/smb: add smbXcli_conn_signing_mandatory()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12997

Signed-off-by: Stefan Metzmacher <metze@samba.org>
6 years agoCVE-2017-12150: auth/credentials: cli_credentials_authentication_requested() should...
Stefan Metzmacher [Tue, 29 Aug 2017 13:24:14 +0000 (15:24 +0200)]
CVE-2017-12150: auth/credentials: cli_credentials_authentication_requested() should check for NTLM_CCACHE/SIGN/SEAL

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12997

Signed-off-by: Stefan Metzmacher <metze@samba.org>
6 years agoCVE-2017-12150: libgpo: make use of SMB_SIGNING_REQUIRED in gpo_connect_server()
Stefan Metzmacher [Mon, 12 Dec 2016 04:49:46 +0000 (05:49 +0100)]
CVE-2017-12150: libgpo: make use of SMB_SIGNING_REQUIRED in gpo_connect_server()

It's important that we use a signed connection to get the GPOs!

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12997

Signed-off-by: Stefan Metzmacher <metze@samba.org>
6 years agoCVE-2017-12150: s3:pylibsmb: make use of SMB_SIGNING_DEFAULT for 'samba.samba3.libsmb...
Stefan Metzmacher [Fri, 9 Dec 2016 08:26:32 +0000 (09:26 +0100)]
CVE-2017-12150: s3:pylibsmb: make use of SMB_SIGNING_DEFAULT for 'samba.samba3.libsmb_samba_internal'

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12997

Signed-off-by: Stefan Metzmacher <metze@samba.org>
6 years agoCVE-2017-12150: s3:lib: get_cmdline_auth_info_signing_state smb_encrypt SMB_SIGNING_R...
Stefan Metzmacher [Thu, 3 Nov 2016 16:16:43 +0000 (17:16 +0100)]
CVE-2017-12150: s3:lib: get_cmdline_auth_info_signing_state smb_encrypt SMB_SIGNING_REQUIRED

This is an addition to the fixes for CVE-2015-5296.

It applies to smb2mount -e, smbcacls -e and smbcquotas -e.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12997

Signed-off-by: Stefan Metzmacher <metze@samba.org>
6 years agoCVE-2017-12150: s3:popt_common: don't turn a guessed username into a specified one
Stefan Metzmacher [Tue, 29 Aug 2017 15:06:21 +0000 (17:06 +0200)]
CVE-2017-12150: s3:popt_common: don't turn a guessed username into a specified one

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12997

Signed-off-by: Stefan Metzmacher <metze@samba.org>
6 years agoCVE-2017-12151: s3:libsmb: make use of cli_state_is_encryption_on()
Stefan Metzmacher [Sat, 17 Dec 2016 09:36:49 +0000 (10:36 +0100)]
CVE-2017-12151: s3:libsmb: make use of cli_state_is_encryption_on()

This will keep enforced encryption across dfs referrals.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12996

Signed-off-by: Stefan Metzmacher <metze@samba.org>
6 years agoCVE-2017-12151: s3:libsmb: add cli_state_is_encryption_on() helper function
Stefan Metzmacher [Mon, 14 Aug 2017 10:13:18 +0000 (12:13 +0200)]
CVE-2017-12151: s3:libsmb: add cli_state_is_encryption_on() helper function

This allows to check if the current cli_state uses encryption
(either via unix extentions or via SMB3).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12996

Signed-off-by: Stefan Metzmacher <metze@samba.org>
6 years agodsdb: Only trigger a re-index once per @INDEXLIST modification
Andrew Bartlett [Mon, 11 Sep 2017 01:53:19 +0000 (13:53 +1200)]
dsdb: Only trigger a re-index once per @INDEXLIST modification

A modify of both @INDEXLIST and @ATTRIBUTES will still trigger two re-index passes
but that is a task for later.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=9527

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Sep 20 12:29:49 CEST 2017 on sn-devel-144

6 years agoselftest: sort dbcheck output to avoid sort order impacting results
Andrew Bartlett [Fri, 25 Aug 2017 05:37:05 +0000 (17:37 +1200)]
selftest: sort dbcheck output to avoid sort order impacting results

The GUID index code will change the returned results order

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>