nivanova/samba-autobuild/.git
3 years agogetncchanges.c: Send linked attributes in each chunk
Tim Beale [Tue, 6 Jun 2017 22:46:47 +0000 (10:46 +1200)]
getncchanges.c: Send linked attributes in each chunk

Instead of sending all the linked attributes at the end, add a
configurable option to send the links in each replication chunk.

The benefits of this approach are:
- it can reduce memory overhead, as we don't have to keep all the links
in memory over the entire replication cycle.
- the client should never end up knowing about objects but not their
links. (Although we're not sure that this has actually resulted in
replication problems, i.e. missing links).

Note that until we support GET_TGT, this approach can mean we now send
a link where the client doesn't know about the target object, causing
the client to siliently drop that linked attribute. Hence, this option
is switched off by default.

Implementation-wise, this code works fairly the same as before. Instead
of sorting the entire getnc_state->la_sorted array at the end and then
splitting it up over chunks, we now split the links up over chunks and
then sort them when we copy them into the message. This should be OK, as
I believe the MS-DRSR Doc says the links in the message should be sorted
(rather than sorting *all* the links overall). Windows behaviour seems
to chunk the links based on USN and then sort them.

getnc_state->la_idx now tracks which links in getnc_state->la_list[]
have already been sent (instead of tracking getnc_state->la_sorted).
This means the la_sorted array no longer needs to be stored in
getnc_state and we can free the array's memory once we've copied the
links into the message. Unfortunately, the link_given/link_total debug
no longer reports the correct information, so I've moved these into
getncchanges_state struct (and now free the struct a bit later so it's
safe to reference in the debug).

The vampire_dc testenv has been updated to use this new behaviour.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Fri Sep 15 10:07:33 CEST 2017 on sn-devel-144

3 years agogetnchanges.c: Avoid unnecessary continue
Tim Beale [Tue, 22 Aug 2017 05:32:32 +0000 (17:32 +1200)]
getnchanges.c: Avoid unnecessary continue

There's not really much after the continue that we're skipping now. We
can just flip the logic and avoid the continue.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agogetncchanges.c: Split out code to get an object for a response
Tim Beale [Tue, 22 Aug 2017 05:18:32 +0000 (17:18 +1200)]
getncchanges.c: Split out code to get an object for a response

Basically, everytime we try to add an object to the response, we want
to:
- Build it (i.e. pack it into an RPC message format)
- Add it to our object-cache if we're keeping one
- Add any ancestors needed for the client to resolve it (if GET_ANC)

GET_TGT is going to use the exact same code, so split this out into a
separate function, rather than duplicating it.

The GET_ANC case also uses almost identical code, but it differs in a
couple of minor aspects. I've left this as is for now, as I'm not sure
if this is by accident or by design.

Because all the memory was talloc'd off the 'obj' variable, we now need
to replace it with a tmp TALLOC_CTX.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agogetncchanges.c: Reduce the parameters to get_nc_changes_build_object()
Tim Beale [Tue, 22 Aug 2017 04:50:38 +0000 (16:50 +1200)]
getncchanges.c: Reduce the parameters to get_nc_changes_build_object()

Fifteen parameters seems a bit excessive. Instead, pass it the structs
containing the information it cares about.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agogetncchanges.c: Remove unused ncRoot_dn parameter
Tim Beale [Tue, 22 Aug 2017 04:29:17 +0000 (16:29 +1200)]
getncchanges.c: Remove unused ncRoot_dn parameter

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agogetncchanges.c: Remove a really old TODO
Tim Beale [Tue, 22 Aug 2017 04:21:47 +0000 (16:21 +1200)]
getncchanges.c: Remove a really old TODO

This TODO was added in 2009 (before Samba supported linked_attributes
in getNCChanges())

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agogetncchanges.c: Replace hard-coded numbers with a define
Tim Beale [Tue, 22 Aug 2017 04:19:54 +0000 (16:19 +1200)]
getncchanges.c: Replace hard-coded numbers with a define

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agogetncchanges.c: Refactor how objects get added to the response
Tim Beale [Tue, 22 Aug 2017 04:17:10 +0000 (16:17 +1200)]
getncchanges.c: Refactor how objects get added to the response

Adding GET_TGT support is going to make things more complicated, and I
think we are going to struggle to do this without refactoring things a
bit.

This patch adds a helper struct to store state related to a single
GetNCChanges chunk. I plan to add to this with things like max_links,
max_objects, etc, which will cutdown on the number of variables/
parameters we pass around.

I found the double-pointer logic where we add objects to the response
confusing - hopefully this refactor simplifies things slightly, and it
allows us to reuse the code for the GET_TGT case.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agogetncchanges.c: Refactor how we add ancestor links
Tim Beale [Tue, 22 Aug 2017 04:00:57 +0000 (16:00 +1200)]
getncchanges.c: Refactor how we add ancestor links

If the current object had already been sent as an ancestor, we were
duplicating the code that added its links and updated the HWM mark.
We want these to occur when we reach the place where the object's USN
naturally occurs.

Instead of duplicating this code, we can just skip the call to
get_nc_changes_build_object() if the object has already been sent.
There is already an existing 'nothing to send'/continue case after we've
updated the highwater mark.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agogetncchanges.c: Add ancestor links when the object normally gets sent
Tim Beale [Tue, 22 Aug 2017 03:45:39 +0000 (15:45 +1200)]
getncchanges.c: Add ancestor links when the object normally gets sent

Currently we add links each time we send an object, but we don't
actually send these links until the end of the replication cycle.

In subsequent patches we want the links to be sent in the same chunk as
their source object, ideally in as close to USN order as possible.
Processing ancestors complicates this a bit, as the ancestor will have a
higher USN than what we're currently up to, and so potentially will the
ancestor's links.

This patch moves where the ancestor's links get added to the
getnc_state->la_list. The ancestor's links now get added when the object
would normally get sent based purely on its USN (we update the highwater
mark at this point too).

This should not affect functionality, i.e. because we send all the links
at the end, it should make no difference at what point they get added to
the list.

This duplicates a tiny bit of code, but this will be cleaned up in the
next patch.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agogetncchanges.c: Split GET_ANC block out into its own function
Tim Beale [Tue, 22 Aug 2017 03:34:04 +0000 (15:34 +1200)]
getncchanges.c: Split GET_ANC block out into its own function

When we add GET_TGT support, it's going to need to reuse all this code
(i.e. to add any ancestors of the link target). This also trims down
the rather large dcesrv_drsuapi_DsGetNCChanges() function a bit.

Note also fixed a compiler warning in the WERR_DS_DRA_INCONSISTENT_DIT
error block which may have caused issues previously (statement was
terminated by a ',' rather than a ';').

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agogetncchanges.c: Split sorting linked attributes into separate function
Tim Beale [Tue, 6 Jun 2017 03:03:33 +0000 (15:03 +1200)]
getncchanges.c: Split sorting linked attributes into separate function

Longer-term we want to split up the links so that they're sent over
multiple GetNCChanges response messages. So it makes sense to split this
code out into its own function. In the short-term, this removes some of
the complexity from dcesrv_drsuapi_DsGetNCChanges() so that the function
is not quite so big.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agogetncchanges.c: Rename anc_cache to obj_cache
Tim Beale [Fri, 2 Jun 2017 02:42:34 +0000 (14:42 +1200)]
getncchanges.c: Rename anc_cache to obj_cache

When we add GET_TGT support we will reuse the ancestor cache and it
should work the same way - if we've already sent an object because it
was needed for resolving a child object or a link target, then there's
no point sending it again.

This just renames anc_cache --> obj_cache.

An extra is_get_anc flag has been added to getnc_state - once GET_TGT
support is added, we can't assume GET_ANC based solely on the existence
of the obj_cache.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agocharset: fix str[n]casecmp_m() by comparing lower case values
Stefan Metzmacher [Wed, 6 Sep 2017 07:47:20 +0000 (09:47 +0200)]
charset: fix str[n]casecmp_m() by comparing lower case values

The commits c615ebed6e3d273a682806b952d543e834e5630d^..f19ab5d334e3fb15761fb009e5de876dfc6ea785
replaced Str[n]CaseCmp() by str[n]casecmp_m().

The logic we had in str[n]casecmp_w() used to compare
the upper cased as well as the lower cased versions of the
characters and returned the difference between the lower cased versions.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13018

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Fri Sep 15 02:23:29 CEST 2017 on sn-devel-144

3 years agocharset/tests: also tests the system str[n]casecmp()
Stefan Metzmacher [Wed, 6 Sep 2017 09:24:28 +0000 (11:24 +0200)]
charset/tests: also tests the system str[n]casecmp()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13018

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
3 years agocharset/tests: add more str[n]casecmp_m() tests to demonstrate the bug
Stefan Metzmacher [Wed, 6 Sep 2017 08:39:00 +0000 (10:39 +0200)]
charset/tests: add more str[n]casecmp_m() tests to demonstrate the bug

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13018

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
3 years agocharset/tests: assert the exact values of str[n]casecmp_m()
Stefan Metzmacher [Wed, 6 Sep 2017 08:38:37 +0000 (10:38 +0200)]
charset/tests: assert the exact values of str[n]casecmp_m()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13018

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
3 years agoInstall dcerpc/__init__.py for all Python environments
Alexander Bokovoy [Wed, 13 Sep 2017 08:37:34 +0000 (11:37 +0300)]
Install dcerpc/__init__.py for all Python environments

Also fix whitespace. We use tabs, not spaces in Python/waf code.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13030

Signed-off-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Thu Sep 14 22:29:39 CEST 2017 on sn-devel-144

3 years agos4-provision: Ensure the dummy main-domain DB used for DLZ has an @INDEXLIST
Andrew Bartlett [Wed, 30 Aug 2017 03:30:04 +0000 (15:30 +1200)]
s4-provision: Ensure the dummy main-domain DB used for DLZ has an @INDEXLIST

The other databases are created from copies of the main provision, but this one
is not, so did not previously get a valid @INDEXLIST.

This is important as otherwise we will not correctly notice support for
the GUID index or new DSDB features in @SAMBA_DSDB as this is gated
on seeing @SAMBA_FEATURES_SUPPORTED in @INDEXLIST.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
3 years agorepl_meta_data: Show failing replicated entry in error code
Andrew Bartlett [Wed, 6 Sep 2017 04:24:35 +0000 (16:24 +1200)]
repl_meta_data: Show failing replicated entry in error code

This re-work of our LDIF printing avoids some of the privacy issue from
printing the full LDIF at level 4, while showing the entry that actually fails.

Instead, with e3988f8f74f4a11e8f26a548e0a33d20f4e863f7 we now print the DN
only at level 4, then the full message at 8.

With this patch on failure, we print the redacted failing message at 5.

While all of the DRS replication data is potentially sensitive
the passwords are most sensitive, and are now not printed unencrypted.

This discourages users from sending the full failing trace, as the
last entry is much more likely the issue.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
3 years agoselftest: reindex in dbcheck-oldrelease after modifying the backend DB
Andrew Bartlett [Tue, 12 Sep 2017 02:17:35 +0000 (14:17 +1200)]
selftest: reindex in dbcheck-oldrelease after modifying the backend DB

Modifying the backend DB is not a supported operation, but helps us create test
situations.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
3 years agoschema: Rework dsdb_schema_set_indices_and_attributes() db operations
Andrew Bartlett [Mon, 11 Sep 2017 03:22:23 +0000 (15:22 +1200)]
schema: Rework dsdb_schema_set_indices_and_attributes() db operations

Commit ec9b1e881c3eef503d6b4b311594113acf7d47d8 did not fully fix this.

There is no value in using dsdb_replace(), we are under the read lock
and replace just confuses things further.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13025

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
3 years agoselftest: Check re-opening sam.ldb corrects the @ATTRIBUTES and @INDEXLIST
Andrew Bartlett [Wed, 13 Sep 2017 04:13:06 +0000 (16:13 +1200)]
selftest: Check re-opening sam.ldb corrects the @ATTRIBUTES and @INDEXLIST

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
3 years agoctdb-protocol: Drop marshalling for monitor controls
Martin Schwenke [Mon, 4 Sep 2017 04:54:47 +0000 (14:54 +1000)]
ctdb-protocol: Drop marshalling for monitor controls

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Thu Sep 14 18:42:28 CEST 2017 on sn-devel-144

3 years agoctdb-client: Drop client code for monitor controls
Martin Schwenke [Mon, 4 Sep 2017 04:51:38 +0000 (14:51 +1000)]
ctdb-client: Drop client code for monitor controls

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
3 years agoctdb-client: Drop old client code for monitor controls
Martin Schwenke [Mon, 4 Sep 2017 04:51:02 +0000 (14:51 +1000)]
ctdb-client: Drop old client code for monitor controls

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
3 years agoctdb-daemon: Remove unused function ctdb_stop_monitoring()
Martin Schwenke [Mon, 11 Sep 2017 00:54:03 +0000 (10:54 +1000)]
ctdb-daemon: Remove unused function ctdb_stop_monitoring()

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
3 years agoctdb-daemon: Drop monitoring mode
Martin Schwenke [Mon, 4 Sep 2017 04:44:16 +0000 (14:44 +1000)]
ctdb-daemon: Drop monitoring mode

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
3 years agoctdb-tests: Drop implementation of monitor controls
Martin Schwenke [Mon, 4 Sep 2017 04:43:41 +0000 (14:43 +1000)]
ctdb-tests: Drop implementation of monitor controls

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
3 years agoctdb-daemon: Drop implementation of monitor controls
Martin Schwenke [Mon, 4 Sep 2017 04:33:17 +0000 (14:33 +1000)]
ctdb-daemon: Drop implementation of monitor controls

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
3 years agoctdb-daemon: Mark monitoring controls obsolete
Martin Schwenke [Mon, 4 Sep 2017 04:22:44 +0000 (14:22 +1000)]
ctdb-daemon: Mark monitoring controls obsolete

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
3 years agoctdb-docs: Drop mention of unimplemented commands
Martin Schwenke [Mon, 4 Sep 2017 04:19:10 +0000 (14:19 +1000)]
ctdb-docs: Drop mention of unimplemented commands

Some of these are only in a comment but git grep finds them.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
3 years agoctdb-tools: Drop monitoring-related ctdb commands
Martin Schwenke [Mon, 4 Sep 2017 04:18:49 +0000 (14:18 +1000)]
ctdb-tools: Drop monitoring-related ctdb commands

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
3 years agoctdb-daemon: Don't explicitly stop monitoring during shutdown
Martin Schwenke [Mon, 11 Sep 2017 00:48:50 +0000 (10:48 +1000)]
ctdb-daemon: Don't explicitly stop monitoring during shutdown

Monitoring is skipped when not in run state RUNNING, so remove the
dependency on the monitoring code.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
3 years agoctdb-daemon: Don't explicitly disable monitoring around recovery
Martin Schwenke [Fri, 1 Sep 2017 02:12:45 +0000 (12:12 +1000)]
ctdb-daemon: Don't explicitly disable monitoring around recovery

Monitoring can fail during recovery due to databases (e.g. registry)
being unavailable.  This has been avoided by explicitly disabling
monitoring around recovery via the START_RECOVERY and END_RECOVERY
controls.  With this approach only there is still a window between
enabling recovery mode and START_RECOVERY when monitoring could be
attempted.  However, explicitly disabling monitoring is unnecessary
because monitoring is not done when a node is in recovery.

So remove the explicit disable/enable of monitoring and rely on
monitoring being skipped when recovery mode is active.

The only possible change of behaviour with this change is that there
is now a window between setting recovery mode to normal and the
END_RECOVERY control where monitoring is enabled.  However, at this
point databases would be available and the "recovered" event will
cancel any in-progress monitoring.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
3 years agoctdb-daemon: Don't explicitly disable monitoring when stopping a node
Martin Schwenke [Tue, 7 Jul 2015 10:41:05 +0000 (20:41 +1000)]
ctdb-daemon: Don't explicitly disable monitoring when stopping a node

Monitoring is now avoided for inactive nodes anyway.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
3 years agoctdb-daemon: Skip monitoring when not in RUNNING runstate
Martin Schwenke [Mon, 4 Sep 2017 04:39:01 +0000 (14:39 +1000)]
ctdb-daemon: Skip monitoring when not in RUNNING runstate

Monitoring does not need to be done in other states.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
3 years agoctdb-daemon: Skip monitoring when node is inactive
Martin Schwenke [Mon, 6 Jul 2015 05:37:23 +0000 (15:37 +1000)]
ctdb-daemon: Skip monitoring when node is inactive

This is currently handled by explicitly disabling monitoring in
various places.  However, those places shouldn't need to know about
monitoring but it is OK for monitoring to know about global node
states.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
3 years agoctdb-tests: Drop unused monitoring status support
Martin Schwenke [Mon, 4 Sep 2017 05:33:54 +0000 (15:33 +1000)]
ctdb-tests: Drop unused monitoring status support

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
3 years agoctdb-client: Initialize ctdb_ltdb_header completely for empty record
Amitay Isaacs [Mon, 11 Sep 2017 04:05:17 +0000 (14:05 +1000)]
ctdb-client: Initialize ctdb_ltdb_header completely for empty record

ctdb_ltdb_fetch() only fills in relevant portion of ctdb_ltdb_header
if the record does not exist.  This can result in uninitialized writes
to ctdb_rec_buffer.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
3 years agoctdb-daemon: Free up record data if a call request is deferred
Amitay Isaacs [Mon, 11 Sep 2017 05:59:19 +0000 (15:59 +1000)]
ctdb-daemon: Free up record data if a call request is deferred

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13029

If a call request for a key (migration request) is in flight, then all
the subsequent call requests for the same key are deferred.  In that case,
the data corresponding to key read from the local tdb is useless and there
is no need to keep it around.  Once the deferred call is reprocessed,
the data corresponding to that key will be fetched again.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
3 years agolibcli: SMB2: NetApps negotiate SMB3_11 but also set the SMB2_CAP_ENCRYPTION flag.
Jeremy Allison [Mon, 11 Sep 2017 23:36:47 +0000 (16:36 -0700)]
libcli: SMB2: NetApps negotiate SMB3_11 but also set the SMB2_CAP_ENCRYPTION flag.

This is a SHOULD not, not a MUST not.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13009

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Steve French <sfrench@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Sep 14 14:48:20 CEST 2017 on sn-devel-144

3 years agovfs_streams_xattr: Fix segfault when running with log level 10
Christof Schmitt [Wed, 13 Sep 2017 23:23:53 +0000 (16:23 -0700)]
vfs_streams_xattr: Fix segfault when running with log level 10

This happens when vfs_streams_xattr is loaded, log level is set to 10
and the default stream of a file or directory is accessed. In that case
streams_xattr_open does not allocate the stream_io fsp extension. The
DBG_DEBUG message in streams_xattr_fstat tries to access the stream_io
before checking for a NULL value, resulting in the crash. Fix this by
moving the debug message after the check for a NULL pointer.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13032

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Thu Sep 14 10:58:12 CEST 2017 on sn-devel-144

3 years agoctdb-tests: Add 31.clamd eventscript unit tests
Martin Schwenke [Sat, 2 Sep 2017 10:59:32 +0000 (20:59 +1000)]
ctdb-tests: Add 31.clamd eventscript unit tests

These test that ctdb_check_unix_socket() is working.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Tue Sep 12 16:14:12 CEST 2017 on sn-devel-144

3 years agoctdb-tests: Enhance ss stub to check for listening Unix domain sockets
Martin Schwenke [Sat, 2 Sep 2017 10:57:56 +0000 (20:57 +1000)]
ctdb-tests: Enhance ss stub to check for listening Unix domain sockets

Generalise command-line parsing, taking hints from old netstat stub,
and use FAKE_NETSTAT_UNIX_LISTEN to specify listening Unix domain
sockets.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
3 years agoctdb-scripts: Switch ctdb_check_unix_socket() to use ss
Martin Schwenke [Sat, 18 Mar 2017 10:55:04 +0000 (21:55 +1100)]
ctdb-scripts: Switch ctdb_check_unix_socket() to use ss

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
3 years agoctdb-scripts: Clean up ctdb_check_unix_socket()
Martin Schwenke [Sat, 18 Mar 2017 10:53:06 +0000 (21:53 +1100)]
ctdb-scripts: Clean up ctdb_check_unix_socket()

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
3 years agoctdb-daemon: Don't release all IPs before "startup" event
Martin Schwenke [Mon, 5 Sep 2016 03:38:18 +0000 (13:38 +1000)]
ctdb-daemon: Don't release all IPs before "startup" event

This doesn't belong in the monitoring/startup code and it is already
done in the 10.interface "init" event.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
3 years agoctdb-recoverd: Abort recovery/takeover if recmaster changes
Amitay Isaacs [Fri, 8 Sep 2017 01:24:27 +0000 (11:24 +1000)]
ctdb-recoverd: Abort recovery/takeover if recmaster changes

Recovery and takeover are run via helper from recovery daemon.  While the
helpers are running, it's possible for the current node to lose election.
If that happens, abort the currently running recovery/takeover helper.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
3 years agoctdb-daemon: GET_DB_SEQNUM should read database conditionally
Amitay Isaacs [Thu, 7 Sep 2017 07:21:03 +0000 (17:21 +1000)]
ctdb-daemon: GET_DB_SEQNUM should read database conditionally

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13021

Once the recovery starts and databases are frozen, then all the record
access is postponed till the recovery is complete except reading the
database sequence number.  Database access for reading sequence number
is done via a control which does not check if the databases are frozen
or not.

If the database is frozen and if the freeze transaction is not started
(this can happen when a node is inactive, or during recovery when the
database is frozen but the transaction has not yet started), then trying
to read sequence number will cause ctdb daemon to deadlock.

Before reading the sequence number, check if the database access is
allowed.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
3 years agoctdb-daemon: Add a function to check if db access is allowed
Amitay Isaacs [Thu, 7 Sep 2017 07:18:18 +0000 (17:18 +1000)]
ctdb-daemon: Add a function to check if db access is allowed

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13021

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
3 years agoctdb-tests: Fix ctdb test binary name in path testing
Amitay Isaacs [Tue, 5 Sep 2017 03:52:47 +0000 (13:52 +1000)]
ctdb-tests: Fix ctdb test binary name in path testing

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13012

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
3 years agoctdb-tests: Wait up to 30 seconds for process to be registered in ctdbd
Martin Schwenke [Tue, 12 Sep 2017 01:51:19 +0000 (11:51 +1000)]
ctdb-tests: Wait up to 30 seconds for process to be registered in ctdbd

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13012

This avoids a potential race where the client is not properly
registered before "ctdb process-exists" is called.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
3 years agos3: vfs: catia: compression get/set must act only on base file, and must cope with...
Jeremy Allison [Fri, 8 Sep 2017 22:28:39 +0000 (15:28 -0700)]
s3: vfs: catia: compression get/set must act only on base file, and must cope with fsp==NULL.

Correctly do filename conversion.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13003

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Tue Sep 12 10:50:57 CEST 2017 on sn-devel-144

3 years agos3: VFS: streams_xattr: Compression is only set/get on base filenames.
Jeremy Allison [Fri, 8 Sep 2017 22:27:37 +0000 (15:27 -0700)]
s3: VFS: streams_xattr: Compression is only set/get on base filenames.

Can be ignored (pass-through) in streams_xattr VFS module.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13003

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
3 years agopackaging: Remove Solaris directory and contents
Rowland Penny [Wed, 6 Sep 2017 13:38:37 +0000 (14:38 +0100)]
packaging: Remove Solaris directory and contents

Signed-off-by: Rowland Penny <rpenny@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Sep 12 06:46:35 CEST 2017 on sn-devel-144

3 years agopackaging: Remove RHEL directory and contents
Rowland Penny [Wed, 6 Sep 2017 13:28:51 +0000 (14:28 +0100)]
packaging: Remove RHEL directory and contents

Signed-off-by: Rowland Penny <rpenny@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
3 years agopackaging: Remove RHEL-CTDB directory and contents
Rowland Penny [Wed, 6 Sep 2017 13:33:58 +0000 (14:33 +0100)]
packaging: Remove RHEL-CTDB directory and contents

Signed-off-by: Rowland Penny <rpenny@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
3 years agos3/smbd: sticky write time offset miscalculation causes broken timestamps
Ralph Boehme [Fri, 8 Sep 2017 10:19:03 +0000 (12:19 +0200)]
s3/smbd: sticky write time offset miscalculation causes broken timestamps

The offset calculation for the offset that got passed to
fetch_write_time_send() in the enumeration loop was wrong as it passed
the offset before smbd_dirptr_lanman2_entry() added required padding.

This resulted in broken timestamps in the find response.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13024

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Tue Sep 12 02:45:46 CEST 2017 on sn-devel-144

3 years agoctdb-tests: Fix ctdb process-exist tests
Amitay Isaacs [Fri, 25 Aug 2017 06:55:34 +0000 (16:55 +1000)]
ctdb-tests: Fix ctdb process-exist tests

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13012

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Sat Sep  9 14:44:57 CEST 2017 on sn-devel-144

3 years agoctdb-tests: Add a dummy ctdb client for testing
Amitay Isaacs [Wed, 30 Aug 2017 03:05:32 +0000 (13:05 +1000)]
ctdb-tests: Add a dummy ctdb client for testing

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13012

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
3 years agoctdb-tests: Fix the implementation of process-exists in fake daemon
Amitay Isaacs [Fri, 25 Aug 2017 06:54:47 +0000 (16:54 +1000)]
ctdb-tests: Fix the implementation of process-exists in fake daemon

Keep track of clients and their pids.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13012

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
3 years agoctdb-daemon: Fix implementation of process_exists control
Amitay Isaacs [Fri, 25 Aug 2017 05:00:59 +0000 (15:00 +1000)]
ctdb-daemon: Fix implementation of process_exists control

Only check processes that are CTDB clients.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13012

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
3 years agoctdb-tools: Fix CID 1414746
Amitay Isaacs [Thu, 7 Sep 2017 01:38:41 +0000 (11:38 +1000)]
ctdb-tools: Fix CID 1414746

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
3 years agoctdb-tools: Use ssize_t instead of int for checking the status of read()
Amitay Isaacs [Tue, 5 Sep 2017 06:42:58 +0000 (16:42 +1000)]
ctdb-tools: Use ssize_t instead of int for checking the status of read()

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
3 years agos3/vfs: move ACE4_ADD_FILE/ACE4_DELETE_CHILD mapping from NFSv4 framework to vfs_zfsacl
Ralph Boehme [Wed, 6 Sep 2017 14:56:47 +0000 (16:56 +0200)]
s3/vfs: move ACE4_ADD_FILE/ACE4_DELETE_CHILD mapping from NFSv4 framework to vfs_zfsacl

This was added in e6a5f11865a55e9644292ae92e4a4b5ec0662ccd to adopt the
NFSv4 framework to follow ZFS permission rules. But this is the wrong
place, other filesystems like GPFS do not allow deletion when the user
has SEC_DIR_ADD_FILE.

This patch therefor moves the change from the NFS4 framework into the
ZFS module.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=6133

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Sep  9 04:59:51 CEST 2017 on sn-devel-144

3 years agovfs_zfsacl: ensure zfs_get_nt_acl_common() has access to stat info
Ralph Boehme [Wed, 6 Sep 2017 14:53:23 +0000 (16:53 +0200)]
vfs_zfsacl: ensure zfs_get_nt_acl_common() has access to stat info

We'll need this in the next commit.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=6133

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
3 years agovfs_zfsacl: pass smb_fname to zfs_get_nt_acl_common
Ralph Boehme [Wed, 6 Sep 2017 14:44:12 +0000 (16:44 +0200)]
vfs_zfsacl: pass smb_fname to zfs_get_nt_acl_common

This is in preperation of moving SMB_ACE4_ADD_FILE /
SMB_ACE4_DELETE_CHILD mapping from the common NFSv4 framework into this
module excusively.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=6133

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
3 years agovfs/nfs4_acls: move special handling of SMB_ACE4_SYNCHRONIZE to vfs_zfsacl
Ralph Boehme [Wed, 6 Sep 2017 14:28:10 +0000 (16:28 +0200)]
vfs/nfs4_acls: move special handling of SMB_ACE4_SYNCHRONIZE to vfs_zfsacl

Commit 99a74ff5e6a9f87ad7a650cb44e0f925f834b3a1 added special handling
of SMB_ACE4_SYNCHRONIZE, always setting it in the access_mask when
fabricating an ACL. While at the same time removing it from the
access_mask when setting an ACL, but this is done direclty in
vfs_zfsacl, not it the common code.

Forcing SMB_ACE4_SYNCHRONIZE to be always set is only needed on ZFS, the
other VFS modules using the common NFSv4 infrastructure should not be
made victims of the special ZFS behaviour.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=7909

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
3 years agosmbd: add missing newline to debug message in daemon_status()
Justin Maggard via samba-technical [Thu, 7 Sep 2017 18:05:45 +0000 (11:05 -0700)]
smbd: add missing newline to debug message in daemon_status()

Signed-off-by: Justin Maggard <jmaggard@netgear.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Sep  8 06:26:52 CEST 2017 on sn-devel-144

3 years agos3-lib: Fix error mapping for EROFS
Sachin Prabhu via samba-technical [Thu, 7 Sep 2017 11:49:49 +0000 (12:49 +0100)]
s3-lib: Fix error mapping for EROFS

EROFS is incorrectly mapped to NT_STATUS_ACCESS_DENIED. This should
instead be mapped to NT_STATUS_MEDIA_WRITE_PROTECTED.

This change has already been done for the client in
unix_nt_errmap in libcli/util/errmap_unix.c
commit 9d055846f225 ("r3278: - rewrote the client side rpc...)")

SMB1 specs for SMB_COM_DELETE also specifies this mapping for EROFS
https://msdn.microsoft.com/en-us/library/ee441772.aspx

RH bz: 1171705
This problem was reported by Red Hat glusterfs QE who encountered
different errors when performing the same operation on a fuse mount and
on a cifs mount of the same underlying gluster filesystem.

Signed-off-by: Sachin Prabhu <sprabhu@redhat.com>
Reviewed-by: Gunther Deschner <gdeschne@redhat.com>
Reported-by: Surabhi Bhalothia <sbhaloth@redhat.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
3 years agopython: Allow debug classes to be specified on the command line for python tools
Andrew Bartlett [Wed, 6 Sep 2017 23:26:04 +0000 (11:26 +1200)]
python: Allow debug classes to be specified on the command line for python tools

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Sep  7 10:43:33 CEST 2017 on sn-devel-144

3 years agolibrpc/dceprc_util.c: Move debug message to DBG_DEBUG()
Andrew Bartlett [Wed, 6 Sep 2017 23:20:27 +0000 (11:20 +1200)]
librpc/dceprc_util.c: Move debug message to DBG_DEBUG()

This message shows up a lot (every packet) at level 6 for the succesful case

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agolibcli/security: Move debug message to DBG_DEBUG()
Andrew Bartlett [Wed, 6 Sep 2017 23:19:01 +0000 (11:19 +1200)]
libcli/security: Move debug message to DBG_DEBUG()

This message shows up a lot at level 6 for no particularly good reason

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agodsdb: Add missing \n to debug
Andrew Bartlett [Wed, 6 Sep 2017 23:13:17 +0000 (11:13 +1200)]
dsdb: Add missing \n to debug

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agodrs repl: Only print raw DRS replication traffic at level 9
Andrew Bartlett [Wed, 6 Sep 2017 04:40:05 +0000 (16:40 +1200)]
drs repl: Only print raw DRS replication traffic at level 9

This can be sensitive even with the passwords still encrypted.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13017
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agodebug: Add new debug class "drs_repl" for DRS replication processing
Andrew Bartlett [Wed, 6 Sep 2017 04:37:34 +0000 (16:37 +1200)]
debug: Add new debug class "drs_repl" for DRS replication processing

This is used in the client and in the server

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agoUse the rpc_parse debug class for PIDL genrated code
Andrew Bartlett [Wed, 6 Sep 2017 04:27:07 +0000 (16:27 +1200)]
Use the rpc_parse debug class for PIDL genrated code

This means that the default print binding string qualifier will now go via this debug class
as will explicit calls to ndr_print_debug() and ndr_print_union_debug().

Calls to ndr_print_debugc() are not changed.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agorepl_meta_data: Re-work printing of replicated entries
Andrew Bartlett [Wed, 6 Sep 2017 04:24:35 +0000 (16:24 +1200)]
repl_meta_data: Re-work printing of replicated entries

This re-work of our LDIF printing avoids some of the privacy issue from
printing the full LDIF at level 4, while showing the entry that actually fails.

Instead, we print the DN only at level 4, then the full message at 8.

While all of the DRS replication data is potentially sensitive
the passwords are most sensitive, and are now not printed unencrypted.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agolinked_attributes: Use ldb_ldif_message_redacted_string() for consistency
Andrew Bartlett [Wed, 6 Sep 2017 03:56:59 +0000 (15:56 +1200)]
linked_attributes: Use ldb_ldif_message_redacted_string() for consistency

This avoids printing un-encrypted secret values in logs, and while links are not likely
secret, this avoids a future copy and paste using ldb_ldif_message_string() again.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agorepl_meta_data: Use ldb_ldif_message_redacted_string() to avoid printing secrets...
Andrew Bartlett [Wed, 6 Sep 2017 03:38:14 +0000 (15:38 +1200)]
repl_meta_data: Use ldb_ldif_message_redacted_string() to avoid printing secrets in logs

This avoids printing un-encrypted secret values in logs

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agoldb: version 1.2.2 ldb-1.2.2
Andrew Bartlett [Wed, 6 Sep 2017 02:26:25 +0000 (14:26 +1200)]
ldb: version 1.2.2

* Bug #13017: Add ldb_ldif_message_redacted_string() to allow debug
              of redacted log messages, avoiding showing secret values

* Bug #13015: Allow re-index of newer databases with binary GUID TDB keys
              (this officially removes support for re-index of the original
              pack format 0, rather than simply segfaulting).
* Avoid memory allocation and so make modify of records in ldb_tdb faster

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agoldb: Add new ldb_ldif_message_redacted_string() with tests
Andrew Bartlett [Tue, 5 Sep 2017 02:05:43 +0000 (14:05 +1200)]
ldb: Add new ldb_ldif_message_redacted_string() with tests

This is designed to be a drop in replacement for
ldb_ldif_message_string() while better protecting privacy.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13017

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agoldb_tdb: Refuse to re-index very old database with no DN in the record
Andrew Bartlett [Thu, 10 Aug 2017 05:08:54 +0000 (17:08 +1200)]
ldb_tdb: Refuse to re-index very old database with no DN in the record

These are not found on any AD DC, and would segfault previous LDB
versions.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13015

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agoldb_tdb: Use braces in ltdb_dn_list_find_val()
Andrew Bartlett [Thu, 10 Aug 2017 04:09:31 +0000 (16:09 +1200)]
ldb_tdb: Use braces in ltdb_dn_list_find_val()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13015

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agoldb_tdb: Check for talloc_strdup() failure in ltdb_index_add1()
Andrew Bartlett [Thu, 10 Aug 2017 02:44:27 +0000 (14:44 +1200)]
ldb_tdb: Check for talloc_strdup() failure in ltdb_index_add1()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13015

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agoldb_tdb: Check for errors during tdb operations in ltdb_reindex()
Andrew Bartlett [Wed, 23 Aug 2017 03:38:01 +0000 (15:38 +1200)]
ldb_tdb: Check for errors during tdb operations in ltdb_reindex()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13015

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agoldb_tdb: Use memcmp rather than strncmp() in ltdb_key_is_record(), re_key() and re_in...
Andrew Bartlett [Thu, 7 Sep 2017 04:23:43 +0000 (16:23 +1200)]
ldb_tdb: Use memcmp rather than strncmp() in ltdb_key_is_record(), re_key() and re_index()

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agoldb_tdb: Create a common ltdb_key_is_record() allowing multiple key forms
Andrew Bartlett [Fri, 1 Sep 2017 02:35:08 +0000 (14:35 +1200)]
ldb_tdb: Create a common ltdb_key_is_record() allowing multiple key forms

If backported, this allows old ldb versions to full-search and re-index newer databases
and in current code allows the transition to and from a GUID or incrementing ID based index

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13016

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agoldb_tdb: Do not trigger the unique index check during a re-index, use another pass
Andrew Bartlett [Fri, 18 Aug 2017 05:01:07 +0000 (17:01 +1200)]
ldb_tdb: Do not trigger the unique index check during a re-index, use another pass

We want to rename the objects, then scan looking for the index values.

This avoids a DB modify during the index scan traverse (the index values
are actually added to an in-memory TDB, written in prepare_commit()).

This allows us to remove the "this might already exist" case in the
index handling, we now know that the entry did not exist in the index
before we add it.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13015

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agoldb_tdb: Use memcmp() to compare TDB keys in re_index()
Andrew Bartlett [Tue, 15 Aug 2017 02:25:59 +0000 (14:25 +1200)]
ldb_tdb: Use memcmp() to compare TDB keys in re_index()

The keys may not always be a null terminated string, they could well
be a binary GUID in a future revision, for efficiency..

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13016

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agoselftest: Avoid a build started just before midnight failing
Andrew Bartlett [Sat, 2 Sep 2017 02:07:11 +0000 (14:07 +1200)]
selftest: Avoid a build started just before midnight failing

By allowing 41 or 42 days, we still test the expiry but are less sensitive to the
current time.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
3 years agocli_credentials: Fix a typo
Volker Lendecke [Wed, 6 Sep 2017 12:05:09 +0000 (14:05 +0200)]
cli_credentials: Fix a typo

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Sep  7 05:56:14 CEST 2017 on sn-devel-144

3 years agoWHATSNEW: Add Using x86_64 Accelerated AES Crypto Instructions section.
Jeremy Allison [Wed, 6 Sep 2017 19:13:32 +0000 (12:13 -0700)]
WHATSNEW: Add Using x86_64 Accelerated AES Crypto Instructions section.

Describes --accel-aes configure time option.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13008

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
3 years agolib: crypto: Add the ability to select Intel AESNI instruction set at configure time.
Jeremy Allison [Wed, 6 Sep 2017 18:59:44 +0000 (11:59 -0700)]
lib: crypto: Add the ability to select Intel AESNI instruction set at configure time.

Add --accel-aes=[none|intelaesni] to select.
Default is none.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13008

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
3 years agolib: crypto: Plumb in the Intel AES instructions.
Jeremy Allison [Wed, 6 Sep 2017 18:40:02 +0000 (11:40 -0700)]
lib: crypto: Plumb in the Intel AES instructions.

Causes:

AES_set_encrypt_key()
AES_set_decrypt_key()
AES_encrypt()
AES_decrypt()

to probe for the Intel AES instructions at runtime (only once)
and then call the hardware implementations if so, otherwise
fall back to the software implementations.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13008

Based on original work by Justin Maggard <jmaggard@netgear.com>

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
3 years agolib: crypt: Prepare the existing code to switch to Intel AES hardware instructions.
Jeremy Allison [Thu, 31 Aug 2017 18:41:32 +0000 (11:41 -0700)]
lib: crypt: Prepare the existing code to switch to Intel AES hardware instructions.

Rename the old struct aes_key as an intermediate struct aes_key_rj
and wrap it in a union so we can chose an alternate aes_key struct
when using Intel AES hardware.

Rename the original software implementations of:

 AES_set_encrypt_key()
 AES_set_decrypt_key()
 AES_encrypt()
 AES_decrypt()

by adding an _rj on the end, and call them via a wrapper
function.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13008

Based on original work by Justin Maggard <jmaggard@netgear.com>

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
3 years agothird_party: Add build capability to aesni-intel.
Jeremy Allison [Wed, 6 Sep 2017 16:58:06 +0000 (09:58 -0700)]
third_party: Add build capability to aesni-intel.

Minor modifications to code to allow building as a Samba
shared library.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13008

Based on original work by Justin Maggard <jmaggard@netgear.com>

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
3 years agothird_party: Add the Intel Add support for AES-NI acceleration.
Jeremy Allison [Wed, 6 Sep 2017 16:44:42 +0000 (09:44 -0700)]
third_party: Add the Intel Add support for AES-NI acceleration.

This commit takes the Linux kernel AES-NI code, and puts it into a
third_party private library. The Linux kernel code is under GPLv2+
so is compatible with Samba.

This can result in massive speed improvements (up to 200% on some
platforms), by using Intel AES-NI instructions.

These are the pristine check-ins of Linux kernel files for Intel AESNI crypto.

git show 8691ccd764f9ecc69a6812dfe76214c86ac9ba06:arch/x86/crypto/aesni-intel_asm.S
git show 2baad6121e2b2fa3428ee6cb2298107be11ab23a:arch/x86/include/asm/inst.h

Show the exact Linux kernel git refspecs we have imported.

These files are not yet used.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13008

Based on original work by Justin Maggard <jmaggard@netgear.com>

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
3 years agopython: Enable execution of samba.tests.security with Python 3.
Lumir Balhar [Tue, 8 Aug 2017 06:50:35 +0000 (08:50 +0200)]
python: Enable execution of samba.tests.security with Python 3.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Sep  6 15:29:58 CEST 2017 on sn-devel-144