kai/samba-autobuild/.git
5 years agoctdb-tests: Shut down transaction_loop clients more cleanly
Martin Schwenke [Wed, 3 Oct 2018 06:39:16 +0000 (16:39 +1000)]
ctdb-tests: Shut down transaction_loop clients more cleanly

A transaction_loop client can exit with a transaction active when its
time limit expires.  This causes a recovery and causes problems with
the test cleanup, which detects unwanted recoveries and fails.

Set a flag when the time limit expires and exit cleanly before the
next transaction is started.

Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
5 years agoctdb-tools: Have onnode pass -n option even when regular ssh not in use
Martin Schwenke [Wed, 3 Oct 2018 09:13:57 +0000 (19:13 +1000)]
ctdb-tools: Have onnode pass -n option even when regular ssh not in use

ONNODE_SSH is really a test hook, so it doesn't need to support
completely random values.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
5 years agoctdb-tests: Support closing of stdin in local daemons ssh stub
Martin Schwenke [Sat, 14 Apr 2018 11:27:20 +0000 (21:27 +1000)]
ctdb-tests: Support closing of stdin in local daemons ssh stub

Not sure this is needed but this makes it behave the same as ssh.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
5 years agoctdb-tests: Be more careful when building public IP addresses
Martin Schwenke [Thu, 20 Sep 2018 05:26:08 +0000 (15:26 +1000)]
ctdb-tests: Be more careful when building public IP addresses

The goal is to allow more local daemons by expanding the address range
rather than generating invalid addresses.

For IPv6, use a separate address space instead of an offset for the
2nd address.

For IPv4, use the last 2 octets with addresses starting at
192.168.100.1 and 192.168.200.1.  Avoid addresses with 0 and 255 in
the last octet by using a maximum of 100 addresses per "subnet"
starting at .1.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
5 years agoctdb-tests: Be more careful when building node addresses
Martin Schwenke [Thu, 20 Sep 2018 05:24:43 +0000 (15:24 +1000)]
ctdb-tests: Be more careful when building node addresses

The goal is to allow more local daemons by expanding the address range
rather than generating invalid addresses.

For IPv6, use all 4 trailing hex digits.

For IPv4, use the last 2 octets.  Although 127.0.0.0 is a /8 network,
avoid unexpected issues due to 0 and 255 in the last octet.  Use a
maximum of 100 addresses per "subnet" starting at .1.  Keep the first
group of addresses in 127.0.0.0/24 to continue to allow a reasonable
number of nodes to be tested with socket-wrapper.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
5 years agoctdb-tests: Don't format IPv4 octets as hex digits
Martin Schwenke [Thu, 20 Sep 2018 04:30:54 +0000 (14:30 +1000)]
ctdb-tests: Don't format IPv4 octets as hex digits

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
5 years agoctdb-tests: Be more efficient about starting/stopping local daemons
Martin Schwenke [Thu, 20 Sep 2018 04:42:42 +0000 (14:42 +1000)]
ctdb-tests: Be more efficient about starting/stopping local daemons

Don't loop, just use onnode all.

For shutting down, use onnode -p all.  This results in a significant
time saving for stopping many deamons because "ctdb shutdown" is now
synchronous.

onnode -p all can be used to start daemons directly because they
daemonize.  However, this does not work under valgrind because the
valgrind process does not exit, so onnode will wait forever for it.
In this case, use onnode without the -p option.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
5 years agoctdb-tests: Do not use ctdbd_wrapper in local daemon tests
Martin Schwenke [Tue, 10 Jul 2018 05:57:19 +0000 (15:57 +1000)]
ctdb-tests: Do not use ctdbd_wrapper in local daemon tests

Run the daemon directly and shut it down using ctdb shutdown.

The wrapper waits for ctdbd to reach >=FIRST_RECOVERY runstate within
a timeout period and shuts ctdbd down if that doesn't happen.  This is
only really used to ensure that ctdbd doesn't exit early after an
apparently successful start.  There are no known cases where ctdbd
will continue running but fail to reach >=FIRST_RECOVERY runstate.

When ctdbd is started in tests, the test code will wait until ctdbd is
in a healthy state on all nodes before proceeding, so there is
effectively no change in behaviour.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
5 years agoctdb-tests: Don't remove non-existent test database directory
Martin Schwenke [Thu, 27 Sep 2018 05:29:43 +0000 (15:29 +1000)]
ctdb-tests: Don't remove non-existent test database directory

This directory is no longer used.  Lack of removal doesn't seem to
cause a problem.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
5 years agoctdb-tests: Drop unused function maybe_stop_ctdb()
Martin Schwenke [Fri, 28 Sep 2018 10:41:45 +0000 (20:41 +1000)]
ctdb-tests: Drop unused function maybe_stop_ctdb()

There are too many functions to start/stop daemons.  Simplify this.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
5 years agoctdb-tests: Explicitly check for local daemons when shutting down
Martin Schwenke [Fri, 28 Sep 2018 10:39:18 +0000 (20:39 +1000)]
ctdb-tests: Explicitly check for local daemons when shutting down

This is clearer if the logic is explicit...  and...

There are too many functions to start/stop daemons.  Simplify this.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
5 years agoctdb-tests: Drop functions daemons_start(), daemons_stop()
Martin Schwenke [Thu, 27 Sep 2018 06:05:38 +0000 (16:05 +1000)]
ctdb-tests: Drop functions daemons_start(), daemons_stop()

There are too many functions to start/stop daemons.  Simplify this.

Inline the functionality into ctdb_start_all() and ctdb_stop_all().

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
5 years agoctdb-tests: Don't used daemons_start()/daemons_stop() directly in tests
Martin Schwenke [Thu, 27 Sep 2018 06:02:54 +0000 (16:02 +1000)]
ctdb-tests: Don't used daemons_start()/daemons_stop() directly in tests

There are too many functions to start/stop daemons.  Simplify this.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
5 years agoctdb-tests: Rename _ctdb_start_all() -> ctdb_start_all()
Martin Schwenke [Thu, 27 Sep 2018 06:00:35 +0000 (16:00 +1000)]
ctdb-tests: Rename _ctdb_start_all() -> ctdb_start_all()

There are too many functions to start/stop daemons.  Simplify this.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
5 years agoctdb-tests: Rename ctdb_start_all() -> ctdb_init()
Martin Schwenke [Thu, 27 Sep 2018 05:54:17 +0000 (15:54 +1000)]
ctdb-tests: Rename ctdb_start_all() -> ctdb_init()

There are too many functions to start/stop daemons.  Simplify this.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
5 years agoctdb-tests: Drop ps_ctdbd()
Martin Schwenke [Thu, 27 Sep 2018 06:23:07 +0000 (16:23 +1000)]
ctdb-tests: Drop ps_ctdbd()

This was used for debugging tests by ensuring that the arguments to
ctdbd were as expected.  It no longer outputs anything useful because
ctdbd is now started without arguments.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
5 years agoctdb-tests: Drop code for RECEIVE_RECORDS control
Amitay Isaacs [Thu, 15 Feb 2018 01:28:36 +0000 (12:28 +1100)]
ctdb-tests: Drop code for RECEIVE_RECORDS control

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
5 years agoctdb-protocol: Drop marshalling code for RECEIVE_RECORDS control
Amitay Isaacs [Thu, 15 Feb 2018 01:21:57 +0000 (12:21 +1100)]
ctdb-protocol: Drop marshalling code for RECEIVE_RECORDS control

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
5 years agoctdb-protocol: Mark RECEIVE_RECORDS control obsolete
Amitay Isaacs [Thu, 15 Feb 2018 02:52:10 +0000 (13:52 +1100)]
ctdb-protocol: Mark RECEIVE_RECORDS control obsolete

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
5 years agoctdb-daemon: Drop implementation of RECEIVE_RECORDS control
Amitay Isaacs [Thu, 15 Feb 2018 01:04:32 +0000 (12:04 +1100)]
ctdb-daemon: Drop implementation of RECEIVE_RECORDS control

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
5 years agoctdb-vacuum: Remove unnecessary check for zero records in delete list
Amitay Isaacs [Wed, 14 Feb 2018 04:23:07 +0000 (15:23 +1100)]
ctdb-vacuum: Remove unnecessary check for zero records in delete list

Since no records are deleted from RB tree during step 1, there is no
need for the check.  Run step 2 unconditionally.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
5 years agoctdb-vacuum: Fix the incorrect counting of remote errors
Amitay Isaacs [Wed, 14 Feb 2018 04:18:17 +0000 (15:18 +1100)]
ctdb-vacuum: Fix the incorrect counting of remote errors

If a node fails to delete a record in TRY_DELETE_RECORDS control during
vacuuming, then it's possible that other nodes also may fail to delete a
record.  So instead of deleting the record from RB tree on first failure,
keep track of the remote failures.

Update delete_list.remote_error and delete_list.left statistics only
once per record during the delete_record_traverse.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
5 years agoctdb-vacuum: Simplify the deletion of vacuumed records
Amitay Isaacs [Wed, 14 Feb 2018 03:50:40 +0000 (14:50 +1100)]
ctdb-vacuum: Simplify the deletion of vacuumed records

The 3-phase deletion of vacuumed records was introduced to overcome
the problem of record(s) resurrection during recovery.  This problem
is now handled by avoiding the records from recently INACTIVE nodes in
the recovery process.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
5 years agoctdb-tests: Add recovery record resurrection test for volatile databases
Martin Schwenke [Mon, 24 Sep 2018 06:17:19 +0000 (16:17 +1000)]
ctdb-tests: Add recovery record resurrection test for volatile databases

Ensure that deleted records and vacuumed records are not resurrected
from recently inactive nodes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
5 years agoctdb-daemon: Invalidate records if a node becomes INACTIVE
Amitay Isaacs [Wed, 14 Feb 2018 03:19:44 +0000 (14:19 +1100)]
ctdb-daemon: Invalidate records if a node becomes INACTIVE

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
5 years agoctdb-daemon: Don't pull any records if records are invalidated
Amitay Isaacs [Wed, 14 Feb 2018 03:27:32 +0000 (14:27 +1100)]
ctdb-daemon: Don't pull any records if records are invalidated

This avoids unnecessary work during recovery to pull records from nodes
that were INACTIVE just before the recovery.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
5 years agoctdb-daemon: Add invalid_records flag to ctdb_db_context
Amitay Isaacs [Wed, 14 Feb 2018 03:29:18 +0000 (14:29 +1100)]
ctdb-daemon: Add invalid_records flag to ctdb_db_context

If a node becomes INACTIVE, then all the records in volatile databases
are invalidated.  This avoids the need to include records from such
nodes during subsequent recovery after the node comes out INACTIVE state.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
5 years agoctdbd_conn: Generalise inaccurate error message
Martin Schwenke [Fri, 5 Oct 2018 04:04:18 +0000 (14:04 +1000)]
ctdbd_conn: Generalise inaccurate error message

Seeing:

  ctdb_read_packet failed: Cannot allocate memory
  [...,  0] ../source3/lib/ctdbd_conn.c:121(cluster_fatal)
  cluster fatal event: ctdbd died

The error is due to a memory allocation failure rather than ctdbd
dying.  However, the error message makes people wonder why ctdbd died.

Another alternative would be to wrap cluster_fatal() and have the
wrapper interpret the return value from ctdb_read_packet() to choose
from a set of more precise messages to pass to cluster_fatal().  For a
memory allocation it isn't strictly necessary to call cluster_fatal(),
but all is probably lost and it is still probably better to try to
exit cleanly as soon as possible instead of crashing somewhere.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Fri Oct  5 16:16:29 CEST 2018 on sn-devel-144

5 years agos4/auth/tests: Fix kerberos test string size
Alexander Bokovoy [Wed, 3 Oct 2018 19:48:00 +0000 (22:48 +0300)]
s4/auth/tests: Fix kerberos test string size

>>> len("user0@samba.example.com")
23

But the string definition does not take a final '\0' into account.
As per Volker's suggestion, use compiler's support to allocate
the string properly.

Signed-off-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
5 years agoleases: Streamline leases_db_key a bit
Volker Lendecke [Thu, 13 Sep 2018 19:05:05 +0000 (21:05 +0200)]
leases: Streamline leases_db_key a bit

We don't need to talloc the blob, it's always the same size

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Oct  3 04:11:59 CEST 2018 on sn-devel-144

5 years agosmbd: Fix a warning
Volker Lendecke [Mon, 10 Sep 2018 12:59:14 +0000 (14:59 +0200)]
smbd: Fix a warning

gcc complains that the "const" is ignored on function return
types. Right now I'm compiling this file a lot, so silence this
warning :-)

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
5 years agolib: Remove unused tdb_pack_append()
Volker Lendecke [Tue, 25 Sep 2018 15:57:17 +0000 (08:57 -0700)]
lib: Remove unused tdb_pack_append()

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
5 years agoincludes: Remove an unused #define
Volker Lendecke [Tue, 25 Sep 2018 11:02:35 +0000 (04:02 -0700)]
includes: Remove an unused #define

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
5 years agoincludes: Fix a typo
Volker Lendecke [Tue, 25 Sep 2018 10:58:29 +0000 (03:58 -0700)]
includes: Fix a typo

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
5 years agoincludes: Remove an unused #define
Volker Lendecke [Tue, 25 Sep 2018 10:58:16 +0000 (03:58 -0700)]
includes: Remove an unused #define

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
5 years agothird_party: Update cmocka to version 1.1.3
Andreas Schneider [Tue, 4 Sep 2018 15:50:15 +0000 (17:50 +0200)]
third_party: Update cmocka to version 1.1.3

* Added function to filter tests (cmocka_set_test_filter)
* Fixed fixture error reporting
* Some improvement for API documentation -> https://api.cmocka.org/
* Fixed subunit output on failures
* Do not abort if a test is skipped

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
5 years agosmbd: Simplify downgrade_share_lease
Volker Lendecke [Sun, 16 Sep 2018 09:25:14 +0000 (11:25 +0200)]
smbd: Simplify downgrade_share_lease

Coalesce the NT_STATUS_OPLOCK_BREAK_IN_PROGRESS case into just one
if-condition

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Tue Oct  2 22:22:37 CEST 2018 on sn-devel-144

5 years agosmbd: Move downgrade_share_lease into downgrade_lease
Volker Lendecke [Fri, 14 Sep 2018 14:41:25 +0000 (16:41 +0200)]
smbd: Move downgrade_share_lease into downgrade_lease

The next step will simplify the logic of the code.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
5 years agosmbd: Simplify downgrade_lease
Volker Lendecke [Fri, 14 Sep 2018 14:10:58 +0000 (16:10 +0200)]
smbd: Simplify downgrade_lease

To me, the "additive" SMB2_LEASE_WRITE|SMB2_LEASE_HANDLE is easier to
read than the negated ~SMB2_LEASE_READ.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
5 years agosmbd: Slightly simplify downgrade_lease()
Volker Lendecke [Fri, 14 Sep 2018 14:03:57 +0000 (16:03 +0200)]
smbd: Slightly simplify downgrade_lease()

As much as I dislike }else{ and prefer early returns, I even more
dislike asking for the same condition in two different ways.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
5 years agosmbd: Use find_share_mode_lease() in downgrade_share_lease
Volker Lendecke [Fri, 14 Sep 2018 11:30:43 +0000 (13:30 +0200)]
smbd: Use find_share_mode_lease() in downgrade_share_lease

Simple simplification: In locking/ we did not have the direct
reference to find_share_mode_lock.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
5 years agosmbd: Move downgrade_share_lease() to smbd/oplock.c
Volker Lendecke [Fri, 14 Sep 2018 11:18:50 +0000 (13:18 +0200)]
smbd: Move downgrade_share_lease() to smbd/oplock.c

This function is pretty closely entangled with its only caller. In
particular the NT_STATUS_OPLOCK_BREAK_IN_PROGRESS triggers acitivity
in the caller, and that's the only case where "*_l" is being set to
non-NULL. Prepare for cleanup

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
5 years agotests: Check pam_winbind pw change with different options
Mathieu Parent [Thu, 31 May 2018 19:16:31 +0000 (21:16 +0200)]
tests: Check pam_winbind pw change with different options

Pair-Programmed-With: Andreas Schneider <asn@samba.org>

Signed-off-by: Mathieu Parent <math.parent@gmail.com>
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Tue Oct  2 17:30:29 CEST 2018 on sn-devel-144

5 years agonsswitch: Add try_authtok option to pam_winbind
Mathieu Parent [Thu, 12 Apr 2018 09:57:15 +0000 (11:57 +0200)]
nsswitch: Add try_authtok option to pam_winbind

Same as the use_authtok option, except that if the new password is not
valid, PAM will prompt for a password.

Bug-Debian: https://bugs.debian.org/858923
Bug-Ubuntu: https://bugs.launchpad.net/ubuntu/+source/samba/+bug/570944

Signed-off-by: Mathieu Parent <math.parent@gmail.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
5 years agothird_party: Add pam_set_items.so from pam_wrapper
Mathieu Parent [Sat, 19 May 2018 12:57:01 +0000 (14:57 +0200)]
third_party: Add pam_set_items.so from pam_wrapper

Pair-Programmed-With: Andreas Schneider <asn@samba.org>
Signed-off-by: Mathieu Parent <math.parent@gmail.com>
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
5 years agothird_party: Update pam_wrapper to version 1.0.7
Andreas Schneider [Thu, 20 Sep 2018 07:13:09 +0000 (09:13 +0200)]
third_party: Update pam_wrapper to version 1.0.7

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
5 years agopython/tests: turn GraphError into failure inside of test_verify()
Stefan Metzmacher [Tue, 2 Oct 2018 10:01:03 +0000 (12:01 +0200)]
python/tests: turn GraphError into failure inside of test_verify()

That test was already marked as flapping in commit
5a73f904e192c44e304850287ac439d0b52f7be5.

However the test generates an UNEXPECTED(error)
instead of an UNEXPECTED(failure).

  [67(594)/75 at 36m28s] samba.tests.kcc.python3(vampire_dc)
  repsFrom source DSA guid (62f01940-a784-4e60-947a-a661c787c8cc) not found
  repsFrom source DSA guid (62f01940-a784-4e60-947a-a661c787c8cc) not found
  repsFrom source DSA guid (62f01940-a784-4e60-947a-a661c787c8cc) not found
  repsFrom source DSA guid (62f01940-a784-4e60-947a-a661c787c8cc) not found
  repsFrom source DSA guid (62f01940-a784-4e60-947a-a661c787c8cc) not found
  UNEXPECTED(error): samba.tests.kcc.python3.samba.tests.kcc.KCCTests.test_verify(vampire_dc)
  REASON: Exception: Exception: Traceback (most recent call last):
    File "bin/python/samba/tests/kcc/__init__.py", line 80, in test_verify
      attempt_live_connections=False)
    File "bin/python/samba/kcc/__init__.py", line 2659, in run
      ('connected',))
    File "bin/python/samba/kcc/__init__.py", line 2513, in plot_all_connections
      vertex_colors=vertex_colours)
    File "bin/python/samba/kcc/graph_utils.py", line 334, in verify_and_dot
      for p, e, doc in errors)))
  samba.kcc.graph_utils.GraphError: The 'dsa_final CN=NTDS Settings,CN=LOCALVAMPIREDC,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=samba,DC=example,DC=com' graph lacks the following properties:
  connected: the graph is not connected, as the following vertices are unreachable:

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
5 years agosmb2_server: set req->do_encryption = true earlier
Stefan Metzmacher [Fri, 17 Aug 2018 09:35:41 +0000 (11:35 +0200)]
smb2_server: set req->do_encryption = true earlier

The STATUS_SESSION_EXPIRED error was returned unencrypted,
if the request was encrypted.

If clients use SMB3 encryption and the kerberos authenticated session
expires, clients disconnect the connection instead of doing a reauthentication.

From https://blogs.msdn.microsoft.com/openspecification/2012/10/05/encryption-in-smb-3-0-a-protocol-perspective/

  The sender encrypts the message if any of the following conditions is
  satisfied:

    - If the sender is sending a response to an encrypted request.
    - If Session.EncryptData is TRUE and the request or response being
      sent is not NEGOTIATE.
    - If Session.EncryptData is FALSE, the request or response being sent
      is not NEGOTIATE or SESSION_SETUP or TREE_CONNECT, and
      <TreeConnect|Share>.EncryptData is TRUE.

[MS-SMB2] 3.3.4.1.4 Encrypting the Message

 If Connection.Dialect belongs to the SMB 3.x dialect family and
 Connection.ClientCapabilities includes the SMB2_GLOBAL_CAP_ENCRYPTION
 bit, the server MUST encrypt the message before sending, if any of the
 following conditions are satisfied:

 - If the message being sent is any response to a client request for which
   Request.IsEncrypted is TRUE.

 - If Session.EncryptData is TRUE and the response being sent is not
   SMB2_NEGOTIATE or SMB2 SESSION_SETUP.

 - If Session.EncryptData is FALSE, the response being sent is not
   SMB2_NEGOTIATE or SMB2 SESSION_SETUP or SMB2 TREE_CONNECT, and
   Share.EncryptData for the share associated with the TreeId in the SMB2
   header of the response is TRUE.

 The server MUST encrypt the message as specified in section 3.1.4.3,
 before sending it to the client.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13624

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Tue Oct  2 14:11:30 CEST 2018 on sn-devel-144

5 years agos4:torture: split smb2.session.expire{1,2} to run with signing and encryptpion
Stefan Metzmacher [Fri, 28 Sep 2018 10:23:37 +0000 (12:23 +0200)]
s4:torture: split smb2.session.expire{1,2} to run with signing and encryptpion

This reproduces the problem we have with expired encrypted sessions.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13624

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
5 years agos3/locking: Fix logging of lock reference count
Anoop C S [Tue, 18 Sep 2018 16:23:54 +0000 (21:53 +0530)]
s3/locking: Fix logging of lock reference count

lock reference count is always increased and reduced by a value of 1.
But lock_ref_count variable holds the old value prior to change and
was being logged wrongly under debug level 10. DEBUG statement must
log lock_ref_count+1 and lock_ref_count-1 respectively when value
gets increased and decreased.

Signed-off-by: Anoop C S <anoopcs@redhat.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
5 years agolibsmb: Remove smb_share_modes.[ch]
Volker Lendecke [Mon, 24 Sep 2018 20:52:25 +0000 (13:52 -0700)]
libsmb: Remove smb_share_modes.[ch]

This was declared nonfunctional in 2014. Finally remove it.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
5 years agos3: smbd: Prevent valgrind errors in smbtorture3 POSIX test.
Jeremy Allison [Thu, 27 Sep 2018 21:12:47 +0000 (14:12 -0700)]
s3: smbd: Prevent valgrind errors in smbtorture3 POSIX test.

Missing fsp talloc free and linked list delete in error
paths in close_directory(). Now matches close_normal_file()
and close_fake_file().

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13633

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Sep 29 05:32:41 CEST 2018 on sn-devel-144

5 years agolibcli: Add debug message if fail to negoatiate SMB protocol
Tim Beale [Wed, 26 Sep 2018 21:53:24 +0000 (09:53 +1200)]
libcli: Add debug message if fail to negoatiate SMB protocol

Currently if the client and server can't negotiate an SMB protocol, you
just get the followiing error on the client-side, which doesn't tell you
much.
ERROR(runtime): uncaught exception - (3221225667, 'The network responded
incorrectly.')

This patch adds a debug message to help highlight what's actually going
wrong.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13621

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Sep 28 11:25:29 CEST 2018 on sn-devel-144

5 years agos3/smbd: Server responds incorrectly if no SMB protocol chosen
Tim Beale [Wed, 26 Sep 2018 21:46:41 +0000 (09:46 +1200)]
s3/smbd: Server responds incorrectly if no SMB protocol chosen

The SMBnegprot response from the server contains the DialectIndex of the
selected protocol from the client's request message. Currently, if no
protocol is selected, the server is responding with a DialectIndex=zero,
which is a valid index (PROTOCOL_CORE by default). The Windows spec, and
historically the code, should return DialectIndex=0xffff if no protocol
is chosen. The following commit changed it recently (presumably
inadvertently), so that it now returns DialectIndex=zero.

06940155f315529c5b5 s3:smbd: Fix size types in reply_negprot()

This results in somewhat confusing error messages on the client side:
ERROR(runtime): uncaught exception - (3221225997, 'The transport
connection has been reset.')

or, when signing is configured as mandatory:
smbXcli_negprot: SMB signing is mandatory and the selected protocol
level (1) doesn't support it.
ERROR(runtime): uncaught exception - (3221225506, '{Access Denied} A
process has requested access to an object but has not been granted those
access rights.')

This patch restores the old behaviour of returning 0xffff.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13621

Pair-Programmed-With: Ralph Boehme <slow@samba.org>
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agonetcmd: Make sure SMB connection is signed when backing up sysvol
Tim Beale [Wed, 26 Sep 2018 05:01:03 +0000 (17:01 +1200)]
netcmd: Make sure SMB connection is signed when backing up sysvol

i.e. protect the client against man-in-the-middle attacks by default.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13621

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agos4/web_server: fix panic from wrong type extracted from conn
Noel Power [Wed, 19 Sep 2018 08:20:02 +0000 (10:20 +0200)]
s4/web_server: fix panic from wrong type extracted from conn

enabling
  server services = +web

and we get a panic

web_server.c:251: Type mismatch: name[struct web_server_data] expected[struct task_server]
smb_panic_default: PANIC (pid 29137): ../source4/web_server/web_server.c:251: Type mismatch: name[struct web_server_data] expected[struct task_server]

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Sep 28 03:57:47 CEST 2018 on sn-devel-144

5 years agoadd new gitlab CI job for building pure python3
Noel Power [Thu, 27 Sep 2018 09:22:44 +0000 (10:22 +0100)]
add new gitlab CI job for building pure python3

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agoscript: add new autobuild task for building pure python3
Noel Power [Thu, 27 Sep 2018 09:22:07 +0000 (10:22 +0100)]
script: add new autobuild task for building pure python3

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agoPY3: We support python3 now, remove error when python3 detected
Noel Power [Tue, 31 Jul 2018 17:00:36 +0000 (18:00 +0100)]
PY3: We support python3 now, remove error when python3 detected

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agobuildtools/wafsamba: Finally fix reference to basestring PY3 error.
Noel Power [Thu, 27 Sep 2018 10:39:44 +0000 (11:39 +0100)]
buildtools/wafsamba: Finally fix reference to basestring PY3 error.

While a previous attempt squashed the error on the config & make
phase, make install threw up this error again.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agodlz_bind9: torture tests exploring rndc reload behaviour
Aaron Haslett [Thu, 9 Aug 2018 05:09:01 +0000 (17:09 +1200)]
dlz_bind9: torture tests exploring rndc reload behaviour

These tests establish that the process triggered by the command 'rndc reload'
does not cause samba's bind9 dlz plugin to crash.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13214

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Sep 27 04:47:16 CEST 2018 on sn-devel-144

5 years agonetcmd: domain backup offline bug fix - ignore sock files
Aaron Haslett [Mon, 24 Sep 2018 04:27:33 +0000 (16:27 +1200)]
netcmd: domain backup offline bug fix - ignore sock files

Ignoring autogenerated .sock files and directories during file system based
offline domain backup.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13604
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agobuildtools/wafsamba: add_manual_dependency needs bytes for value
Noel Power [Wed, 26 Sep 2018 15:26:45 +0000 (16:26 +0100)]
buildtools/wafsamba: add_manual_dependency needs bytes for value

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agobuildtools/wafsamba: fix basestring not defined error in PY3
Noel Power [Wed, 26 Sep 2018 15:22:35 +0000 (16:22 +0100)]
buildtools/wafsamba: fix basestring not defined error in PY3

Test for str first (which exists in py3 & py2) this avoids
the undefined runtime error.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agos4/web_server: PY3: port to python3
Noel Power [Fri, 3 Aug 2018 16:55:09 +0000 (17:55 +0100)]
s4/web_server: PY3: port to python3

Note: Unlike other libraries this library is been only built for the
configured python version. It depends on availability of 'swat' python
module.

The swat module is hosted externally, it seems not to have been modified
for a number of years, I don't think swat is python3 compatabile. These
changes are enough to get allow the samba binary to launch a web server
(which will just display a placeholder page announcing you need to install
swat). It maybe that removing this functionality is what we should do, but
that is a decision that can be made at a later time.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agopython/samba: PY3 add compat function urllib_join to replace urllib.urljoin
Noel Power [Wed, 19 Sep 2018 08:52:52 +0000 (10:52 +0200)]
python/samba: PY3 add compat function urllib_join to replace urllib.urljoin

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agoPY3: md5 related functions need to be passed bytes
Noel Power [Tue, 31 Jul 2018 16:02:54 +0000 (17:02 +0100)]
PY3: md5 related functions need to be passed bytes

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agoPY3: decode bytes in py3 where strings are needed
Noel Power [Fri, 27 Jul 2018 15:54:16 +0000 (16:54 +0100)]
PY3: decode bytes in py3 where strings are needed

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agoPY3: fix "TabError: inconsistent use of tabs and spaces"
Noel Power [Fri, 27 Jul 2018 13:26:35 +0000 (14:26 +0100)]
PY3: fix "TabError: inconsistent use of tabs and spaces"

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agolib/ldb: Ensure ldb.Dn can accept utf8 encoded unicode
Noel Power [Mon, 24 Sep 2018 11:20:20 +0000 (12:20 +0100)]
lib/ldb: Ensure ldb.Dn can accept utf8 encoded unicode

Additionally remove the associated known fail.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13616
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agolib/ldb/tests: add test for ldb.Dn passed utf8 unicode
Noel Power [Mon, 24 Sep 2018 13:37:50 +0000 (14:37 +0100)]
lib/ldb/tests: add test for ldb.Dn passed utf8 unicode

object dn format should be a utf8 encoded string
Note: Currently this fails in python2 as the c python binding for
      the dn string param uses PyArg_ParseTupleAndKeywords() with 's'
      format, this will accept str *or* unicode in the default encoding.
      The default encoding in python2 is... ascii.

Also adding here a knownfail to squash the error produced by the test.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agolib/ldb: Test correct variable for no mem condition
Noel Power [Mon, 24 Sep 2018 10:28:47 +0000 (11:28 +0100)]
lib/ldb: Test correct variable for no mem condition

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agopy3 kcc tests: mark verify test as flapping
Douglas Bagnall [Thu, 20 Sep 2018 01:10:09 +0000 (13:10 +1200)]
py3 kcc tests: mark verify test as flapping

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Sep 26 23:40:58 CEST 2018 on sn-devel-144

5 years agoturn --with-json-audit into global --with-json
Philipp Gesang [Tue, 25 Sep 2018 12:09:18 +0000 (14:09 +0200)]
turn --with-json-audit into global --with-json

Fold the build option --with-json-audit into the toplevel wscript
to reflect the fact that JSON support is no longer local to the
audit subsystem.

Signed-off-by: Philipp Gesang <philipp.gesang@intra2net.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agodocs: Only build vfs manpages if the module is enabled
Andreas Schneider [Fri, 21 Sep 2018 16:58:53 +0000 (18:58 +0200)]
docs: Only build vfs manpages if the module is enabled

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
Autobuild-User(master): Günther Deschner <gd@samba.org>
Autobuild-Date(master): Wed Sep 26 20:10:07 CEST 2018 on sn-devel-144

5 years agosamba-tool domain backup: fix py2.6 incompatible format
Douglas Bagnall [Thu, 20 Sep 2018 23:26:35 +0000 (11:26 +1200)]
samba-tool domain backup: fix py2.6 incompatible format

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Sep 26 10:40:49 CEST 2018 on sn-devel-144

5 years agodomain_backup test: fix py2.6 incompatible format
Douglas Bagnall [Thu, 20 Sep 2018 23:18:19 +0000 (11:18 +1200)]
domain_backup test: fix py2.6 incompatible format

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agojoin: Support site detection when --server is specified
Tim Beale [Tue, 18 Sep 2018 22:44:48 +0000 (10:44 +1200)]
join: Support site detection when --server is specified

When a new DC is joined to the domain, samba-tool would automatically
detect an appropriate site for the new DC. However, it only did this if
the --server option wasn't specified. The new DC's site got
automatically updated as part of the finddc() work, however, this step
gets skipped if we already know the server DC to join to.

In other words, if Default-First-Site-Name doesn't exist and you specify
--server in the join, then you have to also specify --site manually,
otherwise the command fails. This is precisely what's happening in the
join_ldapcmp.sh test, now that the backupfromdc testenv no longer has the
Default-First-Site-Name present.

This patch adds a new find_dc_site() function which uses the same
net.finddc() API (except based on the server-address rather than
domain-name). Assigning DEFAULTSITE has been moved so that it only
gets done if finddc() can't determine the site.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agojoin: Avoid duplicating "Default-First-Site-Name" string
Tim Beale [Tue, 18 Sep 2018 22:21:12 +0000 (10:21 +1200)]
join: Avoid duplicating "Default-First-Site-Name" string

The provision code already defines "Default-First-Site-Name" so we might
as well reuse it.

The join.py already uses a suitable default, so assigning the default in
the domain netcmd code is unnecessary.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agoselftest: Test join against DC with non-default site
Tim Beale [Thu, 20 Sep 2018 01:08:50 +0000 (13:08 +1200)]
selftest: Test join against DC with non-default site

Recent changes around restoring a domain that lacked
Default-First-Site-Name highlighted a problem. Normally when you join a
DC to a domain, samba-tool works out the correct site to use
automatically. However, if the join uses '--server' to select a DC, then
this doesn't work. It defaults back to Default-First-Site-Name, and the
join command fails if this site doesn't exist.

All the testenvs had Default-First-Site-Name present, so this was never
tested. Now the backupfromdc no longer has a Default-First-Site-Name
site, so running a simple join against that DC fails, highlighting the
problem.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agoselftest: Change backup testenvs to use non-default site
Tim Beale [Tue, 18 Sep 2018 04:30:15 +0000 (16:30 +1200)]
selftest: Change backup testenvs to use non-default site

Previously (i.e. up until the last patch) the backup/restore commands
only worked if the Default-First-Site-Name site was present. If this
site didn't exist, then the various restore testenvs would fail to
start. This is now fixed, but this patch changes the backupfrom testenv
so that it uses a non-default site. This will detect the problem if it
is ever re-introduced.

To do this we need to change provision_ad_dc() so the
extra_provision_options can be specified as an argument. (Note that Perl
treats undef the same as an empty array).

By default, the restore will add the new DC into the
Default-First-Site-Name site. This means the backupfromdc and restored
testenvs will now have different sites, so we need to update the ldapcmp
filters to exclude site-specific attributes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13621

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agonetcmd: Re-create default site for backup-restore (if missing)
Tim Beale [Tue, 18 Sep 2018 02:54:51 +0000 (14:54 +1200)]
netcmd: Re-create default site for backup-restore (if missing)

Normally when a new DC joins a domain, samba-tool works out the new
DC's site automatically. However, it does this by querying the existing
DC using CLDAP. In the restore case, there is no DC running. We could
still query the DB on disk and work out the correct site based on the
new DC's IP, however:
- comparing between the CN=Subnet DNs and an IP-address string seems
  like it'd be non-trivial to write, and
- in the lab-domain rename case, chances are the user will want a
  completely different subnet to what's already in the DB.

The restore command now has a --site option so the user can specify an
appropriate site for the restored DC. This patch makes the restore
command work by default (i.e. without a --site option) even if the
default Default-First-Site-Name doesn't exist. Basically the solution is
to just check Default-First-Site-Name exists and create it if it
doesn't. As the recommended workflow is to use the restored DC as a
temporary seed that you'll later throw away, this approach seems
acceptable. Subsequent DCs will then be joined to the running restored
DC, so an appropriate site will be determined using CLDAP. The only
side-effect is potentially an extra Site object.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13621

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agotests: Add test-case for restore into non-default site
Tim Beale [Tue, 18 Sep 2018 05:23:48 +0000 (17:23 +1200)]
tests: Add test-case for restore into non-default site

Add a test-case that exercises the new '--site' restore option and
ensures the restored DC gets added to the correct site.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13621

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agonetcmd: Add --site option when restoring a domain
Tim Beale [Mon, 17 Sep 2018 03:36:21 +0000 (15:36 +1200)]
netcmd: Add --site option when restoring a domain

Restoring a backup only worked if the Default-First-Site-Name site was
still present. When the new restored DC account is created, it was
trying to add the new server's DN under CN=Default-First-Site-Name.
However, if the original domain was setup using a different site, then
the restore would fail because the DN didn't exist.

When running the restore command, you should be able to specify the
site that you want the new/restored DC to be in (same as during a
DC 'join'). Passing the correct --site argument is one way to avoid
this problem. (A subsequent patch will further improve the tool so it
can work around non-default sites automatically).

Note we also need to pass the site through to where the new DNS entries
get registered (in the rename case).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13621

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agonetcmd: Tweak backup-offline output to avoid subunit truncation
Tim Beale [Tue, 18 Sep 2018 03:24:36 +0000 (15:24 +1200)]
netcmd: Tweak backup-offline output to avoid subunit truncation

Currently a backup-offline test is occasionally flapping in autobuild,
however, the output is truncated so we can't see what the actual problem
is. The output only ever contains the list of backup dirs. I suspect
that the ']' character printed at the end of the python list might be
getting interpretted by subunit as the end of *all* the output.

If so, we should be able to avoid the problem by printing the list items
without the '['/']'s, i.e. join the list into a single string.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agos3: Fix the build when compiling without JSON support
Martin Schwenke [Sat, 22 Sep 2018 22:47:28 +0000 (08:47 +1000)]
s3: Fix the build when compiling without JSON support

[3762/3895] Compiling source3/utils/net_ads_gpo.c
../source3/utils/net_ads.c: In function ‘net_ads_cldap_netlogon_json’:
../source3/utils/net_ads.c:311:2: error: parameter name omitted
  (ADS_STRUCT *, const char *,
  ^
../source3/utils/net_ads.c:311:2: error: parameter name omitted
../source3/utils/net_ads.c:312:16: error: parameter name omitted
   const struct NETLOGON_SAM_LOGON_RESPONSE_EX *)
                ^
../source3/utils/net_ads.c: In function ‘net_ads_info_json’:
../source3/utils/net_ads.c:520:1: error: parameter name omitted
 static int net_ads_info_json(ADS_STRUCT *)
 ^

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Mon Sep 24 07:03:09 CEST 2018 on sn-devel-144

5 years agotests/blackbox: add test for net ads JSON output
Philipp Gesang [Mon, 20 Aug 2018 12:50:39 +0000 (14:50 +0200)]
tests/blackbox: add test for net ads JSON output

Implement blackbox tests for

    $ net ads info --json
    $ net ads lookup --json

that validate

    a) JSON wellformedness (by feeding it into the JSON library
       that ships with Python), and
    b) equality of the set of keys printed to that of the
       non-JSON version.

Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Signed-off-by: Philipp Gesang <philipp.gesang@intra2net.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Sat Sep 22 09:20:09 CEST 2018 on sn-devel-144

5 years agos3: net: normalize output of lookup subcommand
Philipp Gesang [Mon, 20 Aug 2018 13:10:31 +0000 (15:10 +0200)]
s3: net: normalize output of lookup subcommand

Use spaces and tabs consistently following the majority of the
printed output: tabs only for indenting, no space before the
colon separator, a single space after the separator.

The irregularities in formatting date back to the original commit
2c029a8b96..

Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Signed-off-by: Philipp Gesang <philipp.gesang@intra2net.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agos3: net: implement json output for ads lookup
Philipp Gesang [Tue, 3 Jul 2018 10:09:17 +0000 (12:09 +0200)]
s3: net: implement json output for ads lookup

Add JSON printer (option '--json') for the 'net ads lookup'
command. This outputs the same information as the plain version,
with integral ({LMNT,LM20} Token, NT Version) and boolean values
(Flags) not stringified.

Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Signed-off-by: Philipp Gesang <philipp.gesang@intra2net.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agos3: net: implement json output for ads info
Philipp Gesang [Mon, 2 Jul 2018 14:21:59 +0000 (16:21 +0200)]
s3: net: implement json output for ads info

Add the switch '--json' to 'net' to format the output as JSON.

The rationale is to supply the information in a machine-readable
fashion to complement the text version of the output which is
neither particularly well defined nor locale-safe.

The output differs from that of plain 'info' in that times are
not formatted as timestamps.

Currently affects only the 'net ads info' subcommand.

Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Signed-off-by: Philipp Gesang <philipp.gesang@intra2net.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agolib/audit_logging: make json_{is_invalid,to_string}() accept a const*
Philipp Gesang [Mon, 9 Jul 2018 07:41:37 +0000 (09:41 +0200)]
lib/audit_logging: make json_{is_invalid,to_string}() accept a const*

Allow for json_is_invalid() and json_to_string() to be used on a
const pointer. Neither function requires for the json object to
be mutable so constraining them to non-const* is unnecessary.

Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Signed-off-by: Philipp Gesang <philipp.gesang@intra2net.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agoselftest: Tweak PSO test-suite name
Tim Beale [Thu, 20 Sep 2018 01:46:32 +0000 (13:46 +1200)]
selftest: Tweak PSO test-suite name

There are 2 different PSO tests:
- make test TESTS=ldap.password_settings
- make test TESTS=samba_tool.passwordsettings

There's also another test that's completely unrelated to PSOs:
- make test TESTS=blackbox.password_settings

This patch renames ldap.password_settings --> ldap.passwordsettings.
This means 'make test TESTS=passwordsettings' will run both PSO tests,
but not the unrelated blackbox test.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Sep 21 22:58:17 CEST 2018 on sn-devel-144

5 years agotests: test ldap search requesting non-existent attribute
Tim Beale [Fri, 14 Sep 2018 02:06:30 +0000 (14:06 +1200)]
tests: test ldap search requesting non-existent attribute

As an after-thought to commit 563e454e8c55e94a950, we thought it
might be a good idea to add a test case that requests an non-existent
attribute in the attribute-filter as well the search-filter.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agoacl_read: Rework Samba code to reflect Windows logic
Tim Beale [Fri, 14 Sep 2018 01:27:56 +0000 (13:27 +1200)]
acl_read: Rework Samba code to reflect Windows logic

This patch should not alter functionality. It is just updating the Samba
code to better match the Windows specification docs.

When fixing Samba BUG #13434, the Microsoft behaviour wasn't clearly
documented, so we made a best guess based on observed behaviour.
The problem was an exception was made to allow "objectClass=*" searches
to return objects, even if you didn't have Read Property rights for the
object's objectClass attribute. However, the logic behind what
attributes were and weren't covered by this exception wasn't clear.

I made a guess that it was attributes belonging to the Public Info
property-set that also have the systemOnly flag set.

Microsoft have confirmed the object visibility behaviour. It turns out
that an optimization is made for the 4 attributes that are always
present for every object (i.e. objectClass, distinguishedName,
name, objectGUID). They're updating their Docs to reflect this.

Now that we know the Windows logic, we can update the Samba code.
This simplifies the code somewhat.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agotests: Minor code cleanups to confidential_attr test
Tim Beale [Wed, 1 Aug 2018 05:30:37 +0000 (17:30 +1200)]
tests: Minor code cleanups to confidential_attr test

+ fix a couple of flake8 warnings
+ add some extra code comments (particularly around the cases where the
  child class overrides a particular method, to avoid confusion when
  browsing the code).
+ assert_not_in_result() was duplicated (it's only needed for the deny
  ACL tests)
+ skip redundant if in dirsync's assert_search_result() (it always has
  to use the base-DN - we never pass it this as an args).

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agopython tests: fix format() strings for Python 2.6
Douglas Bagnall [Fri, 21 Sep 2018 01:22:56 +0000 (13:22 +1200)]
python tests: fix format() strings for Python 2.6

Python 2.6 wants "{0}".format(x), not "{}".format(x).

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agoldb: Fix dependency of ldb module
Amitay Isaacs [Fri, 21 Sep 2018 04:33:42 +0000 (14:33 +1000)]
ldb: Fix dependency of ldb module

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13626
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
5 years agoheimdal-build: Avoid using python str.format()
Amitay Isaacs [Wed, 19 Sep 2018 08:31:34 +0000 (18:31 +1000)]
heimdal-build: Avoid using python str.format()

In python 2.6, the format fields need to be numbered explicitly.  This
causes the build to fail on RHEL6/Centos6 with following error:

 File "/home/amitay/samba.git/source4/heimdal_build/wscript_build", line 87, in HEIMDAL_ASN1
    os.path.join(bld.path.abspath(), option_file)))
ValueError: zero length field name in format

To use str.format() in HEIMDAL_ASN1(),

   "--option-file='{}'"  needs to be   "--options-file='{0}'"

Or this fix which avoids str.format() completely.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Fri Sep 21 03:06:44 CEST 2018 on sn-devel-144

5 years agoFix pdb_samba_dsdb build on Ubuntu 16.04 with -O3 -Werror=strict-overflow
Andrew Bartlett [Mon, 17 Sep 2018 22:34:33 +0000 (15:34 -0700)]
Fix pdb_samba_dsdb build on Ubuntu 16.04 with -O3 -Werror=strict-overflow

../source3/passdb/pdb_samba_dsdb.c: In function ‘pdb_samba_dsdb_set_trusteddom_pw’:
../source3/passdb/pdb_samba_dsdb.c:2778:6: error: assuming signed overflow does not occur when simplifying conditional to constant [-Werror=strict-overflow]
   if (i == 0) {
      ^
cc1: all warnings being treated as errors

gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.10)

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Sep 20 01:29:48 CEST 2018 on sn-devel-144

5 years agos3: nmbd: Stop nmbd network announce storm.
Andrew Bartlett [Mon, 17 Sep 2018 19:45:14 +0000 (12:45 -0700)]
s3: nmbd: Stop nmbd network announce storm.

Correct fix for. On announce, work->lastannounce_time is set
to current time t, so we must check that 't >= work->lastannounce_time',
not 't > work->lastannounce_time' otherwise we end up not
doing the comparison, and always doing the announce.

Reported by Reuben Farrelly

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13620

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Revviewe-by: Andreas Schneider <asn@samba.org>