VERSION: Disable GIT_SNAPSHOT for the 4.9.17 release.

o CVE-2019-14861: Samba AD DC zone-named record Denial of Service in DNS
                  management server (dnsserver).
o CVE-2019-14870: DelegationNotAllowed not being enforced in protocol transition
                  on Samba AD DC.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

WHATSNEW: Add release notes for Samba 4.9.17.

o CVE-2019-14861: Samba AD DC zone-named record Denial of Service in DNS
                  management server (dnsserver).
o CVE-2019-14870: DelegationNotAllowed not being enforced in protocol transition
                  on Samba AD DC.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

CVE-2019-14870: mit-kdc: enforce delegation_not_allowed flag

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14187

Signed-off-by: Isaac Boukris <iboukris@samba.org>

CVE-2019-14870: heimdal: enforce delegation_not_allowed in S4U2Self

Signed-off-by: Isaac Boukris <iboukris@gmail.com>

CVE-2019-14870: heimdal: add S4U test for delegation_not_allowed

Signed-off-by: Isaac Boukris <iboukris@gmail.com>

samba-tool: add user-sensitive command to set not-delegated flag

Signed-off-by: Isaac Boukris <iboukris@gmail.com>

s4-torture: Reduce flapping in SambaToolDrsTests.test_samba_tool_replicate_local

This test often flaps in Samba 4.9 (where more tests and DCs run in the environment)
with obj_1 being 3.  This is quite OK, we just need to see some changes get
replicated, not 0 changes.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
(cherry picked from commit 4ae0f9ce0f5ada99cf1d236377e5a1234c879ae3)

CVE-2019-14861: Test to demonstrate the bug

This test does not fail every time, but when it does it casues a segfault which
takes out the rpc_server master process, as this hosts the dnsserver pipe.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14138

Signed-off-by: Andrew Bartlett <abartlet@samba.org>

CVE-2019-14861: s4-rpc/dnsserver: Avoid crash in ldb_qsort() via dcesrv_DnssrvEnumRecords)

dns_name_compare() had logic to put @ and the top record in the tree being
enumerated first, but if a domain had both then this would break the
older qsort() implementation in ldb_qsort() and cause a read of memory
before the base pointer.

By removing this special case (not required as the base pointer
is already seperatly located, no matter were it is in the
returned records) the crash is avoided.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14138

Signed-off-by: Andrew Bartlett <abartlet@samba.org>

CVE-2019-14861: s4-rpc_server: Remove special case for @ in dns_build_tree()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14138

Signed-off-by: Andrew Bartlett <abartlet@samba.org>

CVE-2019-14861: s4-rpc/dnsserver: Confirm sort behaviour in dcesrv_DnssrvEnumRecords

The sort behaviour for child records is not correct in Samba so
we add a flapping entry.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14138

Signed-off-by: Andrew Bartlett <abartlet@samba.org>

VERSION: Bump version up to 4.9.17...

and re-enable GIT_SNAPSHOT.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

VERSION: Disable GIT_SNAPSHOT for th 4.9.16 release.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

WHATSNEW: Add release notes for Samba 4.9.16.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

Merge tag 'samba-4.9.15' into v4-9-test

samba: tag release samba-4.9.15
Signed-off-by: Karolin Seeger <kseeger@samba.org>

ctdb-tcp: Close inflight connecting TCP sockets after fork

Commit c68b6f96f26 changed the talloc hierarchy such that outgoing TCP sockets
while sitting in the async connect() syscall are not freed via
ctdb_tcp_shutdown() anymore, they are hanging off a longer-running structure.
Free this structure as well.

If an outgoing TCP socket leaks into a long-running child process (possibly the
recovery daemon), this connection will never be closed as seen by the
destination node. Because with recent changes incoming connections will not be
accepted as long as any incoming connection is alive, with that socket leak
into the recovery daemon we will never again be able to successfully connect to
the node that is affected by this leak. Further attempts to connect will be
discarded by the destination as long as the recovery daemon keeps this socket
alive.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14175
RN: Avoid communication breakdown on node reconnect

Signed-off-by: Martin Schwenke <martin@meltin.net>
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit a6d99d9e5c5bc58e6d56be7a6c1dbc7c8d1a882f)

Autobuild-User(v4-9-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-9-test): Wed Nov 20 14:58:33 UTC 2019 on sn-devel-144

ctdb-tcp: Drop tracking of file descriptor for incoming connections

This file descriptor is owned by the incoming queue.  It will be
closed when the queue is torn down.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14175

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit bf47bc18bb8a94231870ef821c0352b7a15c2e28)

ctdb-tcp: Avoid orphaning the TCP incoming queue

CTDB's incoming queue handling does not check whether an existing
queue exists, so can overwrite the pointer to the queue.  This used to
be harmless until commit c68b6f96f26664459187ab2fbd56767fb31767e0
changed the read callback to use a parent structure as the callback
data.  Instead of cleaning up an orphaned queue on disconnect, as
before, this will now free the new queue.

At first glance it doesn't seem possible that 2 incoming connections
from the same node could be processed before the intervening
disconnect.  However, the incoming connections and disconnect occur on
different file descriptors.  The queue can become orphaned on node A
when the following sequence occurs:

1. Node A comes up
2. Node A accepts an incoming connection from node B
3. Node B processes a timeout before noticing that outgoing the queue is writable
4. Node B tears down the outgoing connection to node A
5. Node B initiates a new connection to node A
6. Node A accepts an incoming connection from node B

Node A processes then the disconnect of the old incoming connection
from (2) but tears down the new incoming connection from (6).  This
then occurs until the originally affected node is restarted.

However, due to the number of outgoing connection attempts and
associated teardowns, this induces the same behaviour on the
corresponding incoming queue on all nodes that node A attempts to
connect to.  Therefore, other nodes become affected and need to be
restarted too.

As a result, the whole cluster probably needs to be restarted to
recover from this situation.

The problem can occur any time CTDB is started on a node.

The fix is to avoid accepting new incoming connections when a queue
for incoming connections is already present.  The connecting node will
simply retry establishing its outgoing connection.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14175

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit d0baad257e511280ff3e5c7372c38c43df841070)

ctdb-tcp: Check incoming queue to see if incoming connection is up

This makes it consistent with the reverse case.  Also, in_fd will soon
be removed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14175

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit e62b3a05a874db13a848573d2e2fb1c157393b9c)

VERSION: Bump version up to 4.9.16.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

VERSION: Disable GIT_SNAPSHOT for the 4.9.15 release.

* Bug 14071: CVE-2019-10218: Client code can return filenames containing path
  separators.
* Bug 12438: CVE-2019-14833: Samba AD DC check password script does not receive
  the full password.
* Bug 14040: CVE-2019-14847: User with "get changes" permission can crash AD DC LDAP
  server via dirsync.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

WHATSNEW: Add release notes for Samba 4.9.15.

* Bug 14071: CVE-2019-10218: Client code can return filenames containing path
  separators.
* Bug 12438: CVE-2019-14833: Samba AD DC check password script does not receive
  the full password.
* Bug 14040: CVE-2019-14847: User with "get changes" permission can crash AD DC LDAP
  server via dirsync.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

CVE-2019-14847 dsdb: Correct behaviour of ranged_results when combined with dirsync

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14040

Signed-off-by: Andrew Bartlett <abartlet@samba.org>

CVE-2019-14847 dsdb: Demonstrate the correct interaction of ranged_results style attributes and dirsync

Incremental results are provided by a flag on the dirsync control, not
by changing the attribute name.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14040

Signed-off-by: Andrew Bartlett <abartlet@samba.org>

CVE-2019-14847 dsdb/modules/dirsync: ensure attrs exist (CID 1107212)

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14040

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
(cherry picked from commit 23f72c4d712f8d1fec3d67a66d477709d5b0abe2)

CVE-2019-14833 dsdb: send full password to check password script

utf8_len represents the number of characters (not bytes) of the
password. If the password includes multi-byte characters it is required
to write the total number of bytes to the check password script.
Otherwise the last bytes of the password string would be ignored.

Therefore we rename utf8_len to be clear what it does and does
not represent.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12438

Signed-off-by: Björn Baumbach <bb@sernet.de>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>

CVE-2019-14833: Use utf8 characters in the unacceptable password

This shows that the "check password script" handling has a bug.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12438
Signed-off-by: Andrew Bartlett <abartlet@samba.org>

CVE-2019-10218 - s3: libsmb: Protect SMB2 client code from evil server returned names.

Disconnect with NT_STATUS_INVALID_NETWORK_RESPONSE if so.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14071

Signed-off-by: Jeremy Allison <jra@samba.org>

CVE-2019-10218 - s3: libsmb: Protect SMB1 client code from evil server returned names.

Disconnect with NT_STATUS_INVALID_NETWORK_RESPONSE if so.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14071

Signed-off-by: Jeremy Allison <jra@samba.org>

VERSION: Bump version up to 4.9.15...

and re-enable GIT_SNAPSHOT.

Signed-off-by: Karolin Seeger <kseeger@samba.org>
(cherry picked from commit 5942df0864495dbaea68d2f45b5a6d343f0556ba)

VERSION: Bump version up to 4.9.15...

and re-enable GIT_SNAPSHOT.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

VERSION: Disable GIT_SNAPSHOT for the 4.9.14 release.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

WHATSNEW: Add release notes for Samba 4.9.14.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

spnego: fix server handling of no optimistic exchange

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14106

Signed-off-by: Isaac Boukris <iboukris@redhat.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Sat Oct 12 15:51:42 UTC 2019 on sn-devel-184

Autobuild-User(v4-9-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-9-test): Wed Oct 16 16:47:14 UTC 2019 on sn-devel-144

selftest: add tests for no optimistic spnego exchange

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14106

Signed-off-by: Isaac Boukris <iboukris@redhat.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

spnego: add client option to omit sending an optimistic token

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14106

Signed-off-by: Isaac Boukris <iboukris@redhat.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

ctdb-vacuum: Process all records not deleted on a remote node

This currently skips the last record.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14147
RN: Avoid potential data loss during recovery after vacuuming error

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
(cherry picked from commit 33f1c9d9654fbdcb99c23f9d23c4bbe2cc596b98)

fault.c: improve fault_report message text pointing to our wiki

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14139

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
(cherry picked from commit ec4c5975528f3d3ab9c8813e176c6d1a2f1ca506)

Autobuild-User(v4-9-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-9-test): Thu Sep 26 04:30:04 UTC 2019 on sn-devel-144

ctdb-tools: Stop deleted nodes from influencing ctdb nodestatus exit code

Deleted nodes should simply be ignored.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14129
RN: Stop deleted nodes from influencing ctdb nodestatus exit code

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 32b5ceb31936ec5447362236c1809db003561d29)

Autobuild-User(v4-9-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-9-test): Fri Sep 20 14:09:11 UTC 2019 on sn-devel-144

s3/4: libsmbclient test. Test using smbc_telldir/smbc_lseekdir with smbc_readdir/smbc_readdirplus/smbc_getdents.

Ensure that for file access you can mix any of these
three access methods for directory entries and the
returned names/structs stay in sync across telldir/seekdir
changes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14094

Back-ported from master 3355601fe8541994cc41f5ed800aab9b6a2294f4.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>
Autobuild-User(v4-9-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-9-test): Thu Sep 19 10:40:56 UTC 2019 on sn-devel-144

s3: libsmbclient: Fix smbc_lseekdir() to work with smbc_readdirplus().

If returning files the dir_list and the dirplus_list have exactly the same
entries, we just need to keep the next pointers in sync on seek.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14094

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>
(cherry picked from commit 0d9b1645499ce12a79a137d3482434aa5d2eb47c)

s3: libsmbclient: Ensure SMBC_getdents_ctx() also updates the readdirplus pointers.

If we are returning file entries, we
have a duplicate list in dirplus.

Update dirplus_next also so readdir and
readdirplus are kept in sync.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14094

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>
(cherry picked from commit 754cec7756b2ddb1cfcc3984265f01cb366beb76)

s3: libsmbclient: Ensure SMBC_readdirplus_ctx() also updates the readdir pointers.

If we are returning file entries, we
have a duplicate list in dir_list.

Update dir_next also so readdir and
readdirplus are kept in sync.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14094

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>
(cherry picked from commit 3d82b7d11cd7b78adc6b3642e64e3a8f251de869)

s3: libsmbclient: Ensure SMBC_readdir_ctx() also updates the readdirplus pointers.

If we are returning file entries, we
have a duplicate list in dirplus.

Update dirplus_next also so readdir and
readdirplus are kept in sync.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14094

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>
(cherry picked from commit 4bca8e097f5a909c628daa4dbfa932ddc1725ebc)

libcli/smb: send SMB2_NETNAME_NEGOTIATE_CONTEXT_ID

Note: Unlike the current documentation, the utf16 string
is not null-terminated, that matches Windows Server 1903
as a client.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14055
RN: Add the target server name of SMB 3.1.1 connections
as a hint to load balancers or servers with "multi-tenancy"
support.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Aurelien Aptel <aaptel@suse.com>
(similar to commit 21f6cece543dd791e0f4636458bfe9819823420c)

Autobuild-User(v4-9-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-9-test): Wed Sep 11 13:27:24 UTC 2019 on sn-devel-144

libcli/smb: add new COMPRESSION and NETNAME negotiate context ids

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14055

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Aurelien Aptel <aaptel@suse.com>
(cherry picked from commit e10b90f33bb812600886656a1124e2d434416563)

s3:ldap: Fix join with don't exists machine account

Add check for requested replies of existing machine object during join
machine to domain. This solves regression fail during join with error:
"None of the information to be translated has been translated."

https://bugzilla.samba.org/show_bug.cgi?id=14007

Reviewed-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Günther Deschner <gd@samba.org>
Autobuild-Date(master): Wed Sep  4 17:02:37 UTC 2019 on sn-devel-184

(cherry picked from commit ad4ef1657e9b2a088a3bfadcce196cfcceead1dc)

ctdb: fix compilation on systems with glibc robust mutexes

On older systems like SLES 11 without POSIX robust mutexes, but with glib robust
mutexes where all the functions are available but have a "_np" suffix,
compilation fails in:

ctdb/tests/src/test_mutex_raw.c.239.o: In function `worker':
/root/samba-4.10.6/bin/default/../../ctdb/tests/src/test_mutex_raw.c:129: undefined reference to `pthread_mutex_consistent'
ctdb/tests/src/test_mutex_raw.c.239.o: In function `main':
/root/samba-4.10.6/bin/default/../../ctdb/tests/src/test_mutex_raw.c:285: undefined reference to `pthread_mutex_consistent'
/root/samba-4.10.6/bin/default/../../ctdb/tests/src/test_mutex_raw.c:332: undefined reference to `pthread_mutexattr_setrobust'
/root/samba-4.10.6/bin/default/../../ctdb/tests/src/test_mutex_raw.c:363: undefined reference to `pthread_mutex_consistent'
collect2: ld returned 1 exit status

This could be fixed by using libreplace system/threads.h instead of pthreads.h
directly, but as there has been a desire to keep test_mutex_raw.c standalone and
compilable without other external depenencies then libc and libpthread, make the
tool developer build only. This should get the average user over the cliff.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14038
RN: Fix compiling ctdb on older systems lacking POSIX robust mutexes

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
(cherry picked from commit f5388f97792ac2d7962950dad91aaf8ad49bceaa)

Autobuild-User(v4-9-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-9-test): Thu Sep  5 16:12:34 UTC 2019 on sn-devel-144

vfs_glusterfs: Use pthreadpool for scheduling aio operations

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14098

Signed-off-by: Poornima G <pgurusid@redhat.com>
Reviewed-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(v4-9-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-9-test): Tue Sep  3 16:01:37 UTC 2019 on sn-devel-144

ctdb-recoverd: Fix typo in previous fix

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14085

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Tue Aug 27 15:29:11 UTC 2019 on sn-devel-184

(cherry picked from commit 8190993d99284162bd8699780248bb2edfec2673)

ctdb-tests: Clear deleted record via recovery instead of vacuuming

This test has been flapping because sometimes the record is not
vacuumed within the expected time period, perhaps even because the
check for the record can interfere with vacuuming.  However, instead
of waiting for vacuuming the record can be cleared by doing a
recovery.  This should be much more reliable.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14085
RN: Fix flapping CTDB tests

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Wed Aug 21 13:06:57 UTC 2019 on sn-devel-184

(backported from commit 71ad473ba805abe23bbe6c1a1290612e448e73f3)
Signed-off-by: Martin Schwenke <martin@meltin.net>

ctdb-tests: Strengthen volatile DB traverse test

Check the record count more often, from multiple nodes.  Add a case
with multiple records.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14085

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit ca4df06080709adf0cbebc95b0a70b4090dad5ba)

ctdb-recoverd: Only check for LMASTER nodes in the VNN map

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14085

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 5d655ac6f2ff82f8f1c89b06870d600a1a3c7a8a)

ctdb-tests: Don't retrieve the VNN map from target node for notlmaster

Use the VNN map from the node running node_has_status().

This means that

  wait_until_node_has_status 1 notlmaster 10 0

will run "ctdb status" on node 0 and check (for up to 10 seconds) if
node 1 is in the VNN map.

If the LMASTER capability has been dropped on node 1 then the above
will wait for the VNN map to be updated on node 0.  This will happen
as part of the recovery that is triggered by the change of LMASTER
capability.  The next command will then only be able to attach to
$TESTDB after the recovery is complete thus guaranteeing a sane state
for the test to continue.

This stops simple/79_volatile_db_traverse.sh from going into recovery
during the traverse or at some other inconvenient time.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14085

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 53daeb2f878af1634a26e05cb86d87e2faf20173)

ctdb-tests: Handle special cases first and return

All the other cases involve matching bits.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14085

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit bff1a3a548a2cace997b767d78bb824438664cb7)

ctdb-tests: Inline handling of recovered and notlmaster statuses

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14085

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit bb59073515ee5f7886b5d9a20d7b2805857c2708)

ctdb-tests: Drop unused node statuses frozen/unfrozen

Silently drop unused local variable mpat.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14085

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 9b09a87326af28877301ad27bcec5bb13744e2b6)

ctdb-tests: Reformat node_has_status()

Re-indent and drop non-POSIX left-parenthesis from case labels.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14085

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 52227d19735a3305ad633672c70385f443f222f0)

VERSION: Bump version up to 4.9.14.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

Merge tag 'samba-4.9.13' into v4-9-test

samba: tag release samba-4.9.13

ctdb-daemon: Make node inactive in the NODE_STOP control

Currently some of this is supported by a periodic check in the
recovery daemon's main_loop(), which notices the flag change, sets
recovery mode active and freezes databases.  If STOP_NODE returns
immediately then the associated recovery can complete and the node can
be continued before databases are actually frozen.

Instead, immediately do all of the things that make a node inactive.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14087
RN: Stop "ctdb stop" from completing before freezing databases

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Tue Aug 20 08:32:27 UTC 2019 on sn-devel-184

(cherry picked from commit e9f2e205ee89f4f3d6302cc11b4d0eb2efaf0f53)

Autobuild-User(v4-9-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-9-test): Wed Aug 28 12:04:13 UTC 2019 on sn-devel-144

VERSION: Disable GIT_SNAPSHOT for the 4.9.13 release.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

WHATSNEW: Add release notes for Samba 4.9.13.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14035
CVE-2019-10197 [SECURITY][EMBARGOED] permissions check deny can allow user to
escape from the share.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

CVE-2019-10197: smbd: split change_to_user_impersonate() out of change_to_user_internal()

This makes sure we always call chdir_current_service() even
when we still impersonated the user. Which is important
in order to run the SMB* request within the correct working directory
and only if the user has permissions to enter that directory.

It makes sure we always update conn->lastused_count
in chdir_current_service() for each request.

Note that vfs_ChDir() (called from chdir_current_service())
maintains its own cache and avoids calling SMB_VFS_CHDIR()
if possible.

It means we still avoid syscalls if we get a multiple requests
for the same session/tcon tuple.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14035

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

CVE-2019-10197: test_smbclient_s3.sh: add regression test for the no permission on share root problem

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14035

Signed-off-by: Stefan Metzmacher <metze@samba.org>

CVE-2019-10197: selftest: make fsrvp_share its own independent subdirectory

The next patch will otherwise break the fsrvp related tests.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14035

Signed-off-by: Stefan Metzmacher <metze@samba.org>

CVE-2019-10197: smbd: make sure we reset current_user.{need,done}_chdir in become_root()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14035

Signed-off-by: Stefan Metzmacher <metze@samba.org>

CVE-2019-10197: smbd: make sure that change_to_user_internal() always resets current_user.done_chdir

We should not leave current_user.done_chdir as true if we didn't call
chdir_current_service() with success.

This caused problems in when calling vfs_ChDir() in pop_conn_ctx() when
chdir_current_service() worked once on one share but later failed on another
share.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14035

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

CVE-2019-10197: smbd: separate out impersonation debug info into a new function.

Will be called on elsewhere on successful impersonation.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14035

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

VERSION: Bump version up to 4.9.13...

and re-enable GIT_SNAPSHOT.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

ctdb-daemon: Drop unused function ctdb_local_node_got_banned()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14087

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 91ac4c13d8472955d1f04bd775ec4b3ff8bf1b61)

ctdb-daemon: Switch banning code to use ctdb_node_become_inactive()

There's no reason to avoid immediately setting recovery mode to active
and initiating freeze of databases.

This effectively reverts the following commits:

  d8f3b490bbb691c9916eed0df5b980c1aef23c85
  b4357a79d916b1f8ade8fa78563fbef0ce670aa9

The latter is now implemented using a control, resulting in looser
coupling.

See also the following commit:

  f8141e91a693912ea1107a49320e83702a80757a

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14087

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 0f5f7b7cf4e970f3f36c5e0b3d09e710fe90801a)

ctdb-daemon: Factor out new function ctdb_node_become_inactive()

This is a superset of ctdb_local_node_got_banned() so will replace
that function, and will also be used in the NODE_STOP control.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14087

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit a42bcaabb63722411bee52b80cbfc795593defbc)

ctdb-tcp: Mark node as disconnected if incoming connection goes away

To make it easy to pass the node data to the upcall, the private data
for ctdb_tcp_read_cb() needs to be changed from tnode to node.

RN: Avoid marking a node as connected before it can receive packets
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14084

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Fri Aug 16 22:50:35 UTC 2019 on sn-devel-184

(cherry picked from commit 73c850eda4209b688a169aeeb20c453b738cbb35)

ctdb-tcp: Only mark a node connected if both directions are up

Nodes are currently marked as up if the outgoing connection is
established.  However, if the incoming connection is not yet
established then this node could send a request where the replying
node can not queue its reply.  Wait until both directions are up
before marking a node as connected.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14084

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 8c98c10f242bc722beffc711e85c0e4f2e74cd57)

ctdb-tcp: Create outbound queue when the connection becomes writable

Since commit ddd97553f0a8bfaada178ec4a7460d76fa21f079
ctdb_queue_send() doesn't queue a packet if the connection isn't yet
established (i.e. when fd == -1).  So, don't bother creating the
outbound queue during initialisation but create it when the connection
becomes writable.

Now the presence of the queue indicates that the outbound connection
is up.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14084

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 7f4854d9643a096a6d8a354fcd27b7c6ed24a75e)

ctdb-tcp: Use TALLOC_FREE()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14084

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit d80d9edb4dc107b15a35a39e5c966a3eaed6453a)

ctdb-tcp: Move incoming fd and queue into struct ctdb_tcp_node

This makes it easy to track both incoming and outgoing connectivity
states.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14084

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit c68b6f96f26664459187ab2fbd56767fb31767e0)

ctdb-tcp: Rename fd -> out_fd

in_fd is coming soon.

Fix coding style violations in the affected and adjacent lines.
Modernise some debug macros and make them more consistent (e.g. drop
logging of errno when strerror(errno) is already logged.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14084

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit c06620169fc178ea6db2631f03edf008285d8cf2)

ctdb-daemon: Add function ctdb_ip_to_node()

This is the core logic from ctdb_ip_to_pnn(), so re-implement that
that function using ctdb_ip_to_node().

Something similar (ctdb_ip_to_nodeid()) was recently removed in commit
010c1d77cd7e192b1fff39b7b91fccbdbbf4a786 because it wasn't required.
Now there is a use case.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14084

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 3acb8e9d1c854b577d6be282257269df83055d31)

ctdb-daemon: Replace function ctdb_ip_to_nodeid() with ctdb_ip_to_pnn()

Node ID is a poorly defined concept, indicating the slot in the node
map where the IP address was found.  This signed value also ends up
compared to num_nodes, which is unsigned, producing unwanted warnings.

Just return the PNN because this what both callers really want.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 010c1d77cd7e192b1fff39b7b91fccbdbbf4a786)

undoguididx: blackbox test

This test confirms that running undoguididx causes all GUID keys to be
replaced with DN keys at the KV level

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13978

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
(backported from commits 74d15c9bf76f0a2fb5fa7b7b1d80971d10c4fe45,
ab376a97c972d2d5ebfb912ed90664c787860dc8 and 56400153c8c7052fe319f273c30c6d59556102dc
to avoid changes to TestCaseInTempDir).

ab376a97c972d2d5ebfb912ed90664c787860dc8 was:

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
56400153c8c7052fe319f273c30c6d59556102dc was:

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

undoguididx: Add "or later" to warning about using tools from Samba 4.8

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13978

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
(cherry picked from commit 09f2a187b3d8c161e2c11588499b3256a9dbcc95)

sambaundoguididx: fix for -s

Quick fix running this script with -s instead of -H. samdb_url() returns
a url with a protocol prefix, which causes issues further down in the
script.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13978

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
(cherry picked from commit 40ca8ed5a152ae7c5ec039649c09a037a20a4143)

sambaundoguididx: Add flags=ldb.FLG_DONT_CREATE_DB and port to Python3

In py3 we need to add an extra str() around the returned ldb value to
enable .split() to be used.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13978

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed By: Noel Power <npower@samba.org>

Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Thu May 23 14:25:52 UTC 2019 on sn-devel-184

(cherry picked from commit 1a9da378a1505daff498be6d6355debd73526a1a)

s4/scripting: MORE py3 compatible print functions

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13978

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
(cherry picked from commit 561b654bc5bc2f5e614c5c2ab378193ca94d481a)

s4/scripting/*: py3 compatible print

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
(cherry picked from commit 78f5b6e3999a0bf4a118df36a2aabcb696049792)

ldb: release ldb 1.4.8

* Check for errors from ldb_unpack_data() in ldb_tdb (bug 13959)
* Check for new pack formats during startup (bug 13977)
* Make ldbdump print out pack format info and keys so we have
  low level visibility for testing in python (for bug 13978)

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13959

Signed-off-by: Andrew Bartlett <abartlet@samba.org>

ldb: ldbdump key and pack format version comments

For testing we need to know the actual KV level key of records and each
record's pack format version. This patch makes ldbdump add comments with
that info. We will parse it out in python tests.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13978

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed May 22 05:58:17 UTC 2019 on sn-devel-184

(cherry picked from commit a666a99e4dc594bc153cd26b24cddd547c1cc750)

ldb: baseinfo pack format check on init

We will be adding a new packing format in forthcoming commits and there
may be more versions in the future. We need to make sure the database
contains records in a format we know how to read and write.
Done by fetching the @BASEINFO record and reading the first 4
bytes which contain the packing format version.

NOTE: Configure with --abi-check-disable to build this commit. This
patch is part of a set of LDB ABI changes, and the version update is
done on the last commit.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13977

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
(backported from commit 474e55523224430781ed22aa2d0c8a474306e794)

ldb: Fix segfault parsing new pack formats

We need to check for the errors given by ldb_unpack() et al by preserving
the error code from kv_ctx->parser() called by tdb_parse_record() in
ltdb_parse_record().

Otherwise we will silently accept corrupt records and segfault later.

Likewise new pack formats will confuse the parser but not be
detected except by the incomplete struct ldb_message.

With this patch, the user will see a message like:

 Invalid data for index  DN=@BASEINFO

 Failed to connect to 'st/ad_dc/private/sam.ldb' with backend 'tdb': Unable to load ltdb cache records for backend 'ldb_tdb backend'
 Failed to connect to st/ad_dc/private/sam.ldb - Unable to load ltdb cache records for backend 'ldb_tdb backend'

This can be refined in the future by a specific check for
pack format versions in a higher caller, but this much is
needed regardless to detect corrupt records.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13959

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
(cherry picked from commit a3101b9704f554a350493553336cbbbd7d4ae02e)

ldb: test for parse errors

Parse errors aren't passed up correctly by the tdb backend. This
patch modifies a test to expose the issue, next patch will fix it.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13959

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
(cherry picked from commit 2de0aebed60b8e83508f50e5391ede618ce0e595)

VERSION: Diable GIT_SNAPSHOT for the 4.9.12 release.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

VERSION: Bump version up to 4.9.13...

and re-enable GIT_SNAPSHOT.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

WHATSNEW: Add release notes for Samba 4.9.12.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

vfs_glusterfs: Enable profiling for file system operations

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14093

Signed-off-by: Anoop C S <anoopcs@redhat.com>
Reviewed-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Aug 20 19:25:28 UTC 2019 on sn-devel-184

Autobuild-User(v4-9-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-9-test): Mon Aug 26 13:26:08 UTC 2019 on sn-devel-144

vfs_gpfs: Implement special case for denying owner access to ACL

In GPFS, it is not possible to deny ACL or attribute access through a
SPECIAL_OWNER entry. The best that can be done is mapping this to a
named user entry, as this one can at least be stored in an ACL. The same
cannot be done for inheriting SPECIAL_OWNER entries, as these represent
CREATOR OWNER entries, and the limitation of not being able to deny
owner access to ACL or attributes remains.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14032

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit c1770ed96fd3137f45d584ba9328333d5505e3af)

vfs_gpfs: Move mapping from generic NFSv ACL to GPFS ACL to separate function

This is not functional change. It cleans up the code a bit and makes
expanding this codepath in a later patch easier.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14032

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit fbf3a090a9ec94262b2924461cc1d6336af9919c)

docs: Remove gpfs:merge_writeappend from vfs_gpfs manpage

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14032

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit 8bd79ecc37376dbaa35606f9c2777653eb3d55e3)

vfs_gpfs: Remove merge_writeappend parameter

All supported GPFS versions now support setting WRITE and APPEND in the
ACLs independently. Remove this now unused parameter to simplify the
code.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14032

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit 0aca678fcf1788a76cf0ff11399211c795aa7d2f)