ldap_server: Add the krb5 expiry to conn->limits

Bug: https://bugzilla.samba.org/show_bug.cgi?id=14465

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 77f72fb01faba45babfe6080f805361492ce49e5)

torture: Test ldap session expiry

LDAP connections should time out when the kerberos ticket used to authenticate
expires. Windows does this with a RFC4511 section 4.4.1 message (that as of
August 2020 is encoded not according to the RFC) followed by a TCP disconnect.

ldb sees the section 4.4.1 as a protocol violation and returns
LDB_ERR_PROTOCOL_ERROR.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=14465

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 35c4bb0b0c55a65490fe199edb1a534548104e95)

build: Wrap a long line

There will be another entry in the next commit

Bug: https://bugzilla.samba.org/show_bug.cgi?id=14465

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit c8c2f8ba73324ba43ccef9f6d1c0c726d7ec0d25)

VERSION: Bump version up to 4.13.0rc4...

and re-enable GIT_SNAPSHOT.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

VERSION: Disable GIT_SNAPSHOT for the 4.13.0rc3 release.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

WHATSNEW: Add release notes for Samba 4.13.0rc3.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

ctdb-recoverd: Rename update_local_flags() -> update_flags()

This also updates remote flags so the name is misleading.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14466
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 4aa8e72d60e92951b35190d2ffcfdb1bfb756609)

Autobuild-User(v4-13-test): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(v4-13-test): Thu Aug 27 12:11:01 UTC 2020 on sn-devel-184

ctdb-recoverd: Change update_local_flags() to use already retrieved nodemaps

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14466
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 702c7c4934e79a9161fdc59df70df30ae492d89f)

ctdb-recoverd: Get remote nodemaps earlier

update_local_flags() will be changed to use these nodemaps.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14466
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 910a0b3b747a987ba69b6a0b6256e964b7d85dfe)

ctdb-recoverd: Do not fetch the nodemap from the recovery master

The nodemap has already been fetched from the local node and is
actually passed to this function.  Care must be taken to avoid
referencing the "remote" nodemap for the recovery master.  It also
isn't useful to do so, since it would be the same nodemap.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14466
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit d50919b0cb28f299c9b6985271b29d4f27c5f619)

ctdb-recoverd: Change get_remote_nodemaps() to use connected nodes

The plan here is to use the nodemaps retrieved by get_remote_nodes()
in update_local_flags().  This will improve efficiency, since
get_remote_nodes() fetches flags from nodes in parallel.  It also
means that get_remote_nodes() can be used exactly once early on in
main_loop() to retrieve remote nodemaps.  Retrieving nodemaps multiple
times is unnecessary and racy - a single monitoring iteration should
not fetch flags multiple times and compare them.

This introduces a temporary behaviour change but it will be of no
consequence when the above changes are made.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14466
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 762d1d8a9605f97973a2c1176de5d29fcc61d15a)

ctdb-recoverd: Fix node_pnn check and assignment of nodemap into array

This array is indexed by the same index as nodemap, not the PNN.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14466
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 368c83bfe3bbfff568d14f65e7b1ffa41d5349ac)

ctdb-recoverd: Add fail callback to assign banning credits

Also drop error handling in main_loop() that is replaced by this
change.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14466
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 10ce0dbf1c11eaaab7b28b6bbd014235a36d1962)

ctdb-recoverd: Add an intermediate state struct for nodemap fetching

This will allow an error callback to be added.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14466
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit a079ee31690cf7110f46b41989ffcfb83b7626d6)

ctdb-recoverd: Move memory allocation into get_remote_nodemaps()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14466
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 2eaa0af6160588b6e3364b181d0976477d12b51b)

ctdb-recoverd: Change signature of get_remote_nodemaps()

Change 1st argument to a rec context, since this will be needed later.
Drop the nodemap argument and access it via rec->nodemap instead.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14466
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 3324dd272c7dafa92cd9c3fd0af8f50084bcdaaa)

ctdb-recoverd: Fix a local memory leak

The memory is allocated off the memory context used by the current
iteration of main loop.  It is freed when main loop completes the fix
doesn't require backporting to stable branches.  However, it is sloppy
so it is worth fixing.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14466
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit d2d90f250214582d7124b8137aa2cf5032b2f285)

ctdb-recoverd: Basic cleanups for get_remote_nodemaps()

Don't log an error on failure - let the caller can do this.  Apart
from this: fix up coding style and modernise the remaining error
message.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14466
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 52f520d39cd92e1cf2413fd7e0dd362debd6f463)

ctdb-recoverd: Simplify calculation of new flags

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Fri Jul 24 06:03:23 UTC 2020 on sn-devel-184

(cherry picked from commit 5ce6133a75107abdcb9fcfd93bc7594812dc5055)

ctdb-recoverd: Correctly find nodemap entry for pnn

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 3654e416770cc7521dcc3c15976daeba37023304)

ctdb-recoverd: Do not retrieve nodemap from recovery master

It is already in rec->nodemap.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 9475ab044161e687b9ced3a477746393565b49b1)

ctdb-recoverd: Flatten update_flags_on_all_nodes()

The logic currently in ctdb_ctrl_modflags() will be optimised so that
it no longer matches the pattern for a control function.  So, remove
this function and squash its functionality into the only caller.

Although there are some superficial changes, the behaviour is
unchanged.

Flattening the 2 functions produces some seriously weird logic for
setting the new flags, to the point where using ctdb_ctrl_modflags()
for this purpose now looks very strange.  The weirdness will be
cleaned up in a subsequent commit.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 0c6a7db3ba84b8355359b0a8c52690b234bb866d)

ctdb-recoverd: Move ctdb_ctrl_modflags() to ctdb_recoverd.c

This file is the only user of this function.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit a88c10c5a9afcf0a3dcadef07dd95af498bfa47a)

ctdb-recoverd: Improve a call to update_flags_on_all_nodes()

This should take a PNN, not an array index.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit b1e631ff929fd87392a80895d1c8d265d9df42dc)

ctdb-recoverd: Use update_flags_on_all_nodes()

This is clearer than using the MODFLAGS control directly.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 915d24ac12d27c21649d9e64d201d9df9d583129)

ctdb-recoverd: Introduce some local variables to improve readability

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit f681c0e947741151f8fb95d88edddfd732166dc1)

ctdb-recoverd: Change update_flags_on_all_nodes() to take rec argument

This makes fields such as recmaster and nodemap easily available if
required.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit cb3a3147b7a3a29d7806733791e1fa6ba2e46680)

ctdb-recoverd: Drop unused nodemap argument from update_flags_on_all_nodes()

An unused argument needlessly extends the length of function calls.  A
subsequent change will allow rec->nodemap to be used if necessary.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 6982fcb3e6c940d0047aac3b6bfbc9dfdc8d7214)

docs: Add missing winexe manpage

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14318

Guenther

Signed-off-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Thu Aug 20 12:55:23 UTC 2020 on sn-devel-184

(cherry picked from commit a9b6a8378e5db19d5d5ecd42f08ec3abd49ad5b8)

WHATSNEW: list deprecated parameters

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14460

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Aug 18 01:32:21 UTC 2020 on sn-devel-184

(cherry picked from commit 20606fd0a4c4697ff99da59f748af6908d929901)

Autobuild-User(v4-13-test): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(v4-13-test): Mon Aug 24 15:13:30 UTC 2020 on sn-devel-184

docs: deprecate "raw NTLMv2 auth"

This parameter is appicable only to SMBv1 and we are deprecating SMBv1 specific
authentication options for possible removal.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14460

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 8c9d9441edce2e8d7f0428d0ec5e209ed8a55dbc)

docs: deprecate "client plaintext auth"

This parameter is appicable only to SMBv1 and we are deprecating SMBv1 specific
authentication options for possible removal.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14460

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 37583b19d2c3dbf3e9d0498a39b8b9d9c727e1d4)

docs: deprecate "client NTLMv2 auth"

This parameter is appicable only to SMBv1 and we are deprecating SMBv1 specific
authentication options for possible removal.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14460

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 5543c11c8b007b49641758428af7ba3976683438)

docs: deprecate "client lanman auth"

This parameter is appicable only to SMBv1 and we are deprecating SMBv1 specific
authentication options for possible removal.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14460

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit ac8e5ea22d9f9b16a79f519f69852b46ac798541)

docs: deprecate "client use spnego"

This parameter is appicable only to SMBv1 and we are deprecating SMBv1 specific
authentication options for possible removal.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14460

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 1b85db57e53533ce14beb79f6d949a08f6ef9f91)

docs: Deprecate NT4-like domains and SMBv1-only protocol options

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14460

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit c6aa710f8da9ef92b388f1c0c59b2dd3c602ad2d)

selftest: Do not let deprecated option warnings muck this test up

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14460

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 9e212dd15e6c484d69f236f3c6d7186f0e6353b4)

param: Allow tests to silence deprecation warnings

This helps make output sensitive tests more reliable.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14460

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit d14cc45c98a77fb8a6ac96181eec33f368b8dbd8)

selftest: Add test for suppression of deprecation warnings

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14460

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit d3ff49f48507d8a64b9c4847f79d7939f647e6f0)

util: Add cmocka unit test for directory_create_or_exists

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14166

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Sun Aug 16 07:06:59 UTC 2020 on sn-devel-184

(cherry picked from commit e89ec78e9a262a6e7bb9082323083eb5f1609655)

Autobuild-User(v4-13-test): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(v4-13-test): Wed Aug 19 09:56:13 UTC 2020 on sn-devel-184

util: Allow symlinks in directory_create_or_exist

Commit 9f60a77e0b updated the check to avoid having files or other
objects instead of a directory. This missed the valid case that there
might be a symlink to a directory. Updated the check accordingly to
allow symlinks to directories.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14166

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
(cherry picked from commit 672212cecdd7a7de40acdc81c56e2996ea82c090)

VERSION: Bump version up to 4.13.0rc3...

and re-enable GIT_SNAPSHOT.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

VERSION: Disable GIT_SNAPSHOT for the 4.13.0rc2 release.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

WHATSNEW: Add release notes for Samba 4.13.0rc2.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

Remove depracated "ldap ssl ads" smb.conf option

Signed-off-by: Isaac Boukris <iboukris@samba.org>
Autobuild-User(v4-13-test): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(v4-13-test): Wed Aug 12 11:16:04 UTC 2020 on sn-devel-184

ctdb-tests: Stop cat command failure from causing test failure

In certain circumstance, which aren't obvious, cat(1) can fail when
attempting to write a lot of data.  This is due to something (probably
write(2)) returning EAGAIN.

Given that the -v option should only really be used for test
debugging, ignore the failure instead of spending time debugging it.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14446
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 3ff8765d04c0fb950b7be4f9a049999aeb08223b)

winbind: Fix lookuprids cache problem

Bug: https://bugzilla.samba.org/show_bug.cgi?id=14435
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Thu Jul  9 21:40:52 UTC 2020 on sn-devel-184

(cherry picked from commit cd4122d91e942ca465c03505d5e148117f505ba4)

Autobuild-User(v4-13-test): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(v4-13-test): Mon Aug 10 10:46:37 UTC 2020 on sn-devel-184

winbind: Add test for lookuprids cache problem

When reading entries from gencache, wb_cache_rids_to_names() can
return STATUS_SOME_UNMAPPED, which _wbint_LookupRids() does not handle
correctly.

This test enforces this situation by filling gencache with one wbinfo
-R and then erasing the winbindd_cache.tdb. This forces winbind to
enter the domain helper process, which will then read from gencache
filled with the previous wbinfo -R.

Without having the entries cached this does not happen because
wb_cache_rids_to_names() via the do_query: path calls deep inside
calls dcerpc_lsa_lookup_sids_noalloc(), which hides the
STATUS_SOME_UNMAPPED that came in as lsa_LookupSids result value.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=14435
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
(cherry picked from commit 04eafce653afcff517317d2b190acc4f0cbf4c61)

torture3: Align integer types

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

dbcheck: Allow a dangling forward link outside our known NCs

If we do not have the NC of the target object we can not be really sure
that the object is redundent and so we want to keep it for now
and not (as happened until now) break the dbcheck run made during the
replication stage of a "samba-tool domain backup rename".

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14450

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
(cherry picked from commit 05228c4e07013c0e6f78f1330b3b787271282ca8)

ctdb-scripts: Use nfsconf as a last resort get nfsd thread count

If nfsconf exists then use it as last resort to attempt to extract
[nfsd]:threads from /etc/nfs.conf.

Invocation of nfsconf requires "|| true" because this script uses "set
-e".  Add a stub that always fails to at least test this much.

RN: Use nfsconf utility for variable values in CTDB NFS scripts
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14444
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Mon Jul 27 07:06:58 UTC 2020 on sn-devel-184

(cherry picked from commit 642dc6ded6426ba2fbf3ac1e5cd71aae11ca245b)

ctdb-scripts: Use nfsconf as a last resort to set NFS_HOSTNAME

If nfsconf exists then use it as last resort to attempt to extract
[statd]:name from /etc/nfs.conf.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14444
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
(cherry picked from commit 334dd8cedda6a341e3b89c9adc8102ea5480e452)

docs: Fix documentation for require_membership_of of pam_winbind.conf

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14358

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Isaac Boukris <iboukris@samba.org>
(cherry picked from commit 71b7140fd0a33e7e8c5bf37c2897cea8224b3f01)

docs: Fix documentation for require_membership_of of pam_winbind

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14358

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
(cherry picked from commit 4c74db6978c682f8ba4e74a6ee8157cfcbb54971)

kdc:db-glue: ignore KRB5_PROG_ETYPE_NOSUPP also for Primary:Kerberos

Currently we only ignore KRB5_PROG_ETYPE_NOSUPP for
Primary:Kerberos-Newer-Keys, but not for Primary:Kerberos.

If a service account has msDS-SupportedEncryptionTypes: 31
and DES keys stored in Primary:Kerberos, we'll pass the
DES key to smb_krb5_keyblock_init_contents(), but may get
KRB5_PROG_ETYPE_NOSUPP.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14354

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Isaac Boukris <iboukris@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Tue Jul 28 14:04:26 UTC 2020 on sn-devel-184

(cherry picked from commit 4baa7cc8e473f6b63316b4ae5db34796c0f864c3)

Autobuild-User(v4-13-test): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(v4-13-test): Fri Aug  7 10:39:26 UTC 2020 on sn-devel-184

Add a test with old msDS-SupportedEncryptionTypes

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14354

Signed-off-by: Isaac Boukris <iboukris@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit 07399831794e28c7c2cf0140d0f1d1b5538b5f60)

VERSION: Bump version up to 4.13.0rc2...

and re-enable GIT_SNAPSHOT.

Signed-off-by: Stefan Metzmacher <metze@samba.org>

VERSION: Disable GIT_SNAPSHOT for the 4.13.0rc1 release.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

WHATSNEW: Add release notes for Samba 4.13.0rc1.

Signed-off-by: Karolin Seeger <kseeger@samba.org>

s3:smbd: skip ctdb public ips in fsctl_network_iface_info()

Multi-Channel clients should not connect to ctdb public ip addresses
(which move between nodes).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11898

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Wed Jul  8 17:16:40 UTC 2020 on sn-devel-184

s3:smbd: disconnect the all client connections if a ctdb public ip dropped

For now we keep it simple and any disconnect on a connection that
used a ctdb public address, will disconnect all other remaining
connections.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11898

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s3:smbd: make smbXsrv_client_connection_pass_loop() more robust

Don't leak fds in the error paths.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11898

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s3:smbd: make sure smbXsrv_connection_disconnect_transport() closes the socket fd

I assumed that TALLOC_FREE(xconn->transport.fde) would close the socket,
but until now we didn't use tevent_fd_set_auto_close().

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11898

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s3:smbd: make sure we detect stale smbXsrv_connection pointers in smbXsrv_session_auth0

Pointer values can be reused (yes, I hit that during my testing!).
Introduce a channel_id to identify connections and also add
some timestamps to make debugging easier.

This makes smbXsrv_session_find_auth() much more robust.

This is a similar change as 0cec96526bf4d3209caf36c4a19632ff5d5dd112:
 "smb2_server: make sure we detect stale smbXsrv_connection pointers in smbXsrv_channel_global"

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11898

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s3:smbd: fill in xconn->client early in smbd_add_connection()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11898

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s3:smbd: handle NETWORK_ACCESS_DENIED in smbXsrv_client_connection_pass_loop()

smbd_add_connection() may return a valid connection together with
NT_STATUS_NETWORK_ACCESS_DENIED.

We need additional cleanup for that case.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11898

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s3:smbd: consistently use DLIST_ADD* to fill client->connections in smbd_add_connection()

We should not just overwrite the client->connections pointer if we
reject the connection.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11898

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s3:ctdbd_conn: add ctdbd_control_get_public_ips() and ctdbd_find_in_public_ips()

These will be used in the multi channel code in order to handle
public ip addresses, which can move arround ctdb nodes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11898

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s3:ctdbd_conn: make use of samba_sockaddr in ctdbd_connect()

This avoids compiler warnings like this:
dereferencing type-punned pointer might break strict-aliasing rules [-Wstrict-aliasing]

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11898

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s3:ctdbd_conn: make use of ctdbd_control_local() in ctdbd_register_ips()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11898

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

selftest/Samba3: make use of 'smbd:FSCTL_SMBTORTURE = yes'

This makes sure the lease/oplock break retry logic based on
missing TCP acks is tested.

We're still not able to run multichannel tests automatically,
as socket wrapper doesn't support fd-passing yet.

But this testing this with single channels is a good start.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11897

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s3:smbd: implement FSCTL_SMBTORTURE_FORCE_UNACKED_TIMEOUT

This will be used by smbtorture in order to simulate channel failures
without relying on iptables.

'smbd:FSCTL_SMBTORTURE = yes' is required in order to active this.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11897

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s3:smbd: make use of the new ack infrastructure for oplock/lease breaks

This finally implements the retry of failed oplock/lease breaks.

Before smbd_smb2_break_send/recv completed directly after
sendmsg() passed the pdu to the kernel.

Now the completion is (at least) deferred until the
the next smbXsrv_connection_ack_checker() run happens
and smbd_smb2_send_queue_ack_bytes() found that
all bytes of the break notification left the kernel
send queue (and were TCP acked).

If the connection is disconnected all pending break
notifications are completed with an error, which is
then returned by smbd_smb2_break_recv().
smbXsrv_pending_break_submit() will then submit
another break notification via the next available
connection/channel.

The smbXsrv_connection_ack_checker() runs each
rto_usecs (between 0.2s and 1.0s). smbd_smb2_break_send()
will set a timeout of 6*rto_usecs (between 1.2s and 6s).
If smbXsrv_connection_ack_checker() detects via
smbd_smb2_send_queue_ack_bytes() that a pending break
notification is pending for more than its timeout
we'll disconnect the connection with NT_STATUS_IO_TIMEOUT.
This will be handled as any other disconnect and
will in turn also trigger the retry on the next channel.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11897

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s3:smbd: force multi-channel to be turned off without FreeBSD/Linux support

For now it's safer to disable multi-channel without having support
for TIOCOUTQ/FIONWRITE on tcp sockets.

Using a fixed retransmission timeout (rto) of 1 second would be ok,
but we better require kernel support for requesting for unacked bytes
in the kernel send queue.

"force:server multi channel support = yes" can be used to overwrite
the compile time restriction (mainly for testing).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11897

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s3:smbd: add infrastructure to wait for TCP acks

This will be the core of the logic that allows
us to retry break notifications.

When we start the "pending break cycle" we ask for
the current retransmission timemout (rto) on the TCP connection
and remember how many unacked bytes are in the kernel's
send queue. Each time we send bytes into the kernel
we add them to the unacked bytes.
We use a timer using the rto interval in order
to check the amount of unacked bytes again.
The provides send_queu_entry.ack.req will be completed
with tevent_req_done() when everything is completely acked,
tevent_req_nterror(NT_STATUS_IO_TIMEOUT) when
send_queu_entry.ack.timeout is expired or
tevent_req_nterror(connection_error) when the connection
gets disconnected.

It works with support from the FreeBSD and Linux kernels.
For other platforms we just have a fixed rto of 1 second.
And pretend all bytes are acked when we recheck after 1 second.
So only a connection error could trigger tevent_req_nterror(),
but there's no timeout. A follow up commit will most likely
disable support for multi-channel if we don't have kernel support.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11897

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s3:smbd: add logic to retry break notifications on all available channels

For leases we need to use any available connection with the same
client_guid. That means all connections in the client->connections list.

We try the oldest connection first, as that's what windows is doing.

For oplocks we implement the same as that's what the specification
says. Windows behaves different and we have
'smb2 disable oplock break retry = yes' in order to behave like Windows.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11897

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

docs-xml/smbdotconf: add "smb2 disable oplock break retry"

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s3:smbd: convert smbd_smb2_send_break() into async smbd_smb2_break_send/recv()

This will make it possible to detect errors in order to retry sending
the break on another connection.

For now we always report NT_STATUS_OK, when we delivered the break
notification to the kernel send queue. But that will change in
the following commits.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11897

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s3:smbd: add smbd_smb2_send_queue.sendfile_body_size

The following patches require the size of the full sendfile() pdu.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11897

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s3:smbd: introduce smbXsrv_pending_break infrastructure

This prepares support for oplock/lease break replay from
the server to the client.

We need some state in order to do replays later.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11897

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s3:smbd: avoid dereferencing client->connections

There're typically better ways to get the same information.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11897

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s3:smbd: setup client->global->client_guid even without multichannel support

It's too confusing if client->global->client_guid and
client->connections->smb2.client.guid don't have the same value.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11897

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s3:smbd: pass down smbXsrv_client to smbd_smb2_send_{oplock,lease}_break()

Which connection is actually used should not matter to the main logic.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11897

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s3:smbd: pass smbXsrv_client to downgrade_lease()

This prepares for multichannel support, where breaks are not bound
to a single connection.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11897

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s3:smbd: add smbd_server_disconnect_client[_ex]()

With multichannel things may not happen only on one connection.
We may need to disconnect all connections of a client, when something
bad happens.

The first users of this will be the lease/oplock break code,
if they are not able allocate memory or something similar
we need to bail out.

Having a special smbXsrv_client based function is better than
calling exit_server*() directly.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11897

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s3:smbd: remove unused session,tcon parameters from smbd_smb2_send_oplock_break()

They are no longer used. However we'll make use of
op->compat->vuid in the next commits, as the session id should be part
of oplock breaks.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11897

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s3:smbd: pass down session_id to smbd_smb2_send_break()

Oplock break should contain a valid session id of the open file handle,
as file handles are relative to a session.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11897

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s3:smbd: remove dead code from smbd_smb2_send_break()

Starting with commit 0a924d13cf4bb570cce3955cf0de9d8678b37dbe
("smbd: Send SMB2 oplock breaks unencrypted") we always passed in
session=NULL and tcon=NULL.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11897

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s4:torture/smb2: add smb2.multichannel.oplocks.test3{_windows,specification}

This is similar to the smb2.multichannel.leases.test5,
but it tests the oplock case instead of leases.

With Oplocks Windows only sends a single break on the latest channel,
this is not what the spec says...

Maybe we should have a similar test that would expect the
behavior from the [MS-SMB2] (3/4/2020 rev 60.0)
"3.3.4.6 Object Store Indicates an Oplock Break":

  ...

  If the server implements the SMB 3.x dialect family, SMB2 Oplock Break
  Notification MUST be sent to the client using the first available
  connection in Open.Session.ChannelList where Channel.Connection is not
  NULL. If the server fails to send the notification to the client, the
  server MUST retry the send using an alternate connection, if available,
  in Open.Session.ChannelList.
  ...

Here I add one test that demonstrates the Windows behavior:
 smb2.multichannel.oplocks.test3_windows
and a 2nd test that demonstrates the behavior from MS-SMB2.
 smb2.multichannel.oplocks.test3_specification

Note that Windows 10 seems to behave differently and it's not
possible to open all 32 channel used by this test.

Against remote servers it's required to run iptables as root:

 #> smbtorture //server/torture -Uu%p \
    --option="torture:use_iptables=yes" \
    --option="torture:iptables_command=sudo /sbin/iptables" \
    smb2.multichannel.oplocks.test3_windows

 #> smbtorture //server/torture -Uu%p \
    --option="torture:use_iptables=yes" \
    --option="torture:iptables_command=sudo /sbin/iptables" \
    smb2.multichannel.oplocks.test3_specification

The test will also work against a Samba server
with 'smbd:FSCTL_SMBTORTURE = yes', and won't require iptables
in that case.

Samba will get a "smb2 disable oplock break retry" configuration
option to switch between both behaviors, as it's much more common with Samba
that leases are not supported and clients will fallback to
oplocks together with multichannel.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11897

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s4:torture/smb2: (re-)add smb2.multichannel.leases.test4

This tests 32 channels, which is the maximum Windows Server
versions support. (Note that Windows 10 (a Client OS as SMB server,
seems to support only 20 channels and may differ in other aspects,
so we ignore that for now).

This works at least against Windows Server 2019
and we see lease break notification retries every ~ 1.3 seconds
with ~ 5 TCP retransmissions. At that rate we see the remaining
5 retries after the conflicting SMB2 Create already returned.

Older Windows Server versions use much longer timeouts in the TCP-stack,
they send lease break notification retries less often and only 4 in
total, all other channels get TCP-RST packets because of missing
TCP keepalive packets before they're used.
The intervals between lease break notification retries are
~19 seconds for 2012[_R2] and ~25 seconds for 2016.
It means that only ~2 lease break notifications arrive before
the open returns after ~35 seconds.

Note that Windows 10 seems to behave differently and it's not
possible to open all 32 channel used by this test.

Against remote servers it's required to run iptables as root:

 #> smbtorture //server/torture -Uu%p \
    --option="torture:use_iptables=yes" \
    --option="torture:iptables_command=sudo /sbin/iptables" \
    smb2.multichannel.leases.test4

The test will also work against a Samba server
with 'smbd:FSCTL_SMBTORTURE = yes', and won't require iptables
in that case.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s4:torture/smb2: remove useless 'smb2.multichannel.leases.test4'

Having a test that would only pass against Samba makes things way
to complex, they're already complex and we should try to behave
like windows as much as possible.

The next commit will add a better test that will work against Windows
Servers and the future Samba servers.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11897

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s4:torture/smb2: fix smb2.multichannel.leases.test2 against windows

We still receive the break on the blocked channel,
it's only the response ACKs, which we are blocking (or simulate to
block).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11897

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s4:torture/smb2: split smb2.oplock.batch22 into a and b

batch22a tests the timeout on a valid connection
and batch22b tests the timeout on a broken/blocked connection.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11897

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s4:torture/smb2: move smb2_transport blocking to the generic block.[ch]

We may want to use this in other places too, not only multichannel.c

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11897

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s4:torture/smb2: make use of FSCTL_SMBTORTURE_FORCE_UNACKED_TIMEOUT

This is a way to test without being able to use iptables.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11897

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s4:torture/smb2: refactor block.c to block the OUTPUT path

In order to create useful tests, we should block the outgoing
tcp packets only. That means we're able to see incoming
break notifications, but prevent outgoing TCP ACKs to be delivered
to the server.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11897

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s4:torture/smb2: add break_info.oplock_skip_ack

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11897

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s4:torture/smb2: move interface_info test to smb2.multichannel.generic

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11897

Signed-off-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s4:torture/smb2: make use of transport_options.only_negprot for multichannel connections

This avoid useless session setups and tree connects on the wire.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11897

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

s4:torture/smb2: simplify code to generate list of smb2 channels

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11897

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Guenther Deschner <gd@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>