selftest: Skip s4u2proxy tests, no support yet

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

testprogs: Add MIT Kerberos specific kpasswd blackbox test

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s4-kdc: Start the kpasswd service with MIT KDC

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

waf: Search for MIT kadm-server library

This is needed for plugin registration in the KDC.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s4-kdc: Add MIT Kerberos specific kpasswd code

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s4-torture: Add AES and RC4 enctype checks

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s4-torture: Add TORTURE_KRB5_TEST_CLOCK_SKEW test

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s4-torture: Add TORTURE_KRB5_TEST_BREAK_PW test

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s4-torture: Add TORTURE_KRB5_TEST_PAC_REQUEST test

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s4-torture: Add KDC test harness and first test

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

waf: Only build KRB5 KDC tests when AD_DC build is enabled

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

testprogs: Add test with exported keytab from samba-tool

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

testprogs: Add a kinit trust test for MIT KDC

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

testprogs: Add test_kinit_mit.sh test

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s4-torture: Fix kinit of samba4.blackbox.locktest

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

testprogs: Fix usage printout of bogus blackbox test

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

testprogs: Fix test_chgdcpass blackbox test with MIT

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s4-torture: disable s4u2self/proxy remote pac tests for MIT build for now.

Guenther

Signed-off-by: Günther Deschner <gd@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

selftest: Set clockskew grace time to 5 seconds

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

selftest: Setup configs for MIT KDC

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

selftest: Disable RODC tests with MIT KDC

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

selftest: Start MIT KDC if Kerberos is from MIT

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

waf: Do not disable the ntvfs fileserver when we have MIT DC build

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

param: Add 'mit kdc config' option to smb.conf

This points to the kdc config file created by Samba by default.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s4-kdc: Register the MIT irpc PAC validation service

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s4-kdc: Add MIT KRB5 based irpc service for PAC validation

Pair-Programmed-With: Guenther Deschner <gd@samba.org>

Signed-off-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s4-kdc: Add a MIT Kerberos KDC service

This starts the krb5kdc binary shipped with MIT Kerberos.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

param: Add 'mit kdc command' to change the default.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

waf: Check for MIT KDC binary

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

mit-kdb: Update KDB vtable for DAL version 6

This changed between 1.14 and 1.15. Also the 1.15 change removed the
ability that the KDB module can free memory. This caused issues of
serveral projects. It got fixed with 1.15.1.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

waf: Require MIT Kerberos 1.15.1 for Samba AD

Are build without AD DC still only requried MIT Kerberos 1.9.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

mit-kdb: Zero the db principal when we allocate it

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

samba_dnsupdate: Do not rewrite krb5.conf in selftest

The samba_dnsupdate script is responsible to provision the DNS entries.
The private krb5.conf uses dns lookups to find the KDC to acquire a
Kerberos ticket.  Obviously this will fail because currently we are are
in the process of adding the DNS entries for the KDC.

If we are inside of selftest we need to use the krb5.conf created by
selftest itself.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3-tests: Use common functions in test_smbclient_netbios_aliases.sh

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

testprogs: Add common kinit function

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s4:torture: Fix the remote_pac test

All the Kerberos implementation do not expect an order of the pac
buffer. The buffers are not processed in the oder they are sent but when
required just located.

I confirmed this with MS at the IO Lab.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s4:selftest: Only run auth_log tests with Heimdal

They have only been created against Samba AD with Heimdal Kerberos.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lsa4_srv: Factor out dcesrc_lsa_valid_AccountRight()

The previous code in dcesrv_lsa_AddRemoveAccountRights had the following snippet:

if (sec_privilege_id(rights->names[i].string) == SEC_PRIV_INVALID) {
        if (sec_right_bit(rights->names[i].string) == 0) {
                talloc_free(msg);
                return NT_STATUS_NO_SUCH_PRIVILEGE;
        }
        talloc_free(msg);
        return NT_STATUS_NO_SUCH_PRIVILEGE;
}

If I'm not mistaken, the inner if-statement is essentially dead code,
as regardless of the outcome of the if-condition we execute the same
code. The effect of this is that you can't "net rpc rights grant" a right,
for example SeInteractiveLogonRight. A quick test against a W2k12 server
shows that W2k12 allows this call.

This patch changes the semantics of dcesrv_lsa_AddRemoveAccountRights
to also allow "rights" to be granted and revoked. At the same
time, it centralizes the check for validity of user input from
dcesrv_lsa_EnumAccountsWithUserRight into dcesrc_lsa_valid_AccountRight
too.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Apr 29 09:20:02 CEST 2017 on sn-devel-144

source3 smdb: fix null pointer dereference

Fix the null pointer dereference in smbd, introduced in the auth logging
changes.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Apr 28 07:18:54 CEST 2017 on sn-devel-144

source3 smbd: tests for null pointer dereference

Test case to replicate null pointer dereference in smbd, introduced in
the auth logging changes.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3: smbd: inotify_map_mask_to_filter incorrectly indexes an array.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12760

Signed-off-by: Doug Nazar <nazard@nazar.ca>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Christof Schmitt <cs@samba.org>

tdb: version 1.3.13

* documentation for the tdbbackup -n option
* correctly upgrade F_RDLCK to F_WRLCK locks
* tdbtool: Add "storehex" command
* fix robust mutex detection in threaded applications
  (bug #12593)
* improve debugging of transaction lock failures

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Thu Apr 27 18:50:10 CEST 2017 on sn-devel-144

tdb: Improve debugging in _tdb_transaction_start

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

tdb: Improve debugging when the allrecord lock fails to upgrade

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

tdb: runtime check for robust mutexes may hang in threaded programs

The current runtime check for robust mutexes in
tdb_runtime_check_for_robust_mutexes() is not thread-safe.

When called in a multi-threaded program where any another thread doesn't
have SIGCHLD blocked, we may end up hung in sigsuspend() waiting for a
SIGCHLD of a child procecss and the signal was delivered to another
thread.

Revert to the previous behaviour of waiting for the child instead of
waiting for the SIGCHLD signal.

Ensure the pid we wait for is not reset to -1 in a toctou race with the
signal handler.

Check whether waitpid() returns ECHILD which can happen if the signal
handler is run by more then one thread in parallel (yes, this can
happen) or if tdb_robust_mutex_wait_for_child() and the signal handler
are racing.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=12593

Pair-programmed-with: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Ralph Boehme <slow@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

notify: Fix ordering of events in notifyd

In notifyd_trigger_parser() while initializing notify_event_msg values from
notify_trigger_msg, 'when' value is ignored. So the smbd process does not get
correct 'when' value and this is causing issues during qsort in
notify_marshall_changes(). Because of this issue, smb2.notify.dir test was
failing.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=12756
Signed-off-by: Shilpa Krishnareddy <skrishnareddy@panzura.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Wed Apr 26 17:02:58 CEST 2017 on sn-devel-144

s4-messaging: Add helpful comments

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Wed Apr 26 03:09:19 CEST 2017 on sn-devel-144

process_standard: clean up messaging for children after exit()

This makes sure we remove any messaging sockets if a child dies or calls exit()
without running the talloc destructor for messaging

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

lib: Fix CID 1405493 Error handling issues (CHECKED_RETURN)

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Apr 25 23:07:25 CEST 2017 on sn-devel-144

lib/util: fix a Coverity finding in tfork

If dup2() fails, fd is -1 and is later used in sys_write().

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3-libgpo: Fix the build of the group policy CSEs

Guenther

Signed-off-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Mon Apr 24 23:19:44 CEST 2017 on sn-devel-144

tdbtool: Add "storehex" command

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Apr 22 09:16:16 CEST 2017 on sn-devel-144

secrets: Protect against a non-0-terminated ldap password

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>

vfs_fruit: lp_case_sensitive() does not return a bool

lp_case_sensitive() returns an int, not a bool, so with the default
setting of "Auto" by default we set the AAPL flag
SMB2_CRTCTX_AAPL_CASE_SENSITIVE.

This caused the client to believe the volume is case sensitive where it
wasn't, leading to an error when trying to rename files changing only
the case of the name.

Also fix the existing torture test that verifies AAPL context
negotiation and actually expected the server to return "case sensitive",
while the Samba default is really "case insensitive".

Bug: https://bugzilla.samba.org/show_bug.cgi?id=12749

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

selftest: Do not enable inbound replication during replica_sync

Instead we should use the forced=True to only do a very specific
replication, and so avoid noise from any other DC also live
on the network.  This extra replication in turn causes (and this
patch fixes) flapping replica_sync tests.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12753

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Sat Apr 22 05:19:11 CEST 2017 on sn-devel-144

lib: modules: Change XXX_init interface from XXX_init(void) to XXX_init(TALLOC_CTX *)

Not currently used - no logic changes inside.

This will make it possible to pass down a long-lived talloc
context from the loading function for modules to use instead
of having them internally all use talloc_autofree_context()
which is a hidden global.

Updated all known module interface numbers, and added a
WHATSNEW.

Signed-off-by: Jeremy Allison <jra@samba.org>
Signed-off-by: Ralph Böhme <slow@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Apr 22 01:17:00 CEST 2017 on sn-devel-144

pyrpc: Fix segfault in ClientConnection

Fix segfault when connecting over TCP, the endpoints list in dummy_table
was not initialised this caused a segfault when attempting to connect
over TCP.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Fri Apr 21 16:10:12 CEST 2017 on sn-devel-144

vfs_acl_xattr|tdb: ensure create mask is at least 0666 if ignore_system_acls is set

Bug: https://bugzilla.samba.org/show_bug.cgi?id=12562

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Thu Apr 20 20:50:10 CEST 2017 on sn-devel-144

cleanupdb: Fix a memory read error

Bug: https://bugzilla.samba.org/show_bug.cgi?id=12748

Signed-off-by: Hanno Böck <hanno@hboeck.de>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

lib/util: add a test for samba_runcmd_send()

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

wafsamba: add source directory define SRCDIR to config.h

This will be used in the next commit to prepare the path to a test
script in a smbtorture test.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lib/util: make use of tfork in samba_runcmd_send()

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lib/util: add a test for tfork()

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lib/util: add tfork()

triple-fork to avoid handling SIGCHLD in the parent.

This function is a workaround for the problem of using fork() in
library code. In that case the library should avoid setting a global
signal handler for SIGCHLD, because the application may wants to use its
own handler.

status_fd can be used to wait for the child to exit and get its exit
status.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbldap: Bump version number

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Thu Apr 20 14:23:14 CEST 2017 on sn-devel-144

smbldap: Privatize struct smbldap_state

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>

smbldap: Introduce "smbldap_get_paged_results"

This should be hidden inside smbldap.c, but this is a quick way to
get smbldap_state private to smbldap.c

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>

smbldap: Introduce "smbldap_get_paged_results"

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>

smbldap: Introduce "smbldap_get_ldap"

This is a pretty big boiler-plate change. I've renamed the struct member
temporarily to find all accessors. Not sure where this leads in the end, but
the goal is to make struct smbldap_struct private to smbldap.c

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>

smbldap: Fix a typo

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>

autobuild: Do not require cmocka to be installed for samba-libs to build

cmocka does not have an install target, but is a depencency of ldb for the tests

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Wed Apr 19 19:43:15 CEST 2017 on sn-devel-144

ldb:tests: Unit test the ldb_rename() operation

Signed-off-by: Jakub Hrozek <jakub.hrozek@posteo.se>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>

ldb:tests: Add tests for case insensitive searches

Signed-off-by: Jakub Hrozek <jakub.hrozek@posteo.se>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>

ldb:tests: unit test for ldb_search()

Signed-off-by: Jakub Hrozek <jakub.hrozek@posteo.se>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>

ldb:tests: Add a modify test

Signed-off-by: Jakub Hrozek <jakub.hrozek@posteo.se>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>

ldb:tests: Add a test for ldb transactions

Signed-off-by: Jakub Hrozek <jakub.hrozek@posteo.se>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>

ldb:tests: Add a basic delete test

Signed-off-by: Jakub Hrozek <jakub.hrozek@posteo.se>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>

ldb:tests: Add a basic search test

Signed-off-by: Jakub Hrozek <jakub.hrozek@posteo.se>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>

ldb:tests: A rudimentary ldb_add() test

Signed-off-by: Jakub Hrozek <jakub.hrozek@posteo.se>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>

ldb:tests: Build a ldb test for the tdb backend

Pair-Programmed-With: Andrew Bartlet <abartlet@samba.org>

Signed-off-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Andrew Bartlet <abartlet@samba.org>

ldb:tests: Add a simple cmocka test for ldb_connect()

Signed-off-by: Jakub Hrozek <jakub.hrozek@posteo.se>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>

ldb: Clarify LDB_MODULES_PATH is used

Make it (hopefully more) clear where modules are loaded from.

Signed-off-by: Jakub Hrozek <jakub.hrozek@posteo.se>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>

ldb_tdb: Remove unused function parameter

Signed-off-by: Jakub Hrozek <jakub.hrozek@posteo.se>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>

ldb_tdb: Remove unused function parameter

Signed-off-by: Jakub Hrozek <jakub.hrozek@posteo.se>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>

ldb_tdb: Remove unused function ltdb_add_attr_results

Signed-off-by: Jakub Hrozek <jakub.hrozek@posteo.se>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>

s3-tests: assignement in shell shall have no spaces around equal sign

When assigning value to 'failed', no spaces should be around '=' sign.

Signed-off-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Apr 19 02:53:25 CEST 2017 on sn-devel-144

_netr_ServerPasswordSet2: use info level 26 to set plain text machine password

To support password change for machine or trusted domain accounts in Active
Directory environment we need to pass down actual plain text password
instead of NT hashes. This would allow a backend like ipasam to update
Kerberos keys as well as NT hashes.

By calling samr_SetUserInfo2 info level 26 we ensure PASSDB layer can
actually get the plain text password. If PASSDB backend implements
pdb_update_sam_account() callback, it then gets the plain text password
from samr_SetUserInfo2.

A plain text password is a data blob represented as up to 256 WCHARs. It
is UTF-16 coded on wire and we have its length from the buffer.
SetUserInfo2 SAMR call chain in decode_pw_buffer() does explicitly
expect 512+4 bytes in the buffer. It then calls convert_string_talloc()
to convert it to UNIX charset passing the correct value of the plaintext
password length. However, convert_string_talloc() expects the length of
input string *including* the terminating null and we pass just the
string length.

convert_string_talloc() then explicitly null-terminates the resulting
string by adding two nulls. In most cases UNIX charset is UTF-8, so we
get null-terminated UTF-8 string down to PASSDB layer.

MS-SAMR does not limit what does the password should contain.  It says
it is 'userPassword' value. Either 'userPassword' or 'unicodePwd' cannot
contain null characters according to MS-ADTS 3.1.1.3.1.5 because they
must be proper UTF-8 and UTF-16 strings accordingly.

We are talking to our own SAMR service here.

Signed-off-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

selftest: also run smb2.compound_find against share with async delay set

Add a share with "smbd:find async delay usec" set to 10000 and run the
test smb2.compound_find added in the previous commit against this new
share as well.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s4/torture: add a test for compound SMB2 FIND requests

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s3/smbd: add "smbd:find async delay usec" to SMB2 FIND

This is just a hack for selftest that will be used in subsequent commits
for torturing compound find requests.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s3/smbd: make write time fetching async

Finally use the new async dbwrap_parse_record_send/recv() functions
respectively the fetch_share_mode_send/recv wrappers for fetching the
write time from locking.tdb.

Previously for a directory with n files we would sit idle in the
directory enumeration loop fo n * m seconds waiting for responses from
ctdb, where m is the response time in seconds for a dbwrap request via
ctbd.

This is known to kill performance and we even have a parameter
"smbd:search ask sharemode" that can be used to disable fetching the
write time from locking.tdb.

Using fetch_write_time_send() works this way: in the directory
enumeration loop that calls smbd_dirptr_lanman2_entry() to marshall the
directory entries we

1. call fetch_write_time_send() after calling smbd_dirptr_lanman2_entry
   passing a pointer to the current position in the marshall buffer.

2. If fetch_write_time_send() has set the out parameter "stop", we exit
   the enumeration loop. This is necessary because we only send dbwrap
   requests but don't consume the results. This has the potential to
   deadlock so we must stop sending requests as soon as our ctdb send
   queue is full.

3. In the fetch_write_time_done() callback, if the recv function got a
   locking.tdb record, we push the write time into the marshall buffer
   at the offet saved in the request state.

This new feature is still off by default as it doesn't
give any improvement in the non-clustered usecase.
"smbd:async search ask sharemode" can be used to activate it,
which makes only sense with "clustering = yes" (execept for testing).

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s3/smbd: enable processing SMB2 requests async internally

The idea is to allow the implementation of an SMB2 request to tell the
main SMB2 processing engine that it wants to handle a requests
asynchronously internally.

This has two use cases:

- it allows (internal) async processing of compound requests that would
  otherwise be rejected by the SMB2 processing engine

- it preserves sync semantics at the SMB layer, some clients might not
  expect arbitrary SMB2 requests going async

Not used for now, will be used in laters commit for async SMB2 FIND
requests.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s3/smbd: ask_sharemode is not needed for info_level SMB_FIND_FILE_NAMES_INFO

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s3/smbd: add file_id return arg to smbd_dirptr_lanman2_entry

Not used for now, needed for async write_time updates in
smbd_smb2_query_directory_send().

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s3/locking: add fetch_share_mode_send/recv

The boolean out parameter "queued" tells the caller whether the
async request is blocked in a full send queue:

false := request is dispatched
true  := send queue is full, request waiting to be dispatched

This is useful in a clustered Samba environment where the async dbwrap
request is sent over a socket to the local ctdbd.

If the send queue is full and the caller was issuing multiple async
dbwrap requests in a loop, the caller knows it's probably time to stop
sending requests for now and try again later.

This will be used in subsequent commits in
smbd_smb2_query_directory_send() when implementing async write time
updates. Directories may contain umpteen files so we send many requests
to ctdb without going through tevent and reading the responses which
has the potential to deadlock.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

dbwrap_watch: add parse_record_send/recv wrappers

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

dbwrap: add dbwrap_parse_record_send/recv

The req_state parameter tells the caller whether the async request is
blocked in a full send queue:

req_state >= DBWRAP_REQ_DISPATCHED := request is dispatched
req_state < DBWRAP_REQ_DISPATCHED := send queue is full

This is useful in a clustered Samba environment where the async dbwrap
request is sent over a socket to the local ctdbd.

If the send queue is full and the caller was issuing multiple async
dbwrap requests in a loop, the caller knows it's probably time to stop
sending requests for now and try again later.

This will be used in subsequent commits in
smbd_smb2_query_directory_send() when implementing async write time
updates. Directories may contain umpteen files so we send many requests
to ctdb without going through tevent and reading the responses which has
the potential to deadlock.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

dbwrap_ctdb: implement parse_record_send()/recv()

This mainly works like the sync version, but calls ctdbd_parse_send/recv
instead.

We use one global ctdb connection that is used exclusively for async
requests.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

dbwrap_ctdb: factor out a db_ctdb_try_parse_local_record() function

Pair-programmed-with: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Ralph Boehme <slow@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

ctdb_conn: add ctdbd_parse_send/recv

Implement the ctdb packet layer for async parse send/recv with tevent.

ctdbd_setup_fde() will is used to create an fde from the
connection fd and will be called from dbwrap_ctdb.

ctdbd_parse_send() and ctdbd_parse_recv() will be used by dbwrap_ctdb
for async packet sending and receiving.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>