smbd: Fix async large read

We also do the 128k reads asynchronously, just not the huge 24MB
ones. smb_setlen does not work well for >64k.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

torture: Check messaging_send_all

We must make sure not to receive our own broadcast

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

messaging: Ignore messages from ourselves

For non-clustered messaging this should have never gone through the socket, we
should have caught it before in messaging_send_iov_from.

It can come in on a socket from ctdb when broadcasting in clustered mode. There
ctdb does the broadcasting.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

messaging: Don't do self-sends in messaging_send_all

This leads to cleanupd doing endless MSG_SMB_UNLOCK calls, as it triggers
itself in the send_all. This worked correctly before the serverid.tdb removal
because cleanupd did not register in serverid.tdb (which was a bug, but it
helped us there).

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Remove unsupported colon from configure msg.

- When cross-compiling an answers file must be supplied via
  --cross-compile --cross-answers=<path to answers>.
- The lines in the answer file have the form;
    Config Msg: Answer
- The colon is used to delimit the msg and the answer when reading
  the answers file.
- WAF doesn't support the message containing a colon.
- It's not possible to override this variable so cross compile fails.

Signed-off-by: Zentaro Kavanagh <zentaro@google.com>
Reviewed-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Added smbc_SetLogCallback which lets third party code to capture libsmbclient logs

Signed-off-by: Puran Chand <pchand@vmware.com>
Reviewed-by: Garming Sam <garming@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Sun Dec 10 04:56:23 CET 2017 on sn-devel-144

tests: make password valid in openldap provision test

Test was using an invalid password, which causes test failure with early
password validation patch

Signed-off-by: Jamie McClymont <jamiemcclymont@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

samba-tool: validate password early in `domain provision`

Checks password against default quality and length standards when it is entered,
allowing a second chance to enter one (if interactive), rather than running
through the provisioning process and bailing on an exception

Includes unit tests for the newly-added python wrapper of check_password_quality
plus black-box tests for the checks in samba-tool.

Breaks an openldap test which uses an invalid password.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=9710
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12235

Signed-off-by: Jamie McClymont <jamiemcclymont@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

source4 dsdb: Allow duplicate non local objectSIDs

Remove the unique constraint on the objectSID index, and enable the
unique_object_sids module.

This allows duplicate objectSIDs on foreign security principals, and
disallows duplicates for local objectSIDs

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13004

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>

source4 dsdb modules: Add new module "unique_object_sids"

New module that sets the LDB_FLAG_INTERNAL_UNIQUE_VALUE on all local
objectSIDS and ensure it is cleared for any foreign security principals.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13004

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>

ldb ldb_index: Add an attriubute flag to require a unique value.

Add attribute flag LDB_FLAG_INTERNAL_UNIQUE_VALUE, to request that the
added attribute is unique on the index.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13004

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>

tests dsdb: Add tests for optionally unique objectSID's

It is possible for foreign security principals to have duplicate object
sids, this can be the result of:
 a replication race condition generating conflict resolution objects
 or the foreign security principal being deleted and then re-added on a
 join.

Rather than remove unique check on all objectSIDs we wish to allow
duplicate objectSIDs for foreign security principals.  But enforce the
unique constraint for local objects.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13004

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>

s3:glock: Move sanity check in g_lock_parse()

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Sun Dec 10 00:46:26 CET 2017 on sn-devel-144

pthreadpool: Add a test for the race condition fixed in the last commit

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13179
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

pthreadpool: Fix starvation after fork

After the race is before the race:

1) Create an idle thread
2) Add a job: This won't create a thread anymore
3) Immediately fork

The idle thread will be woken twice before it's actually woken up: Both
pthreadpool_add_job and pthreadpool_prepare_pool call cond_signal, for
different reasons. We must look at pool->prefork_cond first because otherwise
we will end up in a blocking job deep within a fork call, the helper thread
must take its fingers off the condvar as quickly as possible.  This means that
after the fork there's no idle thread around anymore that would pick up the job
submitted in 2). So we must keep the idle threads around across the fork.

The quick solution to re-create one helper thread in pthreadpool_parent has a
fatal flaw: What do we do if that pthread_create call fails? We're deep in an
application calling fork(), and doing fancy signalling from there is really
something we must avoid.

This has one potential performance issue: If we have hundreds of idle threads
(do we ever have that) during the fork, the call to pthread_mutex_lock on the
fork_mutex from pthreadpool_server (the helper thread) will probably cause a
thundering herd when the _parent call unlocks the fork_mutex. The solution for
this to just keep one idle thread around. But this adds code that is not
strictly required functionally for now.

More detailed explanation from Jeremy:

First, understanding the problem the test reproduces:

add a job (num_jobs = 1) -> creates thread to run it.
job finishes, thread sticks around (num_idle = 1).
num_jobs is now zero (initial job finished).

a) Idle thread is now waiting on pool->condvar inside
pthreadpool_server() in pthread_cond_timedwait().

Now, add another job ->

pthreadpool_add_job()
-> pthreadpool_put_job()
This adds the job to the queue.
Oh, there is an idle thread so don't
create one, do:

pthread_cond_signal(&pool->condvar);

and return.

Now call fork *before* idle thread in (a) wakes from
the signaling of pool->condvar.

In the parent (child is irrelevent):

Go into: pthreadpool_prepare() ->
pthreadpool_prepare_pool()

Set the variable to tell idle threads to exit:

pool->prefork_cond = &prefork_cond;

then wake them up with:

pthread_cond_signal(&pool->condvar);

This does nothing as the idle thread
is already awoken.

b) Idle thread wakes up and does:

Reduce idle thread count (num_idle = 0)

pool->num_idle -= 1;

Check if we're in the middle of a fork.

if (pool->prefork_cond != NULL) {

Yes we are, tell pthreadpool_prepare()
we are exiting.

pthread_cond_signal(pool->prefork_cond);

And exit.

pthreadpool_server_exit(pool);
return NULL;
}

So we come back from the fork in the parent with num_jobs = 1,
a job on the queue but no idle threads - and the code that
creates a new thread on job submission was skipped because
an idle thread existed at point (a).

OK, assuming that the previous explaination is correct, the
fix is to create a new pthreadpool context mutex:

pool->fork_mutex

and in pthreadpool_server(), when an idle thread wakes up and
notices we're in the prepare fork state, it puts itself to
sleep by waiting on the new pool->fork_mutex.

And in pthreadpool_prepare_pool(), instead of waiting for
the idle threads to exit, hold the pool->fork_mutex and
signal each idle thread in turn, and wait for the pool->num_idle
to go to zero - which means they're all blocked waiting on
pool->fork_mutex.

When the parent continues, pthreadpool_parent()
unlocks the pool->fork_mutex and all the previously
'idle' threads wake up (and you mention the thundering
herd problem, which is as you say vanishingly small :-)
and pick up any remaining job.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13179
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3-selftest: fix creation of large file

The dd command was skipping the first 20MB of /dev/zero then writing only one,
while indending to create a 20MB sparse file.

Signed-off-by: Jamie McClymont <jamiemcclymont@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Dec  9 02:56:18 CET 2017 on sn-devel-144

s3: test_smbclient_s3: Fix restore of TZ.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>

s3:winbindd: Use a stackframe for memory management in _wbint_QueryGroupList

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Add mdns name configuration option

Add the mdns name configuration variable to control the mdns hostname.
The default is to use the NETBIOS name of the system to match previous
versions which is typically the hostname in all capitals. A value of mdns
can be provided to defer the hostname to the mdns library.

With the recent patch to support time machine being merged this patch
allows for a user to configure the server name that is advertised to
be lower cased through Avahi advertisements.

Signed-off-by: Kevin Anderson <andersonkw2@gmail.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

selftest: Rework samba.dsdb locking test to samba.dsdb_lock

This avoids running the test while samba is modifying and locking the same database,
as this can lead to a deadlock.

The deadlock is not seen in production as the LDB read lock is not held while
waiting for another process, but this test needs to do this to demonstrate
the locking safety.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Fri Dec  8 21:47:55 CET 2017 on sn-devel-144

pthreadpool: Add test for pthread_create failure

This is implemented using cmocka and the __wrap override for
pthread_create.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13170

Signed-off-by: Christof Schmitt <cs@samba.org
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Christof Schmitt <cs@samba.org>
Autobuild-Date(master): Fri Dec  8 13:54:20 CET 2017 on sn-devel-144

wscript: Add check for --wrap linker flag

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13170

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

pthreadpool: Undo put_job when returning error

When an error is returned to the caller of pthreadpool_add_job, the job
should not be kept in the internal job array. Otherwise the caller might
free the data structure and a later worker thread would still reference
it.

When it is not possible to create a single worker thread, the system
might be out of resources or hitting a configured limit. In this case
fall back to calling the job function synchronously instead of raising
the error to the caller and possibly back to the SMB client.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13170

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

pthreadpool: Move creating of thread to new function

No functional change, but this simplifies error handling.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13170

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

s4:kdc: only map SDB_ERR_NOT_FOUND_HERE to HDB_ERR_NOT_FOUND_HERE

HDB_ERR_NOT_FOUND_HERE indicated a very specific error on an RODC.

We should not map any error to HDB_ERR_NOT_FOUND_HERE,
we should just pass errors along unmapped.

Otherwise we'll hit the logic bug in:

    if (ret == KDC_PROXY_REQUEST) {
        uint16_t port;

        if (!sock->kdc_socket->kdc->am_rodc) {
            DEBUG(0,("kdc_udp_call_loop: proxying requested when not RODC"));
                    talloc_free(call);
            goto done;
        }

And just don't send an error message to the client.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13132

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Wed Dec  6 23:16:54 CET 2017 on sn-devel-144

HEIMDAL:kdc: fix dh->q allocation check in get_dh_param()

Thanks to Doug Nazar <nazard@nazar.ca> for spotting this!

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12986

Signed-off-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from heimdal commit a79b59ba27070a015479e8d981b7e685dbe34310)
Reviewed-by: Andreas Schneider <asn@samba.org>

third_party: Update socket_wrapper to version 1.1.9

* Fixed thread - signal deadlock issue
* Added support for openat()
* Added support for open64() and fopen64()
* Always turn on logging
* Increased maximum of wrapped interfaces to 64
* Improved fd duplication code
* Fixed strict-aliasing issues
* Fixed some use after free issues
* Fixed issues on ppc64le

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>

third_party: Fix a typo in the option name

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13174

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>

third_party: Link th aesni-intel library with -z noexecstack

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13174

Signed-off-by: Björn Baumbach <bb@sernet.de>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>

s3: libsmb: Plumb in the new SMB2 get reparse point calls into the cli_readlink_XXXX() calls.

Reparse point symlinks can now be queried over SMB1 and SMB2 from smbclient.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13159

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Wed Dec  6 19:05:34 CET 2017 on sn-devel-144

s3: libsmb: Add SMB2 calls cli_smb2_get_reparse_point_fnum_send()/cli_smb2_get_reparse_point_fnum_recv().

Allow reparse points to be queried over SMB2.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13159

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: libsmb: Do a naive response to SMB2 "stopped on symlink". Assume the last component was the reparse point.

Attempt re-open with FILE_OPEN_REPARSE_POINT. This matches the SMB1
behavior for smbclient.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13159

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>

s3: libsmb: Plumb in the new SMB2 reparse point calls into the cli_symlink_create_XXX() calls.

Reparse point symlinks can now be created over SMB1 and SMB2 from
smbclient.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13159

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>

s3: libsmb: Add SMB2 calls cli_smb2_set_reparse_point_fnum_send()/cli_smb2_set_reparse_point_fnum_recv().

Allow reparse points to be created over SMB2.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13159

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>

s3: libsmb: Make cli_close_send()/cli_close_recv() work for SMB1 and SMB2.

Remove the escape into synchronous smb2 code.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13159

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: libsmb: Rename cli_close_create() -> cli_smb1_close_create().

Move cli_smb1_close_done() next to its caller. This is SMB1 specific.
Prepare to wrap cli_close_send/cli_close_recv to handle SMB2.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13159

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>

vfs: drop casts for VFS_ADD_FSP_EXTENSION() callers

VFS_ADD_FSP_EXTENSION() now returns a properly typed pointer.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Wed Dec  6 03:43:48 CET 2017 on sn-devel-144

vfs: return type via VFS_ADD_FSP_EXTENSION

The type is passed in as a parameter, so should be used to properly type
the returned pointer, instead of using a void *.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

WHATSNEW: Mention removed net serverid subcommands

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Tue Dec  5 17:54:18 CET 2017 on sn-devel-144

s3: test_smbclient_s3: Correctly set and unset TZ variable for test.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Dec  5 08:55:44 CET 2017 on sn-devel-144

lib: Remove unused serverid.tdb

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Dec  5 04:58:26 CET 2017 on sn-devel-144

lib: Use messaging_send_all instead of message_send_all

Just a global search&replace

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lib: Add messaging_send_all

This will replace message_send_all. With messaging_dgm_forall we have
a local broadcast mechanism, and ctdb can also broadcast
everywhere. So there's no need for a separate traverse/send mechanism.

There's no good error reporting mechanism for broadcasting, so make
this function void.

This drops the message_type filtering. I believe that this does not matter in
practice, since messaging is a lot cheaper with dgm instead of the old tdb
based messaging. If someone presents a use case where this matters, nowadays
I'd much rather extend the messaging_dgm lock file format (where the unique id
lives right now) with the filter bits.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

messaging: Always register CTDB_SRVID_SAMBA_PROCESS

This will be used to broadcast to all processes, avoiding the costly
traverse of serverid.tdb.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

messaging: Remove an unused #define

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

net: Remove "net serverid list"

Traversing a clustered tdb is a pretty expensive operation. If someone
really needs this command-line interface, we can re-add it for the local
node using messaging_dgm_forall. If someone needs that globally, there's
the "onnode all" script that could be used. Alternatively, we could
implement an enhanced ping broadcast message also returning a processes
unique id.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

net: Remove "net serverid wipe"

This used to be a hygiene command for clustered node startup. In
clustered mode, CLEAR_IF_FIRST does not work, records can stay alive
by means of recovery. serverid.tdb will soon die, so remove this
command.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbcontrol: Use messaging_dgm_forall

Doing stacktraces can be done locally only anyway

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

messaging_dgm: Use messaging_dgm_forall in dgm_wipe

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

messaging: Add messaging_dgm_forall

This factors out the traversal function from _wipe. It will be used to
replace message_send_all soon.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

messaging: Remove the "n_sent" arg from message_send_all

The only user of this is an informative message in smbcontrol. I don't think
that's worth the effort.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

messaging_dgm: Protect against fork without reinit

In the wake of bug 13150 we've discussed that this could happen even
without clustering. This adds code to make sure that whenever messaging
is used the pid and the files used match.

It's pretty heavy-weight, thus I made it DEVELOPER only. My gut feeling
is that the getsockname is cheap, but the stat call might be a bit too
expensive.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Revert "selftest: mark samba3.smb2.kernel-oplocks as flapping"

The correct fix is in the previous commit, to block the
RT_SIGNAL_LEASE until we're ready to receive it.

This reverts commit 27bb8814a526adbd22452ce58754d18e1b00d426.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Dec  2 03:50:39 CET 2017 on sn-devel-144

s4: torture: Fix race condition in test_smb2_kernel_oplocks8.

The child process gets the kernel lease and then notifies
the parent process to continue by writing a byte up a pipe.
It then sets the alarm and calls pause() to wait for the
parent process to contact the smbd and get it to trigger
the break request using an open call.

It is possible for the parent to run and trigger the break
request after the child has written to the pipe, but *before*
the child calls pause(). We then miss the signal notifying
the child to break the lease.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13121

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>

WHATSNEW: Add new smb.conf parameter

Signed-off-by: Karolin Seeger <kseeger@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

WHATSNEW: Remove old entry about accelerated AES

Signed-off-by: Karolin Seeger <kseeger@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

WHATSNEW: Explain reversal of smbclient symlink parameters against Windows server.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13172

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Thu Nov 30 19:32:21 CET 2017 on sn-devel-144

s3: libsmb: Fix reversing of oldname/newname paths when creating a reparse point symlink on Windows from smbclient.

This happened as smbd doesn't support reparse points so we couldn't test.
This was the reverse of the (tested) symlink parameters in the unix extensions
symlink command.

Rename parameters to link_target instead of oldname so this is clearer.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13172

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

s3: client: Rename <oldname> to <link_target> in cmd_symlink() and cli_posix_symlink().

Stops us from mixing up the old and new names. Only behavior change
is correcting the names printed in the error messages.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13172

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

ctdb-daemon: Send STARTUP control after startup event

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13154

STARTUP control is primarily used to synchronise tcp tickles from running
nodes to a node which has just started up.  Earlier STARTUP control was
sent (using BROADCAST_ALL) after setup event.  Once the other nodes in
the cluster connected to this node, the queued up messages would be sent
and the tcp tickles would get synchronised.

Recent fix to drop messages to disconnected or not-yet-connected nodes,
the STARTUP control was never sent to the remote nodes and the tcp
tickles did not get synchronised.

To fix this problem send the STARTUP control (using BROADCAST_CONNECTED)
after startup event.  By this time all the running nodes in the cluster
are connected.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Thu Nov 30 15:29:48 CET 2017 on sn-devel-144

ctdb-takeover: Send tcp tickles immediately on STARTUP control

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13154

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

ctdb-takeover: Refactor code to send tickle lists for all public IPs

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13154

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

selftest: mark samba3.smb2.kernel-oplocks as flapping

This flaps on sn-devel occaionally, and more often elsewhere

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Nov 30 09:47:22 CET 2017 on sn-devel-144

torture: Use torture_assert{,_int_equal}_goto() in smb2.kernel-oplocks

This allows this test to be added as flapping.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

s3: libsmb: Fix valgrind read-after-free error in cli_smb2_close_fnum_recv().

cli_smb2_close_fnum_recv() uses tevent_req_simple_recv_ntstatus(req), which
frees req, then uses the state pointer which was owned by req.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13171

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Nov 30 05:47:12 CET 2017 on sn-devel-144

testprogs: Fix a typo in the net ads test

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu Nov 30 01:47:24 CET 2017 on sn-devel-144

testprogs: Test net ads keytab list

Test that correct keytab is picked up.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13166

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3:libads: net ads keytab list fails with "Key table name malformed"

When keytab_name is NULL don't call smb_krb5_kt_open use ads_keytab_open
instead, this function will determine the correct keytab to use.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13166

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

winbindd: Name<->SID cache is not sequence number based anymore

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Wed Nov 29 20:57:25 CET 2017 on sn-devel-144

winbindd: Move name<->sid cache to gencache

The mapping from name to sid and vice versa has nothing to
do with a specific domain. It is publically available. Thus put
it into gencache without referring to the domain this was
retrieved from

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

winbindd: Factor out winbindd_domain_init_backend from get_cache()

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

torture3: Test namemap_cache

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

net: Parse namemap_cache in "net cache list"

namemap_cache.c saves these as strv lists: An array of 0-terminated strings.
"net cache list" only printfs the values, so they would be cut off.

We might want to do this with other gencache values too in the future.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

lib: Add namemap_cache

A few functions to maintain lookupname and lookupsid cache in gencache.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

lib: Pass blob instead of &blob to gencache_set_data_blob

Passing a whole DATA_BLOB is cheap enough to simplify the callers: A caller
does not have to create a separate variable.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

lib: Allow parsing a strv from a non-talloc const buf

This will allow parsing a tdb record without having to talloc_memdup it

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

lib: Only call strlen if necessary in strv

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

lib: Pass in "strv_len" to strv_valid_entry

Preparation for a later commit

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

dbwrap_watch: Remove the "prec" parameter from watch_recv

The initial idea was to have some "atomicity" in this API. Every
caller interested in a record would have to do something with
it once it changes. However, only one caller really used this
feature, and that is easily changed to not use it. So
remove the complexity.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

smbd: Avoid using dbwrap_watched_watch_recv's prec argument

This is the only user of the "prec" argument of
dbwrap_watched_watch_recv. The next patch will remove this
functionality, as it's easily replaced here.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

winbindd: let normalize_name_map() call find_domain_from_name_noinit()

Let normalize_name_map fetch the domain itself with
find_domain_from_name_noinit().

This removes two calls to find_domain_from_name_noinit() in the default
configuration of "winbind normalize names = no". The domain is only need
in normalize_name_map if "winbind normalize names" is enabled.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s3/cli_netlogon: remove SEC_CHAN_NULL fallback from rpccli_connect_netlogon()

The caller should handle secure-channel-type SEC_CHAN_NULL. The previous
commit already added handling of SEC_CHAN_NULL to
cm_connect_netlogon_transport.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13167

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

winbindd: restore SEC_CHAN_NULL fallback in cm_connect_netlogon_transport

This partially reverts commit d7e31d9f4d9ce7395e458ac341dd83ac06255a20
"winbindd: Use rpccli_connect_netlogon" and restores handling of SEC_CHAN_NULL.

Without this we fail to enumerate domains in trusted forests so users
from any child-domain (or tree-root) in the trusted forest can't login
via eg SMB.

This is a temporary hack that will go away once we get rid of the trusted domain
list.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13167

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

Replace tabs with spaces to comply with PEP8, reorder list of manpages in the alphanumeric sort order.

Signed-off-by: Timur I. Bakeyev <timur@iXsystems.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>
Autobuild-User(master): Uri Simchoni <uri@samba.org>
Autobuild-Date(master): Wed Nov 29 16:58:32 CET 2017 on sn-devel-144

Add vfs_zfsacl manpage to the list of manpages if we have this module enabled.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12934

Signed-off-by: Timur I. Bakeyev <timur@iXsystems.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

Fix typo in the "wide links" description for the getwd cache.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12934

Signed-off-by: Timur I. Bakeyev <timur@iXsystems.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

pam_winbind: avoid non-literal-format warning

Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Nov 29 12:50:49 CET 2017 on sn-devel-144

vfs_fruit: proper VFS-stackable conversion of FinderInfo

This fixes the problem that conversion failed with
fruit:metadata=stream. Before we were calling ad_set() which stores the
metadata in the Netatalk compatible format.

Rewrite to fully go through the VFS by calling SMB_VFS_CREATE_FILE() and
SMB_VFS_PWRITE().

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13155

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Nov 29 08:38:06 CET 2017 on sn-devel-144

vfs_fruit: add AfpInfo prototypes

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13155

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s4/torture: fruit: in test_adouble_conversion() also check stream list and AFPINFO_STREAM

This reveals that the conversion doesn't work properly with
fruit:metadata=stream.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13155

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s4/torture: fruit: remove use of localdir from test_adouble_conversion test

The previous use of localdir and torture_setup_local_file() was
motivated by the fact that by default vfs_fruit rejects access to files
with a "._" prefix.

Since a previous commit allowed SMB access to ._ files, rewrite the
test_adouble_conversion() test to create the ._ AppleDouble file over
SMB.

This also renders torture_setup_local_file() obsolete.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13155

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

selftest: add "fruit:veto_appledouble = no" to fruit shares

This is needed for a subsequent commit that modifies an existing test to
write a ._ file over SMB instead of using the ugly local creation hack.

SMB acces of ._ files requires "fruit:veto_appledouble = no", so let's
set it.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13155

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s4/torture: let write_stream() deal with stream=NULL

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13155

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

selftest: run AppleDouble sidecar-file conversion test runs against all fruit shares

This needs for work in all possible fruit configs, so test it.

This currently fails with stream_depot, as we don't propely copy over
the resourcefork data from the ._ file to the stream.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13155

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s4/torture: use torture_assert_goto in a vfs.fruit test

No change in behavior.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13155

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s4/torture: rework stream names tests usage of local xattr call

Previously this test, that tests for correct conversion of ':' in stream
names, only worked with streams_xattr with "fruit:metadata" set to
"netatalk".

In order to have test coverage for fruit shares with other configs,
split the test into two:

one test creates the stream over SMB and run against all shares, the
other one is the unmodified existing test and is only run against the
share with streams_xattr and fruit:metadata=netatalk.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13155

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

selftest: add localdir option to fruit subtests

A subsequent commits modifies an existing tests that needs $localdir to
also run against "vfs_fruit_metadata_stream" and
"vfs_fruit_stream_depot". This reveals test failures, those will be
fixed in a subsequent commit.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13155

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

selftest: reorder arguments for fruit tests

This just puts the auth option first matching the first test with the
"vfs_fruit" share directly above the modified lines.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13155

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

systemd: Start processes in forground and without a process group

We should not double fork in notify mode or systemd think something
during startup will be wrong and send SIGTERM to the process. So
sometimes the daemon will not start up correctly.

systemd will also handle the process group.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13129

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Nov 28 15:31:27 CET 2017 on sn-devel-144

s4:samba: Allow samba daemon to run in foreground

We are passing the no_process_group to become_daemon() that setsid() is
not called. In case we are double forking, we run in SysV daemon mode,
setsid() should be called!

See:
https://www.freedesktop.org/software/systemd/man/daemon.html

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13129

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>