s3:smbd: add private option NTCREATEX_OPTIONS_PRIVATE_STREAM_BASEOPEN

This will be used to mark basefile opens of streams opens. This is
needed to later implement a function that can determine if a file has
stream opens.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13451

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s4:torture/vfs/fruit: adjust test testing basefile rename to expect failure

Renaming a basefile that has open streams must fail with
NT_STATUS_ACCESS_DENIED.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13451

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s4:torture/smb2/streams: try to rename basefile while is has open streams

This tests the following:

- create a file with a stream
- open the the stream and keep it open
- on a second connection, try to rename the basefile, this should fail
  with NT_STATUS_ACCESS_DENIED

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13451

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

selftest: run smb2.streams tests against a share with vfs_streams_xattr

The tests are currently only run against streams_depot, where stream IO
is handle based, compared to streams_xattr which is path
based. vfs_streams_xattr is also used much more in real world setups, so
we should run our tests against it.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13451

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

dsdb: Add log when ignoring a replicated object outside of partition

This is probably a note-worthy event for debugging purposes.

(Found while developing the domain rename functionality)

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed May 30 07:03:51 CEST 2018 on sn-devel-144

selftest: Use samba.tests.create_test_ou() in replica_sync tests

This may avoid some flapping tests by ensuring that each part of this
test runs in a unique namespace, no matter what may be left behind
or revived via replication.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

selftest: Use samba.tests.create_test_ou() in repl_move tests

This may avoid some flapping tests by ensuring that each part of this
test runs in a unique namespace, no matter what may be left behind
or revived via replication.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

selftest: Make create_test_ou() return a ldb.Dn

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

dsdb partition.c: Make partition_copy_all aysnc.

partition_copy_all uses ldb_wait to wait for the update to the primary
partition to complete, when updating a special dn.  If a module higher
up the chain inserts a callback, the code blocks in ldb_wait and does
not complete.  This change replaces the ldb_wait logic with a callback.

Currently there is no code that triggers this bug, however the up coming
audit logging changes do trigger this bug.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

ldb: Release ldb 1.4.0

* New LMDB backend (experimental)
* Comprehensive tests for index behaviour
* Enforce transactions for writes
* Enforce read lock use for all reads
* Fix memory leak in paged_results module.
  We hold at most 10 outstanding paged result cookies
  (bug #13362)
* Fix compiler warnings
* Python3 improvements
* Restore --disable-python build
* Fix for performance regression on one-level searches
  (bug #13448)
* Samba's subtree_rename could fail to rename some entries
  (bug #13452)

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

selftest: Add test to show that sam.ldb does not do a full scan in startup

We should add some other more complex operations here.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13448

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

ldb: Add tests for when we should expect a full scan

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13448

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

ldb: One-level search was incorrectly falling back to full DB scan

When no search filter is specified, the code falls back to using
'(|(objectClass=*)(distinguishedName=*)'. ltdb_index_dn() then failed
because matching against '*' is not indexed. The error return then
caused the code to fallback to a full-scan of the DB, which could have a
considerable performance hit.

Instead, we want to continue on and do the ltdb_index_filter() over the
indexed results that were returned.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13448

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

ldb: Explain why an entry can vanish from the index

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

ldb: Indicate that the ltdb_dn_list_sort() in list_union is a bit subtle.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

ldb: Save a copy of the index result before calling the callbacks.

Otherwise Samba modules like subtree_rename can fail as they modify the
index during the callback.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13452

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

subtree_rename: Correct comments

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

dsdb: Remove sort from subtree_delete and add comments.

The sort was written back when the module did not operate recursivly
over the tree. Now it is just confusing, so replace with useful
comments.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

selftest: Lock down the expected parents in BasicTreeDeleteTests

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13448

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

selftest: Rework BasicDeleteTests.test_all() into setUp() and a test

This will allow running multiple tests against the same tree.  This tree
is very similar to the tree produced by the KCC test that simply does a
tree_delete, and I want to lock down the tree_delete behaviour.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13448

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

samldb: Explain why the odd error code is expected.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

samldb: Add useful error string to explain why a group may not be deleted.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

tests: Fix intermittent error in PSO test

Deleting a group fails if the primaryGroupID of a user is set to that of
the group. This can happen in the PSO tests, as we don't clear the
primaryGroupID before cleaning up. Normally it seems to work OK, but
this is relying purely on the subtree delete order.

Update the test to clear the primaryGroupID before the tearDown is
called, to make things more robust.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13448

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

repl_meta_data: Cope with the strange but unusual case of isDeleted: FALSE in replmd_process_linked_attribute()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13448

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

repl_meta_data: Remove el_count from replmd_delete_internals()

Instead, use the actual found attribute (less error prone).

This is an attempt to fix:

./source4/dsdb/repl/replicated_objects.c:945 Failed to prepare commit of transaction:
attribute isDeleted: invalid modify flags on CN=g1_1527558311141,CN=Users,DC=samba,DC=example,DC=com: 0x0

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

s4-repl: Try to give more information in the error codes for prepare_commit failure.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

ldb: Reset error string before running prepare_commit() hook

This ensures that the error string returned to the caller reflects a failure in this call.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

vfs_fruit: delete 0 byte size streams if AAPL is enabled

macOS SMB server uses xattrs as storage backend for streams, directly
exposing xattr get/set characteristics. Setting EOF on a stream to 0
just deletes the xattr as macOS doesn't support 0-byte sized xattrs.

Note that this does not apply to the AFP_AfpInfo and AFP_Resource
streams, they have even stranger semantics and we have other tests
for those.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13441

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed May 30 02:34:29 CEST 2018 on sn-devel-144

s4:torture: test setting EOF of a stream to 0 with enabled AAPL extensions

macOS SMB server uses xattrs as storage backend for streams, directly
exposing xattr get/set characteristics. Setting EOF on a stream to 0
just deletes the xattr as macOS doesn't support 0-byte sized xattrs.

Note that this does not apply to the AFP_AfpInfo and AFP_Resource
streams, they have even stranger semantics and we have other tests
for those.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13441

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s4:torture/vfs/fruit: decrease large resource fork size in test from 1 GB to 64 MB

64 MB is a more realistic value and lets the test pass on FreeBSD with
fruit:resource=stream and vfs_streams_xattr.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

python/samba/netcmd/schema.py: samdb schema update now

When we change schema values, we should trigger a schema update to refresh
the changes applied. This is called after a change is made. A helper to
samdb is added so that it's easier for other locations to call additionally.

Signed-off-by: William Brown <william@blackhats.net.au>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue May 29 08:30:52 CEST 2018 on sn-devel-144

python/samba/tests/samba_tool/schema.py: Improve schema command testing

Assert the correct stdout content of the schema test commands.

Signed-off-by: William Brown <william@blackhats.net.au>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

python/samba/netcmd/schema.py: add schema show_oc for attribute

Often administrators need to add a specific attribute to an object, but
it may not be possible with the objectClasses present. This tool allows
searching "what objectclasses must or may?" take an attribute to help hint
to an administrator what objectclasses can be added to objects to achieve
the changes they want.

Signed-off-by: William Brown <william@blackhats.net.au>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

python/samba/netcmd/schema.py: add schema query and management.

Schema management in active directory is complex and dangerous. Having
a tool that safely wraps administrative tasks as well as allowing query
of the schema will make this complex topic more accessible to administrators.

Signed-off-by: William Brown <william@blackhats.net.au>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

python/samba/netcmd/group.py: add group show

The samba-tool user command can show the ldif of a user. This is
useful for groups also, especially to determine the objectSID and
objectGUID. Add support for group show to samba-tool.

Signed-off-by: William Brown <william@blackhats.net.au>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

python/samba/netcmd/{forest.py,main.py}: add configuration controls

With samba-tool we should expose ways to easily administer and control
common configuration options. This adds the base framework for modifying
forest settings, generally stored in cn=configuration partition.

An example is:

samba-tool forest directory_service show
samba-tool forest directory_service dsheuristics X

Signed-off-by: William Brown <william@blackhats.net.au>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

torture: Cope with WINBINDD_SHOW_SEQUENCE failure for fake trusts

This test has been flapping and we should not be checking the
sequence number of remote trusts in a loop like this.  We can write
a test against the trusts we specifically set up if we want to check
remote sequence numbers (and connectivity) against actually working
domains.

When flapping the previous version of the test gave:

 WARNING!: ../source4/torture/winbind/struct_based.c:812: WINBINDD_SHOW_SEQUENCE returned 0, expected 1: WINBINDD_SHOW_SEQUENCE
 WARNING!: WINBINDD_SHOW_SEQUENCE on domain torturedom failed

 Trust list for WINBINDD_SHOW_SEQUENCE was:
 BUILTIN
 S4MEMBER
 SAMBADOMAIN
 torturedom

 WARNING!: ../source4/torture/winbind/struct_based.c:812: WINBINDD_SHOW_SEQUENCE returned 0, expected 1: WINBINDD_SHOW_SEQUENCE
 WARNING!: WINBINDD_SHOW_SEQUENCE on domain TORTURE302 failed

 Trust list for WINBINDD_SHOW_SEQUENCE was:
 BUILTIN
 S4MEMBER
 SAMBADOMAIN
 TORTURE302
 TORTURE306

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue May 29 02:31:40 CEST 2018 on sn-devel-144

talloc: Fix some typos in the comments

Now with even more typos fixed. Thanks Rowland.

Signed-off-by: Kai Blin <kai@samba.org>
Reviewed-by: Rowland Penny <rpenny@samba.org>
Autobuild-User(master): Kai Blin <kai@samba.org>
Autobuild-Date(master): Mon May 28 16:16:17 CEST 2018 on sn-devel-144

vfs_gpfs: Use full_path_tos instead of talloc_asprintf

full_path_tos avoids the talloc call for most cases; use that instead of
talloc_asprintf.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat May 26 01:43:09 CEST 2018 on sn-devel-144

vfs_gpfs: Remove get_full_smb_filename from is_offline check

No stream information is required here.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs_gpfs: Remove wrong get_full_smb_filename from ntimes function

Updating the timestamps requires the path to the file, but no stream
information.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs_gpfs: Adjust debug level when get_winattrs returns EBADF

This is returned for a get_winattrs call against a non-gpfs file system.
This can happen for the .. entry when listing a share on the file system
root.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Move dfree_info struct

As the struct is no longer used as part of connection_struct, move it to
dfree.c.

This is not backported, as it would change the VFS ABI.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Flush dfree memcache on service reload

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13446

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Cache dfree information based on query path

Sub directories in a SMB share can have different free space information
(e.g. when a different file system is mounted there). Caching the dfree
information per SMB share will return invalid data. Address this by
switching to memcache and store the cached data based on the query path.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13446

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

memcache: Add new cache type for dfree information

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13446

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

selftest: Add test for 'dfree cache'

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13446

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

selftest: Add dfq_cache share with 'dfree cache time' set

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13446

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3: vfs_fake_acls: Correctly implement the chmod/fchmod algorithm on fake acls.

We now pass samba3hide(nt4_dc), so remove it from knownfail.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri May 25 21:29:32 CEST 2018 on sn-devel-144

s3: smbd: Make map_acl_perms_to_permset() extern.

The vfs_fake_acl module will need it to implement chown/fchown.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

s3: smbd: Make unix_perms_to_acl_perms() extern.

The vfs_fake_acl module will need it to implement chown/fchown.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

s3: posix_acls: Remove unused 'connection_struct *conn' parameter to map_acl_perms_to_permset().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

s3: posix_acls: Remove dead functions fchmod_acl()/chmod_acl().

No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

s3: VFS: Remove SMB_VFS_FCHMOD_ACL().

No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

s3: torture: Remove cmd_fchmod_acl().

No longer needed.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

s3: modules: vfs_ceph: Remove FCHMOD_ACL in cephwrap_fchmod().

Now I understand the use of the mask in POSIX ACLs
this extra step is no longer needed.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

s3: modules: vfs_default: Remove FCHMOD_ACL in fchmod.

Now I understand the use of the mask in POSIX ACLs
this extra step is no longer needed.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

s3: VFS: Remove SMB_VFS_CHMOD_ACL().

No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

s3: torture: Remove cmd_chmod_acl().

No longer needed.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

s3: modules: vfs_ceph: Remove CHMOD_ACL in cephwrap_chmod().

Now I understand the use of the mask in POSIX ACLs
this extra step is no longer needed.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

s3: modules: vfs_ceph: Remove CHMOD_ACL in cephwrap_mkdir().

Now I understand the use of the mask in POSIX ACLs
this extra step is no longer needed.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

s3: modules: vfs_default: Remove CHMOD_ACL in chmod.

Now I understand the use of the mask in POSIX ACLs
this extra step is no longer needed.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

s3: modules: vfs_default: Remove CHMOD_ACL in mkdir.

Now I understand the use of the mask in POSIX ACLs
this extra step is no longer needed. If the mkdir
succeeded it's already set the correct mode.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

s3: smbd: Use FCHMOD call, not FCHMOD_ACL call if mode bits reset needed.

This is a behavior change, it will modify the POSIX ACL mask
from a value of rwx instead of modifying the existing ACE
entries to be ANDed with the passed in mode. However it
will have no effect on the underlying permissions, and
better reflects the proper use of POSIX ACLs (i.e. I
didn't understand the use of the mask entry in the
ACL when I first wrote the POSIX ACL code).

In addition, the vfs_acl_common.c module already
filters these calls for all but POSIX opens, which
means the only place this change is exposed to the
client would be a cifsfs unix extensions client doing
posix acl calls (and they would expect the mask to
be set like this on chmod).

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

s3: smbd: Optimization. Only do the FCHMOD_ACL call if mode bits not equal.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

s3: smbd: optimization. Only do the FCHMOD call if needed.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

s3: smbd: Remove use of SMB_VFS_FCHMOD_ACL() in overwrite case.

We have potentially called SMB_VFS_FCHMOD() here in
the file_set_dosmode() call associated with the comment
/* Overwritten files should be initially set as archive */
at line 3755 above, so there is no need to do any POSIX ACL
mask protection.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

s3: smbd: Add clarifying comment on mode change on overwritten files.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

s3: smbd: Remove existing_unx_mode, an unused parameter to open_match_attributes().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

smbd: Call smbXsrv_client_global_init in the parent smbd

Otherwise we're missing the clear-if-first optimization for
smbXsrv_client_global.tdb.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Fri May 25 16:00:08 CEST 2018 on sn-devel-144

vfs_time_audit: Fix a log message

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

autobuild: cover the Gentoo case with python disabled all down the stack

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Fri May 25 13:07:47 CEST 2018 on sn-devel-144

Make ldb configuration --disable-python work as intended

Signed-off-by: Timur I. Bakeyev <timur@iXsystems.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

torture: Give extra information on WINBINDD_SHOW_SEQUENCE failure

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s3:utils: Remove double error check

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
Autobuild-User(master): Günther Deschner <gd@samba.org>
Autobuild-Date(master): Thu May 24 18:07:03 CEST 2018 on sn-devel-144

ldap_server: Fix CID 1435721 Unchecked return value

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu May 24 03:35:00 CEST 2018 on sn-devel-144

kdc: Fix CID 1435720 Unchecked return value

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

ldap_server: Fix CID 1435731 Unchecked return value

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

ctdbd: Fix CID 1435732 Argument cannot be negative

setenv returns its error in "errno"

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

audit_logging: Fix CID 1435739 Dereference null return value

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

ctdb: Fix CID 1435740 Unchecked return value

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

autobuild: Include information of disk free in system-info.txt

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu May 24 00:42:48 CEST 2018 on sn-devel-144

gitlab-ci: Do not hide a possible out-of-space condition by cleaning up the tree

These are VMs anyway and will soon vanish, so a cleanup is totally wasted in any case.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

gitlab-ci: Include system-info.txt to give info on possible failures

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

dsdb: Avoid calculating the PSO multiple times

In a typical user login query, the code tries to work out the PSO 2-3
times - once for the msDS-ResultantPSO attribute, and then again for the
msDS-User-Account-Control-Computed & msDS-UserPasswordExpiryTimeComputed
constructed attributes.

The PSO calculation is reasonably expensive, mostly due to the nested
groups calculation. If we've already constructed the msDS-ResultantPSO
attribute, then we can save ourselves extra work by just re-fetching the
result directly, rather than expanding the nested groups again from
scratch.

The previous patch improves efficiency when there are no PSOs in the
system. This should improve the case where there are PSOs that apply to
the users. (Unfortunately, it won't help where there are some PSOs in
the system, but no PSO applies to the user being queried).

Also updated sam.c so the msDS-ResultantPSO gets calculated first,
before the other constructed attributes.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Autobuild-User(master): Garming Sam <garming@samba.org>
Autobuild-Date(master): Wed May 23 10:09:11 CEST 2018 on sn-devel-144

dsdb: Avoid performance hit if PSOs aren't actually used

The new PSO code adds some additional overhead in extra lookups. To
avoid penalizing existing setups, we can short-circuit the PSO
processing and return early if there are no actual PSO objects in the
DB. The one-level search should be very quick, and it avoids the need to
do more complicated PSO processing (i.e. expanding the nested groups).

The longer-term plan is to rework the tokenGroups lookup so that it only
gets done once, and the result can then be reused by the resultant-PSO
code (rather than computing the nested-groups again). However, in the
short-term, a slight decrease in performance is the price for any users
that want to deploy PSOs.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

tests: Add tests for domain pwdHistoryLength

This is not related to PSOs at all, but there's a minor discrepancy
between Windows and Samba password-history-length behaviour that I
noticed during PSO testing.

When the pwdHistoryLength changes from zero to non-zero, Windows
includes the user's current password as invalid immediately, whereas
Samba only includes it as invalid *after* it next changes. It's a
fairly obscure corner-case, and we might not care enough about it to
fix it. However, I've added a test case to highlight the difference and
marked it as a known-fail for now.

I also added a general pwdHistoryLength test case to show that the
basics work (this didn't seem to be tested anywhere else).

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

tests: Extend passwordsettings tests to cover PSO command options

Add test cases for the new PSO samba-tool command options.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

netcmd: Add samba-tool support for managing PSOs

Add a new command 'samba-tool domain passwordsettings pso', with the
sub-command options: create, delete, set, list, show, show-user, apply,
unapply. The apply and unapply options apply the PSO to a user or group.
The show-user option shows the actual PSO (and its settings) that will
take effect for a given user.

The new commands are pretty self-contained in a new pso.py file. We
decided to add these new commands under the existing 'samba-tool domain
passwordsettings' command, as that's what users would be already
familiar with.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

dsdb: Use PSO maxPwdAge for operational msDS-PasswordExpiryTimeComputed

When calculating the Password-Expiry-Time, we should use the PSO's
max-password-age setting, if one applies to the user.

This is code may be inefficient, as it may repeat the PSO-lookup work
several times (once for each constructed attribute that tries to use
it). For now, I've gone for the simplest code change, and efficiency can
be addressed in a subsequent patch (once we have a good test to measure
it).

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

dsdb: Update password_hash to use PSO settings for password changes

Honour the settings in the PSO when changing the password, i.e.
msDS-PasswordComplexityEnabled, msDS-PasswordHistoryLength, etc.

The password_hash code populates dsdb_control_password_change_status's
domain_data with the password settings to use - these are currently
based on the settings for the domain.

Now, if the password_hash code has worked out that a PSO applies to the
user, we override the domain settings with the PSO's values.

This change means the password_settings tests now pass.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

dsdb: Move anonymous domain_data struct

Anonymous structs and 80 character line-lengths don't mix well. Allow
the struct to be referenced directly.

With the introduction of PSOs, the password-settings are now calculated
per-user rather than per-domain. I've tried to reflect this in the
struct name.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

dsdb: Lookup PSO's lockout settings for password_hash modifies

When a user's password-hash is modified, we need the PSO settings for
that user, so that any lockout settings get applied correctly.

To do this, we query the msDS-ResultantPSO in the user search. Then, if
a PSO applies to the user, we add in a extra search to retrieve the
PSO's settings. Once the PSO search completes, we continue with the
modify operation.

In the event of error cases, I've tried to fallback to logging the
problem and continuing with the default domain settings. However,
unusual internal errors will still fail the operation.

We can pass the PSO result into dsdb_update_bad_pwd_count(), which means
the PSO's lockout-threshold and observation-window are now used. This is
enough to get the remaining lockout tests passing.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

rpc/samr: Fix PSO support in SAMR password_change RPC

To get the SAMR password_lockout test passing, we now just need to query
the msDS-ResultantPSO attribute for the user in the SAMR code. The
common code will then determine that a PSO applies to the user, and use
the PSO's lockout settings.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

dsdb/rpc: Update effective badPwdCount to use PSO settings

The lockOutObservationWindow is used to calculate the badPwdCount. When
a PSO applies to a user, we want to use the PSO's lockout-observation
window rather the the default domain setting.

This is finally enough to get some of the PSO password_lockout tests
to pass.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

dsdb: PSO support for msDS-User-Account-Control-Computed

msDS-User-Account-Control-Computed uses the effective-lockoutDuration to
determine if a user is locked out or not. If a PSO applies to the user,
then the effective-lockoutDuration is the PSO's msDS-LockoutDuration
setting. Otherwise it is the domain default lockoutDuration value.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

dsdb/auth: Use PSO settings for lockOutThreshold/Duration

If a PSO applies to a user, use its lockOutThreshold/Duration settings
instead of the domain setting. When we lookup a user, we now include the
msDS-ResultantPSO attribute. If the attribute is present for a user,
then we lookup the corresponding PSO object to get the lockOutThreshold/
Duration settings.

Note: This is not quite enough to make the PSO lockout tests pass, as
msDS-User-Account-Control-Computed is still constructed based on the
domain lockoutDuration setting rather than the PSO.

Updating the password_hash.c code properly will be done in a subsequent
commit.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

tests: Extend PSO tests to cover password-history/length/complexity

Unhobble the PSO test cases so that they not only check the
msDS-ResultantPSO constructed attribute, but also that the corresponding
PSO's password-history, minimum password length, and complexity settings
are actually used.

The tests now fail once more, as actually using the PSO's settings isn't
implemented yet.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

dsdb: Add msDS-ResultantPSO constructed attribute support

Add support for the msDS-ResultantPSO constructed attribute, which
indicates the PSO (if any) that should apply to a given user. First we
consider any PSOs that apply directly to a user. If none apply directly,
we consider PSOs that apply to any groups the user is a member of. (PSO
lookups are done by finding any 'msDS-PSOAppliesTo' links that apply to
the user or group SIDs we're interested in.

Note: the PSO should be selected based on the RevMembGetAccountGroups
membership, which doesn't include builtin groups. Looking at the spec,
it appears that perhaps our tokenGroups implementation should also
exclude builtin groups. However, in the short-term, I've added a new
ACCOUNT_GROUPS option to the enum, which is only used internally for
PSOs.

The PSO test cases (which are currently only checking the constructed
attribute) now pass, showing that the correct msDS-ResultantPSO value is
being returned, even if the corresponding password-policy settings are
not yet being applied.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

autobuild: build ldb --without-ldb-lmdb

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed May 23 05:11:13 CEST 2018 on sn-devel-144