s3:rpc_client: use cli_credentials based functions in cli_rpc_pipe_open_schannel()

This simplifies the code and allows the previous password to be passed
through the stack.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

s3:rpc_client: remove unused auth_level paramter of cli_rpc_pipe_open_schannel()

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

s3:cli_netlogon: cli_credentials_get_old_nt_hash() in rpccli_setup_netlogon_creds_with_creds()

This way we'll fallback to use the previous machine/trust account password
if required.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

auth/credentials: add cli_credentials_set_old_utf16_password()

This is required to set the previous trust account password.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

auth/credentials: add cli_credentials_[g|s]et_old_nt_hash()

The machine and trust accounts it's important to retry
netr_Authenticate3() with the previous (old) nt_hash.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

auth/credentials: add a missing talloc check to cli_credentials_set_nt_hash()

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

s4:pydsdb: add DSDB_CONTROL_PERMIT_INTERDOMAIN_TRUST_UAC_OID

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

selftest: Change testsuite to use a samAccountName with a space in it

This shows that the previous patch is correct

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

kdc: Ensure we cope with a samAccountName with a space in it

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

dsdb: Ensure we cope with a samAccountName with a space in it in DsCrackName()

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

selftest: Change testsuite to use a UPN with a space in it

This shows that the previous patch is correct

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

selftest: fix the basedn for local accounts in non-DC environments e.g. s4member

open(LDIF, "|$ldbmodify -H $ctx->{privatedir}/sam.ldb");
doesn't generate an error if the command fails...

'testallowed' is a local account here, with a dn of
CN=testallowed,CN=Users,DC=S4MEMBER instead of domain user
CN=testallowed,CN=Users,DC=samba,DC=example,DC=com

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

dsdb: Allow spaces in userPrincipalName values

This is needed to enable a kinit with a UPN that has a space in it

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

heimdal:lib/krb5: let build_logon_name() use KRB5_PRINCIPAL_UNPARSE_DISPLAY

An ENTERPRISE principal should result in 'administrator@S4XDOM.BASE'
instead of 'administrator\@S4XDOM.BASE'.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11142
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

heimdal:lib/krb5: allow enterprise principals in verify_logonname()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11142
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

torture-krb5: Test accepting the ticket to ensure PAC is well-formed

A future test will ask for impersonation to a different user, and
validate returned principal and the PAC matches that user.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11142
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

auth/kerberos: Use KRB5_PRINCIPAL_UNPARSE_DISPLAY in kerberos_create_pac()

This ensures that in the all-Samba PAC creation code, we do not escape a space character if present
in the logon name.  This matches what we do in the Heimdal code in the KDC.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

auth/kerberos: Do a string comparison in kerberos_decode_pac() not a principal comparison

This ensures that if an enterprise principal is used, we do the
comparison properly

This matters as in the enterprise case, which can be triggered by MIT
kinit -E, does not use canonicalization, and so the enterprise name,
with the @ in it, is in the logon name.

Otherwise, we get errors like:
 Name in PAC [TESTALLOWED@WIN2012R2] does not match principal name in ticket

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11142

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

heimdal:krb5.asn1: remove KRB5_PADATA_CLIENT_CANONICALIZED handling

This got removed between draft-ietf-krb-wg-kerberos-referrals-11.txt
and the final rfc6806.txt.

The number 133 was reassigned to PA-FX-COOKIE in rfc6113.txt.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

heimdal:kdc: remove KRB5_PADATA_CLIENT_CANONICALIZED handling

This got removed between draft-ietf-krb-wg-kerberos-referrals-11.txt
and the final rfc6806.txt.

The number 133 was reassigned to PA-FX-COOKIE in rfc6113.txt.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

heimdal:lib/krb5: remove KRB5_PADATA_CLIENT_CANONICALIZED handling

This got removed between draft-ietf-krb-wg-kerberos-referrals-11.txt
and the final rfc6806.txt.

The number 133 was reassigned to PA-FX-COOKIE in rfc6113.txt.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

selftest: also test python.samba.tests.posixacl against plugin_s4_dc_no_nss

Pair-Programmed-With: Guenther Deschner <gd@samba.org>

Signed-off-by: Michael Adam <obnox@samba.org>
Signed-off-by: Guenther Deschner <gd@samba.org>
Autobuild-User(master): Michael Adam <obnox@samba.org>
Autobuild-Date(master): Thu Mar 12 17:12:11 CET 2015 on sn-devel-104

selftest: add a new environment plugin_s4_dc_no_nss

Pair-Programmed-With: Guenther Deschner <gd@samba.org>

Signed-off-by: Michael Adam <obnox@samba.org>
Signed-off-by: Guenther Deschner <gd@samba.org>

selftest: extend setup_plugin_s4_dc to allow for not using nss_winbindd

Pair-Programmed-With: Guenther Deschner <gd@samba.org>

Signed-off-by: Michael Adam <obnox@samba.org>
Signed-off-by: Guenther Deschner <gd@samba.org>

selftest: modify python.samba.test.posixacl to cope with nss_winbind active

It was observed that adding libnss_winbind (via nss_wrapper) lets
the posix acl mapping come out slightly differently with respect
to the owner/domain admin who is not explicitly nailed down in
the original NT acl.

This patch extends the test to react to the presence of
nss_winbind in environment and adapts the expected results.
This in particular fixes the run of the test against the
(changed) plugin_s4_dc environment while keeping the possibility
to successfully run it against an env without nss_winbind.

Pair-Programmed-With: Guenther Deschner <gd@samba.org>

Signed-off-by: Michael Adam <obnox@samba.org>
Signed-off-by: Guenther Deschner <gd@samba.org>

brlock: Use 0 instead of empty initializer list

C does not allow empty initializer lists. Although gcc accepts that, the
SunOS compiler fails in this case with an error.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11153

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Autobuild-User(master): Christof Schmitt <cs@samba.org>
Autobuild-Date(master): Thu Mar 12 02:49:36 CET 2015 on sn-devel-104

lib/util: Include DEBUG macro in internal header files before samba_util.h

It's best practice to include external header files before internal
header files. In this case internal DEBUG macro cannot be defined and
therefore samba version of debug macro will be included
in header file "util/fault.h".

In file included from example.c:27:0:
src/util/util.h:127:0: error: "DEBUG" redefined [-Werror]
 #define DEBUG(level, format, ...) do { \
 ^
In file included from /usr/include/samba-4.0/util/fault.h:29:0,
                 from /usr/include/samba-4.0/samba_util.h:62,
                 from /usr/include/samba-4.0/ndr.h:30,
                 from example.c:24:
/usr/include/samba-4.0/util/debug.h:182:0: note: this is the location of the previous definition
 #define DEBUG( level, body ) \
 ^
  CC       src/providers/ad/libsss_ad_common_la-ad_domain_info.lo
cc1: all warnings being treated as errors

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11033

Signed-off-by: Lukas Slebodnik <lslebodn@redhat.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Mar 11 18:47:22 CET 2015 on sn-devel-104

smbd: Simplify create_token_from_sid()

This if-statement is unnecessary. First, talloc_array returns non-NULL
even if asked for 0 elements. Second, a bit further down we do a

SMB_ASSERT(num_group_sids > 0);

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Simplify create_token_from_sid()

With the previous commit all 3 branches do the same

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Streamline the gids handling in create_token_from_sid()

Usually, I'm all for avoiding talloc. But in this case I believe that this
routine is complex enough to justify this change. For an hour or so I suspect
that the winbind case had an uninitialized "*gid" until I discovered the
sid_to_gid(). This makes it more obvious that *gid is assigned.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Put a variable definition closer to its use

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lib: Avoid a malloc/realloc in getgroups_unix_user

This avoids a malloc/free in the most common case of a user with just a few
group memberships

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lib: Fix whitespace

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

ctdb: Fix CID 1288201 Array compared against 0

"helper_prog" is now declared as a static array

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

idl: define FSCTL_DUPLICATE_EXTENTS_TO_FILE

As specified in the recent 20150129 revision of MS-FSCC.
Add a note regarding the FileHandle field, which was confirmed to
correspond to the volatile part of the fileid:
https://lists.samba.org/archive/samba-technical/2015-February/105454.html

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

tdb: Fix CID 1034842 Resource leak

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ira Cooper <ira@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Wed Mar 11 00:23:20 CET 2015 on sn-devel-104

tdb: Fix CID 1034841 Resource leak

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ira Cooper <ira@samba.org>

lib: Fix CID 1034840 Resource leak

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ira Cooper <ira@samba.org>

lib: Fix CID 1034839 Resource leak

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ira Cooper <ira@samba.org>

lib: Fix CID 1034838 Resource leak

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ira Cooper <ira@samba.org>

smbcontrol: Simplify do_winbind_offline

This saves 128 bytes of .text on x86-64 with -O3. No idea why...

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ira Cooper <ira@samba.org>

libreplace: Fix CID 1034926 Destination buffer too small

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ira Cooper <ira@samba.org>

ctdb: Fix 1125553 Buffer not null terminated

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ira Cooper <ira@samba.org>

registry: Fix CID 240989 Buffer not null terminated

This makes it clearer that we don't really have a string in .hdr

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ira Cooper <ira@samba.org>

registry: Fix CID 241075 Unchecked return value

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ira Cooper <ira@samba.org>

torture4: Fix systems with a 32-bit "long"

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan (metze) Metzmacher <metze@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Tue Mar 10 18:05:13 CET 2015 on sn-devel-104

ctdb-daemon: Use statically allocated arrays for helper paths

The use of talloc with a static variable is somewhat confusing.
Statically allocate an array and use ctdb_set_helper() instead.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Volker Lendecke <vl@samba.org>

ctdb-common: New function ctdb_set_helper()

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Volker Lendecke <vl@samba.org>

Update libwbclient version to 0.12

Increment the minor version of the libwbclient library after new
context functions added. (Major version increase not required as
the only two functions with changed parameters are private to the
library.)

Signed-off-by: Matthew Newton <matthew-git@newtoncomputing.co.uk>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Mar 10 03:24:45 CET 2015 on sn-devel-104

Move wbc global variables into global context instead

There are some global variables in use in the libwbclient
library. Now that we have a context, move these into it so that
they are thread-safe when the wbcCtx* functions are used.

Signed-off-by: Matthew Newton <matthew-git@newtoncomputing.co.uk>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Add context versions of wbclient functions

To make the libwbclient library thread-safe, all functions
that call through to wb_common winbindd_request_response need
to have context that they can use. This commit adds all the
necessary functions.

Signed-off-by: Matthew Newton <matthew-git@newtoncomputing.co.uk>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Add wbcContext to wbcRequestResponse

To enable libwbclient to pass winbindd context through
to the winbind client library in wb_common.

Signed-off-by: Matthew Newton <matthew-git@newtoncomputing.co.uk>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Add wbcContext struct, create and free functions

The basic context structure and functions for libwbclient so that
libwbclient can be made thread-safe.

Signed-off-by: Matthew Newton <matthew-git@newtoncomputing.co.uk>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Use global context for winbindd_request_response

Updating API call in libwbclient, wbinfo, ntlm_auth and
winbind_nss_* as per previous commit to wb_common.c.

Signed-off-by: Matthew Newton <matthew-git@newtoncomputing.co.uk>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Make winbind client library thread-safe by adding context

Rather than keep state in global variables, store the current
context such as the winbind file descriptor in a struct that is
passed in. This makes the winbind client library thread-safe.

Signed-off-by: Matthew Newton <matthew-git@newtoncomputing.co.uk>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

torture/ioctl: add range overflow QAR test

Issue a QAR request with an offset and length that generate an integer
(uint64_t) overflow when summed together. This should result in an
NT_STATUS_INVALID_PARAMETER response, as confirmed against Windows
Server 2012 & 2008.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Mar 10 00:02:18 CET 2015 on sn-devel-104

torture/ioctl: add multi-range QAR test

Write 10 x 64K ranges, with 64K holes punched in between. Afterwards,
check that all ranges are present in the QAR response.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

torture/ioctl: add QAR off-by-one bug paranoia test

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

torture/ioctl: test sparse file operation locking

An exclusively locked file can still be marked sparse. QAR requests
covering the locked-range should also succed. ZERO_DATA requests are
blocked.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3/smbd: fix FSCTL_SET_SPARSE permission checks

On Windows servers (tested against Windows Server 2008 & 2012) the
FSCTL_SET_SPARSE ioctl is processed if FILE_WRITE_DATA,
FILE_WRITE_ATTRIBUTES _or_ SEC_FILE_APPEND_DATA permissions are granted
on the open file-handle.
Fix Samba such that it matches this behaviour, rather than only checking
for FILE_WRITE_DATA or FILE_WRITE_ATTRIBUTES.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

torture/ioctl: add ioctl_sparse_perms test

This test confirms that correct FSCTL_SET_SPARSE permission checks are
in place on the server.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

torture/ioctl: rework and reduce pattern helper IO sizes

check_pattern() currently attempts to read all data in one go. Fix it to
use a 64K maximum IO size so that it works against Windows Server 2008.

Additionally, rework write_pattern() so that it only allocates a buffer
for the largest IO size (now 64K), rather than for the full write
length.

Finally, assert that callers are correctly performing pattern IO in
8-byte increments - copy_chunk_tiny was not, so fix it.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

torture/ioctl: add sparse_punch_invalid test

Attempt to extend a file using ZERO_DATA. The operation should succeed,
but the file should not be extended, as specified in MS-FSCC <58>
Section 2.3.65:
  This FSCTL sets the range of bytes to zero (0) without extending the
  file size.

Also test zero length and invalid BFZ requests.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

torture/ioctl: remove FS specific sparse copy-chunk expectations

NTFS deallocates an entire file when a sparse zero-data request spans
the full length. Other filesystems (e.g. EXT4 and Btrfs) do not.

vfs_btrfs is additionally capable of preserving sparse regions for
copy-chunk, using the BTRFS_IOC_CLONE_RANGE ioctl. This should not be
treated as a failure.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

torture/ioctl: remove FS specific sparse punch check

Samba uses PUNCH_HOLE to zero a range, and subsequently uses fallocate()
to allocate the punched range if the file is marked non-sparse and
"strict allocate" is enabled.

In both cases, the zeroed range will not be detected by SEEK_DATA, so
the range won't be present in QAR responses until the file is marked
non-sparse again.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

torture/ioctl: remove 64K chunk size assumptions

These tests assumed that 4K chunks remain allocated following write at
a subsequent offset. This is not the case for other filesystems (E.g.
XFS, Btrfs, Etc.), which may deallocate the chunk.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3/statvfs: expose FILE_SUPPORTS_SPARSE_FILES capability

Samba now supports:
- FSCTL_SET_SPARSE
- FSCTL_SET_ZERO_DATA, via FALLOC_FL_PUNCH_HOLE
- FSCTL_QUERY_ALLOCATED_RANGES, via SEEK_DATA/SEEK_HOLE

As such, flag support for sparse files, via the
FILE_SUPPORTS_SPARSE_FILES capability flag if FALLOC_FL_PUNCH_HOLE and
SEEK_DATA/SEEK_HOLE are present at configure time.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd/ioctl: add FSCTL_QUERY_ALLOCATED_RANGES support

This change implements support for FSCTL_QUERY_ALLOCATED_RANGES using
the SEEK_HOLE/SEEK_DATA functionality of lseek().

Files marked non-sparse are always reported by the ioctl as fully
allocated, regardless of any potential "strict allocate = no" savings.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

build: check for SEEK_HOLE and SEEK_DATA support

SEEK_HOLE and SEEK_DATA will be used in the implementation of
FSCTL_QUERY_ALLOCATED_RANGES support.

"SEEK_DATA and SEEK_HOLE are nonstandard extensions also present
 in Solaris, FreeBSD, and DragonFly BSD; they are proposed for
 inclusion in the next POSIX revision (Issue 8)."

With Linux they are supported on:
-  Btrfs (since Linux 3.1)
-  OCFS (since Linux 3.2)
-  XFS (since Linux 3.5)
-  ext4 (since Linux 3.8)
-  tmpfs (since Linux 3.8)

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

idl/ioctl: change QAR response array to a DATA_BLOB

[MS-FSCC] specifies:
  The number of FILE_ALLOCATED_RANGE_BUFFER elements returned is
  computed by dividing the size of the returned output buffer (from
  either SMB or SMB2, the lower-layer protocol that carries the FSCTL)
  by the size of the FILE_ALLOCATED_RANGE_BUFFER element.

Ideally, this requirement could be defined in idl with the following:
  [flag(NDR_REMAINING)] file_alloced_range_buf array[];

However, this is not currently supported by PIDL, so just use an opaque
data blob for now.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd/ioctl: add FSCTL_SET_ZERO_DATA support

FSCTL_SET_ZERO_DATA can be used in two ways.
- When requested against a file marked as sparse, it provides a
  mechanism for requesting that the server deallocate the underlying
  disk space for the corresponding zeroed range.
- When requested against a non-sparse file, it indicates that the server
  should allocate and zero the corresponding range.

Both use cases can be handled in Samba using fallocate(). The Linux
specific FALLOC_FL_PUNCH_HOLE flag can be used to deallocate the
underlying disk space. After doing so, a normal fallocate() call can
be used to ensure that the zeroed range is allocated on non-sparse
files.

FSCTL_SET_ZERO_DATA requests must not result in a change to the file
size. The FSCTL_SET_ZERO_DATA handler always calls fallocate() with the
KEEP_SIZE flag set, ensuring that Samba meets this requirement.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

system: add hole punch support to sys_fallocate()

If Samba is configured with FALLOC_FL_PUNCH_HOLE support, then allow
sys_fallocate() to propogate the flag to syscall invocation.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

build: check for fallocate hole-punch support

Add a configure time check for the FALLOC_FL_PUNCH_HOLE Linux specific
fallocate() flag. It's been around since 2.6.38.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3/vfs: change fallocate mode flags from enum->uint32_t

The Linux fallocate syscall offers a mode parameter which can take the
following flags:
FALLOC_FL_KEEP_SIZE
FALLOC_FL_PUNCH_HOLE (since 2.6.38)
FALLOC_FL_COLLAPSE_RANGE (since 3.15)
FALLOC_FL_ZERO_RANGE (since 3.14)

The flags are not exclusive, e.g. FALLOC_FL_PUNCH_HOLE must be specified
alongside FALLOC_FL_KEEP_SIZE.

Samba currently takes a vfs_fallocate_mode enum parameter for the VFS
fallocate hook, taking either an EXTEND_SIZE or KEEP_SIZE value. This
commit changes the fallocate hook such that it accepts a uint32_t flags
parameter, in preparation for PUNCH_HOLE and ZERO_RANGE support.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lib/system: remove useless HAVE_LINUX_FALLOCATE64 logic

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3-winbind: Fix chached user group lookup of trusted domains.

If a user group lookup has aleady been done before with a machine
account we did always return the incomplete information from the cache.
This patch makes sure we return the correct group information from the
netsamlogon cache.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11143

Pair-Programmed-With: Andreas Schneider <asn@samba.org>
Signed-off-by: Michael Adam <obnox@samba.org>
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Mon Mar  9 19:23:25 CET 2015 on sn-devel-104

torture-krb5: Add an initial test for s4u2self behaviour

This test only checks for S4U2Self of the same user, but shows
that a user account is not a valid service for this purpose.

Andrew Bartlett

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Mon Mar  9 12:10:09 CET 2015 on sn-devel-104

kdc: Fix S4U2Self handling with KRB5_NT_ENTERPRISE_PRINCIPAL containing a UPN

This is now handled properly by samba_kdc_lookup_server() and this wrapper actually
breaks things.

Andrew Bartlett

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

talloc: version 2.1.2

Changes:
- Allow destructors to reparent the object
- Allow destructors to remove itself
- Build improvements

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>

talloc: fix _talloc_total_limit_size prototype

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

lib: talloc: Test suite for the new destructor reparent logic.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ira Cooper <ira@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sun Mar  8 20:52:43 CET 2015 on sn-devel-104

lib: talloc: Allow destructors to reparent the object they're called on.

If a destructor returns failure (-1) when freeing a child, talloc
must then reparent the child.

Firstly it tries the owner of any reference, next the parent of the
current object calling _talloc_free_children_internal(), and finally
the null context in the last resort.

If a destructor reparented its own object, which can be a very
desirable thing to do (a destructor can make a decision it isn't
time to die yet, and as the parent may be going away it might
want to move itself to longer-term storage) then this new parent
gets overwritten by the existing reparenting logic.

This patch checks when freeing a child if it already reparented
itself, and if it did doesn't then overwrite the new parent.

Makes destructors more flexible.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ira Cooper <ira@samba.org>

lib: talloc: Fix bug when calling a destructor.

If the destructor itself calls talloc_set_destructor()
and returns -1, the new destructor set is overwritten
by talloc.

Dectect that and leave the new destructor in place.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ira Cooper <ira@samba.org>

Add the definition of FSCTL_SET_INTEGRITY_INFORMATION found from a capture and the Web.

Signed-off-by: Richard Sharpe <rsharpe@samba.org>
Reviewed-by: Ira Cooper <ira@samba.org>
Autobuild-User(master): Ira Cooper <ira@samba.org>
Autobuild-Date(master): Sun Mar  8 00:43:08 CET 2015 on sn-devel-104

torture/fsrvp: remove verification trailer magic field

During initial wire trace analysis, the DCE/RPC PDU verification trailer
was incorrectly identified and tagged in IDL as an FSRVP "magic" blob.

This change removes the incorrectly tagged FSRVP request fields and
corresponding test code - with 1e1b7b1021b16e3ab61c2fca8328c94e60a2c99c
verification trailer parsing is now tested separately.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Mar  7 20:01:20 CET 2015 on sn-devel-104

winbind: Make wb_sids2xids_recv work on an array

The trigger for this is that Coverity got confused by the dual use of &xid
as an array with the implicit length equality between wb_sids2xids_send
and the array passed in to wb_sids2xids_recv for the result.

I don't want to start doing things just for the Coverity scan, but this
makes the code clearer to me by removing this implicit expected array
length equality.

Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: David Disseldorp <ddiss@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Sat Mar  7 15:28:59 CET 2015 on sn-devel-104

vfs_fruit: Fix CID 1273290 Uninitialized scalar variable

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Fri Mar  6 23:56:36 CET 2015 on sn-devel-104

provision: Give a more helpful message when find_provision_key_parameters() fails

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Fri Mar  6 20:11:52 CET 2015 on sn-devel-104

selftest: Improve renamedcs test

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s4-lib/cmdline: Fix help for -P / --machine-pass: this no longer implies -k

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

samba-tool: Add -P to options.CredentialsOptions

This matches our other binaries, and allows samba-tool commands to run with the machine account.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

heimdal: Fix CID 1273430 Double free

I think Coverity is right here: Before the preceding call to
krb5_make_principal we already krb5_free_principal(ctx, tmp_creds.server)
without wiping out tmp_creds.server. The call to krb5_make_principal only
stores something fresh when it also returns 0 a.k.a. success.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: David Disseldorp <ddiss@samba.org>
Autobuild-User(master): David Disseldorp <ddiss@samba.org>
Autobuild-Date(master): Fri Mar  6 17:38:09 CET 2015 on sn-devel-104

script/autobuild.py: build the samba target with --with-profiling-data

In future we may get also runtime tests for profiling...

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Autobuild-User(master): Michael Adam <obnox@samba.org>
Autobuild-Date(master): Mon Dec 15 16:20:14 CET 2014 on sn-devel-104

(cherry picked from commit 4958fcdfa30fd9d8dc51ceafaab35721e61e72c7)

s3:smbprofile: profile the system and user space cpu time

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3:smbprofile: Replace sysv shmem with tdb

What?

This patch gets rid of the central shared memory segment referenced by
"profile_p". Instead, every smbd gets a static profile_area where it collects
profiling data. Once a second, every smbd writes this profiling data into a
record of its own in a "smbprofile.tdb". smbstatus -P does a tdb_traverse on this
database and sums up what it finds.

Why?

At least in my perception sysv IPC has not the best reputation on earth. The
code before this patch uses shmat(). Samba ages ago has developed a good
abstraction of shared memory: It's called tdb.

The main reason why I started this is that I have a request to become
more flexible with profiling data. Samba should be able to collect data
per share or per user, something which is almost impossible to do with
a fixed structure. My idea is to for example install a profile area per
share and every second marshall this into one tdb record indexed by share
name. smbstatus -P would then also collect the data and either aggregate
them or put them into individual per-share statistics. This flexibility
in the data model is not really possible with one fixed structure.

But isn't it slow?

Well, I don't think so. I can't really prove it, but I do believe that on large
boxes atomically incrementing a shared memory value for every SMB does show up
due to NUMA effects. With this patch the hot code path is completely
process-local. Once a second every smbd writes into a central tdb, this of
course does atomic operations. But it's once a second, not on every SMB2 read.

There's two places where I would like to improve things: With the current code
all smbds wake up once a second. With 10,000 potentially idle smbds this will
become noticable. That's why the current only starts the timer when something has
changed.

The second place is the tdb traverse: Right now traverse is blocking in the
sense that when it has to switch hash chains it will block. With mutexes, this
means a syscall. I have a traverse light in mind that works as follows: It
assumes a locked hash chain and then walks the complete chain in one run
without unlocking in between. This way the caller can do nonblocking locks in
the first round and only do blocking locks in a second round. Also, a lot of
syscall overhead will vanish. This way smbstatus -P will have almost zero
impact on normal operations.

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3:smbprofile: specify SMBPROFILE_STATS_SECTION_START() with name vs. display[name]

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Implement TestCase.assertIsNotNone for python < 2.7.

Change-Id: Ieaefdc77495e27bad791075d985a70908e9be1ad
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Mar  6 07:11:43 CET 2015 on sn-devel-104

Implement TestCase.assertIn for older versions of Python.

Change-Id: I17d855166b439c0bd9344a17a454ea5bc6e057aa
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Implement assertIsNone for Python < 2.7.

Change-Id: I3937acb16ca0c5430b70f0af305997878da53c37
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Handle skips when running on python2.6.

Change-Id: I8b0a15760a72f41800d23150232c2b0e59e32c32
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>