Matthew Newton [Sat, 21 Feb 2015 22:30:11 +0000 (22:30 +0000)]
Add context versions of wbclient functions
To make the libwbclient library thread-safe, all functions
that call through to wb_common winbindd_request_response need
to have context that they can use. This commit adds all the
necessary functions.
Signed-off-by: Matthew Newton <matthew-git@newtoncomputing.co.uk>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Matthew Newton [Sat, 24 Jan 2015 00:30:00 +0000 (00:30 +0000)]
Add wbcContext to wbcRequestResponse
To enable libwbclient to pass winbindd context through
to the winbind client library in wb_common.
Signed-off-by: Matthew Newton <matthew-git@newtoncomputing.co.uk>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Matthew Newton [Sat, 21 Feb 2015 00:19:32 +0000 (00:19 +0000)]
Add wbcContext struct, create and free functions
The basic context structure and functions for libwbclient so that
libwbclient can be made thread-safe.
Signed-off-by: Matthew Newton <matthew-git@newtoncomputing.co.uk>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Matthew Newton [Fri, 23 Jan 2015 23:58:53 +0000 (23:58 +0000)]
Use global context for winbindd_request_response
Updating API call in libwbclient, wbinfo, ntlm_auth and
winbind_nss_* as per previous commit to wb_common.c.
Signed-off-by: Matthew Newton <matthew-git@newtoncomputing.co.uk>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Matthew Newton [Fri, 23 Jan 2015 22:35:50 +0000 (22:35 +0000)]
Make winbind client library thread-safe by adding context
Rather than keep state in global variables, store the current
context such as the winbind file descriptor in a struct that is
passed in. This makes the winbind client library thread-safe.
Signed-off-by: Matthew Newton <matthew-git@newtoncomputing.co.uk>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
David Disseldorp [Mon, 9 Mar 2015 15:21:24 +0000 (16:21 +0100)]
torture/ioctl: add range overflow QAR test
Issue a QAR request with an offset and length that generate an integer
(uint64_t) overflow when summed together. This should result in an
NT_STATUS_INVALID_PARAMETER response, as confirmed against Windows
Server 2012 & 2008.
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Mar 10 00:02:18 CET 2015 on sn-devel-104
David Disseldorp [Thu, 26 Feb 2015 00:13:58 +0000 (01:13 +0100)]
torture/ioctl: add multi-range QAR test
Write 10 x 64K ranges, with 64K holes punched in between. Afterwards,
check that all ranges are present in the QAR response.
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
David Disseldorp [Wed, 25 Feb 2015 18:23:49 +0000 (19:23 +0100)]
torture/ioctl: add QAR off-by-one bug paranoia test
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
David Disseldorp [Wed, 25 Feb 2015 14:29:56 +0000 (15:29 +0100)]
torture/ioctl: test sparse file operation locking
An exclusively locked file can still be marked sparse. QAR requests
covering the locked-range should also succed. ZERO_DATA requests are
blocked.
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
David Disseldorp [Mon, 23 Feb 2015 19:27:37 +0000 (20:27 +0100)]
s3/smbd: fix FSCTL_SET_SPARSE permission checks
On Windows servers (tested against Windows Server 2008 & 2012) the
FSCTL_SET_SPARSE ioctl is processed if FILE_WRITE_DATA,
FILE_WRITE_ATTRIBUTES _or_ SEC_FILE_APPEND_DATA permissions are granted
on the open file-handle.
Fix Samba such that it matches this behaviour, rather than only checking
for FILE_WRITE_DATA or FILE_WRITE_ATTRIBUTES.
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
David Disseldorp [Mon, 23 Feb 2015 19:24:11 +0000 (20:24 +0100)]
torture/ioctl: add ioctl_sparse_perms test
This test confirms that correct FSCTL_SET_SPARSE permission checks are
in place on the server.
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
David Disseldorp [Mon, 23 Feb 2015 19:03:19 +0000 (20:03 +0100)]
torture/ioctl: rework and reduce pattern helper IO sizes
check_pattern() currently attempts to read all data in one go. Fix it to
use a 64K maximum IO size so that it works against Windows Server 2008.
Additionally, rework write_pattern() so that it only allocates a buffer
for the largest IO size (now 64K), rather than for the full write
length.
Finally, assert that callers are correctly performing pattern IO in
8-byte increments - copy_chunk_tiny was not, so fix it.
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
David Disseldorp [Mon, 23 Feb 2015 10:29:52 +0000 (11:29 +0100)]
torture/ioctl: add sparse_punch_invalid test
Attempt to extend a file using ZERO_DATA. The operation should succeed,
but the file should not be extended, as specified in MS-FSCC <58>
Section 2.3.65:
This FSCTL sets the range of bytes to zero (0) without extending the
file size.
Also test zero length and invalid BFZ requests.
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
David Disseldorp [Mon, 23 Feb 2015 10:28:17 +0000 (11:28 +0100)]
torture/ioctl: remove FS specific sparse copy-chunk expectations
NTFS deallocates an entire file when a sparse zero-data request spans
the full length. Other filesystems (e.g. EXT4 and Btrfs) do not.
vfs_btrfs is additionally capable of preserving sparse regions for
copy-chunk, using the BTRFS_IOC_CLONE_RANGE ioctl. This should not be
treated as a failure.
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
David Disseldorp [Mon, 23 Feb 2015 10:09:02 +0000 (11:09 +0100)]
torture/ioctl: remove FS specific sparse punch check
Samba uses PUNCH_HOLE to zero a range, and subsequently uses fallocate()
to allocate the punched range if the file is marked non-sparse and
"strict allocate" is enabled.
In both cases, the zeroed range will not be detected by SEEK_DATA, so
the range won't be present in QAR responses until the file is marked
non-sparse again.
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
David Disseldorp [Thu, 19 Feb 2015 15:09:02 +0000 (16:09 +0100)]
torture/ioctl: remove 64K chunk size assumptions
These tests assumed that 4K chunks remain allocated following write at
a subsequent offset. This is not the case for other filesystems (E.g.
XFS, Btrfs, Etc.), which may deallocate the chunk.
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
David Disseldorp [Thu, 19 Feb 2015 15:46:56 +0000 (16:46 +0100)]
s3/statvfs: expose FILE_SUPPORTS_SPARSE_FILES capability
Samba now supports:
- FSCTL_SET_SPARSE
- FSCTL_SET_ZERO_DATA, via FALLOC_FL_PUNCH_HOLE
- FSCTL_QUERY_ALLOCATED_RANGES, via SEEK_DATA/SEEK_HOLE
As such, flag support for sparse files, via the
FILE_SUPPORTS_SPARSE_FILES capability flag if FALLOC_FL_PUNCH_HOLE and
SEEK_DATA/SEEK_HOLE are present at configure time.
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
David Disseldorp [Thu, 19 Feb 2015 15:36:05 +0000 (16:36 +0100)]
smbd/ioctl: add FSCTL_QUERY_ALLOCATED_RANGES support
This change implements support for FSCTL_QUERY_ALLOCATED_RANGES using
the SEEK_HOLE/SEEK_DATA functionality of lseek().
Files marked non-sparse are always reported by the ioctl as fully
allocated, regardless of any potential "strict allocate = no" savings.
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
David Disseldorp [Thu, 19 Feb 2015 14:53:56 +0000 (15:53 +0100)]
build: check for SEEK_HOLE and SEEK_DATA support
SEEK_HOLE and SEEK_DATA will be used in the implementation of
FSCTL_QUERY_ALLOCATED_RANGES support.
"SEEK_DATA and SEEK_HOLE are nonstandard extensions also present
in Solaris, FreeBSD, and DragonFly BSD; they are proposed for
inclusion in the next POSIX revision (Issue 8)."
With Linux they are supported on:
- Btrfs (since Linux 3.1)
- OCFS (since Linux 3.2)
- XFS (since Linux 3.5)
- ext4 (since Linux 3.8)
- tmpfs (since Linux 3.8)
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
David Disseldorp [Thu, 12 Feb 2015 09:58:20 +0000 (10:58 +0100)]
idl/ioctl: change QAR response array to a DATA_BLOB
[MS-FSCC] specifies:
The number of FILE_ALLOCATED_RANGE_BUFFER elements returned is
computed by dividing the size of the returned output buffer (from
either SMB or SMB2, the lower-layer protocol that carries the FSCTL)
by the size of the FILE_ALLOCATED_RANGE_BUFFER element.
Ideally, this requirement could be defined in idl with the following:
[flag(NDR_REMAINING)] file_alloced_range_buf array[];
However, this is not currently supported by PIDL, so just use an opaque
data blob for now.
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
David Disseldorp [Tue, 10 Feb 2015 23:24:16 +0000 (00:24 +0100)]
smbd/ioctl: add FSCTL_SET_ZERO_DATA support
FSCTL_SET_ZERO_DATA can be used in two ways.
- When requested against a file marked as sparse, it provides a
mechanism for requesting that the server deallocate the underlying
disk space for the corresponding zeroed range.
- When requested against a non-sparse file, it indicates that the server
should allocate and zero the corresponding range.
Both use cases can be handled in Samba using fallocate(). The Linux
specific FALLOC_FL_PUNCH_HOLE flag can be used to deallocate the
underlying disk space. After doing so, a normal fallocate() call can
be used to ensure that the zeroed range is allocated on non-sparse
files.
FSCTL_SET_ZERO_DATA requests must not result in a change to the file
size. The FSCTL_SET_ZERO_DATA handler always calls fallocate() with the
KEEP_SIZE flag set, ensuring that Samba meets this requirement.
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
David Disseldorp [Tue, 10 Feb 2015 13:32:07 +0000 (14:32 +0100)]
system: add hole punch support to sys_fallocate()
If Samba is configured with FALLOC_FL_PUNCH_HOLE support, then allow
sys_fallocate() to propogate the flag to syscall invocation.
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
David Disseldorp [Mon, 9 Feb 2015 18:39:06 +0000 (19:39 +0100)]
build: check for fallocate hole-punch support
Add a configure time check for the FALLOC_FL_PUNCH_HOLE Linux specific
fallocate() flag. It's been around since 2.6.38.
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
David Disseldorp [Mon, 9 Feb 2015 17:21:59 +0000 (18:21 +0100)]
s3/vfs: change fallocate mode flags from enum->uint32_t
The Linux fallocate syscall offers a mode parameter which can take the
following flags:
FALLOC_FL_KEEP_SIZE
FALLOC_FL_PUNCH_HOLE (since 2.6.38)
FALLOC_FL_COLLAPSE_RANGE (since 3.15)
FALLOC_FL_ZERO_RANGE (since 3.14)
The flags are not exclusive, e.g. FALLOC_FL_PUNCH_HOLE must be specified
alongside FALLOC_FL_KEEP_SIZE.
Samba currently takes a vfs_fallocate_mode enum parameter for the VFS
fallocate hook, taking either an EXTEND_SIZE or KEEP_SIZE value. This
commit changes the fallocate hook such that it accepts a uint32_t flags
parameter, in preparation for PUNCH_HOLE and ZERO_RANGE support.
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
David Disseldorp [Mon, 9 Feb 2015 14:51:28 +0000 (15:51 +0100)]
lib/system: remove useless HAVE_LINUX_FALLOCATE64 logic
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Michael Adam [Mon, 9 Mar 2015 14:15:37 +0000 (15:15 +0100)]
s3-winbind: Fix chached user group lookup of trusted domains.
If a user group lookup has aleady been done before with a machine
account we did always return the incomplete information from the cache.
This patch makes sure we return the correct group information from the
netsamlogon cache.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11143
Pair-Programmed-With: Andreas Schneider <asn@samba.org>
Signed-off-by: Michael Adam <obnox@samba.org>
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Mon Mar 9 19:23:25 CET 2015 on sn-devel-104
Andrew Bartlett [Sun, 8 Mar 2015 22:12:01 +0000 (11:12 +1300)]
torture-krb5: Add an initial test for s4u2self behaviour
This test only checks for S4U2Self of the same user, but shows
that a user account is not a valid service for this purpose.
Andrew Bartlett
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Mon Mar 9 12:10:09 CET 2015 on sn-devel-104
Andrew Bartlett [Mon, 9 Mar 2015 03:00:56 +0000 (16:00 +1300)]
kdc: Fix S4U2Self handling with KRB5_NT_ENTERPRISE_PRINCIPAL containing a UPN
This is now handled properly by samba_kdc_lookup_server() and this wrapper actually
breaks things.
Andrew Bartlett
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Stefan Metzmacher [Mon, 9 Mar 2015 08:07:24 +0000 (09:07 +0100)]
talloc: version 2.1.2
Changes:
- Allow destructors to reparent the object
- Allow destructors to remove itself
- Build improvements
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Günther Deschner <gd@samba.org>
Stefan Metzmacher [Tue, 27 Jan 2015 12:07:34 +0000 (13:07 +0100)]
talloc: fix _talloc_total_limit_size prototype
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Jeremy Allison [Wed, 4 Mar 2015 01:12:32 +0000 (17:12 -0800)]
lib: talloc: Test suite for the new destructor reparent logic.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ira Cooper <ira@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sun Mar 8 20:52:43 CET 2015 on sn-devel-104
Jeremy Allison [Wed, 4 Mar 2015 01:02:47 +0000 (17:02 -0800)]
lib: talloc: Allow destructors to reparent the object they're called on.
If a destructor returns failure (-1) when freeing a child, talloc
must then reparent the child.
Firstly it tries the owner of any reference, next the parent of the
current object calling _talloc_free_children_internal(), and finally
the null context in the last resort.
If a destructor reparented its own object, which can be a very
desirable thing to do (a destructor can make a decision it isn't
time to die yet, and as the parent may be going away it might
want to move itself to longer-term storage) then this new parent
gets overwritten by the existing reparenting logic.
This patch checks when freeing a child if it already reparented
itself, and if it did doesn't then overwrite the new parent.
Makes destructors more flexible.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ira Cooper <ira@samba.org>
Jeremy Allison [Thu, 5 Mar 2015 20:48:47 +0000 (12:48 -0800)]
lib: talloc: Fix bug when calling a destructor.
If the destructor itself calls talloc_set_destructor()
and returns -1, the new destructor set is overwritten
by talloc.
Dectect that and leave the new destructor in place.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ira Cooper <ira@samba.org>
Richard Sharpe [Sat, 7 Mar 2015 15:46:31 +0000 (07:46 -0800)]
Add the definition of FSCTL_SET_INTEGRITY_INFORMATION found from a capture and the Web.
Signed-off-by: Richard Sharpe <rsharpe@samba.org>
Reviewed-by: Ira Cooper <ira@samba.org>
Autobuild-User(master): Ira Cooper <ira@samba.org>
Autobuild-Date(master): Sun Mar 8 00:43:08 CET 2015 on sn-devel-104
David Disseldorp [Thu, 5 Mar 2015 00:41:47 +0000 (01:41 +0100)]
torture/fsrvp: remove verification trailer magic field
During initial wire trace analysis, the DCE/RPC PDU verification trailer
was incorrectly identified and tagged in IDL as an FSRVP "magic" blob.
This change removes the incorrectly tagged FSRVP request fields and
corresponding test code - with
1e1b7b1021b16e3ab61c2fca8328c94e60a2c99c
verification trailer parsing is now tested separately.
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Mar 7 20:01:20 CET 2015 on sn-devel-104
Volker Lendecke [Thu, 5 Mar 2015 19:59:16 +0000 (20:59 +0100)]
winbind: Make wb_sids2xids_recv work on an array
The trigger for this is that Coverity got confused by the dual use of &xid
as an array with the implicit length equality between wb_sids2xids_send
and the array passed in to wb_sids2xids_recv for the result.
I don't want to start doing things just for the Coverity scan, but this
makes the code clearer to me by removing this implicit expected array
length equality.
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: David Disseldorp <ddiss@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Sat Mar 7 15:28:59 CET 2015 on sn-devel-104
Volker Lendecke [Fri, 6 Mar 2015 09:59:25 +0000 (09:59 +0000)]
vfs_fruit: Fix CID
1273290 Uninitialized scalar variable
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Fri Mar 6 23:56:36 CET 2015 on sn-devel-104
Andrew Bartlett [Mon, 2 Mar 2015 00:22:37 +0000 (13:22 +1300)]
provision: Give a more helpful message when find_provision_key_parameters() fails
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Fri Mar 6 20:11:52 CET 2015 on sn-devel-104
Andrew Bartlett [Fri, 27 Feb 2015 02:56:22 +0000 (15:56 +1300)]
selftest: Improve renamedcs test
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Andrew Bartlett [Wed, 25 Feb 2015 23:24:21 +0000 (12:24 +1300)]
s4-lib/cmdline: Fix help for -P / --machine-pass: this no longer implies -k
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Andrew Bartlett [Wed, 25 Feb 2015 23:23:55 +0000 (12:23 +1300)]
samba-tool: Add -P to options.CredentialsOptions
This matches our other binaries, and allows samba-tool commands to run with the machine account.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Volker Lendecke [Thu, 5 Mar 2015 20:17:31 +0000 (21:17 +0100)]
heimdal: Fix CID
1273430 Double free
I think Coverity is right here: Before the preceding call to
krb5_make_principal we already krb5_free_principal(ctx, tmp_creds.server)
without wiping out tmp_creds.server. The call to krb5_make_principal only
stores something fresh when it also returns 0 a.k.a. success.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: David Disseldorp <ddiss@samba.org>
Autobuild-User(master): David Disseldorp <ddiss@samba.org>
Autobuild-Date(master): Fri Mar 6 17:38:09 CET 2015 on sn-devel-104
Stefan Metzmacher [Mon, 8 Dec 2014 09:30:56 +0000 (10:30 +0100)]
script/autobuild.py: build the samba target with --with-profiling-data
In future we may get also runtime tests for profiling...
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Autobuild-User(master): Michael Adam <obnox@samba.org>
Autobuild-Date(master): Mon Dec 15 16:20:14 CET 2014 on sn-devel-104
(cherry picked from commit
4958fcdfa30fd9d8dc51ceafaab35721e61e72c7)
Stefan Metzmacher [Fri, 14 Nov 2014 11:52:33 +0000 (12:52 +0100)]
s3:smbprofile: profile the system and user space cpu time
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Volker Lendecke [Mon, 29 Sep 2014 16:08:17 +0000 (16:08 +0000)]
s3:smbprofile: Replace sysv shmem with tdb
What?
This patch gets rid of the central shared memory segment referenced by
"profile_p". Instead, every smbd gets a static profile_area where it collects
profiling data. Once a second, every smbd writes this profiling data into a
record of its own in a "smbprofile.tdb". smbstatus -P does a tdb_traverse on this
database and sums up what it finds.
Why?
At least in my perception sysv IPC has not the best reputation on earth. The
code before this patch uses shmat(). Samba ages ago has developed a good
abstraction of shared memory: It's called tdb.
The main reason why I started this is that I have a request to become
more flexible with profiling data. Samba should be able to collect data
per share or per user, something which is almost impossible to do with
a fixed structure. My idea is to for example install a profile area per
share and every second marshall this into one tdb record indexed by share
name. smbstatus -P would then also collect the data and either aggregate
them or put them into individual per-share statistics. This flexibility
in the data model is not really possible with one fixed structure.
But isn't it slow?
Well, I don't think so. I can't really prove it, but I do believe that on large
boxes atomically incrementing a shared memory value for every SMB does show up
due to NUMA effects. With this patch the hot code path is completely
process-local. Once a second every smbd writes into a central tdb, this of
course does atomic operations. But it's once a second, not on every SMB2 read.
There's two places where I would like to improve things: With the current code
all smbds wake up once a second. With 10,000 potentially idle smbds this will
become noticable. That's why the current only starts the timer when something has
changed.
The second place is the tdb traverse: Right now traverse is blocking in the
sense that when it has to switch hash chains it will block. With mutexes, this
means a syscall. I have a traverse light in mind that works as follows: It
assumes a locked hash chain and then walks the complete chain in one run
without unlocking in between. This way the caller can do nonblocking locks in
the first round and only do blocking locks in a second round. Also, a lot of
syscall overhead will vanish. This way smbstatus -P will have almost zero
impact on normal operations.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Stefan Metzmacher [Fri, 28 Nov 2014 08:33:13 +0000 (09:33 +0100)]
s3:smbprofile: specify SMBPROFILE_STATS_SECTION_START() with name vs. display[name]
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Jelmer Vernooij [Thu, 5 Feb 2015 21:04:44 +0000 (22:04 +0100)]
Implement TestCase.assertIsNotNone for python < 2.7.
Change-Id: Ieaefdc77495e27bad791075d985a70908e9be1ad
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Mar 6 07:11:43 CET 2015 on sn-devel-104
Jelmer Vernooij [Thu, 5 Feb 2015 18:57:26 +0000 (19:57 +0100)]
Implement TestCase.assertIn for older versions of Python.
Change-Id: I17d855166b439c0bd9344a17a454ea5bc6e057aa
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Thu, 5 Feb 2015 09:25:53 +0000 (10:25 +0100)]
Implement assertIsNone for Python < 2.7.
Change-Id: I3937acb16ca0c5430b70f0af305997878da53c37
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Wed, 4 Feb 2015 15:40:29 +0000 (16:40 +0100)]
Handle skips when running on python2.6.
Change-Id: I8b0a15760a72f41800d23150232c2b0e59e32c32
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Sun, 1 Feb 2015 22:20:42 +0000 (23:20 +0100)]
Remove another call to addUnexpectedSuccess with too many arguments.
Change-Id: I5f1917e74ffd57b5ae228231c27fbdf70de2992f
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Fri, 30 Jan 2015 01:16:05 +0000 (02:16 +0100)]
Run cleanup after tearDown, for consistency with Python >= 2.7.
Change-Id: Ic3ce975e3b2e4b30e07643efb4496ebf36a93284
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Fri, 30 Jan 2015 01:07:17 +0000 (02:07 +0100)]
TestCase.addUnexpectedSuccess doesn't take an error.
Change-Id: I90f7dd225d9ed3cbc515292de9a37a816ac0639f
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Fri, 30 Jan 2015 01:06:33 +0000 (02:06 +0100)]
Use samba TestCase so we get all compatibility functions on Python < 2.7.
Change-Id: Iba87e3c8fa9331c4d5438ab60a8385379da634d7
Signed-Off-By: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Wed, 28 Jan 2015 22:17:13 +0000 (23:17 +0100)]
Provide TestCase.assertIsInstance for python < 2.7.
Change-Id: Id6d3c8a7dc56cb560eccc3db897a83c638dec7a6
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Wed, 28 Jan 2015 21:17:41 +0000 (22:17 +0100)]
Use Samba TestCase class, as the python 2.6 one doesn't have assertIs, assertIsInstance or addCleanup.
Change-Id: I3daeffade0dac9795f61f91ee0da35fee0143a38
Signed-Off-By: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Tue, 27 Jan 2015 02:44:10 +0000 (03:44 +0100)]
Add replacement addCleanup.
Change-Id: Ie85756effde344fc4cc9b693c4a28611ea091677
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Tue, 27 Jan 2015 02:40:34 +0000 (03:40 +0100)]
Add custom implementations of TestCase.assertIs and TestCase.assertIsNot, for Python2.6.
Change-Id: I3b806abdaf9540b7c39c961c179c2d2b15d327fe
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Mon, 15 Dec 2014 17:35:24 +0000 (17:35 +0000)]
Fix use of TestCase.skipTest on python2.6 now that we no longer use testtools.
Change-Id: I630e4073bf1553dfc77e9fe7e843ee8b71907683
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Mon, 15 Dec 2014 17:33:49 +0000 (17:33 +0000)]
Build python-ntdb bindings if ntdb was found but python-ntdb was not.
Change-Id: I54c248effa63fe65e2bef0d41535d8eec0ef28c0
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Mon, 15 Dec 2014 14:18:30 +0000 (14:18 +0000)]
Drop support for failfast mode, rather than adding support for it everywhere.
Change-Id: I4d6070a0e3b89d5e390f84754dddba9ec17ddf21
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Mon, 15 Dec 2014 00:51:32 +0000 (00:51 +0000)]
Fix use of iso8601.Utc.
Change-Id: Id592571a173eff36e2fb9b395f61a6fb53d580e6
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Mon, 15 Dec 2014 00:51:16 +0000 (00:51 +0000)]
Set failfast property for test reporters that need it.
Change-Id: Ibd632b9f569c23e60bcd13bcff805e367dd2e71c
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Sun, 14 Dec 2014 20:31:45 +0000 (20:31 +0000)]
Remove 'external' python module support code - use the third_party directory instead.
Change-Id: I2f5053bc5a42d3dfe71f5bd027eb6ead7d1b9752
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Sun, 14 Dec 2014 20:30:52 +0000 (20:30 +0000)]
Remove bundled but unused mimeparse.
Change-Id: I245607c0d27dbeae638f2f50c3b4d560eb9b70f8
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Sun, 14 Dec 2014 20:29:49 +0000 (20:29 +0000)]
Remove unused bundled python-extras module.
Change-Id: I6df0561ff4059c827d545d60a0865e17e9fefcf0
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Sun, 14 Dec 2014 20:29:12 +0000 (20:29 +0000)]
Remove bundled testtools.
Change-Id: Ic6ddb352e403c9591cbe4ad3fd36758ffcc2ddb9
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Sun, 14 Dec 2014 20:28:25 +0000 (20:28 +0000)]
Remove bundled subunit.
Change-Id: I18e19c3817de6f97fe2a9fb1b7faa4f8a0d27649
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Sun, 14 Dec 2014 20:27:42 +0000 (20:27 +0000)]
Remove unnecessary python path updates for bundled subunit/testtools.
Change-Id: Idb40fcb564455f16608ea991b086e41e22ae51e3
Signed-Off-By: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Sun, 14 Dec 2014 20:27:03 +0000 (20:27 +0000)]
Support using third party iso8601 module if system doesn't provide one.
Change-Id: I5d035738d244d66d33788636c8ee8b322c227a0e
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Sun, 14 Dec 2014 20:26:19 +0000 (20:26 +0000)]
Bundle pyiso8601 for iso8601 time/date manipulation.
(corrected lib/update-external.sh by Andrew Bartlett)
Change-Id: I0f992b949b1717635ff26fa0db6073675cce4ad7
Signed-Off-By: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Jelmer Vernooij [Sun, 14 Dec 2014 20:17:03 +0000 (20:17 +0000)]
Import UTC definition from utc8601 module.
Change-Id: I3ccd81090c4721b161aff272100aa71bc2f17055
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Sun, 14 Dec 2014 20:10:20 +0000 (20:10 +0000)]
subunithelper: Fix progress support.
Change-Id: I5dd2ca0f3bc02821d5c9e1dc878bba577667d162
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Sun, 14 Dec 2014 20:03:28 +0000 (20:03 +0000)]
Rename TestSkipped to Skiptest, consistent with Python 2.7.
Change-Id: I023df54363328333f1cb6c3ae3c1a406befa8f7b
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Sun, 14 Dec 2014 19:59:13 +0000 (19:59 +0000)]
Avoid importing TestCase and TestSkipped from testtools.
Change-Id: I34488ddf253decd336a67a8634e7039096bdd160
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Sun, 14 Dec 2014 19:55:55 +0000 (19:55 +0000)]
Use iso8601 from the system, rather than the one bundled with subunit.
Change-Id: I681bf79eb9ebe45b6972b3783c8e799eb612400b
Signed-Off-By: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Sun, 14 Dec 2014 19:38:48 +0000 (19:38 +0000)]
format-subunit: Remove import of unnecessary third party modules testtools and subunit.
Change-Id: I3403ceacf8bbdf075c1c540081f7c3e82f4751bc
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Sun, 14 Dec 2014 19:37:46 +0000 (19:37 +0000)]
filter-subunit: Remove import of unnecessary third party modules testtools and subunit.
Change-Id: I6ed0c560c18b4001c7adb9af6f67535640a1b374
Signed-Off-By: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Sun, 14 Dec 2014 19:36:32 +0000 (19:36 +0000)]
tests/sam: Remove unnecessary calls for third party module imports.
Change-Id: Iaa1af59005eaee7ea79f3260b250a2c948e07532
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Sun, 14 Dec 2014 19:34:14 +0000 (19:34 +0000)]
show_test_time: Use system subunit.
Change-Id: I7d799b5f9d6066d988aa2e101e0fe7b6efe74aea
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Sun, 14 Dec 2014 19:32:23 +0000 (19:32 +0000)]
test_samba3dump: Use Samba subunit emitter.
Change-Id: Ie9a115d131624bfc68e6f40822acade70d145735
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Sun, 14 Dec 2014 19:31:48 +0000 (19:31 +0000)]
test_duplicate_symbol: Use Samba subunit emitter.
Change-Id: I280495706ff460674244183103306ca717c419a8
Signed-Off-By: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Sun, 14 Dec 2014 19:25:55 +0000 (19:25 +0000)]
Set default testRunner rather than requiring the user pass it in.
Change-Id: I8b5a5925030049975a83b090e5c7b76d5245c07d
Signed-Off-By: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Sun, 14 Dec 2014 19:25:12 +0000 (19:25 +0000)]
Add basic tap2subunit converter, rather than relying on the one from subunit-tools.
Change-Id: I39ec5ec68c7c3c9d329d8f1a8ce01445b85c7ab8
Signed-Off-By: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Sat, 13 Dec 2014 22:11:06 +0000 (22:11 +0000)]
Inline outputting of subunit in libtorture.
Change-Id: I2c7045c530183a6961cb253540579312c2767330
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Sat, 13 Dec 2014 21:37:51 +0000 (21:37 +0000)]
Fix handling of unexpected failures in subunithelper.
Change-Id: I0da9fcec4a54c43c171b76bb9015ea84389c9bc6
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Thu, 11 Dec 2014 02:30:49 +0000 (02:30 +0000)]
Use samba.subunit in selftest.subunithelper, except for iso8601.
Change-Id: Iec43e0368db91dbe981b39ac8388b50c1eafee0c
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Thu, 11 Dec 2014 02:26:54 +0000 (02:26 +0000)]
Add RemoteTestCase and RemoteError to samba.subunit.
Change-Id: Ib3946cf4eae69f53270a299660f6029290d3791a
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Thu, 11 Dec 2014 02:10:53 +0000 (02:10 +0000)]
Use samba.subunit in selftest/selftest.py.
Change-Id: I804c16cdfcfddc5643a5af158ee62c4ddf478979
Signed-Off-By: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Thu, 11 Dec 2014 01:55:16 +0000 (01:55 +0000)]
Use Samba-only subunit module in selftest/tests/.
Change-Id: I48c61f975c1fa49f6e244ad39dd720fe507db45b
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Thu, 11 Dec 2014 01:33:00 +0000 (01:33 +0000)]
Fix copyright headers for python/samba/subunit.
Change-Id: I5b554051f23f31d20b4f4325debcd6717ba8369e
Signed-Off-By: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Thu, 11 Dec 2014 01:16:38 +0000 (01:16 +0000)]
Use samba.subunit.run to run subunit tests.
Change-Id: Id9bdd33b7ac5e4db40ab8bef4dbe0f04c36d8da7
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Thu, 11 Dec 2014 01:14:08 +0000 (01:14 +0000)]
subunitrun: Update instructions for running subunit tests.
Change-Id: Icaf472198e93e283db2ae6ed99fd7ceae037af87
Signed-Off-By: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Thu, 11 Dec 2014 01:12:11 +0000 (01:12 +0000)]
subunitrun: Use new samba.subunit.run module.
Change-Id: I4a4d2237ec9ef090bf95a0ea57285610047451e5
Signed-Off-By: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Thu, 11 Dec 2014 01:11:41 +0000 (01:11 +0000)]
selftest/tests/*.py: remove use of testtools.
Change-Id: Ia692c6b3037b7d867310c3793980f9f953d31680
Signed-Off-By: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Jelmer Vernooij [Thu, 11 Dec 2014 01:07:36 +0000 (01:07 +0000)]
Add simple subunit runner outputting subunit v1.
This is a short module (< 1k lines) that removes the need to
depend on subunit, testtools, extras and mimeparse. It is
based on an extract from testtools and subunit.
Change-Id: I0a4f3060b25f7bde602a07ed6bef71c8196fca64
Signed-Off-By: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Martin Schwenke [Fri, 6 Mar 2015 00:36:18 +0000 (11:36 +1100)]
ctdb-tests: Check for readable, not executable, script
Scripts in eventscript unit tests are run under an explicitly
specified shell so they do not need to be executable. Checking that
the script is executable breaks on scripts that are installed without
the execute bit set, such as disabled eventscripts.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Fri Mar 6 04:40:07 CET 2015 on sn-devel-104
Volker Lendecke [Thu, 5 Mar 2015 10:12:38 +0000 (11:12 +0100)]
perfcount: Fix CID
1035494 Out-of-bounds read
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ira Cooper <ira@samba.org>
Autobuild-User(master): Ira Cooper <ira@samba.org>
Autobuild-Date(master): Thu Mar 5 18:28:44 CET 2015 on sn-devel-104
Volker Lendecke [Thu, 5 Mar 2015 10:11:59 +0000 (11:11 +0100)]
perfcount: Fix CID
1035493 Out-of-bounds read
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ira Cooper <ira@samba.org>
Volker Lendecke [Thu, 5 Mar 2015 10:10:35 +0000 (11:10 +0100)]
perfcount: Fix CID
1035492 Out-of-bounds read
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ira Cooper <ira@samba.org>
git.samba.org / samba.git / log