Gary Lockyer [Mon, 27 Aug 2018 19:46:59 +0000 (07:46 +1200)]
kdc: Update debug calls
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Gary Lockyer [Wed, 22 Aug 2018 23:26:40 +0000 (11:26 +1200)]
source4 smbd kdc: allow the kdc to run in prefork
Modify the kdc to allow it to run in the prefork process model. The
task_init function has been split up and code moved into the post_fork
function.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Gary Lockyer [Wed, 22 Aug 2018 21:35:52 +0000 (09:35 +1200)]
source4 smdb: Add a post fork hook to the service API
Add a post fork hook to the service API this will be called:
- standard process model
immediately after the task_init.
- single process model
immediately after the task_init
- prefork process model, inhibit_pre_fork = true
immediately after the task_init
- prefork process model, inhibit_pre_fork = false
after each service worker has forked. It is not run on the service
master process.
The post fork hook is not called in the standard model if a new process
is forked on a new connection. It is instead called immediately after
the task_init.
The task_init hook has been changed to return an error code. This ensures
the post_fork code is only run if the task_init code completed successfully.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Gary Lockyer [Wed, 22 Aug 2018 21:29:56 +0000 (09:29 +1200)]
source4 smbd: Make the service_details structure constant.
Make the service_details structure a static const.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Gary Lockyer [Mon, 27 Aug 2018 04:43:00 +0000 (16:43 +1200)]
windbindd: reword error message
Reword the asprintf() out of memory message to make it clear where the
issue is.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Tim Beale [Wed, 24 Oct 2018 02:40:52 +0000 (15:40 +1300)]
replmd: Single DB operation per source object during link processing
Move the source object checks and DB modify operation up a level, so we
only do them once per source object rather than once per link.
This allows LMDB joins to succeed with ~15,000 members in a group.
Previously LMDB would fail with the error:
Failed to apply linked attribute change '(-30792) - MDB_MAP_FULL:
Environment mapsize limit reached at ../lib/ldb/ldb_mdb/ldb_mdb.c:203'
Rewriting the same object ~15000 times seemed to completely fill up
the LMDB 8Gb buffer. Presumably this was because LMDB is 'copy on
write', so it was storing ~15,000 copies of the same object. Strangely,
we don't see this problem writing the backlinks (which this patch won't
have helped with at all, because that's modifying the target object).
Note uSNChanged was only being added to the msg once, so the code has
been modified to replace the usnChanged each time (i.e. remove it and
re-add it).
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Nov 1 23:48:21 CET 2018 on sn-devel-144
Tim Beale [Wed, 24 Oct 2018 00:46:06 +0000 (13:46 +1300)]
replmd: Move talloc context one level up
Eventually we want to combine multiple link attributes, that apply to the
same source object, into a single DB 'modify' operation. This will mean
the memory context needs to hang around until we have performed the DB
operation (instead of allocating a temporary context for each link).
This patch moves the talloc context one level up, so a temp context gets
allocated for each link *group*, instead of for each link *attribute*.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Tim Beale [Wed, 24 Oct 2018 00:25:50 +0000 (13:25 +1300)]
replmd: Split apart source/target checks for links
We've grouped the linked attributes by source-object. Next, we want to
avoid duplicated processing for the source object, i.e. we only need to
check the source object exists once, not once per link.
Before we can do this, we need to tease apart
replmd_extract_la_entry_details(), which is doing both source and target
object processing. Split out extracting the target DSDB-DN so that it's
done separately.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Tim Beale [Tue, 23 Oct 2018 23:30:17 +0000 (12:30 +1300)]
replmd: Group together link attribute processing by source object
Instead of processing each link attribute one at a time, we want to
group them together by source object. This will mean we only have to
look-up the source object once, and only perform one DB 'modify'
operation. With groups with 1000s of members, this will help improve
performance.
This patch takes the first step of group together the links by
source-object. A new 'la_group' struct is added to help track what links
belong to the same source object. The la_list essentially becomes a
'list of lists' now.
Note that only related links *in the same chunk* are only grouped together.
While it is trivial to groups together links that span different
replication chunks, this would be a fairly insignificant efficiency gain,
but seems to have a fairly detrimental memory overhead, once you get
into groups with 10,000+ members.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Tim Beale [Tue, 23 Oct 2018 22:17:38 +0000 (11:17 +1300)]
replmd: Add more debug for replicating links
During a join of a large DB, processing the linked attributes can take a
long time. The join hangs in 'Committing SAM database' for many minutes
with no indication of whether it's making progress or not.
This patch adds some extra debug to show how far through processing the
linked attributes we are, when there are many thousands of links.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Volker Lendecke [Thu, 1 Nov 2018 11:09:14 +0000 (12:09 +0100)]
torture: Fix the clang build
It's used uninitialized if an early torture_assert fails
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Thu Nov 1 17:34:31 CET 2018 on sn-devel-144
Tim Beale [Thu, 1 Nov 2018 03:43:58 +0000 (16:43 +1300)]
selftest: Remove unnecessary code for backup testenvs
setup_namespaces() already gets done for the backupfromdc's domain, so
this step is unnecessary for the restoredc and offlinebackupdc testenvs
(which are based off the backupfromdc's database).
The setup_namespaces() step is still necessary for the renamedc/labdc,
as these don't have the UPN/SPN suffixes for the new realm yet.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Nov 1 13:06:01 CET 2018 on sn-devel-144
Tim Beale [Tue, 30 Oct 2018 01:11:46 +0000 (14:11 +1300)]
dnsupdate: Skip kerberos step if use-file specified
If there's a problem in get_credentials() (getting the machine account
Kerberos credentials), then we fallback to use_samba_tool (essentially
ignoring use-file). However, there's no need to do this, as use-file
shouldn't require Kerberos credentials.
This was making bootstrapping issues starting a testenv harder to debug.
Obviously, Kerberos is dependent on DNS functioning correctly, but
running dnsupdate was also dependent on having a working Kerberos KDC.
In my case, the testenv had a bad krb5.conf file, but the problem
appeared as resolv-wrapper errors (due to a missing RESOLV_WRAPPER_HOSTS
file, which should've been generated by dnsupdate).
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Pair-Programmed-With: Garming Sam <garming@catalyst.net.nz>
Tim Beale [Tue, 30 Oct 2018 00:06:20 +0000 (13:06 +1300)]
dnsupdate: Pass smb.conf through to samba-tool commands
If you call samba_dnsupdate with a --configfile option, this wasn't
passed through to the samba-tool commands the script tries to run.
Normally, samba_dnsupdate would only be run on the DC itself, so it
shouldn't be a big deal, however, this may be a problem if you install
the samba database into a non-default location (i.e. not
/usr/local/samba).
This patch passes through the smb.conf file, if one was specified.
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Tim Beale [Thu, 25 Oct 2018 22:08:54 +0000 (11:08 +1300)]
selftest: Add new customdc testenv that can load any backup-file
This adds a new testenv that can be used for sandpit/manual testing.
This testenv can be based off any backup-file that you like.
The main use case is large databases. Populating 1000s of users is
time-consuming (it can take hours to create a really large DB). Instead
of having to manually add users to the testenv every time you want to
try something, this allows you to populate the users just once, take a
backup/snapshot of the DB, and then spin up the backup multiple times.
In theory this testenv could be useful for other situations too, e.g.
dealing with a corrupted database, testing DB migration (e.g. 4.7 -->
4.8), or if (for some reason) you wanted to create a realistic
lab-domain within a testenv.
To run-up the testenv you need to specify a BACKUP_FILE environment
variable (the same way we specify the SELFTEST_TESTENV), e.g.
BACKUP_FILE=/files/backup-10k-ad_dc.tar.bz2 \
SELFTEST_TESTENV=customdc make testenv
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Douglas Bagnall [Thu, 4 Oct 2018 03:46:34 +0000 (16:46 +1300)]
pytests: allow blackbox subcommands without a shell
When given a list, it will use the list directly as an argument list,
avoiding shell-expansion and the intermediatory process.
This removes shell expansion trouble, and saves the machine a little
bit of work.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Douglas Bagnall [Wed, 10 Oct 2018 23:40:50 +0000 (12:40 +1300)]
samba-tool ntacl: remove unused imports and variables
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Thu Nov 1 09:40:02 CET 2018 on sn-devel-144
Douglas Bagnall [Thu, 11 Oct 2018 00:42:45 +0000 (13:42 +1300)]
python/tests/docs: remove unused import, variables
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Thu, 11 Oct 2018 00:06:16 +0000 (13:06 +1300)]
python/join: remove unused imports and variables
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Douglas Bagnall [Sat, 27 Oct 2018 21:29:06 +0000 (10:29 +1300)]
python/upgradehelpers: remove unused provision imports
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Wed, 10 Oct 2018 23:41:12 +0000 (12:41 +1300)]
samba-tool processes: remove unused imports and variables
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Wed, 10 Oct 2018 23:36:58 +0000 (12:36 +1300)]
samba-tool domain: remove unused variables
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Sat, 27 Oct 2018 21:39:13 +0000 (10:39 +1300)]
py/tests/dcerpc_integer: remove dup tests
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Thu, 11 Oct 2018 03:03:22 +0000 (16:03 +1300)]
autobuild: remove unused variables
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Thu, 11 Oct 2018 01:00:50 +0000 (14:00 +1300)]
python/tests/source: remove useless local variable
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Thu, 11 Oct 2018 00:41:23 +0000 (13:41 +1300)]
python/tests/credentials: remove unused variables
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Thu, 11 Oct 2018 00:14:32 +0000 (13:14 +1300)]
python/xattr: remove unused variable
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Thu, 11 Oct 2018 00:14:02 +0000 (13:14 +1300)]
python/upgradehelpers: remove unused variable
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Wed, 10 Oct 2018 23:35:56 +0000 (12:35 +1300)]
samba-tool dbcheck: remove unused variable
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Wed, 10 Oct 2018 23:52:46 +0000 (12:52 +1300)]
dbchecker: remove unused variables
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Douglas Bagnall [Sat, 27 Oct 2018 11:33:30 +0000 (00:33 +1300)]
samba-tool spn: remove unused variables
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Sat, 27 Oct 2018 11:33:05 +0000 (00:33 +1300)]
samba-tool rodc: remove unused variable
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Thu, 11 Oct 2018 00:40:23 +0000 (13:40 +1300)]
python/samba3: import passdb in the manner it is used
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Thu, 11 Oct 2018 00:38:27 +0000 (13:38 +1300)]
python/kcc/graph: import KCCError, which is used somewhere
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Sat, 27 Oct 2018 22:08:28 +0000 (11:08 +1300)]
tests/rodc_rwdc: fix message format
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Sat, 27 Oct 2018 21:49:24 +0000 (10:49 +1300)]
traffic_replay: Exception has no .message
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Sat, 27 Oct 2018 21:40:12 +0000 (10:40 +1300)]
py/tests/dcerpc_rpc: Py3 compat integer types
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Sat, 27 Oct 2018 21:33:26 +0000 (10:33 +1300)]
waflib: fix syntax error in string format
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Sat, 27 Oct 2018 21:32:35 +0000 (10:32 +1300)]
waflib: add necessary imports
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Fri, 26 Oct 2018 08:03:21 +0000 (21:03 +1300)]
python/remove_dc: fix Exception construction
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Fri, 26 Oct 2018 08:02:46 +0000 (21:02 +1300)]
python/remove_dc: fix formating syntax (missing %s)
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Fri, 26 Oct 2018 08:01:18 +0000 (21:01 +1300)]
provision/backend: LDAPBackendResult takes no creds
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Fri, 26 Oct 2018 08:00:01 +0000 (21:00 +1300)]
provision/backend: fix formating syntax
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Fri, 26 Oct 2018 07:52:05 +0000 (20:52 +1300)]
provision: fix string formatting (number of args)
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Fri, 26 Oct 2018 07:49:38 +0000 (20:49 +1300)]
provision: fix string format syntax
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Fri, 26 Oct 2018 07:41:50 +0000 (20:41 +1300)]
Python provision: remove "ExistingBackend"
This was unused and broken. e.g. here:
- def init(self):
- # Check to see that this 'existing' LDAP backend in fact exists
- ldapi_db = Ldb(self.ldapi_uri)
there is no attribute self.ldapi_uri, so this would always raise an
exception.
It was being left around in case it became useful, but that doesn't
seem to be happening.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Fri, 26 Oct 2018 06:31:26 +0000 (19:31 +1300)]
python dbcheck: use real exception name
and conventional indent
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Fri, 26 Oct 2018 06:27:53 +0000 (19:27 +1300)]
python: avoid useless work in dsdb_Dn.__cmp__
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Fri, 26 Oct 2018 05:38:27 +0000 (18:38 +1300)]
python join: use the sd_utils we imported
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Thu, 11 Oct 2018 04:23:20 +0000 (17:23 +1300)]
s4/dsdb/pytest/sites: do not use variables out of scope
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Thu, 11 Oct 2018 03:48:38 +0000 (16:48 +1300)]
ldb/tests/py/index: remove dup value in dictionary
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Thu, 11 Oct 2018 03:48:12 +0000 (16:48 +1300)]
ldb/tests/py/api: use proper name for ldb.LdbError
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Thu, 11 Oct 2018 01:44:49 +0000 (14:44 +1300)]
python/tests/dnscmd: don't use undefined name
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Thu, 11 Oct 2018 00:43:28 +0000 (13:43 +1300)]
python/tests/dsdb: use correct variable names
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Thu, 11 Oct 2018 00:11:06 +0000 (13:11 +1300)]
python/subnets: use the correct variable name
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Fri, 26 Oct 2018 07:40:16 +0000 (20:40 +1300)]
samba-tool user: fix message format
There were 2 % formats and 3 arguments.
Also reformat for line length
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Fri, 26 Oct 2018 07:20:55 +0000 (20:20 +1300)]
samba-tool: let self.usage() find argv[0] by itself
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Fri, 26 Oct 2018 07:18:03 +0000 (20:18 +1300)]
dbcheck: fix function call (right arguments)
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Fri, 26 Oct 2018 07:08:31 +0000 (20:08 +1300)]
dbcheck: fix message formatting
previously these would have raised an exception
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Fri, 26 Oct 2018 07:36:57 +0000 (20:36 +1300)]
python: PY3 Exceptions don't have .message
but str(e) is the same as str(e.message), so we can use that
on 2 and 3.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Sat, 27 Oct 2018 21:48:42 +0000 (10:48 +1300)]
script/show_test_time: attempt py3 compat
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Sat, 27 Oct 2018 22:17:43 +0000 (11:17 +1300)]
s4/scripting/pfm_verify: PY3: use compat.text_type
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Sat, 27 Oct 2018 22:17:07 +0000 (11:17 +1300)]
s4/scripting/samba_upgradedns: avoid .has_key()
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Sat, 27 Oct 2018 22:16:21 +0000 (11:16 +1300)]
s4/scripting/minschema: whitespace reformat and py3 compatible print
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Sat, 27 Oct 2018 22:12:20 +0000 (11:12 +1300)]
s4/scripting/autoidl: py3 compatible except
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Sat, 27 Oct 2018 22:11:41 +0000 (11:11 +1300)]
s4/script/depfilter.py: use py3 compatible regex import
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Sat, 27 Oct 2018 22:10:58 +0000 (11:10 +1300)]
tests/python/sec_descriptor: safer use of super()
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Sat, 27 Oct 2018 22:02:51 +0000 (11:02 +1300)]
tests/python/notification: safer use of super()
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Sat, 27 Oct 2018 21:56:15 +0000 (10:56 +1300)]
s4/dsdb/pytest/ldap_schema: fix typo in docs
Commit
311727947799e896e05d644103c9db80a665de88 removed a duplicate
test but it removed the wrong one, leaving this dreadful typo.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Sat, 27 Oct 2018 21:50:07 +0000 (10:50 +1300)]
selftest/format_subunit_json: py3 print()
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Douglas Bagnall [Sat, 27 Oct 2018 21:43:42 +0000 (10:43 +1300)]
tests/samba_tool/provision_password_check: follow super inheritance
We were skipping a level in the inheritance chain, which had no effect
in this case (no .setUps or .tearDowns were missed) but it would be
confusing if the parents ever changed.
Note: in python 3, you just call super() with no args, and it works
out the right thing.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
Volker Lendecke [Fri, 5 Oct 2018 14:27:48 +0000 (16:27 +0200)]
nsswitch: Run nsswitch thread test
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Nov 1 05:06:23 CET 2018 on sn-devel-144
Ralph Wuerthner [Fri, 5 Oct 2018 11:53:30 +0000 (13:53 +0200)]
nsswitch: add test for parallel NSS & libwbclient calls
Signed-off-by: Ralph Wuerthner <ralph.wuerthner@de.ibm.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Wuerthner [Tue, 2 Oct 2018 11:41:00 +0000 (13:41 +0200)]
nsswitch: protect access to wb_global_ctx by a mutex
This change will make libwbclient thread safe for all API calls not using a
context. Especially there are no more conflicts with threads using nsswitch
and libwbclient in parallel.
Signed-off-by: Ralph Wuerthner <ralph.wuerthner@de.ibm.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Wuerthner [Tue, 2 Oct 2018 11:35:16 +0000 (13:35 +0200)]
nsswitch: make wb_global_ctx private add add get/put functions to access global context
Signed-off-by: Ralph Wuerthner <ralph.wuerthner@de.ibm.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Wuerthner [Tue, 2 Oct 2018 08:58:12 +0000 (10:58 +0200)]
nsswitch: use goto to have only one function return
Signed-off-by: Ralph Wuerthner <ralph.wuerthner@de.ibm.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Samuel Cabrero [Tue, 30 Oct 2018 17:47:16 +0000 (18:47 +0100)]
s3: winbind: Remove fstring from wb_acct_info struct
The group enumeration backend functions try to allocate an array of
wb_acct_info structs with a number of elements equal to the number of
groups. In domains with a large number of groups this allocation may
fail due to the size of the chunk.
Found while trying to enumerate the groups in a domain with more than
700k groups.
Signed-off-by: Samuel Cabrero <scabrero@suse.de>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Ralph Boehme [Wed, 22 Aug 2018 13:25:26 +0000 (15:25 +0200)]
vfs_fruit: let fruit_open_meta() with O_CREAT return a fake-fd
This is the final step in implementing the needed macOS semantics on the
FinderInfo stream: as long as the client hasn't written a non-zero
FinderInfo blob to the stream, there mustn't be a visible filesystem
entry for other openers.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Nov 1 01:14:23 CET 2018 on sn-devel-144
Ralph Boehme [Sat, 20 Oct 2018 21:50:32 +0000 (23:50 +0200)]
vfs_fruit: don't check for delete-on-close on the FinderInfo stream
macOS SMB server doesn't filter out the FinderInfo stream if it has
delete-on-close set.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Sat, 20 Oct 2018 21:40:14 +0000 (23:40 +0200)]
vfs_fruit: let fruit_pwrite_meta_stream also ftruncate empty FinderInfo
fruit_streaminfo currently filters out the FinderInfo stream is
delete-on-close is set. We set it here internally, but the client may
also set it over SMB. Turns out that the macOS SMB server does NOT
filter out FinderInfo stream with delete-on-close set, so we must change
the way filtering is done in fruit_streaminfo.
Filtering is now done based on the FinderInfo stream being 0-bytes large which
is why I'm adding the ftruncate here.
No idea why the tests that check the filtering passed the commits
leading up to this one, but if you revert this commit after applying the
whole patchset, the "delete AFP_AfpInfo by writing all 0" test will fail.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Sat, 20 Oct 2018 21:46:43 +0000 (23:46 +0200)]
vfs_fruit: pass stream size to delete_invalid_meta_stream()
delete_invalid_meta_stream() is meant to guard against random data being
present in the FinderInfo stream. If the stream size is 0, it's likely a
freshly created stream where no data has been written to yet, so don't
delete it.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Wed, 17 Oct 2018 14:51:34 +0000 (16:51 +0200)]
vfs_fruit: let fruit handle all aio on the FinderInfo metadata stream
This will be required to support using fake fds for the FinderInfo
metadata stream.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Wed, 22 Aug 2018 14:49:23 +0000 (16:49 +0200)]
vfs_fruit: do ino calculation
As we'll start returning fake fds in open shortly, we can't rely on the
next module to calculat correct inode numbers for streams and must take
over that responsibility.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Wed, 22 Aug 2018 13:22:08 +0000 (15:22 +0200)]
vfs_fruit: prepare fruit_pread_meta() for reading on fake-fd
If the read on the stream fails we may have hit a handle on a just
created stream (fio->created=true) with no data written yet.
If that's the case return an empty initialized FinderInfo blob.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Wed, 22 Aug 2018 13:21:08 +0000 (15:21 +0200)]
vfs_fruit: prepare fruit_pwrite_meta() for on-demand opening and writing
This avoid creating files or blobs in our streams backend when a client
creates a stream but hasn't written anything yet. This is the only sane
way to implement the following semantics:
* client 1: create stream "file:foo"
* client 2: open stream "file:foo"
The second operation of client 2 must fail with NT_STATUS_NOT_FOUND.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Wed, 22 Aug 2018 13:22:57 +0000 (15:22 +0200)]
vfs_fruit: prepare struct fio for fake-fd and on-demand opening
Not used for now, that comes in the subsequent commits.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Mon, 22 Oct 2018 14:56:46 +0000 (16:56 +0200)]
vfs_fruit: add fio->created
fio->created tracks whether a create created a stream.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Mon, 15 Oct 2018 16:38:33 +0000 (18:38 +0200)]
vfs_fruit: remove resource fork special casing
Directly unlinking a file with open handles is not good, don't do it.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Wed, 17 Oct 2018 17:07:11 +0000 (19:07 +0200)]
vfs_fruit: add some debugging of dev/ino
Aids in debugging dev/ino mismatch failures in open_file_ntcreate.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Mon, 22 Oct 2018 14:21:21 +0000 (16:21 +0200)]
s4:torture/vfs/fruit: add test "empty_stream"
One to rule them all: consistently test critical operations on all
streams relevant to macOS clients: the FinderInfo stream, the Resource
Fork stream and an arbitrary stream that macOS maps to xattrs when
written to on a macOS SMB server.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Thu, 11 Oct 2018 15:14:50 +0000 (17:14 +0200)]
s4:torture/vfs/fruit: add check_stream_list_handle()
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Wed, 10 Oct 2018 16:45:56 +0000 (18:45 +0200)]
s4:torture/util: add torture_smb2_open()
This seems to be missing: a simple wrapper to just open a file without
fancy options.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Sat, 20 Oct 2018 12:53:50 +0000 (14:53 +0200)]
vfs_fruit: filter empty streams
First step in achieving macOS compliant behaviour wrt to empty streams:
- hide empty streams in streaminfo
- prevent opens of empty streams
This means that we may carry 0-byte sized streams in our streams
backend, but this shouldn't really hurt.
The previous attempt of deleting the streams when an SMB setinfo eof to
0 request came in, turned out be a road into desaster.
We could set delete-on-close on the stream, but that means we'd have to
check for it for every write on a stream and checking the
delete-on-close bits requires fetching the locking.tdb record, so this
is expensive and I'd like to avoid that overhead.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Sat, 20 Oct 2018 13:28:06 +0000 (15:28 +0200)]
vfs_fruit: use check on global_fruit_config.nego_aapl for macOS specific behaviour
Ensure any non MS compliant protocol behaviour targetted at supporting
macOS clients are only effective if the client negotiated AAPL.
Currently this only guards the resource fork which only macOS client are
going to use, but subsequent commits add more this at this place.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Mon, 22 Oct 2018 12:01:34 +0000 (14:01 +0200)]
s4:torture/vfs/fruit: enable AAPL extensions in a bunch of tests
These tests check for macOS SMB server specific behaviour. They work
currently against Samba without enabling AAPL because in vfs_fruit we're
currently don't check whether AAPL has been negotiated in one place. A
subsequent commit will change that and this commit prepares for that
change.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Sat, 20 Oct 2018 12:54:48 +0000 (14:54 +0200)]
vfs_fruit: don't unlink 0-byte size truncated streams
This caused all sort of havoc with subsequent SMB request that acted on
the handle of the then deleted backend storage (file or blob, depending
on the used streams module).
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Sat, 20 Oct 2018 12:52:23 +0000 (14:52 +0200)]
s4:torture/vfs/fruit: write some data to a just created teststream
Doesn't currently make a difference, but this prepares for a later
change in vfs_fruit that will filter out empty streams (which is the
macOS behaviour).
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Mon, 22 Oct 2018 10:43:16 +0000 (12:43 +0200)]
s4:torture/vfs/fruit: expand test "setinfo eof stream"
o Adds checks verifying that after setting eof to 0 on a stream, a
subsequent open gets ENOENT, before and after closing the handle that
had been used to set eof to 0.
o Verify that a write to a handle succeeds after that handle has been
used to set eof to 0 on a stream.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Mon, 22 Oct 2018 10:32:09 +0000 (12:32 +0200)]
vfs_fruit: update handling of read-only creation of resource fork
macOS SMB server versions supports this since 10.12, so we adapt our
behaviour.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Mon, 15 Oct 2018 13:17:08 +0000 (15:17 +0200)]
s4:torture/vfs/fruit: update test "creating rsrc with read-only access" for newer macOS versions
While this operation failed against older macOS versions, it passes
against versions 10.12 and newer. Update the test accordingly, a
subsequent commit will then update our implementation.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
git.samba.org / samba.git / log