samba.git
11 months agopython/tests: remove unused imports
Douglas Bagnall [Thu, 11 Oct 2018 01:33:49 +0000 (14:33 +1300)]
python/tests: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
11 months agopython/tests/samdb: avoid useless local variable
Douglas Bagnall [Thu, 11 Oct 2018 01:00:30 +0000 (14:00 +1300)]
python/tests/samdb: avoid useless local variable

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
11 months agopython/tests/*: remove unused imports
Douglas Bagnall [Thu, 11 Oct 2018 00:58:50 +0000 (13:58 +1300)]
python/tests/*: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
11 months agopython/tests/netbios: remove unused imports
Douglas Bagnall [Thu, 11 Oct 2018 00:50:43 +0000 (13:50 +1300)]
python/tests/netbios: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
11 months agopython/tests/getdcname: remove unused imports
Douglas Bagnall [Thu, 11 Oct 2018 00:45:54 +0000 (13:45 +1300)]
python/tests/getdcname: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
11 months agopython/tests/dsdb_schema_attr: remove unused/duplicate imports
Douglas Bagnall [Thu, 11 Oct 2018 00:44:45 +0000 (13:44 +1300)]
python/tests/dsdb_schema_attr: remove unused/duplicate imports

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
11 months agopython/tests/dns*: remove unused imports
Douglas Bagnall [Thu, 11 Oct 2018 00:42:10 +0000 (13:42 +1300)]
python/tests/dns*: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
11 months agopython/provision: remove unused imports
Douglas Bagnall [Thu, 11 Oct 2018 00:39:06 +0000 (13:39 +1300)]
python/provision: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
11 months agopython/gp_parse/gp_pol: remove unused import
Douglas Bagnall [Thu, 11 Oct 2018 00:23:35 +0000 (13:23 +1300)]
python/gp_parse/gp_pol: remove unused import

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
11 months agopython/schema: remove unused import
Douglas Bagnall [Thu, 11 Oct 2018 00:10:33 +0000 (13:10 +1300)]
python/schema: remove unused import

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
11 months agopython/ntacls: remove unused imports
Douglas Bagnall [Thu, 11 Oct 2018 00:06:40 +0000 (13:06 +1300)]
python/ntacls: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
11 months agopython/gp_ext_loader: remove unused imports
Douglas Bagnall [Thu, 11 Oct 2018 00:05:32 +0000 (13:05 +1300)]
python/gp_ext_loader: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
11 months agopython/forest_update: remove unused imports
Douglas Bagnall [Wed, 10 Oct 2018 23:57:15 +0000 (12:57 +1300)]
python/forest_update: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
11 months agopython/domain_update: remove unused imports
Douglas Bagnall [Wed, 10 Oct 2018 23:55:41 +0000 (12:55 +1300)]
python/domain_update: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
11 months agosamba-tool domain: remove unused imports
Douglas Bagnall [Wed, 10 Oct 2018 23:45:54 +0000 (12:45 +1300)]
samba-tool domain: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
11 months agosamba-tool user: remove unused import
Douglas Bagnall [Wed, 10 Oct 2018 23:41:28 +0000 (12:41 +1300)]
samba-tool user: remove unused import

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
11 months agosamba-tool ldapcmp: remove unused import
Douglas Bagnall [Wed, 10 Oct 2018 23:38:58 +0000 (12:38 +1300)]
samba-tool ldapcmp: remove unused import

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
11 months agosamba-tool drs: remove duplicate and unused imports
Douglas Bagnall [Wed, 10 Oct 2018 23:37:36 +0000 (12:37 +1300)]
samba-tool drs: remove duplicate and unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
11 months agopython/tests/__init__: import what we need
Douglas Bagnall [Thu, 11 Oct 2018 00:49:05 +0000 (13:49 +1300)]
python/tests/__init__: import what we need

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
11 months agopython/tests/gpo: import what we need (errno, not gp_log)
Douglas Bagnall [Thu, 11 Oct 2018 00:47:22 +0000 (13:47 +1300)]
python/tests/gpo: import what we need (errno, not gp_log)

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
11 months agopython/tests/unix: fix spelling and import of text_type
Douglas Bagnall [Thu, 11 Oct 2018 01:43:29 +0000 (14:43 +1300)]
python/tests/unix: fix spelling and import of text_type

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
11 months agos3:smbd: Move a variable declaration closer to its use
Volker Lendecke [Wed, 19 Sep 2018 14:26:09 +0000 (16:26 +0200)]
s3:smbd: Move a variable declaration closer to its use

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu Oct 25 21:44:17 CEST 2018 on sn-devel-144

11 months agosmbd: Slightly optimize delay_rename_for_lease_break
Volker Lendecke [Mon, 24 Sep 2018 13:46:27 +0000 (15:46 +0200)]
smbd: Slightly optimize delay_rename_for_lease_break

Do the checks with increasing cost, possibly avoid more expensive ones

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
11 months agolib: Avoid an "includes.h"
Volker Lendecke [Thu, 18 Oct 2018 03:46:03 +0000 (05:46 +0200)]
lib: Avoid an "includes.h"

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
11 months agosmbd: Replace some GUID_string by GUID_buf_string
Volker Lendecke [Mon, 24 Sep 2018 13:40:46 +0000 (06:40 -0700)]
smbd: Replace some GUID_string by GUID_buf_string

It's only debug statements, but I would like to promote the
stack-allocation routines as good practice where they make sense.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
11 months agotdbtorture: Align integer types
Volker Lendecke [Tue, 23 Oct 2018 03:53:24 +0000 (05:53 +0200)]
tdbtorture: Align integer types

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
11 months agotdbtorture: No transaction with mutexes
Volker Lendecke [Mon, 22 Oct 2018 15:18:43 +0000 (17:18 +0200)]
tdbtorture: No transaction with mutexes

Right now we don't do transactions with mutexed tdbs. tdbtorture -m
locks up. I haven't really investigated why that is the case. The lockup
confused me quite a bit until I figured out it works fine as long as it
does not do transactions, which is all we need right now.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
11 months agotdb: Remove "USE_RIGHT_MERGES" code
Volker Lendecke [Mon, 22 Oct 2018 08:10:24 +0000 (10:10 +0200)]
tdb: Remove "USE_RIGHT_MERGES" code

This has not been activated by default for ages and can be very
inefficient. With check_merge_with_left_record() we have an
alternative that will merge freelist records while we walk it
anyway. This has reduced fragmentation significantly

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
11 months agotdb: Fix a typo
Volker Lendecke [Mon, 22 Oct 2018 08:14:23 +0000 (10:14 +0200)]
tdb: Fix a typo

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
11 months agotdb: Use explicit initialization
Volker Lendecke [Mon, 22 Oct 2018 05:08:58 +0000 (07:08 +0200)]
tdb: Use explicit initialization

Let the compiler figure out the optimal code

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
11 months agotdb: Avoid casts
Volker Lendecke [Mon, 22 Oct 2018 05:42:14 +0000 (07:42 +0200)]
tdb: Avoid casts

We have %PRIu32 and %zu these days

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
11 months agoCI/Autobuild: Remove samba-none-env-py3 test
Noel Power [Thu, 18 Oct 2018 08:49:24 +0000 (09:49 +0100)]
CI/Autobuild: Remove samba-none-env-py3 test

We now run a purepython3 none-env test, later when the whole
build is running under python3 we will resurrect this job
but as (samba-none-env-py2) for python2

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Oct 23 09:10:22 CEST 2018 on sn-devel-144

11 months agoAdd new CI job for new purepy3-none-env autobuild task
Noel Power [Tue, 16 Oct 2018 18:39:41 +0000 (19:39 +0100)]
Add new CI job for new purepy3-none-env autobuild task

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agoscript: Add new (temporary) autobuild task for none-env tests.
Noel Power [Tue, 16 Oct 2018 18:25:26 +0000 (19:25 +0100)]
script: Add new (temporary) autobuild task for none-env tests.

Ideally we want all the tests to run under python3 by default (no
special task for this) and then convert the existing '-py3' tasks
to run the python tests with python3.
However at the moment the convertion process is not ready to do this,
for a while we need to run separate autobuild tasks for this.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agopython/samba/netcmd: PY3 fix CI error for samba.tests.samba_tool.help
Noel Power [Wed, 17 Oct 2018 17:06:34 +0000 (18:06 +0100)]
python/samba/netcmd: PY3 fix CI error for samba.tests.samba_tool.help

Strangely the test was failing on CI only, looks like there is an
issue with order of elements returned from dict.items() with python3.4
(version of python in CI docker instance) and python3.6 (version on my
development machine). Changed code to sort the keys so order of help
printed out should be the same for each invocation.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agopython/samba/provision: PY3 PY3 port samba4.blackbox.provision-backend
Noel Power [Wed, 10 Oct 2018 11:25:09 +0000 (12:25 +0100)]
python/samba/provision: PY3 PY3 port samba4.blackbox.provision-backend

Enclose filter with list as filter object has no 'len' method

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agopython/samba: PY3 port samba4.blackbox.provision-backend
Noel Power [Wed, 10 Oct 2018 11:21:10 +0000 (12:21 +0100)]
python/samba: PY3 port samba4.blackbox.provision-backend

convert ldif content to str when necessary

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agopython/samba/provision: PY3 port samba4.blackbox.provision-backend
Noel Power [Wed, 10 Oct 2018 11:18:32 +0000 (12:18 +0100)]
python/samba/provision: PY3 port samba4.blackbox.provision-backend

Fix some attibrutes that need to be treated as str

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agopython/samba/netcmd: PY3 make sure get_testparm_var returns 'str'
Noel Power [Wed, 10 Oct 2018 10:18:13 +0000 (11:18 +0100)]
python/samba/netcmd: PY3 make sure get_testparm_var returns 'str'

part of PY3 port samba4.blackbox.upgrade.samba3-upgrade*

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agopython/samba/samba3: PY3 tdb.Tdb.get method expects bytes in PY3
Noel Power [Wed, 10 Oct 2018 10:16:40 +0000 (11:16 +0100)]
python/samba/samba3: PY3 tdb.Tdb.get method expects bytes in PY3

part of PY3 port samba4.blackbox.upgrade.samba3-upgrade*

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agopython/samba/netcmd: PY3 fix samba.tests.domain_backup_offline
Noel Power [Tue, 9 Oct 2018 20:16:08 +0000 (21:16 +0100)]
python/samba/netcmd: PY3 fix samba.tests.domain_backup_offline

Fix attributes that need to be treated as str not bytes

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agopython/samba: PY3 port samba4.blackbox.functionalprep
Noel Power [Mon, 8 Oct 2018 14:21:45 +0000 (15:21 +0100)]
python/samba: PY3 port samba4.blackbox.functionalprep

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agopython/samba/netcmd: PY3 Fix error in samba4.blackbox.schemaupgrade
Noel Power [Mon, 8 Oct 2018 11:27:56 +0000 (12:27 +0100)]
python/samba/netcmd: PY3 Fix error in samba4.blackbox.schemaupgrade

Getting Exception: must be str, not ldb.bytes error in scheme_upgrade
phase of test
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agoselftest/filter: PY3 Make filter-subunit forgiving of decoding errors
Noel Power [Fri, 5 Oct 2018 15:49:45 +0000 (16:49 +0100)]
selftest/filter: PY3 Make filter-subunit forgiving of decoding errors

samba4.local.ndr for one is a test that outputs string in an encoding
that stdin.readline() guesses to be utf8 (but it isn't) filter subunit
can afford to be forgiving of some random text that can't be decoded as
utf8 so lets do that.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agopython/samba: PY3 fix samba4.blackbox.upgradeprovision.release-4-0-0
Noel Power [Fri, 5 Oct 2018 14:10:52 +0000 (15:10 +0100)]
python/samba: PY3 fix samba4.blackbox.upgradeprovision.release-4-0-0

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agos4/scripting/bin: blobs needs to be bytes
Noel Power [Thu, 4 Oct 2018 17:29:37 +0000 (18:29 +0100)]
s4/scripting/bin: blobs needs to be bytes

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agos4/scripting/bin: PY3 fix samba4.blackbox.upgradeprovision.alpha13
Noel Power [Wed, 3 Oct 2018 14:19:09 +0000 (15:19 +0100)]
s4/scripting/bin: PY3 fix samba4.blackbox.upgradeprovision.alpha13

11 months agos4/scripting/bin: PY3 make sure GUID result of format is string
Noel Power [Tue, 2 Oct 2018 13:45:53 +0000 (14:45 +0100)]
s4/scripting/bin: PY3 make sure GUID result of format is string

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agos4/scripting: Py2/Py3 for samba4.blackbox.upgradeprovision.alpha13
Noel Power [Mon, 27 Aug 2018 19:29:53 +0000 (20:29 +0100)]
s4/scripting: Py2/Py3 for samba4.blackbox.upgradeprovision.alpha13

fixup source4/scripting/bin/samba_upgradedns
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agos4/scripting: PY2/PY3 port for samba4.blackbox.upgradeprovision.current
Noel Power [Mon, 27 Aug 2018 12:08:26 +0000 (13:08 +0100)]
s4/scripting: PY2/PY3 port for samba4.blackbox.upgradeprovision.current

o Fix various ldb attribute that need to be converted to string
o dict has no 'has_key' method
o ndr_unpack needs bytes not string
o b64encode needs bytes (so open file with binary mode)
o StandardError was removed in python3 use Exception instead
o fix octal literals
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agopython/samba: samba4.blackbox.dbcheck.release-4-1-0rc3
Noel Power [Sat, 25 Aug 2018 11:50:58 +0000 (12:50 +0100)]
python/samba: samba4.blackbox.dbcheck.release-4-1-0rc3

* Various string related fixed py2/py3
* Fix strange double decode followed by encode code (see comments
  in change)
* Added dump_attr_values, simply printing attribute values (from
  sequence) doesn't work (when using string '%s' format codes in
  existing string). We need to print out string from bytes in PY3
  and fallback to repr(which will print b'blah') if we get a decode error

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agosamba_tool: Enclose iterator with list
Noel Power [Sat, 25 Aug 2018 10:39:04 +0000 (11:39 +0100)]
samba_tool: Enclose iterator with list

Really strange bug caused by map being updated while being iterated.
This caused keys to be skipped and inconsistent and incorrect
results from ldapcmp.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agopython/samba: misc use of str for ldb.bytes
Noel Power [Thu, 23 Aug 2018 16:59:12 +0000 (17:59 +0100)]
python/samba: misc use of str for ldb.bytes

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agopython/samba: ldb attribute string fix for wafsamba.tests
Noel Power [Mon, 1 Oct 2018 13:47:06 +0000 (14:47 +0100)]
python/samba: ldb attribute string fix for wafsamba.tests

11 months agoPY3: ensure StringIO usage is py2/py3 compatible
Noel Power [Mon, 6 Aug 2018 12:39:15 +0000 (13:39 +0100)]
PY3: ensure StringIO usage is py2/py3 compatible

11 months agoPY3: relative import fixes
Noel Power [Mon, 6 Aug 2018 14:02:18 +0000 (15:02 +0100)]
PY3: relative import fixes

11 months agos4/scripting/bin: PY3 Fix exception tuple assignments.
Noel Power [Fri, 28 Sep 2018 15:49:27 +0000 (16:49 +0100)]
s4/scripting/bin: PY3 Fix exception tuple assignments.

In Python3 to access the exception arguments you need to now use
Exception.args, in Python2 you could access these direcly with the
'except' declaration.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agoPY3: port samba.tests.samba3sam
Noel Power [Sun, 5 Aug 2018 20:08:33 +0000 (21:08 +0100)]
PY3: port samba.tests.samba3sam

11 months agopython/samba: use PY3 version of ConfigParser
Noel Power [Wed, 5 Sep 2018 13:59:19 +0000 (14:59 +0100)]
python/samba: use PY3 version of ConfigParser

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agopython/samba/gp_parse: PY2/PY3 compat porting for gp_init.py
Noel Power [Mon, 15 Oct 2018 09:58:23 +0000 (10:58 +0100)]
python/samba/gp_parse: PY2/PY3 compat porting for gp_init.py

Fixes
1) use compat versions of ConfigParser and StringIO
2) open file needs to be opened in binary mode as write_pretty_xml
   routine uses BytesIO() object.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agopython/samba: add alias for ConfigParser for PY2/PY3 compatability
Noel Power [Fri, 24 Aug 2018 08:55:08 +0000 (09:55 +0100)]
python/samba: add alias for ConfigParser for PY2/PY3 compatability

ConfigParser module changed name to configParser in PY3, additionally
the behaviour regarding interpolation has changed. ConfigParser now
has a default interpolation param whose behaviour demands that '%' is
escaped. To maintain behaviour with the python2 version this default
param needs to be changed. Add some alias(s) and 'shim' Configparser
symbol in samba.compat to cater for this.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agoPY3: convert samba.tests.strings to Py2/Py3
Noel Power [Sat, 4 Aug 2018 20:00:06 +0000 (21:00 +0100)]
PY3: convert samba.tests.strings to Py2/Py3

Previously the py2 api for strcasecmp_m/strstr_m required strings/unicode
 but couldn't actually handle unicode with anything other than the default
encoding (e.g. ascii). The c-api as been fixed and the encoding steps
(which were unnecessary and causing errors in PY3) have been removed

11 months agopython: py_strcasecmp_m & py_strstr_m don't handle unicode properly
Noel Power [Tue, 7 Aug 2018 15:21:35 +0000 (16:21 +0100)]
python: py_strcasecmp_m & py_strstr_m don't handle unicode properly

py_strcasecmp_m & py_strstr_m use PyArg_ParseTuple() with 's' which
in Py2 tries to decode string with the default (e.g. ascii) encoding

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agopython/samba/tests: PY2/PY3 port samba.tests.dcerpc.integer
Noel Power [Sat, 4 Aug 2018 16:01:24 +0000 (17:01 +0100)]
python/samba/tests: PY2/PY3 port samba.tests.dcerpc.integer

Python3 no longer has a long type so the 'L' postfix is no
longer valid. Additionally in python2 an int that exceeds will
be transparently converted into a long when necessary

11 months agopython/samba/netcmd: PY3 only possible to decode bytes
Noel Power [Thu, 27 Sep 2018 17:16:49 +0000 (18:16 +0100)]
python/samba/netcmd: PY3 only possible to decode bytes

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agoPY3: Only decode when necessary
Noel Power [Sat, 4 Aug 2018 13:23:28 +0000 (14:23 +0100)]
PY3: Only decode when necessary

11 months agos4/setup/tests: PY3 samba-tool needs to be called with correct python ver.
Noel Power [Wed, 10 Oct 2018 10:14:59 +0000 (11:14 +0100)]
s4/setup/tests: PY3 samba-tool needs to be called with correct python ver.

Ensure samba-tool python version defined by $PYTHON

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agos4/scripting: PY3 Ensure python scripts are run with correct python ver.
Noel Power [Wed, 10 Oct 2018 09:15:31 +0000 (10:15 +0100)]
s4/scripting: PY3 Ensure python scripts are run with correct python ver.

As part of port samba4.blackbox.samba3dump to python2/3
make sure test_samba3dump.sh runs samba3dump with $PYTHON which should
define the correct python version.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agotestprogs/blackbox: Use PYTHON env variable for calling python scripts
Noel Power [Mon, 27 Aug 2018 16:06:37 +0000 (17:06 +0100)]
testprogs/blackbox: Use PYTHON env variable for calling python scripts

Ensure samba-tool is called with correct python that is
defined by $PYTHON

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agopython/samba/tests: make sure samba-tool is called with ${PYTHON}
Noel Power [Thu, 27 Sep 2018 15:08:34 +0000 (16:08 +0100)]
python/samba/tests: make sure samba-tool is called with ${PYTHON}

Ensure python scripts are called with the python version that
is defined by $PYTHON

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agoselftest/target: Make sure samba-tool is called with ${PYTHON}
Noel Power [Thu, 27 Sep 2018 08:30:40 +0000 (09:30 +0100)]
selftest/target: Make sure samba-tool is called with ${PYTHON}

Ensure python scripts are called with the python version that
is defined by $PYTHON

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agoPY3: In a pure PY3 build filter-subunit was getting called without 'python'
Noel Power [Sat, 4 Aug 2018 14:38:40 +0000 (15:38 +0100)]
PY3: In a pure PY3 build filter-subunit was getting called without 'python'

tests were getting called with "| ${src}/selftest/filter-subunit" which
resulted in filter-subunit getting execve'd without a calling python. This
resulted in /usr/bin/python (default python) getting called and subsequent
imports failing.

11 months agos4/scripting: PY3 need to convert cmp funct to key func for sort.
Noel Power [Mon, 15 Oct 2018 09:36:19 +0000 (10:36 +0100)]
s4/scripting: PY3 need to convert cmp funct to key func for sort.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agopython/samba/gp_parse: PY3 fdeploy_sids needs to use key method for sort
Noel Power [Mon, 15 Oct 2018 15:04:25 +0000 (16:04 +0100)]
python/samba/gp_parse: PY3 fdeploy_sids needs to use key method for sort

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agosamba-tool: PY3 dict view doesn't have sort method,
Noel Power [Mon, 15 Oct 2018 15:23:07 +0000 (16:23 +0100)]
samba-tool: PY3 dict view doesn't have sort method,

Can't sort a dict view, create a list from view then use list.sort
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agopython/samba/tests: PY3 iterable has no sort method
Noel Power [Wed, 26 Sep 2018 16:22:16 +0000 (17:22 +0100)]
python/samba/tests: PY3 iterable has no sort method

map in python3 returns an iterable, in python2 it returned
a list. Iterable has no sort method, use sort function instead or
construct a list from the iterable so you can use list.sort

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agos4/scripting/bin: PY3 Make sure print statements are enclosed by '()'
Noel Power [Thu, 27 Sep 2018 17:15:49 +0000 (18:15 +0100)]
s4/scripting/bin: PY3 Make sure print statements are enclosed by '()'

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
11 months agoselftest: Add expected-value testing for userParameters
Andrew Bartlett [Tue, 23 Oct 2018 03:27:29 +0000 (16:27 +1300)]
selftest: Add expected-value testing for userParameters

This does not means that bugs like https://bugzilla.samba.org/show_bug.cgi?id=11881
are fixed, however we do not wish to cause further issues
without noticing it, eg during python3 fixes for dbcheck.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
11 months agoctdb-daemon: Fix valgrind hit in event code
Martin Schwenke [Wed, 17 Oct 2018 06:19:06 +0000 (17:19 +1100)]
ctdb-daemon: Fix valgrind hit in event code

==25741== Syscall param write(buf) points to uninitialised byte(s)
==25741==    at 0x4939291: write (write.c:27)
==25741==    by 0x4868285: sys_write (sys_rw.c:68)
==25741==    by 0x13915D: sock_queue_trigger (sock_io.c:316)
==25741==    by 0x4DE6478: tevent_common_invoke_immediate_handler (in /usr/lib/x86_64-linux-gnu/libtevent.so.0.9.37)
==25741==    by 0x4DE64A2: tevent_common_loop_immediate (in /usr/lib/x86_64-linux-gnu/libtevent.so.0.9.37)
==25741==    by 0x4DEBE5A: ??? (in /usr/lib/x86_64-linux-gnu/libtevent.so.0.9.37)
==25741==    by 0x4DEA2D6: ??? (in /usr/lib/x86_64-linux-gnu/libtevent.so.0.9.37)
==25741==    by 0x4DE57E3: _tevent_loop_once (in /usr/lib/x86_64-linux-gnu/libtevent.so.0.9.37)
==25741==    by 0x15D1BA: ctdb_event_script_args (eventscript.c:821)
==25741==    by 0x13B437: ctdb_start_daemon (ctdb_daemon.c:1315)
==25741==    by 0x110642: main (ctdbd.c:393)
==25741==  Address 0x57888a4 is 100 bytes inside a block of size 144 alloc'd
==25741==    at 0x48357BF: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==25741==    by 0x4B9B7C0: talloc_named_const (in /usr/lib/x86_64-linux-gnu/libtalloc.so.2.1.14)
==25741==    by 0x15CCC6: eventd_client_write (eventscript.c:430)
==25741==    by 0x15CCC6: eventd_client_run (eventscript.c:556)
==25741==    by 0x15CCC6: ctdb_event_script_run (eventscript.c:649)
==25741==    by 0x15D198: ctdb_event_script_args (eventscript.c:812)
==25741==    by 0x13B437: ctdb_start_daemon (ctdb_daemon.c:1315)
==25741==    by 0x110642: main (ctdbd.c:393)
==25741==

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13659

Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Mon Oct 22 09:27:15 CEST 2018 on sn-devel-144

11 months agoctdb-event: Check the return status of sock_daemon_set_startup_fd
Amitay Isaacs [Wed, 10 Oct 2018 07:19:32 +0000 (18:19 +1100)]
ctdb-event: Check the return status of sock_daemon_set_startup_fd

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13659

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
11 months agoctdb-common: Set close-on-exec for startup fd
Amitay Isaacs [Wed, 10 Oct 2018 07:16:33 +0000 (18:16 +1100)]
ctdb-common: Set close-on-exec for startup fd

The startup_fd should not be propagated to the child processes created
from a daemon.  It should only be used in the daemon code to return the
status of the startup.  Another use of startup_fd is to notify the
parent if the daemon process has exited.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13659

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
11 months agoctdb-daemon: Exit if eventd goes away
Martin Schwenke [Thu, 11 Oct 2018 00:26:06 +0000 (11:26 +1100)]
ctdb-daemon: Exit if eventd goes away

ctdbd enters a broken state if eventd goes away.  A clean shutdown is
not possible because that involves running events.  Restarting eventd
is possible but this might mask a serious problem and it is possible
that eventd might keep on disappearing.  Just exit.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13659

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
11 months agoctdb-daemon: Return early when refusing to run an event script
Martin Schwenke [Wed, 10 Oct 2018 02:35:00 +0000 (13:35 +1100)]
ctdb-daemon: Return early when refusing to run an event script

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13659

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
12 months agos3:smbcontrol: Simplify the return code check
Andreas Schneider [Wed, 26 Sep 2018 12:47:20 +0000 (14:47 +0200)]
s3:smbcontrol: Simplify the return code check

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Oct 20 02:17:56 CEST 2018 on sn-devel-144

12 months agos4:torture: Fix the scope of the req variable in drsuapi test
Andreas Schneider [Wed, 26 Sep 2018 12:34:07 +0000 (14:34 +0200)]
s4:torture: Fix the scope of the req variable in drsuapi test

Found by covscan.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
12 months agondr: Init variables of GUID_from_data_blob()
Andreas Schneider [Wed, 26 Sep 2018 12:30:32 +0000 (14:30 +0200)]
ndr: Init variables of GUID_from_data_blob()

Found by covscan.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
12 months agos3:registry: Avoid a double-free in reg_perfcount
Andreas Schneider [Wed, 26 Sep 2018 12:29:50 +0000 (14:29 +0200)]
s3:registry: Avoid a double-free in reg_perfcount

Found by covscan.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
12 months agotalloc: deprecate talloc_set_memlimit()
David Disseldorp [Tue, 16 Oct 2018 17:06:48 +0000 (19:06 +0200)]
talloc: deprecate talloc_set_memlimit()

The memlimit functionality was never utilized by Samba. It adds unneeded
complexity, so flag it as deprecated.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
12 months agolib: Remove gencache.h from proto.h
Volker Lendecke [Thu, 18 Oct 2018 19:53:36 +0000 (21:53 +0200)]
lib: Remove gencache.h from proto.h

It's a pain to recompile the world if gencache.h changes

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Fri Oct 19 18:52:50 CEST 2018 on sn-devel-144

12 months agos3:secrets: clean up sid before storing
Philipp Gesang [Thu, 4 Oct 2018 07:25:14 +0000 (09:25 +0200)]
s3:secrets: clean up sid before storing

SIDs may contain non-zero memory beyond SubAuthorityCount:

    {
    key(15) = "SECRETS/SID/FOO"
    data(68) = "\01\04\00\00\00\00\00\05\15\00\00\00}u@\8C\08\A3\06nx\95\16\FE\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00`F\92\B7\03\00\00\00\18e\92\B7\03\00\00\00@H\92\B7\00\00\00\00"
    }

These parts are lost when converting to ``string format syntax``
so a roundtrip conversion does not result in the same binary
representation.

Ensure that these never reach the tdb by using an initialized
copy. This allows bitwise comparisons of secrets.tdb after
dumping SIDs as text and reading them back.

Signed-off-by: Philipp Gesang <philipp.gesang@intra2net.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Oct 19 13:59:04 CEST 2018 on sn-devel-144

12 months agodsdb encrypted_secrets: Allow "ldb:// and "mdb://" in file path
Gary Lockyer [Mon, 15 Oct 2018 03:02:40 +0000 (16:02 +1300)]
dsdb encrypted_secrets: Allow "ldb:// and "mdb://" in file path

Correctly handle "ldb://" and "mdb://" schemes in the file path when
determining the path for the encrypted secrets key file.

When creating a new user and specifying the local file path of the
sam.ldb DB, it was possible to create an account that you could not
login with. The path for the key file was incorrectly calculated
for the "ldb://" and "mdb://" schemes, the scheme was not stripped from
the path and the subsequent open of the key file failed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13653

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Oct 19 09:34:46 CEST 2018 on sn-devel-144

12 months agodsdb encrypted_secrets tests: Allow "ldb://" in file path
Gary Lockyer [Mon, 15 Oct 2018 03:01:47 +0000 (16:01 +1300)]
dsdb encrypted_secrets tests: Allow "ldb://" in file path

When creating a new user and specifying the local file path of the
sam.ldb DB, it's possible to create an account that you can't actually
login with.

This commit contains tests to verify the bug.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13653

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
12 months agopython tests Blackbox: add random_password
Gary Lockyer [Tue, 16 Oct 2018 20:10:10 +0000 (09:10 +1300)]
python tests Blackbox: add random_password

Add the random_password method to the BlackboxTestCase class and remove
duplicated copies from other test cases. Also use SystemRandom so that
the generated passwords are more cryptographically sound.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
12 months agoldb_ldif: avoid strlen(NULL)
Douglas Bagnall [Thu, 18 Oct 2018 21:21:21 +0000 (10:21 +1300)]
ldb_ldif: avoid strlen(NULL)

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Fri Oct 19 03:43:58 CEST 2018 on sn-devel-144

12 months agoldb_ldif: be less horribly efficient in debugging
Douglas Bagnall [Wed, 17 Oct 2018 03:28:25 +0000 (16:28 +1300)]
ldb_ldif: be less horribly efficient in debugging

perf said all the time was in strlen.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Thu Oct 18 13:17:30 CEST 2018 on sn-devel-144

12 months agopy3_tests/kcc : test_verify can hit KCCError as well as GraphError
Douglas Bagnall [Wed, 17 Oct 2018 04:21:09 +0000 (17:21 +1300)]
py3_tests/kcc : test_verify can hit KCCError as well as GraphError

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
12 months agopy3/tests/kcc: turn error into failure for flapping.d/kcc
Douglas Bagnall [Tue, 16 Oct 2018 20:50:41 +0000 (09:50 +1300)]
py3/tests/kcc: turn error into failure for flapping.d/kcc

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
12 months agoselftest: add tests for samba-tool drs uptodateness
Joe Guo [Thu, 4 Oct 2018 02:37:49 +0000 (15:37 +1300)]
selftest: add tests for samba-tool drs uptodateness

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13658

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Oct 18 10:02:19 CEST 2018 on sn-devel-144

12 months agonetcmd/drs: add cmd_drs_uptodateness with json support
Joe Guo [Wed, 3 Oct 2018 22:28:44 +0000 (11:28 +1300)]
netcmd/drs: add cmd_drs_uptodateness with json support

Add cmd to print uptodateness summary with json support.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13658

12 months agouptodateness: add get_utdv_summary function
Joe Guo [Wed, 3 Oct 2018 22:24:33 +0000 (11:24 +1300)]
uptodateness: add get_utdv_summary function

Get utdv summary from distances matrix and support attr filters.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13658