Volker Lendecke [Sun, 9 Dec 2018 11:15:05 +0000 (12:15 +0100)]
lib: Use dom_sid_str_buf
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Volker Lendecke [Sun, 9 Dec 2018 10:56:59 +0000 (11:56 +0100)]
auth: Use dom_sid_str_buf
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Volker Lendecke [Sat, 8 Dec 2018 14:52:31 +0000 (15:52 +0100)]
passdb: dom_sid_str_buf
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Volker Lendecke [Sun, 9 Dec 2018 10:52:16 +0000 (11:52 +0100)]
pdb_ldap: Fix typos
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Noel Power [Mon, 10 Dec 2018 07:36:15 +0000 (07:36 +0000)]
CI: Adjust CI tasks for new python3 autobuild.py default
Now that autobuild has defaulted to python3 (via shebang) we no longer
need to explicity call autobuild.py with 'python3'
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Mon Dec 10 14:32:24 CET 2018 on sn-devel-144
Noel Power [Mon, 10 Dec 2018 07:28:31 +0000 (07:28 +0000)]
autobuild: We should run autobuild with python3 by default
sn-devel autobuild runing autobuild.py (via git hooks I suppose) but
if run directly (e.g. depending on script shebang) then 'python' aka
python2 will run. This will cause an error when building some targets
because the autobuild script itself sometimes builds paths based
on the version of python executing the script e.g ${PYTHON_PREFIX}.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Mon, 3 Dec 2018 17:08:51 +0000 (17:08 +0000)]
selftest/flapping.d: Add python2 versions for flapping tests
Post build & test running under python3 we now run with
'--extra-python=/usr/bin/python2', these tests will get
python2 appended to the test name so we need also to create
new flapping*/* entries for these. We will keep the python3
versions in case we create some CI job(s)
with
PYTHON=python configure.developer --extra-python=/usr/bin/python3
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Fri, 30 Nov 2018 14:36:14 +0000 (14:36 +0000)]
selftest/knownfail: Add python2 version of known fails
Post build & test running under python3 we now run with
'--extra-python=/usr/bin/python2', these tests will get
python2 appended to the test name so we need also to create
new knownfails for these. We will keep the python3 versions
in case we create (and we probably should) some CI job(s)
with
PYTHON=python configure.developer --extra-python=/usr/bin/python3
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Thu, 29 Nov 2018 21:28:11 +0000 (21:28 +0000)]
CI: Add new py2 CI jobs to replace old py3 ones
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Thu, 29 Nov 2018 21:21:58 +0000 (21:21 +0000)]
autobuild: Convert old py3 tasks to py2
Now that we are building with python3 by default we need to
convert the old python3 test tasks to python2 (e.g. reverse how
we used do it)
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Thu, 29 Nov 2018 21:18:07 +0000 (21:18 +0000)]
selftest: don't hardcode '.python3' for extra-python tests
Instead of hardcoding '.python3' we now hardcode the extra python
exe (which will be python2 for a default python3 build)
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Thu, 29 Nov 2018 16:51:50 +0000 (16:51 +0000)]
buildtools/wafsamba: Support --extra-python with python2
Relax restriction on extra_python version
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Sat, 24 Nov 2018 11:13:47 +0000 (11:13 +0000)]
autobuild: Adjust autobuild for PY2/PY3 get_python_lib behaviour
The resuls of get_python_lib are different between python2 & python3
and this results in autobuild generating the wrong PYTHONPATH with
python3.
python2
=======
print ("%s" % get_python_lib(standard_lib=1, prefix='/my/prefix'))
/my/prefix/lib64/python2.7
python3
print ("%s" % get_python_lib(standard_lib=1, prefix='/my/prefix'))
/my/prefix/lib/python3.6
But with addition of plat_specific param the results are the same
python2
=======
print ("%s" % get_python_lib(plat_specific=1, standard_lib=0, prefix='/my/prefix'))
/my/prefix/lib64/python2.7/site-packages
python3
=======
print ("%s" % get_python_lib(plat_specific=1, standard_lib=0, prefix='/my/prefix'))
/my/prefix/lib64/python3.6/site-packages
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Thu, 29 Nov 2018 15:10:40 +0000 (15:10 +0000)]
CI: convert the purepy3 job to a py2 one
With patches to have python3 as default we need a
pure python2 build
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Thu, 29 Nov 2018 14:21:27 +0000 (14:21 +0000)]
autobuild: Modify old samba_buildpy3_only job to python2
Since autobuild now builds python3 by default we need to change
the previously buildpy3 only job to python2
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Wed, 5 Dec 2018 16:37:17 +0000 (16:37 +0000)]
script: Fix failing build_xc job
build_xc job uses compare_cc_results.py to compare cache
files, the cache files are stringified hash maps, the results
in python 3.4 don't compare well due to inconsistent order of
dict key/value pairs when the cache files are created. While
comparing the file contents works fine in python3.6 it fails
with python3.4. This patch detects problematic dict lines and
rewrites the value for comparison
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Thu, 29 Nov 2018 14:18:05 +0000 (14:18 +0000)]
CI: Run autobuild jobs with python3
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Sat, 24 Nov 2018 15:27:45 +0000 (15:27 +0000)]
PY3: switch current build to use python3
Make sure default make and configure for all now defaults
to building with python3.
To build a samba (or sub component e.g. talloc etc.) with python3
./configure && make
To build a samba (or sub component e.g. talloc etc.) with python2
PYTHON=python ./configure && PYTHON=python make
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Thu, 29 Nov 2018 15:40:48 +0000 (15:40 +0000)]
buildtools/wafsamba Provide proper pc file
To support building by default python3 we need to fix assumption
that only extra-python with py3 has valid PYTHON_SO_ABI_FLAG used to
create alt name for pc file.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Thu, 29 Nov 2018 15:12:10 +0000 (15:12 +0000)]
buildtools/wafsamba: Ensure we detect the correct python.
In order to support a default python3 build we need to ensure
we detect python3 if no PYTHON env variable is set up.
Currently we detect python with
conf.find_program('python', var='PYTHON', mandatory=mandatory)
which uses PYTHON as as hint and falls back to the program name
'python' otherwise
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Mon, 26 Nov 2018 14:55:02 +0000 (14:55 +0000)]
selftest: Always set PYTHON to something sensible (if it isn't set)
Lots of test scripts need to run with the correct version
of python. With the correct shebang the script should run with the
correct version, the problem is that not all scripts are part
of the installation, some scripts are part of the source code,
and the shebang is not dynamically generated as yet.
It is safer if we are somewhat version neutral at the moment and
ignore the shebang and always run scripts from the test environment
with the python version (determined by PYTHON env variable) If this
env variable isn't set then set it according to the python version
that is running the tests
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Sun, 25 Nov 2018 00:59:04 +0000 (00:59 +0000)]
buildtools/wafsamba: Decode result of Popen as unicode/string
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Sat, 24 Nov 2018 16:41:53 +0000 (16:41 +0000)]
adjust ldb config for py3 build
waf code is making assumptions that main build is py2 and extra build
is py3, this results in wrong library names being used.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Sat, 24 Nov 2018 15:27:21 +0000 (15:27 +0000)]
PY3 pure build fix some str/bytes wobblies
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Thu, 29 Nov 2018 13:52:04 +0000 (13:52 +0000)]
CI: Remove the remainder py3 jobs
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Thu, 29 Nov 2018 13:46:55 +0000 (13:46 +0000)]
autobuild: Remove temporary purepy3 tasks
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Thu, 6 Dec 2018 14:18:08 +0000 (14:18 +0000)]
auth/credentials/tests: Python 3.6 avoid deepcopy error
In PY3 both deepcopy & (shallow)copy fail with
Traceback (most recent call last):
File "auth/credentials/tests/bind.py", line 42, in <module>
creds_machine = copy.copy(creds)
File "/usr/lib64/python3.6/copy.py", line 96, in copy
rv = reductor(4)
TypeError: can't pickle credentials.Credentials objects
This patch avoids the nasty copies but creating and populating the
Credential objects instead of copying
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Thu, 29 Nov 2018 17:02:53 +0000 (17:02 +0000)]
s4/selftest: Reenable samba.tests.gpo to run with --extra-python
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Thu, 29 Nov 2018 12:31:26 +0000 (12:31 +0000)]
CI: Remove purepy3 temporary jobs
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Wed, 28 Nov 2018 17:57:43 +0000 (17:57 +0000)]
CI: Add new CI jobs for samba-purepy3 & samba-purepy3-nt4
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Wed, 28 Nov 2018 17:52:30 +0000 (17:52 +0000)]
CI: Add new (TEMP) pure python3 autobuild jobs for samba-build & samba-nt4
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Wed, 21 Nov 2018 12:39:56 +0000 (12:39 +0000)]
s4/dsdb/tests/python: PY3 allow test samba4.ldap.sort pass
Adjust test data to remove use of embedded NULLs.
We are getting the following exception when running the test
under python3
Exception: Exception: Traceback (most recent call last):
File "source4/dsdb/tests/python/sort.py", line 181, in setUp
key=cmp_to_key_fn(locale.strcoll))
ValueError: embedded null character
Looking at the source code for locale.strcoll the lhs & rhs
params are processed using 'PyUnicode_AsWideCharString(s, NULL)'
Documentation for PyUnicode_AsWideCharString states:
"Convert the Unicode object to a wide character string.
The output string always ends with a null character. If size is
not NULL, write the number of wide characters (excluding the
trailing null termination character) into *size. Note that the
resulting wchar_t string might contain null characters, which
would cause the string to be truncated when used with most C
functions. If size is NULL and the wchar_t* string contains null
characters a ValueError is raised."
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Wed, 21 Nov 2018 12:34:55 +0000 (12:34 +0000)]
s4/dsdb/tests/python: partial PY3 port for samba4.ldap.sort
Test still fails after applying the following fixes
a) only decode byte or (py2) str objects
b) fix sorted function no longer use cmp func, use compat
cmp_to_key_fn instead
c) convert ldb.bytes returned from ldb search results
d) convert sort_functions.keys() to list so array slicing works
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Tue, 20 Nov 2018 17:02:25 +0000 (17:02 +0000)]
python/samba: PY3 port samba.tests.samba_tool.visualize_drs
* Fix calling samba-tool with correct PYTHON version
* Fix integer division needs '//' operator (this was causing
'uncaught exception - list indices must be integers or slices,
not float'
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Tue, 20 Nov 2018 16:20:49 +0000 (16:20 +0000)]
python/samba/netcmd: PY3 port for samba4.drs.samba_tool_drs_showrepl
Fix various ldb.bytes that need to be stringified in order to get
tests to pass
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Tue, 20 Nov 2018 15:30:07 +0000 (15:30 +0000)]
s4/dsdb/tests/python: PY3 port samba4.deletetest
Fix misc ldb.bytes needing to be stringified for tests to succeeed
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Tue, 20 Nov 2018 15:10:35 +0000 (15:10 +0000)]
s4/dsdb/tests/python: PY3 Port samba4.ldap.password_lockout
use Exception.args member as exception is no longer supports
indexing.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Tue, 20 Nov 2018 14:41:20 +0000 (14:41 +0000)]
s4/dsdb/tests/python: PY3 port for samba4.ldap.rodc_rwdc.python
* Make sure samba-tool is called with correct PYTHON version
* Convert ldb.bytes results to string for comparison with expected
results
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Tue, 20 Nov 2018 14:05:58 +0000 (14:05 +0000)]
s4/dsdb/tests/python: PY3 port for samba4.ldap.linked_attributes
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Tue, 20 Nov 2018 13:10:25 +0000 (13:10 +0000)]
s4/dsdb/tests: PY3 port samba4.ldap.vlv
Misc changes mostly around use of ldb.bytes, bytes and string to get
the test to work. Additionally results of a range cannot be combined
with a list using the '+' operator (need to enclose the range with
list).
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Mon, 19 Nov 2018 18:38:46 +0000 (18:38 +0000)]
s4/dsdb/tests/python: PY3 port samba4.ldap.sites
Fix comparison of ldb.bytes with string
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Mon, 19 Nov 2018 18:29:23 +0000 (18:29 +0000)]
s4/dsdb/tests: PY3 fix failing samba4.ldap.notification
Convert ldb.bytes object to string for further processing
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Mon, 19 Nov 2018 18:07:16 +0000 (18:07 +0000)]
lib/ldb-samba/tests: PY3 port samba4.ldap.match_rules
Various fixes
a) schema_format_value can return bytes (esp. for objectSid,
objectGUID
b) embedded NULL now cause ValueError in py3 (was TypeError in py2)
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Mon, 19 Nov 2018 15:04:21 +0000 (15:04 +0000)]
s4/dsdb/tests: PY3 port samba4.user_account_control test
Fix dict_items are not combinable with '+' operator error, need
to convert to list first.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Mon, 19 Nov 2018 14:04:39 +0000 (14:04 +0000)]
s4/dsdb/tests/python: PY3 port samba4.tokengroups.krb5
gensec.Security.update takes bytes as param not string with py3
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Mon, 19 Nov 2018 11:20:29 +0000 (11:20 +0000)]
s4/dsdb/tests: Port PY3 ldap tests
Fix various assertTrue/assertEquals so ldb.bytes (PY3)
comparaisons work both in PY3 & PY2
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Mon, 19 Nov 2018 10:50:29 +0000 (10:50 +0000)]
python/samba/blackbox: PY3 port for samba.tests.blackbox.traffic_learner
The order of the values in the TrafficModel is different,
but... also unfortunately output of json.dump is also
different (even when using sorted versions of the associated
dictionaries before dumping), these changes reimport the output
files into TrafficModel objects rather than comparing the actual
raw files.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Mon, 19 Nov 2018 10:49:40 +0000 (10:49 +0000)]
script: PY3 port traffic_learner
Use python3 compatable print
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Mon, 19 Nov 2018 09:39:06 +0000 (09:39 +0000)]
python/samba/tests: PY3 make sure traffic_learner is called with correct python
Also path to traffic_learner is not in the normal 'bin' path so
also adjusted the insertion of PYTHON version to cover this
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Fri, 16 Nov 2018 19:58:50 +0000 (19:58 +0000)]
python/samba/test: Make sure traffic_replay is called with correct python
PY3 fix samba4.blackbox.rfc2307_mapping
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Fri, 16 Nov 2018 15:06:52 +0000 (15:06 +0000)]
CI: Remove build_samba_ad_dc_py3 CI job
We now run a purepython3 ad-dc test job, later when the whole
build is running under python3 we will resurrect build_samba_ad_dc_py3
but as (build_samba_ad_dc_py2) for python2
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Fri, 16 Nov 2018 15:05:56 +0000 (15:05 +0000)]
CI: Add new CI job for new build_samba_purepy3-ad-dc job
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Fri, 16 Nov 2018 14:59:01 +0000 (14:59 +0000)]
script: Add new (temporary) pure python3 ad-dc test
Ideally we want all the tests to run under python3 by default (no
special task for this) and then convert the existing '-py3' tasks
to run the python tests with python3.
However at the moment the convertion process is not ready to do this,
for a while we need to run separate autobuild tasks for this.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Fri, 16 Nov 2018 12:46:16 +0000 (12:46 +0000)]
python/samba/tests: Fix auth_log messaging problems in py3
Some tests (especially samba.tests.auth_log_netlogon_bad_creds) are
failing due to not receiving expected messages. There seems to be
some timing issue or race around the messaging bus being set up and
getting the expected events resulting from the failed netlogon.
Specifically the the order of destruction of the messaging.Messaging()
c-py objects is different under python2. Under python2 all of the
messaging.Messaging() objects are destructed *after* all the tests
are run. Note: each instance of the TestCase has it's own Messaging()
instance which is created by TestCaseXYZ.setUp, so it appears the unittest
destroys the test instances when all the tests have run whereas in
python3 we see each messaging.Messaging() instance destroyed after
each test runs.
Ok, what difference does that make ? well it seems in python3 because
each Messaging() instance is destructed after a test runs that the
associated messaging_dgm_destroy() also runs, this destroys the
global_dgm_context context which means when the next test runs the whole
messaging infrastructure needs to be built again when the next Messaging()
object is created. On the server-side this seems to result in attempts
to send messages to the listener failing first with
get_event_server: Failed to find 'auth_event' registered on the message bus to send JSON audit events to: NT_STATUS_CONNECTION_REFUSED
and subsequently with
get_event_server: Failed to find 'auth_event' registered on the message bus to send JSON audit events to: NT_STATUS_UNSUCCESSFUL
client doesn't get any more messages, test fails :-(
So, what's the difference in python2, well because the destructors for the
(4 in the case of netlogon_bad_creds) instances of Messagaging() don't run
till the end of the tests this doesn't happen and the global_dgm_context
never gets destroyed untill all the tests complete. There is some race
condition at play here, a simple sleep at the start of a failing test
fixes the problem. But... ok that isn't a possible solution here, instead
I have adjusted the base auth tests to store the Messaging() objects in a
global list forcing them to remain in scope until the tests are complete.
This ensure the behaviour is consistent across python2 & python3.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Thu, 15 Nov 2018 16:11:09 +0000 (16:11 +0000)]
python/samba/tests: PY3 port failing samba.tests.auth_log_samlogon.py
Make sure correctly encode password to utf16 and not use
unicode (which doesn't exist in PY3)
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Thu, 15 Nov 2018 16:06:15 +0000 (16:06 +0000)]
python/samba/tests: PY3 port failing samba.tests.auth_log_netlogon
Fix password encoding
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Wed, 14 Nov 2018 16:29:07 +0000 (16:29 +0000)]
python/samba: PY3 fix failing py3 samba.tests.group_audit test
Fix bytes being compared against ldb.bytes
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Thu, 8 Nov 2018 18:47:59 +0000 (18:47 +0000)]
python/samba/kcc: PY3 fix some str versus ldb.bytes comparisons
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Tue, 6 Nov 2018 19:58:48 +0000 (19:58 +0000)]
python/samba: PY3 don't call str for bytes (or str)
Note: Fix needed also for gpo.apply
minPwdAge, maxPwdAge, minPwdLength & set_pwdProperties all
have a line like
value = str(value).encode('utf8')
this is a generic type statement I guess to convert int, float etc
to utf8 encoded bytes representing the string value for those.
This worked fine in PY2 but in py3 some routine already are passing
bytes into these methods, in these cases e.g. b'200' will get converted
to "b'200'", this change only performs the conversion above for non
bytes (or str) types by replacing the above with
if not isinstance(value, binary_type):
value = str(value).encode('utf8')
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Tue, 6 Nov 2018 19:55:22 +0000 (19:55 +0000)]
python/samba: PY3 port gpo.apply smbtorture test
1) configparser.set requires string values
2) self.gp_db.store() etc. neex to pass str object for
xml.etree.ElementTree.Element text attribute which needs
to be text
3) tdb delete method needs bytes key
4) configparser.write needs a file opened in text mode
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Tue, 6 Nov 2018 19:50:00 +0000 (19:50 +0000)]
s4/torture/gpo: Use existing code to 'run' external commands
Noticed when the smb.conf defined 'gpo update command' contained
the $PYTHON version then the exec_wait function failed to run the
command. Seems there is some issue with the arg handling. Also
there is already existing code (samba_runcmd_send) that works fine
in similar situation (e.g. when running dnsupdate etc.) so replaced
the homebrewed exec_wait functionality with the samba_runcmd util
function.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Tue, 6 Nov 2018 19:47:14 +0000 (19:47 +0000)]
python/samba: fix default params for PY3 ConfigParser
The default params for the python3 version of the compat ConfigParser
are not correct. Code like
foo = ConfigParser()
fails because of this.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Mon, 5 Nov 2018 21:03:39 +0000 (21:03 +0000)]
s4/dsdb/tests: PY3 port samba4.ldap.passwordsettings test
* Fix various assertEquals comparing ldb.bytes with string
when running with PY3
* Fix a couple of tuple assignments to exception (not supported
in PY3)
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Mon, 5 Nov 2018 20:43:42 +0000 (20:43 +0000)]
s4/dsdb/tests/python: PY3 port samba4.sam.python test
Misc changes to ensure samba4.sam.python test will run under
python2/python3
* various objectSID values when formatted need to be treated
as strings for tests.
* DOMAIN_RID_USERS, DOMAIN_RID_DOMAIN_MEMBERS, DOMAIN_RID_DCS,
DOMAIN_RID_READONLY_DCS are all integers (but attibutes are
ldb.bytes in PY3, need to adust various assertEquals.
* Make sure password is encoded correctly
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Mon, 5 Nov 2018 13:56:48 +0000 (13:56 +0000)]
python/samba/netcmd: PY3 fix samba4.blackbox.trust_utils test
In python3 we are using ldb.bytes where we need strings
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Mon, 5 Nov 2018 09:38:23 +0000 (09:38 +0000)]
python/samba/tests: Py3 port for samba.tests.auth_log_netlogon_bad_creds.samba
fix unicode doesn't exist error in PY3
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Wed, 28 Nov 2018 14:15:23 +0000 (14:15 +0000)]
PY3: wrap filter calls with list where list is expected
filter in PY2 returns list in PY3 it returns an iterator
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Sun, 4 Nov 2018 17:32:19 +0000 (17:32 +0000)]
python/samba/tests: PY3 port samba.tests.dns
Misc hanges needed to get make test TEST=samba.tests.dns &
samb.tests.dns_fowarder to run and pass under PY3
* socket.send needs bytes not string
* rec.dwTimeStamp expects int not float (in PY3 / operator
will give float results, for int use '//' instead)
* re.match using bytes needs a bytes search term
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Wed, 28 Nov 2018 14:06:54 +0000 (14:06 +0000)]
PY3: net.change_password & net.set_password take string not bytes
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Wed, 28 Nov 2018 13:58:49 +0000 (13:58 +0000)]
python/samba: PY3 Credential.set_password takes string
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Thu, 8 Nov 2018 15:03:52 +0000 (15:03 +0000)]
auth/credentials: PY3 set_password should decode from unicode 'utf8'
set_password processes input using ParseTuple with "s" format, this
accepts string or unicode but...
Some py2 code is incorrectly using code like
credentials.set_password(pass.encode('utf8'))
however that won't work in PY3. We should just make sure the string
retrieved from unicode passed in is encoded with 'utf8'
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Wed, 28 Nov 2018 15:35:33 +0000 (15:35 +0000)]
python/samba: Py3 Use new compat.sockerserver symbol
SocketServer symbol changed in PY3 to socketserver so
we need to use a compat symbol for PY2/PY3 code.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Wed, 28 Nov 2018 15:33:28 +0000 (15:33 +0000)]
python/samba: PY3 compat py2/p3 symbol for SocketServer/socketserver
SocketServer was renamed to socketserver in Py3, this patch
create a samba.compat.SocketServer which can be used in py2 or
py3
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Fri, 2 Nov 2018 16:16:39 +0000 (16:16 +0000)]
python/samba: PY3 ord needs 'str' type not int
string_to_byte_array returns not a bytearray (as the name suggests)
but a list of byte values (int). Some code expects the list so even
using a 'real' bytearray wont work.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Tue, 20 Nov 2018 15:33:32 +0000 (15:33 +0000)]
s4/setup/tests: make sure samba-tool is called with correct py version
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Fri, 16 Nov 2018 19:32:31 +0000 (19:32 +0000)]
s4/utils/test: PY3 make sure we call correct python version for samba-tool
fixes samba4.blackbox.samba_tool blackbox test
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Fri, 16 Nov 2018 19:44:08 +0000 (19:44 +0000)]
nsswitch/tests: PY3 samba_tool call correct python
fix samba4.blackbox.rfc2307_mapping
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Mon, 5 Nov 2018 14:12:24 +0000 (14:12 +0000)]
testprogs/blackbox: make sure samba-tool is called with correct python
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Noel Power [Fri, 2 Nov 2018 15:42:11 +0000 (15:42 +0000)]
python/samba.tests: Ensure samba-tool is called with correct python ver.
* remove unnecessary 'bin/' part of path as base BlackBox class
will do this anyway and also ensure correct detection that
command needs to have 'PYTHON=blah'
addeded
* modify shell script so PYTHON variable if set is prepended
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Christof Schmitt [Mon, 22 May 2017 23:28:40 +0000 (16:28 -0700)]
ctdb: Remove <file> parameter from pfetch usage info
The code does not implement saving the record data to a file, so update
the usage info accordingly.
Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Mon Dec 10 05:02:13 CET 2018 on sn-devel-144
Christof Schmitt [Mon, 22 May 2017 19:31:35 +0000 (12:31 -0700)]
ctdb: Fix hex to int conversion in h2i
Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Volker Lendecke [Fri, 7 Dec 2018 09:32:08 +0000 (10:32 +0100)]
rpcclient: Use dom_sid_str_buf
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Dec 8 02:43:48 CET 2018 on sn-devel-144
Volker Lendecke [Fri, 7 Dec 2018 09:25:09 +0000 (10:25 +0100)]
libnet: Use dom_sid_str_buf
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Volker Lendecke [Fri, 7 Dec 2018 09:20:38 +0000 (10:20 +0100)]
net: Use dom_sid_str_buf
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Volker Lendecke [Thu, 6 Dec 2018 19:36:09 +0000 (20:36 +0100)]
pdb_ldap: Use dom_sid_str_buf
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Volker Lendecke [Thu, 6 Dec 2018 18:14:20 +0000 (19:14 +0100)]
winbindd: Use dom_sid_str_buf
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Volker Lendecke [Thu, 6 Dec 2018 18:12:07 +0000 (19:12 +0100)]
idmap_tdb: Use dom_sid_str_buf
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Volker Lendecke [Thu, 6 Dec 2018 17:25:02 +0000 (18:25 +0100)]
winbindd_cache: Use dom_sid_str_buf
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Volker Lendecke [Thu, 6 Dec 2018 17:20:06 +0000 (18:20 +0100)]
net_usershare: Use dom_sid_str_buf
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Volker Lendecke [Thu, 6 Dec 2018 16:53:24 +0000 (17:53 +0100)]
net_rpc: Use dom_sid_str_buf
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Volker Lendecke [Thu, 6 Dec 2018 16:43:05 +0000 (17:43 +0100)]
net_rpc: Use dom_sid_equal where appropriate
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Swen Schillig [Mon, 12 Mar 2018 16:56:21 +0000 (17:56 +0100)]
ctdb: Adding memory pool for queue callback
The received packet is copied into a newly allocated memory chunk for further
processing by the assigned callback. Once this is done, the memory is free'd.
This is repeated for each received packet making the memory allocation / free
an expensive task. To optimize this process, a memory pool is defined which
is sized identically to the queue's buffer.
During tests it could be seen that more than 95% of all messages were sized
below the standard buffer_size of 1k.
Signed-off-by: Swen Schillig <swen@vnet.ibm.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Christof Schmitt <cs@samba.org>
Autobuild-User(master): Christof Schmitt <cs@samba.org>
Autobuild-Date(master): Fri Dec 7 23:27:16 CET 2018 on sn-devel-144
Swen Schillig [Mon, 12 Mar 2018 10:00:55 +0000 (11:00 +0100)]
ctdb: Introduce buffer.offset to avoid memmove
The memmove operation is quite expensive, therefore,
a new buffer attribute "offset" is introduced to support
an optimized buffer processing.
The optimization is to "walk" through the buffer and process
each packet until the buffer is fully processed (empty)
without requiring any memmove.
Only if a packet is in-complete, the buffer content is moved
and the new data is read from the queue.
This way almost all memmove operations are eliminated.
Signed-off-by: Swen Schillig <swen@vnet.ibm.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Christof Schmitt <cs@samba.org>
Andreas Schneider [Thu, 6 Dec 2018 08:35:15 +0000 (09:35 +0100)]
librpc:ndr: Give the optimizer hints for ndr_push_bytes()
Also remove the redundant check in ndr_push_DATA_BLOB.
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Fri Dec 7 15:33:38 CET 2018 on sn-devel-144
Daniel Southward-Ellis [Mon, 3 Dec 2018 01:42:14 +0000 (14:42 +1300)]
Removed dead groups link from Mail List Etiquette
Signed-off-by: Daniel Southward-Ellis <danielsouthwardellis@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Dec 7 10:12:42 CET 2018 on sn-devel-144
Daniel Southward-Ellis [Mon, 3 Dec 2018 01:36:15 +0000 (14:36 +1300)]
Updated "About Samba" Info in README
Signed-off-by: Daniel Southward-Ellis <danielsouthwardellis@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Daniel Southward-Ellis [Sun, 2 Dec 2018 23:43:27 +0000 (12:43 +1300)]
Changed GitHub info to GitLab
Signed-off-by: Daniel Southward-Ellis <danielsouthwardellis@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Daniel Southward-Ellis [Sun, 2 Dec 2018 22:57:04 +0000 (11:57 +1300)]
Updated Website section
Signed-off-by: Daniel Southward-Ellis <danielsouthwardellis@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Daniel Southward-Ellis [Sun, 2 Dec 2018 22:55:03 +0000 (11:55 +1300)]
Added link to How to do Samba: Nicely
Signed-off-by: Daniel Southward-Ellis <danielsouthwardellis@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Daniel Southward-Ellis [Sun, 2 Dec 2018 22:52:30 +0000 (11:52 +1300)]
Changed web page to webpage
Signed-off-by: Daniel Southward-Ellis <danielsouthwardellis@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>