buildtools/wafsamba: Decode result of Popen as unicode/string

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

adjust ldb config for py3 build

waf code is making assumptions that main build is py2 and extra build
is py3, this results in wrong library names being used.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

PY3 pure build fix some str/bytes wobblies

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

CI: Remove the remainder py3 jobs

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

autobuild: Remove temporary purepy3 tasks

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

auth/credentials/tests: Python 3.6 avoid deepcopy error

In PY3 both deepcopy & (shallow)copy fail with

  Traceback (most recent call last):
    File "auth/credentials/tests/bind.py", line 42, in <module>
      creds_machine = copy.copy(creds)
    File "/usr/lib64/python3.6/copy.py", line 96, in copy
      rv = reductor(4)
  TypeError: can't pickle credentials.Credentials objects

This patch avoids the nasty copies but creating and populating the
Credential objects instead of copying

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/selftest: Reenable samba.tests.gpo to run with --extra-python

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

CI: Remove purepy3 temporary jobs

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

CI: Add new CI jobs for samba-purepy3 & samba-purepy3-nt4

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

CI: Add new (TEMP) pure python3 autobuild jobs for samba-build & samba-nt4

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/dsdb/tests/python: PY3 allow test samba4.ldap.sort pass

Adjust test data to remove use of embedded NULLs.

We are getting the following exception when running the test
under python3

Exception: Exception: Traceback (most recent call last):
  File "source4/dsdb/tests/python/sort.py", line 181, in setUp
    key=cmp_to_key_fn(locale.strcoll))
ValueError: embedded null character

Looking at the source code for locale.strcoll the lhs & rhs
params are processed using 'PyUnicode_AsWideCharString(s, NULL)'

Documentation for PyUnicode_AsWideCharString states:

"Convert the Unicode object to a wide character string.
 The output string always ends with a null character. If size is
 not NULL, write the number of wide characters (excluding the
 trailing null termination character) into *size. Note that the
 resulting wchar_t string might contain null characters, which
 would cause the string to be truncated when used with most C
 functions. If size is NULL and the wchar_t* string contains null
 characters a ValueError is raised."

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/dsdb/tests/python: partial PY3 port for samba4.ldap.sort

Test still fails after applying the following fixes

a) only decode byte or (py2) str objects
b) fix sorted function no longer use cmp func, use compat
   cmp_to_key_fn instead
c) convert ldb.bytes returned from ldb search results
d) convert sort_functions.keys() to list so array slicing works

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/samba: PY3 port samba.tests.samba_tool.visualize_drs

* Fix calling samba-tool with correct PYTHON version
* Fix integer division needs '//' operator (this was causing
  'uncaught exception - list indices must be integers or slices,
   not float'

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/samba/netcmd: PY3 port for samba4.drs.samba_tool_drs_showrepl

Fix various ldb.bytes that need to be stringified in order to get
tests to pass

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/dsdb/tests/python: PY3 port samba4.deletetest

Fix misc ldb.bytes needing to be stringified for tests to succeeed

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/dsdb/tests/python: PY3 Port samba4.ldap.password_lockout

use Exception.args member as exception is no longer supports
indexing.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/dsdb/tests/python: PY3 port for samba4.ldap.rodc_rwdc.python

* Make sure samba-tool is called with correct PYTHON version
* Convert ldb.bytes results to string for comparison with expected
  results

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/dsdb/tests/python: PY3 port for samba4.ldap.linked_attributes

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/dsdb/tests: PY3 port samba4.ldap.vlv

Misc changes mostly around use of ldb.bytes, bytes and string to get
the test to work. Additionally results of a range cannot be combined
with a list using the '+' operator (need to enclose the range with
list).

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/dsdb/tests/python: PY3 port samba4.ldap.sites

Fix comparison of ldb.bytes with string
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/dsdb/tests: PY3 fix failing samba4.ldap.notification

Convert ldb.bytes object to string for further processing

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

lib/ldb-samba/tests: PY3 port samba4.ldap.match_rules

Various fixes

a) schema_format_value can return bytes (esp. for objectSid,
   objectGUID
b) embedded NULL now cause ValueError in py3 (was TypeError in py2)

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/dsdb/tests: PY3 port samba4.user_account_control test

Fix dict_items are not combinable with '+' operator error, need
to convert to list first.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/dsdb/tests/python: PY3 port samba4.tokengroups.krb5

gensec.Security.update takes bytes as param not string with py3

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/dsdb/tests: Port PY3 ldap tests

Fix various assertTrue/assertEquals so ldb.bytes (PY3)
comparaisons work both in PY3 & PY2

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/samba/blackbox: PY3 port for samba.tests.blackbox.traffic_learner

The order of the values in the TrafficModel is different,
but... also unfortunately output of json.dump is also
different (even when using sorted versions of the associated
dictionaries before dumping), these changes reimport the output
files into TrafficModel objects rather than comparing the actual
raw files.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

script: PY3 port traffic_learner

Use python3 compatable print

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/samba/tests: PY3 make sure traffic_learner is called with correct python

Also path to traffic_learner is not in the normal 'bin' path so
also adjusted the insertion of PYTHON version to cover this

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/samba/test: Make sure traffic_replay is called with correct python

PY3 fix samba4.blackbox.rfc2307_mapping

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

CI: Remove build_samba_ad_dc_py3 CI job

We now run a purepython3 ad-dc test job, later when the whole
build is running under python3 we will resurrect build_samba_ad_dc_py3
but as (build_samba_ad_dc_py2) for python2

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

CI: Add new CI job for new build_samba_purepy3-ad-dc job

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

script: Add new (temporary) pure python3 ad-dc test

Ideally we want all the tests to run under python3 by default (no
special task for this) and then convert the existing '-py3' tasks
to run the python tests with python3.
However at the moment the convertion process is not ready to do this,
for a while we need to run separate autobuild tasks for this.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/samba/tests: Fix auth_log messaging problems in py3

Some tests (especially samba.tests.auth_log_netlogon_bad_creds) are
failing due to not receiving expected messages. There seems to be
some timing issue or race around the messaging bus being set up and
getting the expected events resulting from the failed netlogon.

Specifically the the order of destruction of the messaging.Messaging()
c-py objects is different under python2. Under python2 all of the
messaging.Messaging() objects are destructed *after* all the tests
are run. Note: each instance of the TestCase has it's own Messaging()
instance which is created by TestCaseXYZ.setUp, so it appears the unittest
destroys the test instances when all the tests have run whereas in
python3 we see each messaging.Messaging() instance destroyed after
each test runs.
Ok, what difference does that make ? well it seems in python3 because
each Messaging() instance is destructed after a test runs that the
associated messaging_dgm_destroy() also runs, this destroys the
global_dgm_context context which means when the next test runs the whole
messaging infrastructure needs to be built again when the next Messaging()
object is created. On the server-side this seems to result in attempts
to send messages to the listener failing first with

get_event_server: Failed to find 'auth_event' registered on the message bus to send JSON audit events to: NT_STATUS_CONNECTION_REFUSED

and subsequently with

get_event_server: Failed to find 'auth_event' registered on the message bus to send JSON audit events to: NT_STATUS_UNSUCCESSFUL

client doesn't get any more messages, test fails :-(

So, what's the difference in python2, well because the destructors for the
(4 in the case of netlogon_bad_creds) instances of Messagaging() don't run
till the end of the tests this doesn't happen and the global_dgm_context
never gets destroyed untill all the tests complete. There is some race
condition at play here, a simple sleep at the start of a failing test
fixes the problem. But... ok that isn't a possible solution here, instead
I have adjusted the base auth tests to store the Messaging() objects in a
global list forcing them to remain in scope until the tests are complete.
This ensure the behaviour is consistent across python2 & python3.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/samba/tests: PY3 port failing samba.tests.auth_log_samlogon.py

Make sure correctly encode password to utf16 and not use
unicode (which doesn't exist in PY3)

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/samba/tests: PY3 port failing samba.tests.auth_log_netlogon

Fix password encoding

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/samba: PY3 fix failing py3 samba.tests.group_audit test

Fix bytes being compared against ldb.bytes

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/samba/kcc: PY3 fix some str versus ldb.bytes comparisons

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/samba: PY3 don't call str for bytes (or str)

Note: Fix needed also for gpo.apply

minPwdAge, maxPwdAge, minPwdLength & set_pwdProperties all
have a line like

value = str(value).encode('utf8')

this is a generic type statement I guess to convert int, float etc
to utf8 encoded bytes representing the string value for those.

This worked fine in PY2 but in py3 some routine already are passing
bytes into these methods, in these cases e.g. b'200' will get converted
to "b'200'", this change only performs the conversion above for non
bytes (or str) types by replacing the above with

        if not isinstance(value, binary_type):
            value = str(value).encode('utf8')

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/samba: PY3 port gpo.apply smbtorture test

1) configparser.set requires string values
2) self.gp_db.store() etc. neex to pass str object for
   xml.etree.ElementTree.Element text attribute which needs
   to be text
3) tdb delete method needs bytes key
4) configparser.write needs a file opened in text mode

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/torture/gpo: Use existing code to 'run' external commands

Noticed when the smb.conf defined 'gpo update command' contained
the $PYTHON version then the exec_wait function failed to run the
command. Seems there is some issue with the arg handling. Also
there is already existing code (samba_runcmd_send) that works fine
in similar situation (e.g. when running dnsupdate etc.) so replaced
the homebrewed exec_wait functionality with the samba_runcmd util
function.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/samba: fix default params for PY3 ConfigParser

The default params for the python3 version of the compat ConfigParser
are not correct. Code like
   foo = ConfigParser()
fails because of this.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/dsdb/tests: PY3 port samba4.ldap.passwordsettings test

* Fix various assertEquals comparing ldb.bytes with string
  when running with PY3
* Fix a couple of tuple assignments to exception (not supported
  in PY3)

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/dsdb/tests/python: PY3 port samba4.sam.python test

Misc changes to ensure samba4.sam.python test will run under
python2/python3

* various objectSID values when formatted need to be treated
as strings for tests.

* DOMAIN_RID_USERS, DOMAIN_RID_DOMAIN_MEMBERS, DOMAIN_RID_DCS,
  DOMAIN_RID_READONLY_DCS are all integers (but attibutes are
  ldb.bytes in PY3, need to adust various assertEquals.
* Make sure password is encoded correctly

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/samba/netcmd: PY3 fix samba4.blackbox.trust_utils test

In python3 we are using ldb.bytes where we need strings
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/samba/tests: Py3 port for samba.tests.auth_log_netlogon_bad_creds.samba

fix unicode doesn't exist error in PY3

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

PY3: wrap filter calls with list where list is expected

filter in PY2 returns list in PY3 it returns an iterator

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/samba/tests: PY3 port samba.tests.dns

Misc hanges needed to get make test TEST=samba.tests.dns &
samb.tests.dns_fowarder to run and pass under PY3

* socket.send needs bytes not string
* rec.dwTimeStamp expects int not float (in PY3 / operator
  will give float results, for int use '//' instead)
* re.match using bytes needs a bytes search term

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

PY3: net.change_password & net.set_password take string not bytes

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/samba: PY3 Credential.set_password takes string

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

auth/credentials: PY3 set_password should decode from unicode 'utf8'

set_password processes input using ParseTuple with "s" format, this
accepts string or unicode but...

Some py2 code is incorrectly using code like

   credentials.set_password(pass.encode('utf8'))

however that won't work in PY3. We should just make sure the string
retrieved from unicode passed in is encoded with 'utf8'
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/samba: Py3 Use new compat.sockerserver symbol

SocketServer symbol changed in PY3 to socketserver so
we need to use a compat symbol for PY2/PY3 code.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/samba: PY3 compat py2/p3 symbol for SocketServer/socketserver

SocketServer was renamed to socketserver in Py3, this patch
create a samba.compat.SocketServer which can be used in py2 or
py3

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/samba: PY3 ord needs 'str' type not int

string_to_byte_array returns not a bytearray (as the name suggests)
but a list of byte values (int). Some code expects the list so even
using a 'real' bytearray wont work.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/setup/tests: make sure samba-tool is called with correct py version

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/utils/test: PY3 make sure we call correct python version for samba-tool

fixes samba4.blackbox.samba_tool blackbox test

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

nsswitch/tests: PY3 samba_tool call correct python

fix samba4.blackbox.rfc2307_mapping

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

testprogs/blackbox: make sure samba-tool is called with correct python

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/samba.tests: Ensure samba-tool is called with correct python ver.

* remove unnecessary 'bin/' part of path as base BlackBox class
  will do this anyway and also ensure correct detection that
  command needs to have 'PYTHON=blah' addeded
* modify shell script so PYTHON variable if set is prepended

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

ctdb: Remove <file> parameter from pfetch usage info

The code does not implement saving the record data to a file, so update
the usage info accordingly.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Mon Dec 10 05:02:13 CET 2018 on sn-devel-144

ctdb: Fix hex to int conversion in h2i

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

rpcclient: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Dec  8 02:43:48 CET 2018 on sn-devel-144

libnet: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

net: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

pdb_ldap: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

winbindd: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

idmap_tdb: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

winbindd_cache: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

net_usershare: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

net_rpc: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

net_rpc: Use dom_sid_equal where appropriate

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

ctdb: Adding memory pool for queue callback

The received packet is copied into a newly allocated memory chunk for further
processing by the assigned callback. Once this is done, the memory is free'd.
This is repeated for each received packet making the memory allocation / free
an expensive task. To optimize this process, a memory pool is defined which
is sized identically to the queue's buffer.
During tests it could be seen that more than 95% of all messages were sized
below the standard buffer_size of 1k.

Signed-off-by: Swen Schillig <swen@vnet.ibm.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Christof Schmitt <cs@samba.org>
Autobuild-User(master): Christof Schmitt <cs@samba.org>
Autobuild-Date(master): Fri Dec  7 23:27:16 CET 2018 on sn-devel-144

ctdb: Introduce buffer.offset to avoid memmove

The memmove operation is quite expensive, therefore,
a new buffer attribute "offset" is introduced to support
an optimized buffer processing.
The optimization is to "walk" through the buffer and process
each packet until the buffer is fully processed (empty)
without requiring any memmove.
Only if a packet is in-complete, the buffer content is moved
and the new data is read from the queue.
This way almost all memmove operations are eliminated.

Signed-off-by: Swen Schillig <swen@vnet.ibm.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Christof Schmitt <cs@samba.org>

librpc:ndr: Give the optimizer hints for ndr_push_bytes()

Also remove the redundant check in ndr_push_DATA_BLOB.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Fri Dec  7 15:33:38 CET 2018 on sn-devel-144

Removed dead groups link from Mail List Etiquette

Signed-off-by: Daniel Southward-Ellis <danielsouthwardellis@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Dec  7 10:12:42 CET 2018 on sn-devel-144

Updated "About Samba" Info in README

Signed-off-by: Daniel Southward-Ellis <danielsouthwardellis@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Changed GitHub info to GitLab

Signed-off-by: Daniel Southward-Ellis <danielsouthwardellis@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Updated Website section

Signed-off-by: Daniel Southward-Ellis <danielsouthwardellis@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Added link to How to do Samba: Nicely

Signed-off-by: Daniel Southward-Ellis <danielsouthwardellis@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Changed web page to webpage

Signed-off-by: Daniel Southward-Ellis <danielsouthwardellis@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

ldb: complex expression testing

Tests that prepare complex ldap expressions and equivalent python expressions,
then compare the results of the two.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
Autobuild-User(master): Gary Lockyer <gary@samba.org>
Autobuild-Date(master): Fri Dec  7 07:07:08 CET 2018 on sn-devel-144

wscript_configure_system_mitkrb5: reject a system heimdal krb5-config

Review with: git show -w

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Thu Dec  6 16:53:33 CET 2018 on sn-devel-144

selftest: Don't run KCC on backup testenvs (to avoid flappiness)

KCC onthe backup domain (i.e. backupfromdc, restoredc, offlinebackupdc)
can establish new connections for replication. Depending on timing,
this can cause the join_ldapcmp test to fail, because there's an extra
object under the NTDS Settings, at the point the ldapcmp is done.

We don't need any replication to happen on the backup domain. The
backup/restore workflow in the real world should mean that the restored DC
is never run in the same network as the original DC.

This patch updates the default KCC command for the backup testenvs to be
a no-op, so the DCs won't create new connection objects.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Dec  6 12:03:53 CET 2018 on sn-devel-144

librpc:ndr: Fix undefined behavior in ndr_basic

librpc/ndr/ndr_basic.c:723:2: runtime error: null pointer passed as
argument 2, which is declared to never be null

The following triggered the undefined behavior:

(gdb) bt
    at librpc/gen_ndr/ndr_drsuapi.c:2318
    fn=0x7ffff6e72983 <ndr_push_drsuapi_DsReplicaObjectIdentifier3Binary>) at ../../librpc/ndr/ndr.c:1337
    at ../../source4/dsdb/schema/schema_syntax.c:2136
    drs_str=<optimized out>) at ../../source4/dsdb/schema/tests/schema_syntax.c:122
    already_setup=<optimized out>, restricted=restricted@entry=0x0) at ../../lib/torture/torture.c:442
    at ../../lib/torture/torture.c:507
    suite=0x5555563d9490, matched=0x7fffffffcef7) at ../../source4/torture/smbtorture.c:93
    matched=0x7fffffffcef7) at ../../source4/torture/smbtorture.c:95
    at ../../source4/torture/smbtorture.c:143
(gdb) f 1
1335            NDR_CHECK(ndr_push_bytes(ndr, blob.data, blob.length));
(gdb) p blob
$2 = {data = 0x0, length = 0}

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Gary Lockyer <gary@samba.org>
Autobuild-Date(master): Thu Dec  6 08:48:28 CET 2018 on sn-devel-144

WHATSNEW: document changes in SMB server parametric options

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Dec  6 05:14:03 CET 2018 on sn-devel-144

smbd: use lp_smbd_getinfo_ask_sharemode()

Counterpart for "smbd:search ask sharemode" for getinfo.

Pair-Programmed-With: Volker Lendecke <vl@samba.org>
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

docs-xml: add "smbd getinfo ask sharemode"

Counterpart for "smbd search ask sharemode" for getinfo.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:smbd: use lp_smbd_max_async_dosmode()

Parametric options have a performance impact, use the normal options
added in the previous commit.

"aio max threads" can only be calculated at run time and requires a
handle to a pthreadpool_tevent which loadparm will never have.

Because of that lp_smbd_max_async_dosmode() will always return 0 as
default and it's up to us to calculate "aio max threads * 2" if
lp_smbd_max_async_dosmode() returns 0.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

docs-xml: add "smbd max async dosmode"

The parameter is added to the lists of ignored-paremteres in the
samba.docs tests, as the given default "aio max threads * 2" works only
as manpage string.

"aio max threads" can only be calculated at run time and requires a
handle to a pthreadpool_tevent which loadparm will never have.

Because of that lp_smbd_max_async_dosmode() will always return 0 as
default and it's up to the caller to calculate "aio max threads * 2" if
lp_smbd_max_async_dosmode() returns 0. Cf the next commit.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:smbd: use lp_smbd_async_dosmode()

Parametric options have a performance impact, use the normal options
added in the previous commit.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

docs-xml: add "smbd async dosmode"

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:smbd: use lp_smbd_search_ask_sharemode()

Parametric options have a performance impact, use the normal options
added in the previous commit.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

docs-xml: add "smbd search ask sharemode"

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

tests:docs: add a exceptions set

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

tests:docs: reindent special_cases to one by line

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Added redirect from GitHub to GitLab

Signed-off-by: Daniel Southward-Ellis <danielsouthwardellis@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Wed Dec  5 16:35:33 CET 2018 on sn-devel-144

ctdb/wscript: make use of MODE_{644,744,755,777}

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

wafsamba: add MODE_{744,_777}

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

ctdb/wscript: use python 3.6 compatible functions

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

buildtools: remove unused buildtools/bin/waf-1.9

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

winbindd: Route predefined domains through the BUILTIN domain child

Without this eg "NT Authority" didn't work:

  $ bin/wbinfo -n "NT Authority/Authenticated Users"
  failed to call wbcLookupName: WBC_ERR_DOMAIN_NOT_FOUND
  Could not lookup name NT Authority/Authenticated Users

  $ bin/wbinfo --group-info="NT Authority/Authenticated Users"
  failed to call wbcGetgrnam: WBC_ERR_DOMAIN_NOT_FOUND
  Could not get info for group NT Authority/Authenticated Users

With the patch:

  $ bin/wbinfo -n "NT Authority/Authenticated Users"
  S-1-5-11 SID_WKN_GROUP (5)

  $ bin/wbinfo --group-info="NT Authority/Authenticated Users"
  NT AUTHORITY\authenticated users:x:10002:

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12164

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: David Mulder <dmulder@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Wed Dec  5 11:27:22 CET 2018 on sn-devel-144