tests/match_rules: Use system privilege for msDS-RevealedUsers

Must be done before the systemOnly attribute is enforced.

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

dbcheck: Improve dbcheck to find (and may fix) dangling msDS-RevealedUsers

We cannot add missing backlinks because of the duplicate checking. There
seems to be no trivial way to add the bypass.

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

getncchanges: include object SID in tokenGroups calculation for repl secret

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests/repl_rodc: Test the direct allow/deny attribute works

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

getncchanges: Reorder and comment code for clarity

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

getncchanges: Prevent a small, but possible race condition in build_object

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

getncchanges: Refactor filter_attrs from build_object

This makes it easier to have a transaction around it.

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

getncchanges: Tie destination DSA GUID to authenticating RODC for REPL_SECRET

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests/repl_rodc: Ensure that the machine account is tied to the destination DSA

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

getncchanges: Implement functionality for msDS-RevealedUsers

This multi-valued DN+Binary linked attribute is present on the server object
for an RODC. A link to an object is added to it whenever secret
attributes from that object are replicated to an RODC to serve as an
audit trail.

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Pair-programmed-with: Bob Campbell <bobcampbell@catalyst.net.nz>

getncchanges: Do not filter secrets by PAS in EXOP_REPL_SECRET

This conforms with Windows' behaviour.

Signed-off-by: Bob Campbell <bobcampbell@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>

replmd: Include extra data on DN in search if it exists

This is important for multi-valued DN+Binary (or DN+String) attributes,
as otherwise they will be considered duplicates.

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Pair-programmed-with: Bob Campbell <bobcampbell@catalyst.net.nz>

replmd: Ensure that binary blobs in links are ordered in the database

This is required if we are to search them with a binsearch.

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

getncchanges: Let security of RWDC+ manually replicate secrets to RODCs

This correctly passes has_get_all_changes through to repl_secrets.

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Pair-programmed-with: Bob Campbell <bobcampbell@catalyst.net.nz>

drsblobs: Add decode for replPropertyMetaData1

Signed-off-by: Bob Campbell <bobcampbell@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>

tests/repl_rodc: Duplicate msDS-RevealedUsers test for RODC machine acct

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/tests: Add repl_rodc test

Currently, this tests the msDS-RevealedUsers feature, which we don't
support at the moment.

Signed-off-by: Bob Campbell <bobcampbell@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>

getncchanges: Return correct denied REPL_SECRET error code

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

drsbase: use credentials if supplied

Pair-programmed-with: Bob Campbell <bobcampbell@catalyst.net.nz>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/dsdb_dn: Add a generic get_bytes method on DNs

Pair-programmed-with: Bob Campbell <bobcampbell@catalyst.net.nz>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

ldb_tdb: Add better comments for duplicate attr values

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Pair-programmed-with: Andrew Bartlett <abartlet@samba.org>

ldb_tdb: Do not check for duplicate values during a rename

This is not the time to be pretending to be dbcheck, and there are
exceptions to the single-value rules in Samba. This is needed for
the same reasons as the modify case.

(Note: this error was triggered with the demote of an RODC with links)

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Pair-programmed-with: Andrew Bartlett <abartlet@samba.org>

ldb_tdb: Do not care about duplicates if single value check disabled

This behaviour of ignoring duplicates with the flag
LDB_FLAG_INTERNAL_DISABLE_SINGLE_VALUE_CHECK is also used in the replace
case here.

When we add a forward DN+Binary link with a duplicate DN, this prevents
us from not being able to add the backlink because it appears to be a
duplicate here.

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Pair-programmed-with: Bob Campbell <bobcampbell@catalyst.net.nz>

samba-tool/domain: Correctly re-enable replication

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

werror: Correct the error code checking

Broken in commit ea3c3f10edac2b6e7e1900b4e75f4be4d70d369a

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

typo: uppon -> upon

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Correct "ommited" typos.

Signed-off-by: Chris Lamb <chris@chris-lamb.co.uk>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

doc: update "ea support" section of the smb.conf manpage

This section was badly outdated.

Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Sun Mar 12 21:04:11 CET 2017 on sn-devel-144

winbindd: avoid multiple wbint_LookupSids/lsa_LookupSids calls to the same domain

find_lookup_domain_from_sid() returns the same domain for all non local
sids on a domain member. We should not chunk one wb_lookupsids_send/recv
into multiple wbint_LookupSids_send/recv to the same 'lookup' domain,
just because the requested SIDs don't all belong to the same domain.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Sun Mar 12 00:56:14 CET 2017 on sn-devel-144

winbindd: remove unused find_root_domain()

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

winbindd: remove bogus fallback to the forest root in wb_lookupsid*()

It's the job of the domain controller in our domain
to traverse the trust chain.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

winbindd: remove bogus fallback to the forest root in wb_lookupname*()

It's the job of the domain controller in our domain
to traverse the trust chain.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

selftest: Do not plan samba3.base.delaywrite twice

This test is already slow. We should not run it twice!

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Sat Mar 11 04:25:14 CET 2017 on sn-devel-144

lib/pthreadpool: fix a memory leak

When copying large files from the server to the client with aio enabled
we noticed that smbd kept growing RSS and VSZ.

valgrind was reporting:

==2503== 4,093,440 bytes in 6,560 blocks are possibly lost in loss record 460 of 460
==2503==    at 0x4C299CE: calloc (vg_replace_malloc.c:711)
==2503==    by 0x4011C24: _dl_allocate_tls (in /usr/lib64/ld-2.17.so)
==2503==    by 0x4E3C960: pthread_create@@GLIBC_2.2.5 (in /usr/lib64/libpthread-2.17.so)
==2503==    by 0x9B298AE: pthreadpool_add_job (in /usr/lib64/samba/libmessages-dgm-samba4.so)
==2503==    by 0x9B29FDC: pthreadpool_tevent_job_send (in /usr/lib64/samba/libmessages-dgm-samba4.so)
==2503==    by 0x56A78EF: ??? (in /usr/lib64/samba/libsmbd-base-samba4.so)
==2503==    by 0x55D86B7: smb_vfs_call_pread_send (in /usr/lib64/samba/libsmbd-base-samba4.so)
==2503==    by 0x55F7543: schedule_smb2_aio_read (in /usr/lib64/samba/libsmbd-base-samba4.so)
==2503==    by 0x5608F57: smbd_smb2_request_process_read (in /usr/lib64/samba/libsmbd-base-samba4.so)
==2503==    by 0x55FCB6C: smbd_smb2_request_dispatch (in /usr/lib64/samba/libsmbd-base-samba4.so)
==2503==    by 0x55FD7DC: ??? (in /usr/lib64/samba/libsmbd-base-samba4.so)
==2503==    by 0x641B977: ??? (in /usr/lib64/samba/libtevent.so.0.9.31)

The problem seems to be caused by worked threads that are not properly
started in detached state and thus their tls is not reclaimed upon
thread termination.

In pthreadpool.c we prepare a pthread attribute with
PTHREAD_CREATE_DETACHED, but we don't pass it to pthread_create().

Bug: https://bugzilla.samba.org/show_bug.cgi?id=12624

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Fri Mar 10 22:06:02 CET 2017 on sn-devel-144

objectclass_attrs: Remove schema copy shallow from attr_handler2

This appears quite expensive (particularly in provision), and also
unnecessary.

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Fri Mar 10 15:34:39 CET 2017 on sn-devel-144

s4:kdc: disable principal based autodetected referral detection

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12554

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

HEIMDAL:kdc: make it possible to disable the principal based referral detection

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12554

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

s3:gse: Correctly handle external trusts with MIT

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12554

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>

s3:gse: Check if we have a target_princpal set we should use

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12554

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>

s3:gse: Move setup of service_principal to update function

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12554

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>

s3:gse: Pass down the gensec_security pointer

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12554

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>

krb5_wrap: Remove obsolete smb_krb5_get_principal_from_service_hostname()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12554

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s3:gse: Use smb_krb5_get_realm_from_hostname()

With credentials for administrator@FOREST1.EXAMPLE.COM
this patch changes the target_principal for
the ldap service of host dc2.forest2.example.com
from

  ldap/dc2.forest2.example.com@FOREST1.EXAMPLE.COM

to

  ldap/dc2.forest2.example.com@FOREST2.EXAMPLE.COM

Typically ldap/dc2.forest2.example.com@FOREST1.EXAMPLE.COM
should be used in order to allow the KDC of FOREST1.EXAMPLE.COM
to generate a referral ticket for
krbtgt/FOREST2.EXAMPLE.COM@FOREST1.EXAMPLE.COM.

The problem is that KDCs only return such referral tickets
if there's a forest trust between FOREST1.EXAMPLE.COM
and FOREST2.EXAMPLE.COM. If there's only an external domain
trust between FOREST1.EXAMPLE.COM and FOREST2.EXAMPLE.COM
the KDC of FOREST1.EXAMPLE.COM will respond with S_PRINCIPAL_UNKNOWN
when being asked for ldap/dc2.forest2.example.com@FOREST1.EXAMPLE.COM.

In the case of an external trust the client can still ask
explicitly for krbtgt/FOREST2.EXAMPLE.COM@FOREST1.EXAMPLE.COM
and the KDC of FOREST1.EXAMPLE.COM will generate it.

From there the client can use the
krbtgt/FOREST2.EXAMPLE.COM@FOREST1.EXAMPLE.COM
ticket and ask a KDC of FOREST2.EXAMPLE.COM for a
service ticket for ldap/dc2.forest2.example.com@FOREST2.EXAMPLE.COM.

With Heimdal we'll get the fallback on S_PRINCIPAL_UNKNOWN behavior
when we pass ldap/dc2.forest2.example.com@FOREST2.EXAMPLE.COM as
target principal. As _krb5_get_cred_kdc_any() first calls
get_cred_kdc_referral() (which always starts with the client realm)
and falls back to get_cred_kdc_capath() (which starts with the given realm).

MIT krb5 only tries the given realm of the target principal,
if we want to autodetect support for transitive forest trusts,
we'll have to do the fallback ourself.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12554

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>

s4:gensec_gssapi: Correctly handle external trusts with MIT

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12554

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>

s4:gensec_gssapi: Use smb_krb5_get_realm_from_hostname()

With credentials for administrator@FOREST1.EXAMPLE.COM
this patch changes the target_principal for
the ldap service of host dc2.forest2.example.com
from

  ldap/dc2.forest2.example.com@FOREST1.EXAMPLE.COM

to

  ldap/dc2.forest2.example.com@FOREST2.EXAMPLE.COM

Typically ldap/dc2.forest2.example.com@FOREST1.EXAMPLE.COM
should be used in order to allow the KDC of FOREST1.EXAMPLE.COM
to generate a referral ticket for
krbtgt/FOREST2.EXAMPLE.COM@FOREST1.EXAMPLE.COM.

The problem is that KDCs only return such referral tickets
if there's a forest trust between FOREST1.EXAMPLE.COM
and FOREST2.EXAMPLE.COM. If there's only an external domain
trust between FOREST1.EXAMPLE.COM and FOREST2.EXAMPLE.COM
the KDC of FOREST1.EXAMPLE.COM will respond with S_PRINCIPAL_UNKNOWN
when being asked for ldap/dc2.forest2.example.com@FOREST1.EXAMPLE.COM.

In the case of an external trust the client can still ask
explicitly for krbtgt/FOREST2.EXAMPLE.COM@FOREST1.EXAMPLE.COM
and the KDC of FOREST1.EXAMPLE.COM will generate it.

From there the client can use the
krbtgt/FOREST2.EXAMPLE.COM@FOREST1.EXAMPLE.COM
ticket and ask a KDC of FOREST2.EXAMPLE.COM for a
service ticket for ldap/dc2.forest2.example.com@FOREST2.EXAMPLE.COM.

With Heimdal we'll get the fallback on S_PRINCIPAL_UNKNOWN behavior
when we pass ldap/dc2.forest2.example.com@FOREST2.EXAMPLE.COM as
target principal. As _krb5_get_cred_kdc_any() first calls
get_cred_kdc_referral() (which always starts with the client realm)
and falls back to get_cred_kdc_capath() (which starts with the given realm).

MIT krb5 only tries the given realm of the target principal,
if we want to autodetect support for transitive forest trusts,
we'll have to do the fallback ourself.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12554

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>

s4:gensec_gssapi: Move setup of service_principal to update function

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12554

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>

s4:gensec-gssapi: Create a helper function to setup server_principal

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12554

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>

krb5_wrap: Make smb_krb5_get_realm_from_hostname() public

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12554

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>

krb5_wrap: pass client_realm to smb_krb5_get_realm_from_hostname()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12554

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>

krb5_wrap: Try to guess the correct realm from the service hostname

If we do not get a realm mapping from the krb5.conf or from the Kerberos
library try to guess it from the service hostname. The guessing of the
realm from the service hostname is already implemented in Heimdal. This
makes the behavior of smb_krb5_get_realm_from_hostname() consistent
with both MIT and Heimdal.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12554

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>

krb5_wrap: Do not return an empty realm from smb_krb5_get_realm_from_hostname()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12554

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>

testprogs: Add kinit_trusts tests with smbclient4

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12554

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

testprogs: Use smbclient by default in test_kinit_trusts

This is the tool we use by default and we should test with it.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12554

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

waf: disable-python - don't include python.h in test_headers.c

Signed-off-by: Ian Stakenvicius <axs@gentoo.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Mar 10 11:24:13 CET 2017 on sn-devel-144

autobuild: Add nopython environment to test --disable-python builds (but without tests)

This ensures we keep this option building as we extend our use of python.

The rule is that new features and changes to existing features that
require python are most welcome, they just need to be disabled for the
minimalistic targets we still ecourage Samba on, that typically just
want smbd

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

waf: disable-python - don't build torture bits

samba-net being disabled causes a chain of dependency or proto.h-based
missing code issues that require a number of modules or subsystems
to be disabled in samba4/torture.

Signed-off-by: Ian Stakenvicius <axs@gentoo.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

waf: disable-python - don't build samba-policy

samba-policy requires samba-net which requires PROVISION, which
is disabled when python isn't available.

Signed-off-by: Ian Stakenvicius <axs@gentoo.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

waf: disable-python - don't build samba-net

samba-net requires PROVISION, which is disabled when python isn't available.

Signed-off-by: Ian Stakenvicius <axs@gentoo.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

waf: disable-python - don't build pyrpc_util, dcerpc.py

Signed-off-by: Ian Stakenvicius <axs@gentoo.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

waf: disable-python - don't build PROVISION, pyparam_util

Signed-off-by: Ian Stakenvicius <axs@gentoo.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

waf: disable-python - don't build python/

Signed-off-by: Ian Stakenvicius <axs@gentoo.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

waf: disable-python - align tdb's wscript

Drop the configure option for --disable-python as it is now
global in wafsamba.

If samba is set to use a system copy of tdb, and tdb wasn't built
with python support, then the system pytevent will not be found.  If
samba is being built without python support then pytdb is not needed,
so do not bother to try and find it.

Signed-off-by: Ian Stakenvicius <axs@gentoo.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

waf: disable-python - align tevent wscript

Drop the configure option for --disable-python as it is now
global in wafsamba.

If samba is set to use a system copy of tevent, and tevent wasn't built
with python support, then the system pytevent will not be found.  If
samba is being built without python support then pytevent is not needed,
so do not bother to try and find it.

Signed-off-by: Ian Stakenvicius <axs@gentoo.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

waf: disable-python - align ldb's wscript

If samba is set to use a system copy of ldb, and ldb wasn't built with
python support, then no system pyldb-util will be found.  If samba is
being built without python support then pyldb-util isn not needed, so
do not bother to try and find it.

The system ldb check had to be duplicated due to the earlier commits
which changed order of ldb and pyldb-util checks, and by association
also added a dependency of pyldb-util onto ldb.  This seemed cleaner
than messing with variables.

The build configuration for pyldb-util needs to exist even if it's
not being built, so that dependency resolution can occur throughout
the rest of the samba build system -- this required dropping the higher
level conditional and using the enabled= parameter instead.

Signed-off-by: Ian Stakenvicius <axs@gentoo.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

waf: disable-python - align talloc's wscript

Drop the configure option for --disable-python as it is now
global in wafsamba

If samba is set to use a system copy of talloc, and talloc wasn't built
with python support, then the system pytalloc-util will not be found.
If samba is being built without python support then pytalloc-util is not
needed, so do not bother to try and find it.

The build configuration for pytalloc-util needs to exist even if it's
not being built, so that dependency resolution can occur throughout
the rest of the samba build system -- this required dropping the higher
level conditional and using the enabled= parameter instead.

Signed-off-by: Ian Stakenvicius <axs@gentoo.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

waf: disable-python - configuration adjustments

Adjust configuration to accomodate when --disable-python is set:

- Error when AD-DC is still enabled (and others later as needed)

- Set mandatory=false on SAMBA_CHECK_PYTHON_HEADERS

Signed-off-by: Ian Stakenvicius <axs@gentoo.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

waf: disable-python - add option globally to build system

This commit adds --disable-python as an option to the build system.
It adds PYTHON_BUILD_IS_ENABLED() to bld, to be used with enabled=
on other modules, and adjusts SAMBA_PYTHON() to set enabled=False
if PYTHON_BUILD_IS_ENABLED() is false.

Signed-off-by: Ian Stakenvicius <axs@gentoo.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

waf: disable-python - fix ctdb configuration

When ctdb is built in standalone mode, it turned off the python
requirement for submodules by setting Options.options.disable_python
to True before checking for its own (non-optional) python support.

Ad ctdb does not need python for itself or any of the submodules
it is built against, the safest solution seems to be to allow
the python and python-headers checks to not find python.

Signed-off-by: Ian Stakenvicius <axs@gentoo.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

python: Port the samba.net module to Python 3

Signed-off-by: Petr Viktorin <pviktori@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

python: Remove unused import PY3

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

python: selftest: Add possibility to run old Python test suites with Python 3

Add possibility to execute old Python test suites with Python 3
and enable tests with Python 3 of ported samba.gensec module.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

python: samba.gensec: Port module to Python 3 compatible form

Port samba.gensec and samba.tests.gensec modules to Python 3
compatible form, enable execution of tests with Python 3 and
remove unused import of samba.gensec from samba.tests module
__init__.py file.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

python: samba.gensec: Fix error handling in set_credentials() function

Add `return NULL;` to error handling part of `set_credentials()`
function.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

python: selftests: Enable samba.getopt tests execution with Python 3

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

python: samba.getopt: Port module to Python 3 compatible form

Port samba.getopt module to Python 3 compatible form.

Remove unused and untested `get_hostconfig()` function. Andrew Bartlett
suggested this removal because it is the simpliest way how to break
a long dependency line of Python modules which have to be ported
at once.
More info: https://lists.samba.org/archive/samba-technical/2017-January/118150.html

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

python: samba.tests.core: Port and enable core tests in Python 3

Port samba core tests to Python 3 compatible form and enable their
execution with Python 3.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

python: samba.tests: Move import of ported modules out of PY3 condition

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

python: samba._ldb: Port of samba._ldb to Python 3 compatible form

Port of samba._ldb Python module to Python 3 compatible form.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

python: samba.tests.auth: Add tests for samba.auth module

Add some tests which test that `system_session` object has
correct attributes and methods.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

python: samba.auth: Port samba.auth to Python 3 compatible form

Port samba.auth Python module to Python 3 compatible form and
enable tests execution with Python 3.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

python: wscript_build: Build some DCE/RPC modules with Python 3

Samba.auth Python module depends on a lot of DCE/RPC modules which
have to be built with Python 3 to make port of samba.auth to
Python 3 possible.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

python: samba.dcerpc: Port security module to Python 3 comp. form

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

dcerpc/misc tests: asset GUID ordering in python 2 and 3

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python: samba.tests.dcerpc.misc: Port and enable tests

Port tests of samba.dcerpc.misc module to Python 3 compatible form
and enable their execution with Python 3.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

python: samba.dcerpc: Port RPC related stuff to Python 3

Port RPC related stuff like samba.dcerpc.misc and samba.dcerpc
Python modules and pyrpc_util to Python 3 compatible form.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-by: Andrew Bartlett <abartlet@samba.org>

python: pidl: Port Python interface generator

Port PIDL generator of Python interfaces to generate interfaces in
Python 3 compatible form.

Python 2.7 is now required, so we can use PyCapsule in both versions.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-by: Andrew Bartlett <abartlet@samba.org>

python: samba.tests: Enable Python 3 tests for ported modules

Enable tests with Python 3 for Python 3 compatible modules.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

buildtools: Work around a . being in the target name when building python3 helpers

The pyparam_util module becomes pyparam_util.cpython_35m_x86_64_linux_gnu but
the command line parser for -D stops at the first .

That we even set -DSTATIC_subsystem_MODULES_PROTO for these subsystems without
any modules ever declared is left for another time

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

python: wscript_build: Build some modules for Python 3

Update a few wscript_build files to build Python 3-compatible modules
for Python 3.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

python: Make top-level samba modules Python 3 compatible

New file compat.py will help with porting to Python 3. For now, it
contains only PY3 variable based on six.PY3 which simplifies
condition mentioned below.

The added `if not PY3` conditions enable us to bootstrap running
tests with Python 3 even if most modules are not ported yet.
The plan is to move modules outside this condition as they are ported.
The `PY3` condition is currently used only in tests and for
the samba._ldb module which is not ported yet and has a lot of
dependencies.

The other changes are related to differences between Python 2 and 3.
Python 2.6 introduced the `0o` prefix for octal literals as an
alternative to plain `0`. In Python 3, support for plain `0` is
dropped and octal literals have to start with `0o` prefix.
Python 2.6 introduced a clearer `except` syntax:
`except ExceptionType as target:` instead of
`except ExceptionType, target:`. In Python 3, the old syntax
is no longer allowed.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

python: samba.tests.dcerpc: Move Class RawDCERPCTest to separated file.

The class is quite big, used in only one place, and it complicates
situation around bootstrapping of Python 3 port.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

python: samba.tests.glue: Add new tests for samba._glue.

Add new file with tests of samba._glue module.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

python: samba._glue: Port samba._glue module to Python 3.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

python: samba.tests.param: Add missing tests

Add some new tests of samba.param Python bindings.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

python: samba.param: Port param module to Python 3

Port Python bindings of samba.param module to
Python3-compatible form.

Because native Python file objects are officially
no longer backed by FILE*, API of some _dump()
functions is changed. File argument is now
optional and contains only name of file. Stdout
is default if no file name is specified. Otherwise
opening and closing files is done on C layer
instead of Python.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

python: samba.tests.credentials: Python 3 compatible tests

Port test of pycredentials to Python 3 compatible form.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

python: samba.credentials: Port pycredentials.c to Python3-compatible form.

Port Python bindings of samba.credentials module to
Python3-compatible form using macros from py3compat.h.

Signed-off-by: Lumir Balhar <lbalhar@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

lib/ldb: Enable use of a python3 pyldb-util system library

To do this, we have to install a .pc file for the python3 pyldb-util

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Incorportaing fixes by Petr Viktorin <pviktori@redhat.com>

Signed-off-by: Petr Viktorin <pviktori@redhat.com>

talloc: use the system pytalloc-util for python3 as well

This involves installing a .pc file for the python3 library as well

To get the .pc file generated and installed is quite a mission, we
have to rework the talloc build system to ensure that the second 'env'
created for EXTRA_PYTHON has everything set up on it, the
TALLOC_VERSION in particular.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Incorportaing fixes by Petr Viktorin <pviktori@redhat.com>

Signed-off-by: Petr Viktorin <pviktori@redhat.com>

scripts/traffic_summary: documentation typo

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>