samba.git
5 years agoexamples/VFS/skel_transparent: add missing durable handle functions
Ralph Boehme [Tue, 24 Jul 2018 20:03:01 +0000 (22:03 +0200)]
examples/VFS/skel_transparent: add missing durable handle functions

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
5 years agoexamples/VFS/skel_transparent: add missing audit_file_fn
Ralph Boehme [Tue, 24 Jul 2018 16:58:59 +0000 (18:58 +0200)]
examples/VFS/skel_transparent: add missing audit_file_fn

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
5 years agoexamples/VFS/skel_opaque: call smb_vfs_assert_all_fns()
Stefan Metzmacher [Mon, 23 Jul 2018 07:02:52 +0000 (09:02 +0200)]
examples/VFS/skel_opaque: call smb_vfs_assert_all_fns()

This template should always implement all calls.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
5 years agoexamples/VFS/skel_opaque: add missing durable handle functions
Ralph Boehme [Tue, 24 Jul 2018 19:56:26 +0000 (21:56 +0200)]
examples/VFS/skel_opaque: add missing durable handle functions

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
5 years agoexamples/VFS/skel_opaque: add missing audit_file_fn
Ralph Boehme [Tue, 24 Jul 2018 16:58:45 +0000 (18:58 +0200)]
examples/VFS/skel_opaque: add missing audit_file_fn

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
5 years agoautobuild: add some basic tests for the all static build
Ralph Boehme [Tue, 24 Jul 2018 10:30:33 +0000 (12:30 +0200)]
autobuild: add some basic tests for the all static build

This makes sure each module is at least loaded once
and registers itself as a module.

It means that the skel_opaque and skel_transparent vfs examples
are loaded.

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Ralph Boehme <slow@samba.org>
5 years agowinbind: Move variable declarations close to their use
Volker Lendecke [Thu, 3 May 2018 09:48:45 +0000 (11:48 +0200)]
winbind: Move variable declarations close to their use

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Jul 24 23:31:43 CEST 2018 on sn-devel-144

5 years agowinbind: Align integer types
Volker Lendecke [Fri, 4 May 2018 19:07:35 +0000 (21:07 +0200)]
winbind: Align integer types

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
5 years agonsswitch: Correct users of "ctx->is_privileged"
Volker Lendecke [Fri, 4 May 2018 13:23:18 +0000 (15:23 +0200)]
nsswitch: Correct users of "ctx->is_privileged"

winbindd_context->is_privileged is a bool

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
5 years agonsswitch: Make two functions static
Volker Lendecke [Fri, 4 May 2018 12:40:12 +0000 (14:40 +0200)]
nsswitch: Make two functions static

nss_irix was the only external user

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
5 years agonsswitch: Remove IRIX support
Volker Lendecke [Fri, 4 May 2018 12:33:55 +0000 (14:33 +0200)]
nsswitch: Remove IRIX support

According to wikipedia, IRIX has seen the last patch update in August 2006. As
of now, www.sgi.com is unreachable. Probably this code has not been built in
years. If someone wants to revive it, it can be found in the git history.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
5 years agolib: Fix prototype of srprs_str
Volker Lendecke [Fri, 4 May 2018 20:28:43 +0000 (22:28 +0200)]
lib: Fix prototype of srprs_str

Many callers use "-1" as the "len" argument. That's what ssize_t is for.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
5 years agolib: Remove an #include "includes.h"
Volker Lendecke [Wed, 25 Apr 2018 09:53:00 +0000 (11:53 +0200)]
lib: Remove an #include "includes.h"

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
5 years agolib: Remove an #include "includes.h"
Volker Lendecke [Wed, 25 Apr 2018 09:51:36 +0000 (11:51 +0200)]
lib: Remove an #include "includes.h"

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
5 years agos3:util_sec: add a cache to set_thread_credentials()
Stefan Metzmacher [Thu, 3 May 2018 14:08:06 +0000 (16:08 +0200)]
s3:util_sec: add a cache to set_thread_credentials()

Calling set_thread_credentials() with the same values,
skips syscalls the 2nd time.

We only do this if '__thread' is supported to provide
thread local storage.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Tue Jul 24 20:35:17 CEST 2018 on sn-devel-144

5 years agolib/util: rename USE_LINUX_THREAD_CREDENTIALS to HAVE_LINUX_THREAD_CREDENTIALS
Ralph Boehme [Thu, 28 Jun 2018 12:28:34 +0000 (14:28 +0200)]
lib/util: rename USE_LINUX_THREAD_CREDENTIALS to HAVE_LINUX_THREAD_CREDENTIALS

The define reflects the results of a feature test, not a configure
option.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
5 years agopthreadpool: test cancelling and freeing jobs of a wrapped pthreadpool_tevent
Ralph Boehme [Mon, 18 Jun 2018 14:57:18 +0000 (16:57 +0200)]
pthreadpool: test cancelling and freeing jobs of a wrapped pthreadpool_tevent

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
5 years agopthreadpool: implement pthreadpool_tevent_wrapper_create() infrastructure
Stefan Metzmacher [Fri, 20 Apr 2018 15:12:07 +0000 (17:12 +0200)]
pthreadpool: implement pthreadpool_tevent_wrapper_create() infrastructure

This can be used implement a generic per thread impersonation
for thread pools.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
5 years agopthreadpool: add pthreadpool_restart_check[_monitor_{fd,drain}]()
Stefan Metzmacher [Mon, 16 Jul 2018 12:43:01 +0000 (14:43 +0200)]
pthreadpool: add pthreadpool_restart_check[_monitor_{fd,drain}]()

This makes it possible to monitor the pthreadpool for exited worker
threads and may restart new threads from the main thread again.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
5 years agopthreadpool: add tests for pthreadpool_tevent_[current_job_]per_thread_cwd()
Stefan Metzmacher [Wed, 18 Jul 2018 08:21:22 +0000 (10:21 +0200)]
pthreadpool: add tests for pthreadpool_tevent_[current_job_]per_thread_cwd()

Note this currently this doesn't enforce the support for
unshare(CLONE_FS) as some contraint container environment
(e.g. docker) reject the whole unshare() system call.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
5 years agopthreadpool: add pthreadpool_tevent_[current_job_]per_thread_cwd()
Stefan Metzmacher [Thu, 21 Jun 2018 23:02:41 +0000 (01:02 +0200)]
pthreadpool: add pthreadpool_tevent_[current_job_]per_thread_cwd()

This can be used to check if worker threads run with
unshare(CLONE_FS).

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
5 years agopthreadpool: call unshare(CLONE_FS) if available
Ralph Boehme [Tue, 13 Mar 2018 15:59:32 +0000 (16:59 +0100)]
pthreadpool: call unshare(CLONE_FS) if available

This paves the way for pthreadpool jobs that are path based.

Callers can use pthreadpool_per_thread_cwd() to check if
the current pool supports it.

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Ralph Boehme <slow@samba.org>
5 years agoconfigure: check for Linux specific unshare() with CLONE_FS
Ralph Boehme [Tue, 13 Mar 2018 15:58:49 +0000 (16:58 +0100)]
configure: check for Linux specific unshare() with CLONE_FS

Note we still need some kind of runtime detection as
it can fail in some constraint container setups, which
reject the whole unshare() syscall instead of just the
once used for container features.

In case unshare(CLONE_FS) works, we can have a per thread
current working directory and use [f]chdir() safely in
worker threads.

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Ralph Boehme <slow@samba.org>
5 years agopthreadpool: test cancelling and freeing pending pthreadpool_tevent jobs/pools
Ralph Boehme [Mon, 18 Jun 2018 13:32:30 +0000 (15:32 +0200)]
pthreadpool: test cancelling and freeing pending pthreadpool_tevent jobs/pools

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Ralph Boehme <slow@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
5 years agopthreadpool: add a comment about a further optimization in pthreadpool_tevent_job_des...
Stefan Metzmacher [Fri, 22 Jun 2018 15:22:10 +0000 (17:22 +0200)]
pthreadpool: add a comment about a further optimization in pthreadpool_tevent_job_destructor()

This seems to be a really rare race, it's likely that the immediate
event will still trigger and cleanup.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
5 years agopthreadpool: maintain a list of job_states on each pthreadpool_tevent_glue
Stefan Metzmacher [Fri, 22 Jun 2018 15:14:31 +0000 (17:14 +0200)]
pthreadpool: maintain a list of job_states on each pthreadpool_tevent_glue

We should avoid traversing a linked list within a thread without holding
a mutex!

Using a mutex would be very tricky as we'll likely deadlock with
the mutexes at the raw pthreadpool layer.

So we use somekind of spinlock using atomic_thread_fence in order to
protect the access to job->state->glue->{tctx,ev} in
pthreadpool_tevent_job_signal().

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
5 years agopthreadpool: add helgrind magic to PTHREAD_TEVENT_JOB_THREAD_FENCE_*()
Stefan Metzmacher [Thu, 21 Jun 2018 10:43:08 +0000 (12:43 +0200)]
pthreadpool: add helgrind magic to PTHREAD_TEVENT_JOB_THREAD_FENCE_*()

This avoids the expected helgrind/drd warnings on the job states which
are protected by the thread fence.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
5 years agolib/replace: also check for valgrind/helgrind.h
Stefan Metzmacher [Thu, 21 Jun 2018 10:46:48 +0000 (12:46 +0200)]
lib/replace: also check for valgrind/helgrind.h

This will be used in lib/pthreadpool/pthreadpool_tevent.c
in order to avoid extected helgrind/drd warnings.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
5 years agos3:wscript: don't check for valgrind related headers twice
Stefan Metzmacher [Thu, 21 Jun 2018 10:46:06 +0000 (12:46 +0200)]
s3:wscript: don't check for valgrind related headers twice

We already check them in lib/replace/wscript.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
5 years agopthreadpool: add some lockless coordination between the main and job threads
Stefan Metzmacher [Wed, 20 Jun 2018 11:38:19 +0000 (13:38 +0200)]
pthreadpool: add some lockless coordination between the main and job threads

In the direction from the main process to the job thread, we have:

- 'maycancel', which is set when tevent_req_cancel() is called,
- 'orphaned' is the job request, tevent_context or pthreadpool_tevent
  was talloc_free'ed.

The job function can consume these by using:

   /*
    * return true - if tevent_req_cancel() was called.
    */
   bool pthreadpool_tevent_current_job_canceled(void);

   /*
    * return true - if talloc_free() was called on the job request,
    * tevent_context or pthreadpool_tevent.
    */
   bool pthreadpool_tevent_current_job_orphaned(void);

   /*
    * return true if canceled and orphaned are both false.
    */
   bool pthreadpool_tevent_current_job_continue(void);

In the other direction we remember the following points
in the job execution:

- 'started'  - set when the job is picked up by a worker thread
- 'executed' - set once the job function returned.
- 'finished' - set when pthreadpool_tevent_job_signal() is entered
- 'dropped'  - set when pthreadpool_tevent_job_signal() leaves with orphaned
- 'signaled' - set when pthreadpool_tevent_job_signal() leaves normal

There're only one side writing each element,
either the main process or the job thread.

This means we can do the coordination with a full memory
barrier using atomic_thread_fence(memory_order_seq_cst).
lib/replace provides fallbacks if C11 stdatomic.h is not available.

A real pthreadpool requires pthread and atomic_thread_fence() (or an
replacement) to be available, otherwise we only have pthreadpool_sync.c.
But this should not make a real difference, as at least
__sync_synchronize() is availabe since 2005 in gcc.
We also require __thread which is available since 2002.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
5 years agoreplace: add checks for atomic_thread_fence(memory_order_seq_cst) and add possible...
Stefan Metzmacher [Thu, 21 Jun 2018 12:17:35 +0000 (14:17 +0200)]
replace: add checks for atomic_thread_fence(memory_order_seq_cst) and add possible fallbacks

This implements a full memory barrier.
On ubuntu amd64 with results in an 'mfence' instruction.

This is required to syncronization between threads, where
there's typically only one write of a memory that should be
synced between all threads with the barrier.

Much more details can be found here:
https://gcc.gnu.org/onlinedocs/gcc-7.3.0/gcc/_005f_005fatomic-Builtins.html#g_t_005f_005fatomic-Builtins
https://gcc.gnu.org/onlinedocs/gcc-7.3.0/gcc/_005f_005fsync-Builtins.html#g_t_005f_005fsync-Builtins

The main one we use seems to be in C11 via stdatomic.h,
the oldest fallback is __sync_synchronize(), which is available
since 2005 in gcc.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
5 years agothird_party/*_wrapper/wscript: remove redundant configure checks
Stefan Metzmacher [Wed, 18 Jul 2018 06:54:22 +0000 (08:54 +0200)]
third_party/*_wrapper/wscript: remove redundant configure checks

HAVE___THREAD and HAVE_DESTRUCTOR_ATTRIBUTE are already checked
as part of Samba.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
5 years agolib/replace: check for __thread support
Stefan Metzmacher [Wed, 18 Jul 2018 06:44:48 +0000 (08:44 +0200)]
lib/replace: check for __thread support

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
5 years agopthreadpool: replace assert_return_code(ret, 0); with assert_int_equal(ret, 0);
Stefan Metzmacher [Mon, 23 Jul 2018 21:24:22 +0000 (23:24 +0200)]
pthreadpool: replace assert_return_code(ret, 0); with assert_int_equal(ret, 0);

We need to assert the exact value!

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
5 years agopthreadpool: test pthreadpool_tevent_max_threads() returns the expected result
Stefan Metzmacher [Wed, 18 Jul 2018 08:17:51 +0000 (10:17 +0200)]
pthreadpool: test pthreadpool_tevent_max_threads() returns the expected result

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
5 years agopthreadpool: make sure a pthreadpool is marked as stopped in child processes
Stefan Metzmacher [Mon, 16 Jul 2018 15:17:59 +0000 (17:17 +0200)]
pthreadpool: make sure a pthreadpool is marked as stopped in child processes

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
5 years agotevent: use talloc_zero_size() for the private state in tevent_context_wrapper_create()
Stefan Metzmacher [Sat, 14 Jul 2018 08:55:02 +0000 (10:55 +0200)]
tevent: use talloc_zero_size() for the private state in tevent_context_wrapper_create()

This is watch tevent_req_create() uses and what callers of
tevent_context_wrapper_create() would therefore also expect.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
5 years agosamba-tool trust: support discovery via netr_GetDcName
Alexander Bokovoy [Sat, 24 Feb 2018 12:34:44 +0000 (14:34 +0200)]
samba-tool trust: support discovery via netr_GetDcName

In case a remote DC does not support netr_DsRGetDCNameEx2(),
use netr_GetDcName() instead.

This should help with FreeIPA where embedded smbd runs as a domain
controller but does not implement full Active Directory compatibility.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13538

Signed-off-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Alexander Bokovoy <ab@samba.org>
Autobuild-Date(master): Tue Jul 24 09:55:23 CEST 2018 on sn-devel-144

5 years agos3:selftest: run rpc.lsa.lookupsids also with explicit [smb1] and [smb2]
Stefan Metzmacher [Thu, 19 Jul 2018 05:34:11 +0000 (07:34 +0200)]
s3:selftest: run rpc.lsa.lookupsids also with explicit [smb1] and [smb2]

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
5 years agos4:librpc: autonegotiate SMB1/2/3
Stefan Metzmacher [Wed, 18 Jul 2018 14:55:33 +0000 (16:55 +0200)]
s4:librpc: autonegotiate SMB1/2/3

Windows Server 1709 defaults to SMB2 and does not have SMB1 enabled.
When establishing trust, samba-tool does not specify SMB protocol
version and fail by default.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
5 years agopython/tests: use explicit "client ipc max protocol = NT1" for samba.tests.net_join_n...
Stefan Metzmacher [Sun, 22 Jul 2018 22:17:35 +0000 (00:17 +0200)]
python/tests: use explicit "client ipc max protocol = NT1" for samba.tests.net_join_no_spnego

The tests rely on SMB1.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
5 years agotests/auth_log: Permit SMB2 service description if empty binding is used for kerberos...
Alexander Bokovoy [Thu, 19 Jul 2018 11:07:39 +0000 (14:07 +0300)]
tests/auth_log: Permit SMB2 service description if empty binding is used for kerberos authentication

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Alexander Bokovoy <ab@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
5 years agos4:libcli: add smb_connect_nego_{send,recv}()
Stefan Metzmacher [Wed, 18 Jul 2018 12:52:43 +0000 (14:52 +0200)]
s4:libcli: add smb_connect_nego_{send,recv}()

This can be used to create a connection up to a negotiated
smbXcli_conn.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
5 years agos4:libcli: allow a fallback to NTLMSSP if SPNEGO is not supported locally
Stefan Metzmacher [Thu, 19 Jul 2018 21:04:33 +0000 (23:04 +0200)]
s4:libcli: allow a fallback to NTLMSSP if SPNEGO is not supported locally

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
5 years agos4:libcli: add fallback_to_anonymous to smb2_connect_send()
Stefan Metzmacher [Wed, 18 Jul 2018 21:52:30 +0000 (23:52 +0200)]
s4:libcli: add fallback_to_anonymous to smb2_connect_send()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
5 years agos4:libcli: allow passing an already negotiated connection to smb2_connect_send()
Stefan Metzmacher [Wed, 18 Jul 2018 14:44:16 +0000 (16:44 +0200)]
s4:libcli: allow passing an already negotiated connection to smb2_connect_send()

It will just do the session setup and tree connect steps.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
5 years agos4:libcli: split out smb2_connect_session_start()
Stefan Metzmacher [Wed, 18 Jul 2018 14:43:32 +0000 (16:43 +0200)]
s4:libcli: split out smb2_connect_session_start()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
5 years agos4:libcli: add smb2_transport_raw_init()
Stefan Metzmacher [Wed, 18 Jul 2018 14:43:04 +0000 (16:43 +0200)]
s4:libcli: add smb2_transport_raw_init()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
5 years agos4:libcli: allow passing an already negotiated connection to smb_composite_connect()
Stefan Metzmacher [Wed, 18 Jul 2018 13:34:55 +0000 (15:34 +0200)]
s4:libcli: allow passing an already negotiated connection to smb_composite_connect()

It will just do the session setup and tree connect steps.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
5 years agos4:libcli: use talloc_zero() for struct smb_composite_connect in fetchfile.c
Stefan Metzmacher [Wed, 18 Jul 2018 13:01:50 +0000 (15:01 +0200)]
s4:libcli: use talloc_zero() for struct smb_composite_connect in fetchfile.c

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
5 years agos4:libcli: add smbcli_transport_raw_init()
Stefan Metzmacher [Wed, 18 Jul 2018 13:36:52 +0000 (15:36 +0200)]
s4:libcli: add smbcli_transport_raw_init()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
5 years agos4:libcli: split out smb_raw_negotiate_fill_transport()
Stefan Metzmacher [Wed, 18 Jul 2018 13:36:31 +0000 (15:36 +0200)]
s4:libcli: split out smb_raw_negotiate_fill_transport()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
5 years agolibrpc: add binding handle support for [smb1]
Stefan Metzmacher [Thu, 19 Jul 2018 05:32:08 +0000 (07:32 +0200)]
librpc: add binding handle support for [smb1]

This will be used to force smb1.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
5 years agopython/samba/tests: make sure samba.tests can be imported without SamDB
Alexander Bokovoy [Sat, 21 Jul 2018 09:05:15 +0000 (12:05 +0300)]
python/samba/tests: make sure samba.tests can be imported without SamDB

We are using samba.tests Python module __init__.py file as a catch-all
for all types of helpers. Some of these helpers are only usable with
Samba AD DC targets.

When SamDB is not available in a non-Samba AD DC target, provide a
dummy replacement that simply returns None. This allows to complete
initialization for non-Samba AD DC target tests which do not use
connect_samdb() helper.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13542

Signed-off-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
Autobuild-User(master): Günther Deschner <gd@samba.org>
Autobuild-Date(master): Mon Jul 23 19:36:56 CEST 2018 on sn-devel-144

5 years agos4-dsdb: only build dsdb Python modules for AD DC
Alexander Bokovoy [Fri, 20 Jul 2018 09:32:20 +0000 (12:32 +0300)]
s4-dsdb: only build dsdb Python modules for AD DC

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13542

Signed-off-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
5 years agos4-dns_server: Only build dns server Python code for AD DC
Alexander Bokovoy [Fri, 20 Jul 2018 09:31:20 +0000 (12:31 +0300)]
s4-dns_server: Only build dns server Python code for AD DC

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13542

Signed-off-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
5 years agos3: smbd: Fix AIX sendfile() for SMB2. Ensure we don't spin on EAGAIN.
Jeremy Allison [Wed, 18 Jul 2018 22:49:29 +0000 (15:49 -0700)]
s3: smbd: Fix AIX sendfile() for SMB2. Ensure we don't spin on EAGAIN.

For SMB2 the socket is set non-blocking. Ensure sendfile()
calls complete if they return EAGAIN by saving the socket state,
setting it blocking, doing the sendfile until completion and then
restoring the socket state.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13537

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Fri Jul 20 15:14:24 CEST 2018 on sn-devel-144

5 years agos3: smbd: Fix FreeBSD sendfile() for SMB2. Ensure we don't spin on EAGAIN.
Jeremy Allison [Wed, 18 Jul 2018 22:44:34 +0000 (15:44 -0700)]
s3: smbd: Fix FreeBSD sendfile() for SMB2. Ensure we don't spin on EAGAIN.

For SMB2 the socket is set non-blocking. Ensure sendfile()
calls complete if they return EAGAIN by saving the socket state,
setting it blocking, doing the sendfile until completion and then
restoring the socket state.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13537

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
5 years agos3: smbd: Fix HPUX sendfile() for SMB2. Ensure we don't spin on EAGAIN.
Jeremy Allison [Wed, 18 Jul 2018 22:36:47 +0000 (15:36 -0700)]
s3: smbd: Fix HPUX sendfile() for SMB2. Ensure we don't spin on EAGAIN.

For SMB2 the socket is set non-blocking. Ensure sendfile()
calls complete if they return EAGAIN by saving the socket state,
setting it blocking, doing the sendfile until completion and then
restoring the socket state.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13537

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
5 years agos3: smbd: Fix Solaris sendfile() for SMB2. Ensure we don't spin on EAGAIN.
Jeremy Allison [Wed, 18 Jul 2018 22:29:37 +0000 (15:29 -0700)]
s3: smbd: Fix Solaris sendfile() for SMB2. Ensure we don't spin on EAGAIN.

For SMB2 the socket is set non-blocking. Ensure sendfile()
calls complete if they return EAGAIN by saving the socket state,
setting it blocking, doing the sendfile until completion and then
restoring the socket state.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13537

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
5 years agos3: smbd: Fix Linux sendfile() for SMB2. Ensure we don't spin on EAGAIN.
Jeremy Allison [Wed, 18 Jul 2018 20:32:49 +0000 (13:32 -0700)]
s3: smbd: Fix Linux sendfile() for SMB2. Ensure we don't spin on EAGAIN.

For SMB2 the socket is set non-blocking. Ensure sendfile()
calls complete if they return EAGAIN by saving the socket state,
setting it blocking, doing the sendfile until completion and then
restoring the socket state.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13537

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
5 years agodns wildcards: fix BUG 13536
Gary Lockyer [Wed, 18 Jul 2018 03:33:26 +0000 (15:33 +1200)]
dns wildcards: fix BUG 13536

The current position in the dns name was not advanced past the '.'
character

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Jul 20 04:40:31 CEST 2018 on sn-devel-144

5 years agodns wildcards: tests to confirm BUG 13536
Gary Lockyer [Wed, 18 Jul 2018 03:29:21 +0000 (15:29 +1200)]
dns wildcards: tests to confirm BUG 13536

DNS wildcard matching failing if more than one label to the left of the
wildcard. This commits adds tests to confirm the bug.

Wildcard entry: *.example.org
bar.example.com matches
foo.bar.example.com does not, but it it should.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
5 years agoctdb-tests: Loosen match against pstree output in simple test
Martin Schwenke [Wed, 18 Jul 2018 05:24:05 +0000 (15:24 +1000)]
ctdb-tests: Loosen match against pstree output in simple test

As per previous commit, pstree can truncate output if it gets too
wide.  Instead of matching against the script's full path and
arguments, just match against the script name.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13531

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Wed Jul 18 14:53:39 CEST 2018 on sn-devel-144

5 years agoctdb-tests: Simplify pstree output in eventd unit tests
Martin Schwenke [Wed, 18 Jul 2018 02:01:27 +0000 (12:01 +1000)]
ctdb-tests: Simplify pstree output in eventd unit tests

pstree truncates output when it exceeds a maximum width - the default
is 132 columns.  A couple of recent
commits (12fd8d7a5c5d14d403aac6cd9e318afcd0a8e159,
b23f3f996038626f618c5b5aa552686c1b852f44) lengthened the command
string in the output so that it is more likely to exceed this limit
and be truncated, as below:

==================================================
Running "cat /memdisk/autobuild/fl/b1851760/ctdb/ctdb/tests/var/eventd/debug_script.log"
--------------------------------------------------
Output (Exit status: 0):
--------------------------------------------------
02.enabled.scri,PID /memdisk/autobuild/fl/b1851760/ctdb/ctdb/tests/var/eventd/events/random/02.enabled.script ...
  `-sleep,PID 99
01.disabled          DISABLED
02.enabled           TIMEDOUT   DATETIME
  OUTPUT: Sleeping for 99 seconds
--------------------------------------------------
Required output (Exit status: 0):
--------------------------------------------------
02.enabled.scri,PID /memdisk/autobuild/fl/b1851760/ctdb/ctdb/tests/var/eventd/events/random/02.enabled.script verbosetimeout
  `-sleep,PID 99
01.disabled          DISABLED
02.enabled           TIMEDOUT   DATETIME
  OUTPUT: Sleeping for 99 seconds

FAILED

It isn't clear that the above example exceeds 132 characters, given
that the PID has been filtered into a fixed string, but it certainly
goes close.  Whether or not it is truncated probably depends on the
width of the PID in the unfiltered output.  This would explain why the
test flaps.

Avoid the output truncation by dropping the -a and -p options to
simplify the pstree output.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13531

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
5 years agodsdb: Fix the 32-bit build
Volker Lendecke [Fri, 13 Jul 2018 14:31:21 +0000 (16:31 +0200)]
dsdb: Fix the 32-bit build

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Jul 18 03:12:02 CEST 2018 on sn-devel-144

5 years agokcc: Fix the 32-bit build
Volker Lendecke [Fri, 13 Jul 2018 14:31:07 +0000 (16:31 +0200)]
kcc: Fix the 32-bit build

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
5 years agos3: smbd: fix path check in smbd_smb2_create_durable_lease_check()
Ralph Boehme [Tue, 17 Jul 2018 13:40:04 +0000 (15:40 +0200)]
s3: smbd: fix path check in smbd_smb2_create_durable_lease_check()

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13535

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
5 years agos4: torture: run test_durable_v2_open_reopen2_lease() in a subdirectory
Ralph Boehme [Tue, 17 Jul 2018 13:56:05 +0000 (15:56 +0200)]
s4: torture: run test_durable_v2_open_reopen2_lease() in a subdirectory

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13535

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
5 years agotevent: fix CID 1437974 dereference after null check
Ralph Boehme [Tue, 17 Jul 2018 10:59:50 +0000 (12:59 +0200)]
tevent: fix CID 1437974 dereference after null check

Probably a copy/paste error from the tevent_debug() statement a few
lines above as at this place we want to pass main_ev directly to
tevent_debug() anyway.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Tue Jul 17 16:21:21 CEST 2018 on sn-devel-144

5 years agotevent: fix CID 1437976 dereference before null check
Ralph Boehme [Tue, 17 Jul 2018 10:54:29 +0000 (12:54 +0200)]
tevent: fix CID 1437976 dereference before null check

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
5 years agodbwrap: determine basename once instead of three times
David Disseldorp [Tue, 10 Jul 2018 11:35:30 +0000 (13:35 +0200)]
dbwrap: determine basename once instead of three times

Currently determined twice in the clear-if-first codepath and once in
the ctdb code path. Do it once at the top of the function.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Mon Jul 16 21:11:30 CEST 2018 on sn-devel-144

5 years agos3-tldap: do not install test_tldap
Günther Deschner [Thu, 12 Jul 2018 21:32:36 +0000 (23:32 +0200)]
s3-tldap: do not install test_tldap

Guenther

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13529

Signed-off-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Sat Jul 14 03:09:00 CEST 2018 on sn-devel-144

5 years agos3: torture: adjust SMB1 cli_splice() test sizes
David Disseldorp [Fri, 13 Jul 2018 11:58:04 +0000 (13:58 +0200)]
s3: torture: adjust SMB1 cli_splice() test sizes

The test writes 20M and then splices just over 1M (13M is intended,
but there's a 1024*0124 typo). Fix the type and reduce the size of the
dataset to make it run faster - cli_splice works with 1M chunks
(SPLICE_BLOCK_SIZE), and the reproducer only requires that the splice
size is not chunk-aligned.

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Jul 14 00:14:13 CEST 2018 on sn-devel-144

5 years agowafsamba/samba_abi: always hide ABI symbols which must be local
Alexander Bokovoy [Thu, 12 Jul 2018 07:19:41 +0000 (10:19 +0300)]
wafsamba/samba_abi: always hide ABI symbols which must be local

binutils 2.31 is going to change how shared libraries are linked, such
that they always provide their own local definitions of the _end, _edata
and _bss_start symbols.  This would all be fine, except for shared
libraries that export all symbols be default.  (Rather than just
exporting those symbols that form part of their API).

According to binutils developers, we should only export the symbols we
explicitly want to be used. We don't use this principle for all our
libraries and deliberately don't want to have ABI versioning control for
all of them, so the change I introduce here is to explicitly mark those
symbols that will always be added by default linker configuration with
binutils 2.31 as local. Right now these are '_end', '_edata', and
'__bss_start' symbols.

Signed-off-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Jul 13 17:45:36 CEST 2018 on sn-devel-144

5 years agos3: libsmbclient: Fix cli_splice() fallback when reading less than a complete file.
Jeremy Allison [Thu, 12 Jul 2018 19:18:50 +0000 (12:18 -0700)]
s3: libsmbclient: Fix cli_splice() fallback when reading less than a complete file.

We were always asking for SPLICE_BLOCK_SIZE even when the
remaining bytes we wanted were smaller than that. This works
when using cli_splice() on a complete file, as the cli_read()
terminated the read at the right place. We always have the
space to read SPLICE_BLOCK_SIZE bytes so this isn't an overflow.

Found by Bailey Berro <baileyberro@google.com>

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13527

Signed-off-by: Bailey Berro <baileyberro@google.com>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: David Disseldorp <ddiss@samba.org>
Autobuild-User(master): David Disseldorp <ddiss@samba.org>
Autobuild-Date(master): Fri Jul 13 14:57:14 CEST 2018 on sn-devel-144

5 years agos3: torture: Test SMB1 cli_splice() fallback path when doing a non-full file splice.
Jeremy Allison [Thu, 12 Jul 2018 19:15:12 +0000 (12:15 -0700)]
s3: torture: Test SMB1 cli_splice() fallback path when doing a non-full file splice.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13527

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: David Disseldorp <ddiss@samba.org>
5 years agodocs: mention that the echo handler is for SMB1 only
Björn Jacke [Thu, 12 Jul 2018 07:39:00 +0000 (09:39 +0200)]
docs: mention that the echo handler is for SMB1 only

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Karolin Seeger <kseeger@samba.org>
Autobuild-User(master): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(master): Fri Jul 13 11:44:02 CEST 2018 on sn-devel-144

5 years agopython/samba/netcmd: Fix wrong exception referenced in code
Noel Power [Wed, 27 Jun 2018 11:01:18 +0000 (12:01 +0100)]
python/samba/netcmd: Fix wrong exception referenced in code

post commit: 52729d35495db638c84caa8cc6f5ffdf0b670353 wrong
exception name was referenced.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Jul 13 04:01:59 CEST 2018 on sn-devel-144

5 years agopython/samba/kcc: md5 needs to be passed bytes in py3
Noel Power [Tue, 19 Jun 2018 10:22:50 +0000 (11:22 +0100)]
python/samba/kcc: md5 needs to be passed bytes in py3

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
5 years agopython/samba/tests: various py3 porting for ord/chr
Noel Power [Mon, 18 Jun 2018 17:02:01 +0000 (18:02 +0100)]
python/samba/tests: various py3 porting for ord/chr

various messages are lists of ints that need converting to str or bytes
depending on py2/py3, others are str/bytes that need modification and
are converted to lists or string char or ints for modificate (and then
reconstructed as str/bytes again)

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
5 years agos4/torure/drs/python: Fix incorrect use of unicode which doesn't exist in py3
Noel Power [Fri, 15 Jun 2018 15:26:28 +0000 (16:26 +0100)]
s4/torure/drs/python: Fix incorrect use of unicode which doesn't exist in py3

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
5 years agos4/torture/drs/python: xrange -> range for py2/py3 compat
Noel Power [Fri, 15 Jun 2018 14:13:56 +0000 (15:13 +0100)]
s4/torture/drs/python: xrange -> range for py2/py3 compat

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
5 years agos4/torture/drs/python: long is not used in py3
Noel Power [Fri, 15 Jun 2018 13:29:42 +0000 (14:29 +0100)]
s4/torture/drs/python: long is not used in py3

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
5 years agos4/torture/drs/python: use cmp_fn for cmp (for py2/py3 compat)
Noel Power [Fri, 15 Jun 2018 12:06:13 +0000 (13:06 +0100)]
s4/torture/drs/python: use cmp_fn for cmp (for py2/py3 compat)

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
5 years agos4/torture/drs/python: use cmp_fn and key=cmp_to_key_fn for py2/py3
Noel Power [Fri, 15 Jun 2018 12:04:13 +0000 (13:04 +0100)]
s4/torture/drs/python: use cmp_fn and key=cmp_to_key_fn for py2/py3

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
5 years agopython/samba: Add cmp_fn and cmp_to_key_fn functions for py2/py3
Noel Power [Fri, 15 Jun 2018 12:00:15 +0000 (13:00 +0100)]
python/samba: Add cmp_fn and cmp_to_key_fn functions for py2/py3

the cmp function and the cmp paramater (e.g. to sort functions)
no longer exist in python3.

cmp_fn is provides the missing functionality of the py2 cmp builtin
function.

cmp_to_key_fn allows the key paramater (e.g. for sort) to use the
old py2 cmp function for sorting. Note: the cmp_to_key is present in
since 2.7 (hence the inclusion of the source code for this function pre
that version)

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
5 years agos4/torture/drs/python: py2/py3 port map / ord usage
Noel Power [Fri, 15 Jun 2018 09:58:13 +0000 (10:58 +0100)]
s4/torture/drs/python: py2/py3 port map / ord usage

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
5 years agos4/torture/drs: ndr_upack needs bytes in py3 (samba4.drs.repl_move)
Noel Power [Fri, 15 Jun 2018 09:18:30 +0000 (10:18 +0100)]
s4/torture/drs: ndr_upack needs bytes in py3 (samba4.drs.repl_move)

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
5 years agos4/dsdb/tests/python: base64.b64encode returns bytes
Noel Power [Thu, 31 May 2018 15:13:28 +0000 (16:13 +0100)]
s4/dsdb/tests/python: base64.b64encode returns bytes

adjust to unicode for py2/py3 compat needed as part of changes
to ensure samba4.ldap.password_settings will work with PY3

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
5 years agos4/librpc/ndr: allow GUID to accept unicode also
Noel Power [Fri, 18 May 2018 16:15:44 +0000 (17:15 +0100)]
s4/librpc/ndr: allow GUID to accept unicode also

This needed since _GUID_string method change
(in source4/torture/drs/python/drs_base.py) which makes use use
a unicode guid at times now

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
5 years agopython/samba/netcmd: Fix relative module import
Noel Power [Fri, 18 May 2018 11:12:44 +0000 (12:12 +0100)]
python/samba/netcmd: Fix relative module import

Part of future changes needed to enable samba4.drs.replica_sync_rodc
for PY3.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
5 years agopython/samba/netcmd: Protect variable that can be None
Noel Power [Thu, 17 May 2018 14:25:29 +0000 (15:25 +0100)]
python/samba/netcmd: Protect variable that can be None

In py3 None variable cannot be compared with '>' '<' etc operators

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
5 years agopython/samba/netcmd: fix py2/py3 bytes usage for replace
Noel Power [Thu, 17 May 2018 14:23:38 +0000 (15:23 +0100)]
python/samba/netcmd: fix py2/py3 bytes usage for replace

base64.b64encode returns bytes in py3 make sure associated replace
uses 'b' for strings passed to replace

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
5 years agos4/torture/drs/python: Py2/Py2 fix tab/space also incorrect unicode usage
Noel Power [Tue, 15 May 2018 14:32:35 +0000 (15:32 +0100)]
s4/torture/drs/python: Py2/Py2 fix tab/space also incorrect unicode usage

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
5 years agolib/ldb: Implement a bytes derived object for attributes py2/py3
Noel Power [Fri, 11 May 2018 12:48:29 +0000 (13:48 +0100)]
lib/ldb: Implement a bytes derived object for attributes py2/py3

ldb attributes are either bytes (py3) or str (py2)

Some places in the code do str(res[0]['attribute'][0])
which results in
   'result' (py2)
  b'result' (py3)

or more commonly the attribute is used to construct a string e.g.
   "blah=" + res[0]['attribute'][0] + ",foo,bar=...."

giving
   "blah=result,foo,bar=...." (py2)
and very unhelpfully
   "blah=b'result',foo,bar=...." (py3)

lots of code already constructs various strings for passing to other
api using the above. To avoid many excessive
    res[0]['attribute'][0].decode('utf8')

code like 'res[0]['attribute'][0]'

will now return LdbBytes (a new object subclassing 'bytes') in py3
instead of bytes. This object has a custom '__str__' method which
attempts to return a string decoded to uft8. In Py2 this will behave as
it did previously (this is the safer option at the moment)

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
5 years agopython/samba/tests: remove Py2 specific imports.
Noel Power [Thu, 10 May 2018 12:14:22 +0000 (13:14 +0100)]
python/samba/tests: remove Py2 specific imports.

Remove some python2 specific import, probably this was due to
previous unavailability for some c-modules in python3

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
5 years agopython/samba/emulate: Fix some more missed exception tuple assignments
Noel Power [Wed, 9 May 2018 19:17:30 +0000 (20:17 +0100)]
python/samba/emulate: Fix some more missed exception tuple assignments

In python3 we need to change

    except LdbError as e:
-        (status, _) = e
to
    except LdbError as e:
+        (status, _) = e.args

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
5 years agopython/samba/tests: Py2/Py3 port for hexdump
Noel Power [Tue, 1 May 2018 18:58:36 +0000 (19:58 +0100)]
python/samba/tests: Py2/Py3 port for hexdump

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
5 years agopython/samba/tests: Py2/Py3 allow import of ndr_(un)pack to work
Noel Power [Mon, 30 Apr 2018 17:43:54 +0000 (18:43 +0100)]
python/samba/tests: Py2/Py3 allow import of ndr_(un)pack to work

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>