Fix typos and ban the rams from samba

Signed-off-by: Swen Schillig <swen@linux.ibm.com>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <Volker.Lendecke@SerNet.DE>

selftest: Remove duplicate dependency definition

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

dsdb: Print strerror in addition to errno

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

dns: Do NULL checks in dns_sign_tsig

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:libsmb: cli_smb2_list() can sometimes fail initially on a connection

cli_smb2_list() appears to be a slightly unique SMB operation in that it
specifies the max transaction size for the response buffer size. The
Python bindings highlighted a problem where if cli_smb2_list() were one
of the first operations performed on the SMBv2 connection, it would fail
due to insufficient credits. Because the response buffer size is
(potentially) so much larger, it requires more credits (128) compared
with other SMB operations.

When talking to a samba DC, the connection credits seem to start off at
1, then increase by 32 for every SMB reply we receive back from the
server. After cli_full_connection(), the connection has 65 credits. The
cli_smb2_create_fnum() in cli_smb2_list() adds another 32 credits, but
this is still less than the 128 that smb2cli_query_directory() requires.

This problem doesn't happen for smbclient because the cli_cm_open() API
it uses ends up sending more messages, and so the connection has more
credits.

This patch changes cli_smb2_list(), so it requests a smaller response
buffer size if it doesn't have enough credits available for the max
transaction size. smb2cli_query_directory() is already in a loop, so it
can span multiple SMB messages if for some reason the transaction size
isn't big enough for the listings.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13736

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Thu Jan 10 02:40:16 CET 2019 on sn-devel-144

libcli: Add error log if insufficient SMB2 credits

Although it's unusual to hit this case, I was seeing it happen while
working on the SMB python bindings. Even with debug level 10, there was
nothing coming out to help pin down the source of the
NT_STATUS_INTERNAL_ERROR.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13736

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

s3:utils:net: Print debug message about Netbios

With a preceding patch, cli_connect_nb() will return
NT_STATUS_NOT_SUPPORTED when 'disable netbios' is set in smb.conf.

Print an informative error message to indicate Netbios is disabled
if this occurs.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13727

Signed-off-by: Justin Stephenson <jstephen@redhat.com>
Reviewed-by: Noel Power <nopower@suse.com>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Wed Jan  9 22:38:21 CET 2019 on sn-devel-144

s3:smbpasswd: Print debug message about Netbios

With a preceding patch, cli_connect_nb() will return
NT_STATUS_NOT_SUPPORTED when 'disable netbios' is set in smb.conf.

Print an informative error message to indicate Netbios is disabled
if this occurs.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13727

Signed-off-by: Justin Stephenson <jstephen@redhat.com>
Reviewed-by: Noel Power <nopower@suse.com>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:libsmb: Print debug message about Netbios

With a preceding patch, cli_connect_nb() will return
NT_STATUS_NOT_SUPPORTED when 'disable netbios' is set in smb.conf.

Print an informative error message to indicate Netbios is disabled
if this occurs.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13727

Signed-off-by: Justin Stephenson <jstephen@redhat.com>
Reviewed-by: Noel Power <nopower@suse.com>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:libsmb: Check disable_netbios in socket connect

If the disable_netbios option is set then return NT_STATUS_NOT_SUPPORTED
for a port 139 connection in the low level socket connection code.

Signed-off-by: Justin Stephenson <jstephen@redhat.com>
Reviewed-by: Noel Power <nopower@suse.com>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:libsmb: Revert SMB Py bindings name back to libsmb_samba_internal

In order to make it clear that the APIs in these Python bindings are
unstable and should not be used by external consumers, this patch
changes the name of the Python bindings back to libsmb_samba_internal.

To make the Python code that uses these bindings (i.e. samba-tool, etc)
look a little cleaner, we can just change the module name as we import
it, e.g.

  from samba.samba3 import libsmb_samba_internal as libsmb

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Wed Jan  9 14:30:31 CET 2019 on sn-devel-144

netcmd/user: python[3]-gpgme unsupported and replaced by python[3]-gpg

python[3]-gpgme is deprecated since ubuntu 1804 and debian 9.
use python[3]-gpg instead, and adapt the API.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13728

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

xml_docs: update traffic script documentation

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Tim Beale <timbeale@samba.org>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Wed Jan  9 03:53:58 CET 2019 on sn-devel-144

traffic_replay: use packets per second as primary scale

The old -S/--scale-traffic is relative to the original model, which made
its relationship to true traffic volumes quite opaque

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic: assert json equality in learner test

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic tests: use TestCaseInTempdir.mktemp()

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/tests: add TestCaseInTempdir.mktemp()

This gives you a name of a temporary file within the test case's tempdir.
Use it like this:

  with self.mktemp() as filename:
     self.check_run("samba-tool foo --output %s" % filename)
     self.assertStringsEqual(open(filename).read(), expected)

and filename will flick out of existence when the with block ends.

This is based on an idea used in the traffic_runner tests, which will
soon be adapted to use this method.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic_replay: --old-scale to mimic the old traffic_replay

traffic_replay had a broken sense of traffic scale. That is fixed, but
in order to compare old and new tests, it helps to be able to
approximate the old behaviour.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic: add option to reanimate dying conversations

The traffic model is generated from a window in time, which makes
conversations appear to start and stop unnaturally at the window
boundaries. When the window is short compared to the traffic replay
time and the true expected conversation length, this has a significant
distorting effect, leading to more conversations than would be
expected to generate a given number of packets.

To offset this slightly we add the --conversation-persistence option
which tries to convert apparent death into a longish wait.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic_model: don't report generation errors as parse errors

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic: improve debug messages in traffic_replay

tracebacks and less nonsense at higher debug levels.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic: Conversation.add_short_packet is discerning about packets

If the packets really wouldn't do anything, we might as well not add them.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic: remove useless branch in stats report

This completes the work of 68c64c634ad7576b2ab5ef838c203320afd90f44,
but differs from that in that it makes no actual change because isatty
was not being called so was always evaluated as true.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic: generate more statistics in replay

Add more "header" values indicating the progress of the run as a
whole.

The new fields are:

Max sleep miss            - the longest sleep() oversleep. Indicates client load.
Maximum lag               - the longest gap between a planned packet
                            time and its actual time.
Start lag                 - the longest gap between intended and actual
                            conversation start.
Planned conversations     - how many conversations we meant to have.
Planned packets           - how many "packets" we thought we were making. Not
                            all "packets" result in actual operations or packets.
Unfinished conversations  - how many conversations had not finished
                            when they were killed.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic: avoid bare except: clauses

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic: simplify tests for badpassword_frequency

x <= 0 will fail one or both of the other test clauses.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic: assign context domain at start

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic: traffic_replay --latency-timeout to control final wait

Conversations that haven't finished within some acceptable margin of
on-time can be said to have failed. This is where you specify that
margin.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic_replay: --stop-on-any-error option to not ignore client trouble

Sometimes you want to know if any client is crashing for any reason.
In those times use --stop-on-any-error for an early exit.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic: evoke OS error codes by name, not number

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic: reinitialise RNG in each child

Otherwise they all replay using the same random sequence.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic: avoid generating conversations without packets

Some "packets" don't generate any actual traffic. If we have a
conversation consisting only of those, we can avoid forking a client
for it.

This *slightly* increases the load over that which would be generated
otherwise for a given traffic rate, but that's OK.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic: rework conversation generation to better use memory

Use less memory altogether and don't allocated shared mutable before
the fork.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic: generate sparser descriptions of conversations

Rather than building all the packets at this point, we stick to the
barest details of the packets (which is all the model gives us
anyway).

The advantage is that will take a lot less memory, which matters
because this process forks into many clients that were sharing and
mutate the conversation list.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic: new version of model with packet_rate, version number

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic: fix hash non-determinism when loading JSON

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Tim Beale <timbeale@samba.org>

traffic: Conversation gets given id, list of short packets

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic: rename packet_rate -> replay_speed for accuracy and room

We are soon going to have a self.packet_rate, and replay_speed is more
accurate in this case.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic: initialise conversation specific lists after the fork

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic: split is_a_real_packet() function out of class

So we can use it to determine whether a packet should be a Packet before
making the leap.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic: give Packet __slots__ for reduced memory

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic: make random_colour_print more deterministic, silent on demand

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic: delete empty temp directories

even if someone asked to keep the temporary data, they don't want
to see 5000 empty directries. Non-empty directories will remain.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic: use namedtuple for ConversationAccounts namespace class

Saving memory, which reduces fork overhead.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic: fix mk_masked_dir doc and comments

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic: Packet.from_line classmethod uses cls var

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic learner: avoid truncated output files on error

add_argument(type=argparse.FileType('w'), ...) will open the file
and leave it empty if the script fails.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic_learner: use samba.logger, not print(file=sys.stderr)

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic_learner: return an error code

And use it in tests, rather than expecting exact strings.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic: use default value for --duration

We could not do this when we replayed traffic summaries (as opposed to
models), but now this script does not do that.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic_replay: drop summary replay

The traffic_replay script has been able to replay a replay log as well
as a model, which was not used in practice and complicated the script.

If we want that feature, we can make a new script for it.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

traffic replay: allow --random-seed=0

Zero is a valid seed.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/traffic: remove duplicate test file

The other one is python/samba/tests/blackbox/testdata/traffic-sample-very-short.model

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/tests/blackbox: check_exit_code() explains by default

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

idmap: In _wbint_Sids2UnixIDs, pass back what we have

SOME_UNMAPPED does not mean that nothing worthwhile is in here. We
need to pass what we have.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Tue Jan  8 13:15:35 CET 2019 on sn-devel-144

idmap_tdb: If one SID fails to map, try the rest

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

selftest: Test sids-to-xids with one failing sid

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>

waf: fix tri-state of --with-sendfile-support being auto the default

--with-sendfile-support should be used automatically if it's found.

Configure now also fails reliably for all platforms when sendfile was
explicitly requested but failed to be detected.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=9707

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Jan  8 07:03:49 CET 2019 on sn-devel-144

waf: fix waf errors with sendfail detection on Solaris

Signed-off-by: Bjoern Jacke <bjacke@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3: net: Do not set NET_FLAGS_ANONYMOUS with -k

This affects net rpc getsid and net rpc changetrustpw commands.
This avoids an anonymous IPC connection being made when -k is used,
this only affects net rpc getsid and net rpc changetrustpw commands.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13726

Signed-off-by: Justin Stephenson <jstephen@redhat.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Noel Power <npower@samba.org>

libnet4: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

winbind4: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

pysecurity: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

auth4: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

ntpd: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

passdb: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

winbind: Fix a typo

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

pydsdb: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libndr: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbclient: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libcli: Simplify smblsa_lookup_name

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

winbindd: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

auth3: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lib: Remove StrnCpy

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Use strlcpy instead of StrnCpy

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

nmbd: Use strlcpy instead of StrnCpy

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libsmb: Use strlcpy instead of StrnCpy

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s4:torture:base:bench-readwrite: fix uninitialized memory causing segfault

Allocation of 'struct smb_composite_connect' happens without zeroing
so that smb_composite_connect_send() thinks that connection
already exists and fails on its handling.

Signed-off-by: Aliaksei Karaliou <akaraliou@panasas.com>
Reviewed-by: Douglas Bagnall <dbagnall@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

netcmd: Small refactor to SMB connection in domain backup

Rework the code so we only do this in one place.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Tim Beale <timbeale@samba.org>
Autobuild-Date(master): Tue Jan  8 03:13:48 CET 2019 on sn-devel-144

netcmd: Refactor duplicated SMB connect in GPO commands

Do the SMB connection in a single helper function.

Note: this highlights that perhaps we want all SMB connections to be
signed, but we can fix that up separately.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

tests: Avoid hardcoding domain in GPO tests

The realm varies by testenv. Currently the GPO tests will only run on
the ad_dc testenv.

A slightly better solution is to use the REALM environmental variable,
so that the test can run on any testenv.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

python/ntacls: Convert ntacls to use s3 flags

This helper code is just using the flags defined by the Python bindings.
Convert it over to use s3 bindings instead of s4.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

s3:pylibsmb: Add flags used by .list() to SMB Py bindings

These flags are exposed by the s4 code. Python code that calls .list()
checks the returned attribs/mode for the directory listing, e.g. to work
out whether something is a sub-directory:

  if item['attrib'] & libsmb.FILE_ATTRIBUTE_DIRECTORY...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

s3:pylibsmb: Rename 'credentials' param to match s4

s4 just calls it 'creds'. Renaming it will make it easier to convert
existing SMB connections over to use the new bindings.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

tests: Rename libsmb_samba_internal test to libsmb

So that it better matches the updated Python bindings name.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

s3:pylibsmb: Rename libsmb_samba_internal Py bindings to libsmb

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

tests: Run SMB Py bindings tests against testenv with SMBv1-disabled

Sanity-check that the SMBv2 connection actually works by running it
against a testenv with SMBv1 disabled.

I've dropped 'local' from the ad_dc target, because it shouldn't be
needed. We're trying to test the client-side SMB connection, so running
it without 'local' is probably a better test.

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Tim Beale <timbeale@samba.org>
Autobuild-Date(master): Mon Jan  7 04:29:51 CET 2019 on sn-devel-144

tests: Avoid hardcoding domain in test

Currently the sysvol domain directory is hard-coded, so the tests can
only ever run on the ad_dc.

This patch makes things marginally better by using the REALM
environmental variable instead. This allows us to run it against other
testenvs (like the SMBv2-only restoredc).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

tests: Completely replace s4 connection in smb tests

This test now uses the s3 python bindings completely, so we can remove
the s4 connection.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s3:pylibsmb: Add .deltree() API to SMB py bindings

This basically re-uses the underlying functionality of existing APIs in
order to support a .deltree() API, i.e.
- we use the .list() functionality (i.e. do_listing()) to traverse every
  item in the given directory.
- we then use either .unlink() (i.e. unlink_file()) or .rmdir() (i.e.
  remove_dir()) to delete the individual item.
- sub-directories are handled recursively, by repeating the process.

Note that the .deltree() API is currently only really used for testing
(and deleting GPO files). So the recursion is never going to be
excessive.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s3:pylibsmb: Minor refactor to py_cli_list() variables

Add a define for the file attribute mask (we'll reuse it in a subsequent
patch), and make the variable type explicit.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s3:libsmb: Avoid duplicated code by making cli_read_sink() public

cli_read_sink() and pull_helper() were essentially identical. By making
cli_read_sink() non-static, we can delete the latter.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

s3:pylibsmb: Add .loadfile() API to SMB py bindings

Add a .loadfile API to read a file's contents. This provides a
convenient way to read a file and is consistent with the existing
source4 API, which is used by things like the GPO python code and the
ntacls backup.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

s3:pylibsmb: Add .savefile() API to SMB py bindings

This provides a simple API for writing a file's contents and makes the
s3 API consistent with the s4 API.

All the async APIs here support SMBv2 so we don't need to use the sync
APIs at all.

Note that we have the choice here of using either cli_write_send() or
cli_push_send(). I chose the latter, because that's what smbclient uses.
It also appears to handle writing a large file better (i.e. one that
exceeds the max write size of the underlying connection).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

s3:pylibsmb: Free async .list() memory

finfos was being allocated but never freed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s3:pylibsmb: Make s3 and s4 listings return the same dict

Make the python dictionary generated by the s3 .list() use the same keys
as the current source4 dict. The reason for using the source4 dict is
that other python code depends on these keys (e.g. ntacls.py), whereas
the source3 API is currently unused.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

s3:pylibsmb: Don't return '.'/'..' in .list()

The source4 .list() API wasn't doing this. This makes source3 and source4
have *almost* equivalent functionality, so we can start usign the
source3 API in the tests.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s3:pylibsmb: Make .list() work for SMBv2

In order for .list() to work on an SMBv2 connection we need to
use the synchronous API.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s3:pylibsmb: Split code out into do_listing() helper

We'll want to reuse the directory listing function for the deltree API.
This patch splits the bulk of the work out into a do_listing() helper
function.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s3:pylibsmb: Split out code into list_helper()

Refactor out the code so we can re-use it for the synchronous API as
well.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>