librpc: Add test for ndr_string_length()

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Dec 20 09:01:30 UTC 2019 on sn-devel-184

librpc: Fix string length checking in ndr_pull_charset_to_null()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14219

Pair-Programmed-With: Guenther Deschner <gd@samba.org>
Signed-off-by: Guenther Deschner <gd@samba.org>
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

upgradedns: ensure lmdb lock files linked

Ensure that the '-lock' files for the dns partitions as well as the data
files are linked when running
  samba_dnsupgrade --dns-backend=BIND9_DLZ
failure to create these links can cause corruption of the corresponding
data file.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14199

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

test upgradedns: ensure lmdb lock files linked

Add tests to check that the '-lock' files for the dns partitions as well as
the data files are linked when running
    samba_dnsupgrade --dns-backend=BIND9_DLZ
failure to create these links can cause corruption of the corresponding
data file.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14199

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

selftest: don't use NTVFS fileserver in chgdcpass

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Fri Dec 20 07:34:42 UTC 2019 on sn-devel-184

selftest: don't use NTVFS fileserver in rodc

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

selftest: don't use NTVFS fileserver in fl2008r2dc

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

selftest: don't use NTVFS fileserver in fl2003dc

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

selftest: don't use NTVFS fileserver in fl2000dc

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

selftest: don't use NTVFS fileserver in vampire_dc

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

selftest: don't use NTVFS fileserver in promoted_dc

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

selftest: make fl2008dc an alias for ad_dc, not ad_dc_ntvfs

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

selftest: make ad_dc_slowtests an alias for ad_dc, not ad_dc_ntvfs

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

selftest: make ad_dc_default an alias for ad_dc, not ad_dc_ntvfs

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

selftest: run samba.tests.samba_tool.user against ad_dc_ntvfs:local explicitly

No change in behaviour.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

selftest: run samba.ldap.referrals against ad_dc_ntvfs explicitly

No change in behaviour.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

selftest: run samba4.ldap.dipython against ad_dc_ntvfs explicitly

No change in behaviour.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

selftest: run samba.tests.dcerpc.srvsvc against ad_dc_ntvfs explicitly

No change in behaviour.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

selftest: run rpc.srvsvc and rpc.mgmt against ad_dc_ntvfs explicitly

No change in behaviour.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

selftest: hardcode ad_dc_ntvfs for the rpc.netlogon testsuite

The rpc.netlogon testsuite has a test that verifies LSA over netlogon which is
only enabled in the ad_dc_ntvfs env.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

selftest: make testenv name logic more flexible for the rpc testcases

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

selftest: use ad_dc_ntvfs env instead of ad_dc_default for samba4.ldb.ldaps

ad_dc_default is currently an alias for ad_dc_ntvfs, so this is currently no
change in behaviour, but this is going to change.

As the ad_dc_ntvfs env specifies "ldap server require strong auth =
allow_sasl_over_tls" and this is needed for the test, we have to let the test
use the ad_dc_ntvfs env explicitly.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

libsmbclient: If over SMB1 first try to do a posix stat on the file.

Disable in future, if server doesn't support this.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14101

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu Dec 19 15:44:25 UTC 2019 on sn-devel-184

s3:libsmb: Add a setup_stat_from_stat_ex() function

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14101

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:libsmb: Return a 'struct stat' buffer for SMBC_getatr()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14101

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:libsmb: Add try_posixinfo to SMBSRV struct. Only enable for SMB1 with UNIX for now.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14101

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:libsmb: Generate the inode only based on the path component

Currently we use the full smb url which includes also username and
password.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14101

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:script: Try to fix a Perl warning

Scalar value @ENV{"BASH_ENV"} better written as $ENV{"BASH_ENV"} at
/tmp/samba-testbase/b23/samba-ad-dc-1/source3/script/tests/printing/modprinter.pl
line 134.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

source4/utils/oLschema2ldif: include stdint.h before cmocka.h

This fix the following build failure:

In file included from /home/fabrice/buildroot/output/host/opt/ext-toolchain/lib/gcc/mips64el-buildroot-linux-uclibc/5.5.0/include/stdint.h:9:0,
                 from /home/fabrice/buildroot/output/host/mips64el-buildroot-linux-uclibc/sysroot/usr/include/inttypes.h:27,
                 from ../../lib/replace/../replace/replace.h:64,
                 from ../../source4/include/includes.h:23,
                 from ../../source4/utils/oLschema2ldif/test.c:25:
/home/fabrice/buildroot/output/host/mips64el-buildroot-linux-uclibc/sysroot/usr/include/stdint.h:122:27: error: conflicting types for ‘uintptr_t’
 typedef unsigned long int uintptr_t;
                           ^
In file included from ../../source4/utils/oLschema2ldif/test.c:23:0:
/home/fabrice/buildroot/output/host/mips64el-buildroot-linux-uclibc/sysroot/usr/include/cmocka.h:132:28: note: previous declaration of ‘uintptr_t’ was here
       typedef unsigned int uintptr_t;

Fixes:
 - http://autobuild.buildroot.org/results/9507739b3d5d51024ee9c60b74c2f85d5004e7e2

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14218

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Reviewed-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Wed Dec 18 16:57:52 UTC 2019 on sn-devel-184

lib/ldb/tests: include stdint.h before cmocka.h

This fix the following build failures:

[2466/3864] Linking bin/default/lib/ldb/ldbmodify
In file included from /home/buildroot/autobuild/instance-0/output-1/host/opt/ext-toolchain/lib/gcc/mips64el-buildroot-linux-uclibc/5.5.0/include/stdint.h:9:0,
                 from ../../lib/tevent/tevent.h:31,
                 from ../../lib/ldb/include/ldb.h:51,
                 from ../../lib/ldb/tests/test_ldb_dn.c:25:
/home/buildroot/autobuild/instance-0/output-1/host/mips64el-buildroot-linux-uclibc/sysroot/usr/include/stdint.h:122:27: error: conflicting types for 'uintptr_t'
 typedef unsigned long int uintptr_t;
                           ^
In file included from ../../lib/ldb/tests/test_ldb_dn.c:23:0:
/home/buildroot/autobuild/instance-0/output-1/host/mips64el-buildroot-linux-uclibc/sysroot/usr/include/cmocka.h:132:28: note: previous declaration of 'uintptr_t' was here
       typedef unsigned int uintptr_t;
                            ^

In file included from /home/buildroot/autobuild/instance-0/output-1/host/opt/ext-toolchain/lib/gcc/mips64el-buildroot-linux-uclibc/5.5.0/include/stdint.h:9:0,
                 from ../../lib/tevent/tevent.h:31,
                 from ../../lib/ldb/tests/ldb_key_value_test.c:48:
/home/buildroot/autobuild/instance-0/output-1/host/mips64el-buildroot-linux-uclibc/sysroot/usr/include/stdint.h:122:27: error: conflicting types for 'uintptr_t'
 typedef unsigned long int uintptr_t;
                           ^
In file included from ../../lib/ldb/tests/ldb_key_value_test.c:43:0:
/home/buildroot/autobuild/instance-0/output-1/host/mips64el-buildroot-linux-uclibc/sysroot/usr/include/cmocka.h:132:28: note: previous declaration of 'uintptr_t' was here
       typedef unsigned int uintptr_t;
                            ^

Fixes:
 - http://autobuild.buildroot.org/results/9507739b3d5d51024ee9c60b74c2f85d5004e7e2

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14218

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Reviewed-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

Avoiding bad call flags with python 3.8, using METH_NOARGS instead of zero.

(C) SBE network solutions GmbH

Signed-off-by: Torsten Fohrer <torsten.fohrer@sbe.de>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Wed Dec 18 14:33:58 UTC 2019 on sn-devel-184

samba_kcc: avoid ValueError when local connections are less than 2

Signed-off-by: Björn Baumbach <bb@sernet.de>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Björn Baumbach <bb@sernet.de>
Autobuild-Date(master): Wed Dec 18 11:37:53 UTC 2019 on sn-devel-184

lib/replace: prefer <sys/xattr.h> over <attr/xattr.h>

This prevents the following compile error that may happens if "system/filesys.h"
is included before "system/capability.h" on Ubuntu 16.04:

  [1802/4407] Compiling source3/lib/system.c
  In file included from ../../lib/replace/system/filesys.h:112:0,
                   from ../../source3/include/vfs.h:29,
                   from ../../source3/include/smb.h:150,
                   from ../../source3/include/includes.h:284,
                   from ../../source3/lib/system.c:23:
  /usr/include/x86_64-linux-gnu/sys/xattr.h:32:3: error: expected identifier before numeric constant
     XATTR_CREATE = 1, /* set value, fail if attr already exists.  */
     ^

The above error is from compiling a source tree which includes a change that
adds an include "system/filesys.h" to the top of "source3/include/vfs.h".

"source3/lib/system.c" has the following includes:

  #include "includes.h"
  #include "system/syslog.h"
  #include "system/capability.h"
  #include "system/passwd.h"
  #include "system/filesys.h"
  #include "../lib/util/setid.h"

The first include of "includes.h" pulls in "vfs.h" which will pull in
"system/filesys.h" with the mentioned change. "system/filesys.h" pulls in
<attr/xattr.h> which has this define

  #define XATTR_CREATE  0x1

Later in "source3/lib/system.c" "system/capability.h" is included which includes
<sys/xattr.h> on Ubuntu 16.04 (not in later versions of glibc). This defines the
XATTR_* values as an enum:

  enum {
    XATTR_CREATE = 1,     /* set value, fail if attr already exists.  */
    XATTR_REPLACE = 2     /* set value, fail if attr does not exist.  */
  };

The previous define of XATTR_CREATE as 1 makes this

  enum {
    1 = 1,     /* set value, fail if attr already exists.  */
    2 = 2     /* set value, fail if attr does not exist.  */
  };

which is invalid C. The compiler error diagnostic is a bit confusing, as it
prints the original enum from the include file.

See also:

<https://bugs.freedesktop.org/show_bug.cgi?id=78741>
<https://bugs.launchpad.net/ubuntu/+source/attr/+bug/1288091>
<https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=756097>

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Björn Baumbach <bb@samba.org>

librpc: Move winstation.idl to the top level and exclude from fuzzing

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Dec 18 08:05:05 UTC 2019 on sn-devel-184

lib/fuzzing and librpc: Do not generate fuzzers for pointless targets

We need to focus the fuzzing effort on reachable code, and these IDL
are just historical artifacts, many are entirely [todo] and have
no samba client nor server.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

lib/fuzzer: Allow coverage build for oss-fuzz

This still does not seem to be enough but it is one step towards a working
coverage build.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

lib ldb: fix use after free

Fix ASAN detected use after free.  No security implications as  the
talloc_free is followed immediately by the print statement and the value
printed is an integer

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

librpc: Fix manually written printer for drsuapi_DsAttributeValue

Credit to OSS-Fuzz

Found using the ndr_fuzz_X target.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

sefltest: Demonstrate crash in manually written printer for drsuapi_DsAttributeValue

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

lib/fuzzing: Allow load of fuzz inputs as files on the command line

This is easier to put under gdb.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

lib/fuzzing: Initialise st buffer in fuzz_ndr_X

An NDR pull of a function will fill in either the in. or out.
elements of this structure, but never both.

However, some structures have size_is() in the out. that reference
the in. elements.  This is the reason for the --context-file option
in ndrdump.

We have a special handler in the fuzzing case embedded in the
pidl-generated output to cope with this, by filling in pointers
for elements declared [ref,in] but it relies on the in-side
(at least) of the buffer being zeroed.

So zero the buffer before we start.  Sadly this means things
like valgrind can not find a use of uninitialised data, but that
is a price we have to pay.

Credit to OSS-Fuzz

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

smbd: Convert share_mode_data->num_share_modes into a boolean8

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Dec 18 00:05:13 UTC 2019 on sn-devel-184

smbd: Don't store "num_share_modes" in locking.tdb

With the last commit we don't store the share mode entry count
anymore. With this commit we go one step further and avoid storing
it. If there's valid record in locking.tdb, there is a corresponding
record in share_entries.tdb, so there's no point storing that once
more explicitly.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Use share_mode_data->num_share_modes as a boolean

This is a micro-commit showing that we don't actually need
share_mode_data->num_share_modes as a number *counting* the share mode
entries in share_entries.tdb anymore. Instead, we are only using it as
an indication for share_mode_lock_destructor() to see whether share
entries are around or not, i.e. whether it's worth keeping or deleting
the record in locking.tdb.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Avoid a direct access to share_mode_data->num_share_modes

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Introduce share_mode_have_entries()

This hides a use of share_mode_data->num_share_modes in
share_mode_lock.c

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Avoid a reference to share_mode_data->num_share_modes

share_mode_data->num_share_modes will go away soon, count the values
directly while walking the array.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Avoid a reference to share_mode_data->num_share_modes

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Pass num_share_modes to share_mode_entry_do() callback

mark_share_mode_disconnected_fn() will need this, the information is
easily available and should not hurt the other callers.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

net: Use share_mode_count_entries()

Avoid a reference to share_mode_data->num_share_modes

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Add share_mode_count_entries()

In order to not write the share mode on every open/close, we need to get rid of
share_mode_data->num_share_modes. "net tdb" needs this information precisely
though, and it's pretty cheap to calculate.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs_ceph_snapshots: fix root relative path handling

For file paths relative to root, ceph_snap_get_parent_path() may return
an empty parent dir string, in which case the CephFS snashot path should
be ".snap".

Bug: https://bugzilla.samba.org/show_bug.cgi?id=14216

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3: VFS: glusterfs: Reset nlinks for symlink entries during readdir

On receiving an already initialized stat_ex buffer for readdir() call we
invoke readdirplus() GlusterFS API, an optimized variant of readdir(),
which then returns stat information along with dir entry result. But for
symlink entries we don't know if link or target info is needed. In that
case it is better to leave this decision back to caller by resetting
nlinks value inside stat information to make it invalid.

This was also preventing us from displaying msdfs link as directories
inside the share.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14182

Signed-off-by: Anoop C S <anoopcs@redhat.com>
Reviewed-by: Günther Deschner <gd@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Dec 17 21:53:07 UTC 2019 on sn-devel-184

mit-kdb: Fix license on header file

Signed-off-by: Simo Sorce <idra@samba.org>
Signed-off-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Tue Dec 17 09:24:56 UTC 2019 on sn-devel-184

s3-rpc_server: always print the full PDU.

Guenther

Signed-off-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

smbd: Use explicit share_mode_wakeup_waiters()

We don't need to unparse the locking.tdb record, we just need to wake
up waiters when removing an oplock

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Mon Dec 16 20:27:59 UTC 2019 on sn-devel-184

smbd: Use explicit share_mode_wakeup_waiters()

We don't need to unparse the locking.tdb record, we just need to wake
up waiters when downgrading a lease

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Explicitly state when modifying share_mode_data

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

torture4: Fix a typo

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Fix a typo

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

smbd: Avoid casts in open_directory()

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3: smbd: msdfs: Factor out the code to create a msdfs:referral,list into a separate function.

This will allow it to be called from other places once the get/set_msdfs
calls are moved into being first class VFS functions.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Mon Dec 16 15:32:08 UTC 2019 on sn-devel-184

s3: smbd: cleanup. Change 'int referral_count' -> 'size_t referral_count' in struct junction_map.

This is a non-negative count. Fix remaing code to not mix int and size_t.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: smbd: msdfs: Cleanup, don't mix int and size_t types for a count variable.

Add integer wrap check.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

lib/replace: work around an API conflict between ncurses and XFS xattr API

Compile error:

  [4530/4693] Compiling source3/utils/regedit_list.c
  In file included from ../../source3/utils/regedit_list.h:24,
                   from ../../source3/utils/regedit_list.c:20:
  /usr/include/curses.h:611:28: error: conflicting types for ‘attr_get’
    611 | extern NCURSES_EXPORT(int) attr_get (attr_t *, NCURSES_PAIRS_T *, void *); /* generated */
        |                            ^~~~~~~~
  compilation terminated due to -Wfatal-errors.

Both ncurses and XFS xattr API provide a get_attr() function. As a workaround
avoid including <sys|attr/attributes.h> if <attr|sys/xattr.h> is present.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Dec 12 20:22:51 UTC 2019 on sn-devel-184

libndr: Return enum ndr_err_code from ndr_{pull,push}_steal_switch_value()

This breaks the ABI so we merge this into the unreleased libndr-1.0.0.

The advantage of the new functions is there (except for print, which
is unchanged) is an error raised when the token is not found, so
we can be confident in the changes to the token behaviour.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Dec 12 03:56:23 UTC 2019 on sn-devel-184

librpc/ndr: Remove ndr_{push,pull}_get_switch_value()

By removing this we know we do not need to worry about this list
growing without bounds.  We merge this into the recently created but
not yet released ABI 1.0.0

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

librpc: Remove last callers of ndr_push_get_switch_value()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

librpc: Remove last callers of ndr_pull_get_switch_value()

By removing this we know we do not need to worry about this list
growing without bounds.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

librpc: Do not follow a NULL pointer when calculating the size of a union

Found by Douglas Bagnall using Hongfuzz and the new fuzz_ndr_X
fuzzer.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

pidl: Mismatch between set and get of relative base pointers

The set was within the switch, the get was before the switch.

The difference is shown when there is an empty default element.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

librpc: Do not follow a NULL pointer when calculating the size of a structure

Found by Douglas Bagnall using Hongfuzz and the new fuzz_ndr_X
fuzzer.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

pidl: Mismatch between set and get of relative base pointers

The set was within the switch, the get was before the switch.

The difference is shown when there is an empty default element.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

pidl: Add and use ndr_print_steal_switch_value(), removing ndr_print_get_switch_value()

This avoids really long token lists for switch values
that will not be needed past this point.

The function name is changed to clarify what exactly is being
done here, and the old function is removed to ensure it is
not being used anywhere else.

Merge the removal of ndr_print_get_switch_value into
just-tagged librpc/ABI/ndr-1.0.0.sigs as this
has not been put into any release yet.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

librpc/ndr: Add ndr_push_steal_switch_value()

This will allow generated code to instead push and pop union values onto the
switch_list stack, which is more memory efficient than creating a single large
list to be scannned and eventually discarded.

Merge into unreleased ABI 1.0.0

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

pidl: Generate and consume the switch level token for both NDR_SCALARS and NDR_BUFFERS in ndr_pull()

This means what was previously a list becomes a single variable that
could be passed as a function paraemter, but this is avoided for now
because it would change the ABI and be more intrusive.

Before this, a client could cause a NDR token containing the swith level
to be allocated for each and every element in the array that they
promised they were sending (without having to actually send them).

Found by Michael Hanselmann using Honggfuzz and an fuzzer for Samba's
NDR layer.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

librpc: Set the switch_value before NDR_BUFFERS to prepare for new libndr behaviour

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

negoex: Set the switch_value before NDR_BUFFERS to prepare for new libndr behaviour

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

s4-libcli/rap: Set the switch_value before NDR_BUFFERS to prepare for new libndr behaviour

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

ndr: Restrict size of ndr_token lists to avoid memory abuse by malicious clients

This is designed to stop a very large number of tokens from being stored for
arrays of structures containing relative pointers in particular.

This was one part of the minimum patch for CVE-2019-14908 before
being downgraded as not a security-release worthy issue.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

libndr: Do not overwrite token list with NULL on allocation failure

This was one part of the minimum patch for CVE-2019-14908 before
being downgraded as not a security-release worthy issue.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

selftest: Add test for ndr_size_struct() faulting on a NULL pointer

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

selftest: Add test for ndr_size_union() faulting on a NULL pointer

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

selftest: Add example xattr_NTACL packets to demonstrate switch/union behaviour

This is a good example with both buffers and scalars in the union.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

selftest: Add test for structure with NDR_BUFFERS only in a union

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13876

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

selftest: Do not force the endpoint for fsrvp tests

The test suite will bind to the srvsvc interface, let it find the
correct endpoint through the endpoint mapper.

Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Dec 12 02:00:19 UTC 2019 on sn-devel-184

s4:torture/rpc: Fix torture comment in mdssvc.c

Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

pidl:NDR/Server: Allow to define endpoint server shutdown functions

The next commits will register legacy api_struct when the endpoint server
is initialized. This commit adds a shutdown function which will be used
to unregister the legacy api_struct.

The shutdown function will be also used to replace the rpc_srv_callbacks
struct shutdown member used, for example, by the spoolss service to
cleanup before exiting.

Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

librpc:core: Add a function to reinitialize the dcesrv_context

Clears all registered endpoints and interfaces, association groups and
broken connections.

To be used by S3 forked daemons.

Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

librpc:core: Add public functions to initialize endpoint servers

The dcesrv_init_registered_ep_servers() will be used by the S3 server to
initialize all registered endpoint servers (for embedded services), and
the dcesrv_init_ep_server() function will be used by the external
daemons to initialize the required ones.

As serveral S3 services may require to initialize another one before
itself (svcctl and eventlog for example require winreg) a boolean flag is
added to track the initialization status.

Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

librpc:core: Split dcesrv context init and endpoint servers init

The S4 server will initialize the endpoint servers specified in smb.conf,
but the S3 server need to initialize all registered endpoint servers (the
embedded ones).

Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

librpc/idl/dnsserver.idl: Ensure DnsProperty id matches what is pulled from the stored buffer

There are two concerns here, assuming the attacker can place arbitary values
in a dnsProperty attribute over LDAP (eg is a DNS administrator).

This comes from the fact that id is used as the switch value at the C layer
but at the NDR layer the wDataLength value is considered first.

One concern is that a pull/push round-trip could include server memory:

 The previous switch_is() behaviour could store the server memory back
 into the attribute.

 However this pattern of pull/push only happens in ndrdump and fuzzing tools, as
 dnsserver_db_do_reset_dword() operates only on the uint32/bitmap union
 arms, and fully initialises those.

The other is that a pull of the attacker-supplied value could
cause the server to expose memory.

 This would be over the network via DNS or the RPC dnsserver protocols.
 However at all times the ndr_pull_struct_blob is passed zeroed memory.

The final concern (which fuzz_ndr_X found) is that in the ndr_size_dnsPropertyData()
the union descriminent is only id.

 This has no impact as only zeroed memory is used so there will be a
 zero value in all scalars, including data->d_ns_servers.AddrArray.

 Therefore the server will not crash processing the attacker-supplied blob

[MS-DNSP] 2.3.2.1 dnsProperty has no mention of this special behaviour.
https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-dnsp/445c7843-e4a1-4222-8c0f-630c230a4c80

This was known as CVE-2019-14908 before being triaged back to a normal bug.

Found by Douglas Bagnall using Hongfuzz and the new fuzz_ndr_X fuzzer.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14206
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@samba.org>

librpc dnsp test: Ensure length matches union selector

Ensure that a dnsp_DnsProperty is rejected if the length data does not not
correspond to the length indicated by the union id.  It was possible for
the union to be referencing memory past the end of the structure.

Found by Douglas Bagnall using Hongfuzz and the new fuzz_ndr_X fuzzer.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=14206
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

lib/krb5_wrap: Remove unused smb_krb5_get_allowed_weak_crypto()

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Isaac Boukris <iboukris@samba.org>
Autobuild-User(master): Isaac Boukris <iboukris@samba.org>
Autobuild-Date(master): Wed Dec 11 22:18:47 UTC 2019 on sn-devel-184

lib/fuzzing: Fix argument order to ldb_filter_from_tree in fuzz_ldb_parse_tree

Found by the oss-fuzz CI tooling.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Dec 11 04:21:28 UTC 2019 on sn-devel-184

lib/fuzzing: Split up automatically build fuzzers into TYPE_{IN,OUT,STRUCT}

The advise is that a fuzz target should be as small as possible
so we split this up.  Splitting up by function would build too
many fuzzers, but this should help a little.

See for example:
https://github.com/google/fuzzing/blob/master/docs/good-fuzz-target.md#large-apis

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@samba.org>

lib/fuzzing: Ensure mem_ctx is freed each time fuzz_ldb_parse_tree is run

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@samba.org>

autobuild: extend autobuild with samba-fuzz job to build the fuzzers in AFL mode using oss-fuzz scripts

This helps ensure the build_samba.sh file keeps working and the fuzzers build
(because they are excluded from the main build).

This is not in the default autobuild because it uses too much
space on sn-devel (4GB).

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@samba.org>

autobuild.py: Avoid listing jobs twice

We use the tasks table instead, to avoid the issue shown in the previous commit.

Now we just have to keep .gitlab-ci.yml and the tasks table in sync.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@samba.org>

autobuild.py: Add missing samba-simpleserver job

This was missed when the job was split out in f0e8dd1a08698884209873bb84002d7b34db016c.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Gary Lockyer <gary@samba.org>