Merge tag 'ldb-1.1.24' into master

ldb: tag release ldb-1.1.24

smbd: make "hide dot files" option work with "store dos attributes = yes"

When using "store dos attributes = yes", the function that reads the
attributes from the xattr get_ea_dos_attribute() will overwrite the
attribute previously set for "hide dot files".

According to smb.conf, "store dos attributes = yes" should only
overwrite the "map XXX" options, but not "hide dot files".

Bug: https://bugzilla.samba.org/show_bug.cgi?id=11645

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Dec 16 07:21:10 CET 2015 on sn-devel-104

librpc: Fix typos

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ira Cooper <ira@samba.org>

lib: Remove ntstatus.h from gencache.h

No clue why I put it there, sorry for the noise...

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>

vfs_glusterfs: Attach missing destructor.

This activates the new AIO code's cancellation logic.

Signed-off-by: Ira Cooper <ira@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Dec 15 23:33:12 CET 2015 on sn-devel-104

smbd: Fix CID 1343333 Uninitialized variables

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

dns_server: Remove unused handle_question

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Kai Blin <kai@samba.org>
Autobuild-User(master): Kai Blin <kai@samba.org>
Autobuild-Date(master): Tue Dec 15 17:50:32 CET 2015 on sn-devel-104

dns_server: Add handle_authoritative_send()

An async version of handle_question

Bug: https://bugzilla.samba.org/show_bug.cgi?id=9409
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Kai Blin <kai@samba.org>

dns_server: Add add_dns_res_rec()

Same as add_response_rr(), but it copies over a dns_res_rec

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Kai Blin <kai@samba.org>

dns_server: Convert "ask_forwarder" params

Usually we have mem_ctx and ev first when doing a _send function

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Kai Blin <kai@samba.org>

dns_server: Simplify array length handling

talloc objects carry an implicit length

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Kai Blin <kai@samba.org>

dns_server: Simplify talloc handling

By making sure that the answers are always allocated, we don't have
to pass an explicit mem_ctx anymore

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Kai Blin <kai@samba.org>

dns_server: Consolidate talloc_realloc

This puts the talloc_realloc into add_response_rr instead of before
create_response_rr. It is a bit less efficient, but as we do not expect
hundreds of answers, I think this code is a bit easier to understand.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Kai Blin <kai@samba.org>

Fix bug 10881 Wrong keytab permissions when joining additional DC with BIND backend

BUG: https://bugzilla.samba.org/show_bug.cgi?id=10881
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Dec 15 11:47:21 CET 2015 on sn-devel-104

samba_upgradedns: Set correct permissions on secrets.keytab for BIND9

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

samba_upgradedns: Improve search for existing accounts in secrets.ldb

We should actually check for the combination of both an account in secrets.ldb
and sam.ldb, but this is at least an improvement.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

samba_dnsupdate: Simplify logic and add more verbose debugging

By reducing the intendation this code is a little clearer

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

samba_dnsupdate: Expand output when --verbose is set

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

python: Give a more helpful error message when we do not have an smb.conf

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

password_lockout: test creds.get_kerberos_state()

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Garming Sam <garming@samba.org>
Autobuild-Date(master): Tue Dec 15 03:17:52 CET 2015 on sn-devel-104

auth: keep track of lastLogon and lastLogonTimestamp

lastLogon is supposed to be updated for every interactive or kerberos
login, and (according to testing against Windows2012r2) when the bad
password count is non-zero but the lockout time is zero. It is not
replicated.

lastLogonTimestamp is updated if the old value is more than 14 -
random.choice([0, 1, 2, 3, 4, 5]) days old, and it is replicated. The
14 in this calculation is the default, stored as
"msDS-LogonTimeSyncInterval", which we offer no interface for
changing.

The authsam_zero_bad_pwd_count() function is a convenient place to
update these values, as it is called upon a successful logon however
that logon is performed. That makes the function's name inaccurate, so
we rename it authsam_logon_success_accounting(). It also needs to be
told whet5her the login is interactive.

The password_lockout tests are extended to test lastLogon and
lasLogonTimestamp.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Ralph Boehme <slow@samba.org>

password_lockout tests: add assertLoginFailure()

In a few places where a login should fail in a particular way, an
actual login success would not have triggered a test failure -- only
the wrong kind of login failure was caught.

This makes a helper function to deal with them all.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Ralph Boehme <slow@samba.org>

auth: increase resolution for password grace period calculation

This changes the resolution of "now" from 1s to 100ns.

It should have little effect in practice, unless users are in the
habit of playing chicken with the grace period.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Ralph Boehme <slow@samba.org>

pycredentials: add get_kerberos_state() method

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Ralph Boehme <slow@samba.org>

s4:torture/winbind: add more debug output to samba4.winbind.struct.domain_info

With this we hopefully find the reason for the following flakey test:

  [1566(10157)/1882 at 1h47m18s] samba4.winbind.struct(ad_member:local)
  Running WINBINDD_DOMAIN_INFO (struct based)
  DOMAIN 'BUILTIN' => '' [ ] [S-1-5-32]
  DOMAIN 'LOCALADMEMBER' => '' [ ] [S-1-5-21-4121020324-2900821022-46155812]
  DOMAIN 'SAMBADOMAIN' => 'samba.example.com' [ PR AD NA ] [S-1-5-21-929009974-669086582-3038401809]
  DOMAIN 'TORTURE300' => 'torturedom300.samba._none_.example.com' [ AD NA ] [S-1-5-21-97398-379795-300]
  DOMAIN 'TORTURE301' => 'torturedom301.samba._none_.example.com' [ AD NA ] [S-1-5-21-97398-379795-301]
  DOMAIN 'TORTURE302' => 'torturedom302.samba._none_.example.com' [ AD NA ] [S-1-5-21-97398-379795-302]
  DOMAIN 'TORTURE303' => 'torturedom303.samba._none_.example.com' [ AD NA ] [S-0-0]
  UNEXPECTED(failure): samba4.winbind.struct.domain_info(ad_member:local)
  REASON: Exception: Exception: ../source4/torture/winbind/struct_based.c:460:
  Expression `ok' failed: SID's doesn't match

With the changes we get:

  [1566(10158)/1882 at 1h47m51s] samba4.winbind.struct(ad_member:local)
  Running WINBINDD_DOMAIN_INFO (struct based)
  LIST[0] 'BUILTIN' => '' [S-1-5-32]
  LIST[1] 'LOCALADMEMBER' => '' [S-1-5-21-734569583-677146317-1850798319]
  LIST[2] 'SAMBADOMAIN' => 'samba.example.com' [S-1-5-21-1870621479-3245899124-866531092]
  LIST[3] 'TORTURE300' => 'torturedom300.samba._none_.example.com' [S-1-5-21-97398-379795-300]
  LIST[4] 'TORTURE301' => 'torturedom301.samba._none_.example.com' [S-1-5-21-97398-379795-301]
  LIST[5] 'TORTURE302' => 'torturedom302.samba._none_.example.com' [S-1-5-21-97398-379795-302]
  LIST[6] 'TORTURE303' => 'torturedom303.samba._none_.example.com' [S-1-0-0]
  LIST[7] 'TORTURE304' => 'torturedom304.samba._none_.example.com' [S-1-0-0]
  LIST[8] 'TORTURE305' => 'torturedom305.samba._none_.example.com' [S-1-0-0]
  LIST[9] 'TORTURE306' => 'torturedom306.samba._none_.example.com' [S-1-5-21-97398-379795-306]
  LIST[10] 'TORTURE307' => 'torturedom307.samba._none_.example.com' [S-1-5-21-97398-379795-307]
  LIST[11] 'TORTURE308' => 'torturedom308.samba._none_.example.com' [S-1-5-21-97398-379795-308]
  LIST[12] 'TORTURE309' => 'torturedom309.samba._none_.example.com' [S-1-5-21-97398-379795-309]
  LIST[13] 'TORTURE310' => 'torturedom310.samba._none_.example.com' [S-1-5-21-97398-379795-310]
  LIST[14] 'TORTURE311' => 'torturedom311.samba._none_.example.com' [S-1-5-21-97398-379795-311]
  DOMAIN[0] 'BUILTIN' => '' [ ] [S-1-5-32]
  DOMAIN[1] 'LOCALADMEMBER' => '' [ ] [S-1-5-21-734569583-677146317-1850798319]
  DOMAIN[2] 'SAMBADOMAIN' => 'samba.example.com' [ PR AD NA ] [S-1-5-21-1870621479-3245899124-866531092]
  DOMAIN[3] 'TORTURE300' => 'torturedom300.samba._none_.example.com' [ AD NA ] [S-1-5-21-97398-379795-300]
  DOMAIN[4] 'TORTURE301' => 'torturedom301.samba._none_.example.com' [ AD NA ] [S-1-5-21-97398-379795-301]
  DOMAIN[5] 'TORTURE302' => 'torturedom302.samba._none_.example.com' [ AD NA ] [S-1-5-21-97398-379795-302]
  DOMAIN[6] 'TORTURE303' => 'torturedom303.samba._none_.example.com' [ AD NA ] [S-0-0]
  UNEXPECTED(failure): samba4.winbind.struct.domain_info(ad_member:local)
  REASON: Exception: Exception: ../source4/torture/winbind/struct_based.c:471: Expression `ok' failed: SID's doesn't match [S-1-0-0] != [S-0-0]

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Mon Dec 14 23:26:40 CET 2015 on sn-devel-104

lib: Remove unused talloc_append_blob

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

gencache: Refactor gencache_set_data_blob

Replace 3 calls into talloc with 1. Add an overflow check.

With this change, it will be easier to avoid the talloc call for small
blobs in the future and do it on the stack.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lib: Separate out xx_path() & callers

We should not have to #include proto.h just for cache_path() or so

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lib: Use directory_create_or_exist in xx_path

directory_create_or_exist is a little different: It does the lstat first and
sets the umask properly, but I think this is more correct than the xx_path()
version before.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

gencache: True->true, False->false

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lib: Add gencache.h

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

docs-xml: Update idmap_rfc2307 manpage for new realm handling

Now there is only "realm" as a config option; it replaces "cn_realm" and
"ldap_realm".

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Mon Dec 14 15:43:55 CET 2015 on sn-devel-104

idmap_rfc2307: Fix handling of cn realm

When cn_realm was set, the idmap_rfc2307 module tried to determine the
realm from the AD connection struct. In case of referring to a different
domain using the ldap_domain config option, the wrong realm was used.

Since the LDAP-server case already requires having the realm in the
config, extend that to the AD case to fix the issue: Having LDAP records
with @realm in the cn, now always requires having the realm in the
config.

Now cn_realm and ldap_realm always would have to be specified together,
so replace the two options with a single "realm" option.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

s3: smbd: When requesting posix open in open_file_ntcreate() we need to set all posix flags.

Fixes POSIX rename problem introduced in d698cec1c7e700e57cab46d33df0dde13303b318

Signed-off-by: Ralph Boehme <rb@sernet.de>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Mon Dec 14 02:03:12 CET 2015 on sn-devel-104

s3: smbd: Moving lp_posix_pathnames() out of the lower-level code.

Remove lp_posix_pathnames() out of ms_has_wild().

NB. The usage of ms_has_wild() inside set_namearray()
should *never* have been looking at lp_posix_pathnames()
anyway, as this is a config option that needs to look
at wildcards. This was probably an old (but never
triggered) bug.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: smbd: Moving lp_posix_pathnames() out of the lower-level code.

Prepare to remove lp_posix_pathnames() out of ms_has_wild().
Check before calls to ms_has_wild().

Fix smbd_smb2_query_directory_send().

No SMB2 client uses unix extensions yet, but this is a placeholder
for when we move the POSIX pathnames bit into the SMB2 request
when moving to handle based code.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: smbd: Moving lp_posix_pathnames() out of the lower-level code.

Prepare to remove lp_posix_pathnames() out of ms_has_wild().
Check before calls to ms_has_wild().

Fixup reply_search().

Don't think any client makes this call with POSIX extensions
on, but this keeps the same old behavior.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: smbd: Moving lp_posix_pathnames() out of the lower-level code.

Prepare to remove lp_posix_pathnames() out of ms_has_wild().
Check before calls to ms_has_wild().

Fix open_file().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: smbd: Moving lp_posix_pathnames() out of the lower-level code.

Prepare to remove lp_posix_pathnames() out of ms_has_wild().
Check before calls to ms_has_wild().

Fixup reply_ntrename().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: smbd: Moving lp_posix_pathnames() out of the lower-level code.

Prepare to remove lp_posix_pathnames() out of ms_has_wild().
Check before calls to ms_has_wild().

Fixup unix_convert().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: smbd: Moving lp_posix_pathnames() out of the lower-level code.

Prepare to remove lp_posix_pathnames() out of ms_has_wild().
Check before calls to ms_has_wild().

Fixup check_parent_exists().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: smbd: Moving lp_posix_pathnames() out of the lower-level code.

Prepare to remove lp_posix_pathnames() out of ms_has_wild().
Check before calls to ms_has_wild().

Fix determine_path_error().

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

s3: smbd: Moving lp_posix_pathnames() out of the lower-level code.

Ensure we set posix_pathnames early.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

ctdb-client: Only get capabilities from active nodes

This is used by the recovery daemon to validate the current recovery
master.  Don't risk being unable to elect a new master if the current
master is inactive but unresponsive.

Note that this client call is currently not used by any other callers.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Sun Dec 13 03:17:10 CET 2015 on sn-devel-104

vfs_glusterfs: Move vfs_gluster_write and vfs_gluster_pwrite.

Move the functions to a more logical location.

Signed-off-by: Ira Cooper <ira@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Dec 12 01:03:40 CET 2015 on sn-devel-104

vfs_glusterfs: Add white space so vfs_glusterfs_pread_send and vfs_glusterfs_pwrite_send match.

These two functions are basically the same thing, so they should be
formatted the same.

Signed-off-by: Ira Cooper <ira@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

vfs_glusterfs: Fix AIO crash on smb.conf reload.

This fixes an issue where we couldn't handle cancellation properly
so when smb.conf was reloaded we crashed.

Signed-off-by: Ira Cooper <ira@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

docs: Fix typos in man vfs_gpfs.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11641
Duplicate "acl map full control" entry in man vfs_gpfs

Signed-off-by: Karolin Seeger <kseeger@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s4-torture: Remove obsolte code in backupkey_heimdal rpc test

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu Dec 10 11:54:00 CET 2015 on sn-devel-104

s4-torture: Improve backupkey test to validate the self signed cert

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

s4-torture: Add a GnuTLS based backupkey rpc test

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

s4-torture: Rename backupkey test to backupkey_heimdal

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

s4-rpc_server: Add a GnuTLS based backupkey implementation

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

waf: Check for GnuTLS 3.4.7

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

s4-rpc-bkrp: Do not set the ca status

Windows doesn't have any CA data set on the certificate.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

s4-rpc_server: Rename dcesrv_backupkey to dcesrv_backupkey_heimdal

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

s4-torture: make sure we always verify ndr pull and push of bkrp_exported_RSA_key_pair struct.

Guenther

Signed-off-by: Günther Deschner <gd@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

ping_pong: add -l option

Add a new option -l to check whether POSIX byte range locks are
working. Usage:

node1$ touch /path/to/cluster-fs/FILE

node1$ ./bin/ping_pong -l /path/to/cluster-fs/FILE
Holding lock, press any key to continue...
You should run the same command on another node now.

node2$ ./bin/ping_pong -l /path/to/cluster-fs/FILE

Output can either be:

  Holding lock, press any key to continue...

This means POSIX byte range locks are *not* working.

If you see this instead:

  file already locked, calling check_lock to tell us who has it locked...:
  check_lock failed: lock held: pid='27375', type='1', start='0', len='0'
  Working POSIX byte range locks

Congrats, you have a cluster fs with functional byte range locks!

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Dec 10 08:48:38 CET 2015 on sn-devel-104

ldb: version 1.1.24

* fix for CVE-2015-5330, bug 11599
* fix for CVE-2015-3223, bug 11325
* move ldb_(un)pack_data into ldb_module.h for testing
* fix installation of _ldb_text.py
* fix propagation of LDB errors through TDB
* fix bug triggered by having an empty message in database during search

Bug: https://bugzilla.samba.org/show_bug.cgi?id=11325
Bug: https://bugzilla.samba.org/show_bug.cgi?id=11599
Bug: https://bugzilla.samba.org/show_bug.cgi?id=11636

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

CVE-2015-5330: ldb_dn_explode: copy strings by length, not terminators

That is, memdup(), not strdup(). The terminators might not be there.

But, we have to make sure we put the terminator on, because we tend to
assume the terminator is there in other places.

Use talloc_set_name_const() on the resulting chunk so talloc_report()
remains unchanged.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=11599

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-with: Andrew Bartlett <abartlet@samba.org>
Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>
Pair-programmed-with: Stefan Metzmacher <metze@samba.org>
Pair-programmed-with: Ralph Boehme <slow@samba.org>

CVE-2015-5330: next_codepoint_handle_ext: don't short-circuit UTF16 low bytes

UTF16 contains zero bytes when it is encoding ASCII (for example), so we
can't assume the absense of the 0x80 bit means a one byte encoding. No
current callers use UTF16.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=11599

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-with: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

CVE-2015-5330: strupper_talloc_n_handle(): properly count characters

When a codepoint eats more than one byte we really want to know,
especially if the string is not NUL terminated.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=11599

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-with: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

CVE-2015-5330: Fix handling of unicode near string endings

Until now next_codepoint_ext() and next_codepoint_handle_ext() were
using strnlen(str, 5) to determine how much string they should try to
decode. This ended up looking past the end of the string when it was not
null terminated and the final character looked like a multi-byte encoding.
The fix is to let the caller say how long the string can be.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=11599

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-with: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

CVE-2015-5330: ldb_dn_escape_value: use known string length, not strlen()

ldb_dn_escape_internal() reports the number of bytes it copied, so
lets use that number, rather than using strlen() and hoping a zero got
in the right place.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=11599

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-with: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

CVE-2015-5330: ldb_dn: simplify and fix ldb_dn_escape_internal()

Previously we relied on NUL terminated strings and jumped back and
forth between copying escaped bytes and memcpy()ing un-escaped chunks.
This simple version is easier to reason about and works with
unterminated strings. It may also be faster as it avoids reading the
string twice (first with strcspn, then with memcpy).

Bug: https://bugzilla.samba.org/show_bug.cgi?id=11599

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-with: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

CVE-2015-3223: lib: ldb: Use memmem binary search, not strstr text search.

Values might have embedded zeros.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11325

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

CVE-2015-3223: lib: ldb: Cope with canonicalise_fn returning string "", length 0.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11325

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

lib/param: handle (ignore) substitution variable in smb.conf

BUG: https://bugzilla.samba.org/show_bug.cgi?id=10722

The function handle_include returns false when trying to include
files that have a substitution variable in filename (like %U),
this patch makes handle_include to ignore this case, to make
samba-tool work when there is such include in samba's configuration.

Error was :
root@ubuntu:/usr/local/samba# grep 'include.*%U' etc/smb.conf
include = %U.conf
root@ubuntu:/usr/local/samba# ./bin/samba-tool user list
Can't find include file %U.conf
ERROR(runtime): uncaught exception - Unable to load default file

Signed-off-by: Quentin Gibeaux <qgibeaux@iris-tech.fr>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Dec  9 02:05:30 CET 2015 on sn-devel-104

libdns: Small cleanup

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libdns: Convert dns_udp_request to 0/errno

Replaces 5 calls to unix_to_werror with just one

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libdns: Properly set ENOMEM

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

libdns: tsocket returns -1 and sets errno

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lib: Use GUID_buf_string in discover_dc_dns

One talloc call less..

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lib: Lift lp_disable_netbios one level

This should fix an error code when neither DS_IS_FLAT_NAME nor
DS_IS_DNS_NAME are specified. If netbios is disabled and the DC
can't be found via DNS we should not return NOT_SUPPORTED but
DOMAIN_CONTROLLER_NOT_FOUND.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lib: make debug_dsdcinfo_flags static

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

lib: Avoid a includes.h

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Revert "s3: smbd: Tear down global_smbXsrv_client in the correct order."

Wrong fix for the problem that was actually fixed in the dbwrap_rbt
code with commits:

590507951fc514a679f44b8bfdd03c721189c3fa
0f46da08e160e6712e5282af14e1ec4012614fc7

This reverts commit 8024e19b70047865249305bceddd4473d6e60051.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Mon Dec  7 21:09:04 CET 2015 on sn-devel-104

samba-tool: user create examples show 'add' instead of 'create'

Signed-off-by: Rowland Penny <repenny241155@gmail.com>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

samba-tool: fsmo.py throws an uncaught exception if no

fSMORoleOwner attribute

This will fix bug 11613 where a user got the uncaught exception when trying
to seize an FSMO role that didn't have the required attribute.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11613

Signed-off-by: Rowland Penny <repenny241155@gmail.com>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>
Autobuild-User(master): Uri Simchoni <uri@samba.org>
Autobuild-Date(master): Sun Dec  6 00:33:10 CET 2015 on sn-devel-104

ntlm_auth: Add --offline-logon

Signed-off-by: Wolfgang Ocker <weo@recco.de>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Sat Dec  5 01:24:56 CET 2015 on sn-devel-104

docs-xml: Document range parameter for idmap_autorid

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Autobuild-User(master): Christof Schmitt <cs@samba.org>
Autobuild-Date(master): Fri Dec  4 22:09:09 CET 2015 on sn-devel-104

ctdb-ipalloc: Rename top level IP allocation algorithm functions

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Fri Dec  4 12:25:14 CET 2015 on sn-devel-104

ctdb-ipalloc: Rename ctdb_takeover_run_core() to ipalloc()

It just does IP allocation...

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-ipalloc: Fold force_rebalance_candidates into IP allocation state

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-ipalloc: Fold all IPs list into IP allocation state

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-ipalloc: Tidy up some of the IP allocation functions

Shorter temporary variables for compactness/readability.  "tmp_ip" is
5 characters longer than "t".  In each for statement it is used 4
times, so costs 20 characters.  Save those extra characters so that
future edits will avoid going over 80 columns.

Tweak whitespace for readability, rewrap some code.

No functional changes.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-daemon: Don't delete connection information for released IP

As per the comment:

  If the IP address is hosted on this node then remove the connection.

  Otherwise this function has been called because the server IP
  address has been released to another node and the client has exited.
  This means that we should not delete the connection information.
  The takeover node processes connections too.

This doesn't matter at the moment, since the empty connection list for
an IP address that has been released will never be pushed to another
node.  However, it matters if the connection information is stored in
a real replicated database.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-daemon: Move VNN lookup out of ctdb_remove_tcp_connection()

In a subsequent commit ctdb_takeover_client_destructor_hook() needs to
know the VNN.  So just have both callers of
ctdb_remove_tcp_connection() do the lookup and pass in the VNN.

This should cause no change in behaviour.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-daemon: Do not process tickle updates for hosted IP addresses

Tickle list updates are broadcast to all connected nodes and are
accepted even when received on the same node that sent them.  This
could actually lead to lost connection information when information
about new connections is received while an update is in-flight.

Instead, return early when the IP is hosted on the current node, since
it is the only one that could have sent the update.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-docs: Rewrite event script documentation

Move information about TCP connection tracking and resetting into
ctdb.7.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-scripts: Add exportfs cache to NFS Linux kernel callout

exportfs can hang when, for example, DNS is flakey.  Given that
exports don't change much, it makes sense to cache them.

Don't try to add error handling when exportfs fails but do print a
warning.  Proper error handling can be added separately.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-recoverd: Drop function unban_all_nodes()

It hasn't worked since commit cda5f02c7c3491917d831ee23b93278dfaa5c82b
in 2009, which reworked the banning code.  Since then
ctdb_control_modflags() has contained a comment saying:

  /* we don't let other nodes modify our BANNED status */

Unbanning all nodes originally occurred here when the recovery master
role moved to a new node.  The logic could have been meant for the
case when the old recovery master was malfunctioning, so got banned.
If any other nodes had been banned by this recovery master then they
would be unbanned.  However, this would also unban the old recovery
master, which is probably suboptimal.  The logic would also trigger if
a node was banned for a good reason and then the recovery master was
stopped.  So, apart from doing nothing, the logic is too simplistic so
might as well be removed.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-daemon: Drop handling of ban control sent to unexpected node

The banning code caters for the case where the node specified in the
bantime data is not the node receiving the control.  This never
happens.  There are 2 places where ctdb_ctrl_set_ban() is called: the
ctdb CLI tool and the recovery daemon.  Both pass the same node in the
bantime data that they are sending the control to.  There are no plans
to do anything more elaborate, so just delete the handling of this
special case.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ldb torture: Test ldb unpacking and printing

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11602
Signed-off-by: Adrian Cochrane <adrianc@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Autobuild-User(master): Garming Sam <garming@samba.org>
Autobuild-Date(master): Fri Dec  4 09:16:22 CET 2015 on sn-devel-104

ldb: Move ldb_(un)pack_data into ldb_module.h for testing

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11602
Signed-off-by: Adrian Cochrane <adrianc@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

ldb: Fix installation of _ldb_text.py

_ldb_text.py is installed as part of the ldb package and also if you
compile Samba with the system ldb version. This way we have have the
file twice in the same location and run into file confilcts.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

Fix little typo in README file

Reviewed-by: Bjoern Jacke <bj@sernet.de>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

Fix propagation of LDB errors through TDB.

Returning a non-zero value from a tdb_traverse callback indicates that tdb_traverse
should stop traversing the database. This error code IS NOT propagated back to the
caller, so LTDB must record the error otherwise. This patch corrects LTDB for this
misunderstanding.

Naturally exposing these errors changes the behaviour of some tests. This commit fixes
that as well.

Signed-off-by: Adrian Cochrane <adrianc@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

ldb: Fix bug triggered by having an empty message in database during search.

Previously if the message had 0 elements, Talloc would reallocate the projected
array to NULL, fooling LDB into thinking that it failed to reallocate. This fix
corrects LDB to be able to handle the case where the message has no attributes
in common with the filter.

Also the realloc call resized the array to the number of elements in the message,
not the number of elements in common with the filter -- it essentially did nothing.

Unlike talloc_realloc, talloc_array always returns a non-null pointer. This would
help protect against possible errors.

Signed-off-by: Adrian Cochrane <adrianc@catalyst.net.nz>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

clitar: cope with functions of older versions of libarchive

the feature of automatic decompression in extract mode is only supported in
libarchive 3, so we use this only when available now.

The changed config checks for that also fix this bug:

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11296

Signed-off-by: Bjoern Jacke <bj@sernet.de>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Björn Jacke <bj@sernet.de>
Autobuild-Date(master): Thu Dec  3 16:09:08 CET 2015 on sn-devel-104