git.samba.org - metze/samba/wip.git/log

messaging_dgm: avoid GCC snprintf warnings in messaging_dgm_out_create

We are trying to put something that (in theory) could be 109 bytes
long, into the sockaddr_un.sun_path field which has a fixed size of
108 bytes. The "in theory" part is that one of the components is a
pid, which although stored as 32 bits is in practice 16 bits, so the
maximum size is not actually hit.

This is all very annoying, because the length is checked anyway and
all this achieves is silencing a warning.

Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Feb 10 09:05:31 CET 2017 on sn-devel-144

shadow_copy_get_shadow_copy_data: fix GCC snprintf warning

GCC 7 warns about snprintf truncating a dirent d_name (potentially 255 bytes) to 25 bytes,
even though we have checked that it is 25 long in shadow_copy_match_name().

Using strlcpy instead of snprintf lets us check it again, JUST TO BE SURE.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python provision: fix indenting of doc string

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

ntlmssp: fix compilation with -O2 -fno-inline

Without inlining the function, GCC doesn't know that
gensec_ntlmssp->ntlmssp_state->role always has a valid value.

With inlining, this is obviously redundant but GCC clearly knows
enough to detect this and elide the default case.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

getncchanges script: use library code, not copied functions.

These functions were duplicates. To be exact, the diff -ub between what
getncchanges had, and what drs_uitls now has is this:

|@@ -1,4 +1,5 @@
|-def do_DsBind(drs):
|+def drs_DsBind(drs):
|     '''make a DsBind call, returning the binding handle'''
|     bind_info = drsuapi.DsBindInfoCtr()
|     bind_info.length = 28
|@@ -32,7 +33,8 @@
|     bind_info.info.supported_extensions |= drsuapi.DRSUAPI_SUPPORTED_EXTENSION_GETCHGREPLY_V7
|     bind_info.info.supported_extensions |= drsuapi.DRSUAPI_SUPPORTED_EXTENSION_VERIFY_OBJECT
|     (info, handle) = drs.DsBind(misc.GUID(drsuapi.DRSUAPI_DS_BIND_GUID), bind_info)
|-    return handle
|+
|+    return (handle, info.info.supported_extensions)
|
|
| def drs_get_rodc_partial_attribute_set(samdb):
|@@ -43,7 +45,7 @@
|     attids = []
|
|     # the exact list of attids we send is quite critical. Note that
|-    # we do ask for the secret attributes, but set set SPECIAL_SECRET_PROCESSING
|+    # we do ask for the secret attributes, but set SPECIAL_SECRET_PROCESSING
|     # to zero them out
|     schema_dn = samdb.get_schema_basedn()
|     res = samdb.search(base=schema_dn, scope=ldb.SCOPE_SUBTREE,
|@@ -71,3 +73,4 @@
|     partial_attribute_set.attids         = attids
|     partial_attribute_set.num_attids = len(attids)
|     return partial_attribute_set

while the drs_utils code has changed in moving
drs_get_rodc_partial_attribute_set() out of the class.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

lib/replace tests: prevent GCC fretting over snprintf sizes

These tests deliberately use snprintf for truncating strings, which is
fine for tests. This has the effect of leaving the warning in place
but preventing it from becoming a fatal error.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

fix blackbox_supported_features: mkdir -p its directory

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

docs/smbconf: update log level list in man page

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

selftest: show multiple arguments for --help

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

waf --test-list takes a filename argument

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

docs: Add missing spaces in man smb.conf.

Signed-off-by: Karolin Seeger <kseeger@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Feb 9 23:58:02 CET 2017 on sn-devel-144

s3-vfs: Only walk the directory once in open_and_sort_dir()

On a slow filesystem or network filesystem this can make a huge
difference.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12571

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Move pthreadpool to top of the tree.

Signed-off-by: Matthieu Patou <mat@matws.net>
Reviewed-by: Jeremy Allison <jra@samba.org>

wafsamba: Remove 2010 comments that seems not accurate anymore

In my tests default value is correctly used and if we provide explicitly
a --with it will comply with the store_true and if we provide --without
then it will comply with the store_false

Change-Id: I820a7f2f08c51ec23b694bce7009c3891d4ab8ef
Reviewed-by: Jeremy Allison <jra@samba.org>

wafsamba: Move command line option function labelled as 'samba3' to the common set of functions

It allows to be used for things that are not 'samba3' only (or more
accurately things not in common and not related to the AD DC
implementation)

Signed-off-by: Matthieu Patou <mat@matws.net>
Reviewed-by: Jeremy Allison <jra@samba.org>

Switch on the sortedLinks Flag on new databases

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Thu Feb 9 07:07:43 CET 2017 on sn-devel-144

replmd: check for the sortedLinks feature flag

If it is there, we assume linked attributes are stored in a sorted
order.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

dsdb: Honour @SAMBA_FEATURES_SUPPORTED flag in @IDXATTR

This allows us to detect modification by a Samba version prior to
the introduction of the compatibleFeatures logic as this flag will
be stripped by the schema load code of older Samba versions.
Therefore if it is not present, then remove all
compatibleFeatures.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Pair-programmed-with: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

schema: Set flag into @INDEXLIST to indicate we support feature flags

Because @INDEXLIST is rewritten by all Samba versions, we can detect
that we have opened the database with an older version that does not
support the feature flags by the absense of this in @INDEXLIST

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

samba_dsdb: Use and maintain compatibleFeatures and requiredFeatures in @SAMBA_DSDB

This will allow us to introduce new database features that are
backward compatible from the point of view of older versions of Samba,
but which will be damaged by modifying the database with such a
version.

For example, if linked attributes are stored in sorted order in 4.7,
and this change, without any values in current_supportedFeatures is
itself included in 4.6, then our sortedLinks are backward compatible
to that release.

That is with 4.6 (including this patch) which doesn't care about
ordering -- but a downgraded 4.7 database used by 4.6 will be broken
when later used with 4.7. If we add a 'sortedLinks' feature flag in
compatibleFeatures, we can detect that.

This will allow us to determine if the database still contains
unsorted links, as that information allows us to make the code
handling links much more efficient.

We won't add the actual flag until all the code is in place.

Andrew wrote the actual code and Douglas wrote the tests, and they
cross-reviewed.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Piar-programmed-with: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
selftest: check for database features flags

repl_md: get links in sorted order in replmd_add_fix_la

This is where forward links get added when they get added with an
object.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-with: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

replmd: treat a zero GUID as not present in get_parsed_dns

This roughly follows the pattern in the 2009 commit
0d5d7f58473c989bff4 by the Andrews Tridgell and Bartlett, which dealt
with zero GUIDs in replmd_add_fix_la(). That function is about to use
get_parsed_dns() [see next commit], and the other users of
get_parsed_dns don't really want to see zero guids, so it is simpler
to test here.

This makes hitting the GUID_all_zero branch of parsed_dn_find() even
more unlikely.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-with: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

replmd: keep links sorted in replmd_process_linked_attribute

This is where linked attributes get added during a replication.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

replmd linked_attributes: maintain sorted links in replace

We use a merge-like algorithm, which gives us a slight algorithmic
improvement (O(m + n) vs O(m log(n) + n log(m))) and keeps the results
sorted.

Here's an example. There are existing links to {A C D* F*} where D*
and F* represent deleted links, and we want to replace them with {B C
E F}.

existing:       A     C  D* E  F*
                      |     |  |
replacements:      B  C     E  F

result:         A* B  C  D* E  F

This is what happens to each link:

A  gets deleted to A*.
B  gets added.
C  is retained, with possible extended DN changes.
D* stays in the list as a deleted link
E  is retained like C
F  is undeleted.

Backlinks are created in the case of B and F
The backlink for A is deleted
The backlinks are not changed for C and E or D* (D* has none)

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-with: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

replmd linked attributes: use really_parse_trusted_dn everywhere

This function fills out the DN and GUID fields of an unparsed
parsed_dn struct, which was happening in a few other places already.

In some places the GUID was not being filled out, which would probably
cause problems if the sorted_links switch was turned on.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

replmd: simplify and optimise replmd_modify_la_delete

With the old binary search, we didn't get a pointer to the found
value, just a yes or no answer as to its existence. That meant we
ended up searching in both directions to find the links to be deleted.
As a consequence we needed to parse out the GUID of every existing
link, even if it wasn't being deleted.

Here we do it in one pass.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

replmd: rearrange nothing-to-delete logic

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

repl_meta_data: linked attributes use DRS sort order

Links come over the wire as if sorted by memcmp() on the binary blobs,
not as sorted by GUID_compare(). Until a few patches ago, a newly
joined DC would have its linked attributes in the memcmp order. This
restores that behaviour.

This comparison could be made more efficient by storing the GUID in
the original state, but it does not seem to be a bottleneck.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Pair-programmed-with: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

replmd: rework replmd_modify_la_add to merge efficiently

Because both the list of added links and the list of existing links
are sorted, it is possible to interlace the two and obtain a merged
sorted list.

We avoid a great amount of talloc_realloc()ing by observing that the
merged list can't be longer than the sum of the two lists.

In the (common) case where there are many existing links but few being
added, we avoid parsing most of the existing link DNs and GUIDs if the
sorted_links feature flag is set.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

replmd linked attrs: fully parse dn for upgrade check

Elsewhere we use the dsdb_dn pointer as a flag indicating parsed-ness,
so we have to be consistent.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

replmd linked attributes: lazy parsing for trusted DNs

If we know that links from the database are in sorted order (via the
replmd_private->sorted_links flag), we can avoid actually parsing them
until it is absolutely necessary.

In many cases we are adding a single link to a long list. The location
of the single link is found via a binary search, so we end up parsing
log(N) DNs instead of N.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

replmd: Add placeholder sorted_links to struct replmd_private

This will be initialised to false (zero) by default and will later come
from the compatibleFeatures in @SAMBA_DSDB

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

replmd: replmd_check_upgrade_links() needs to first parses DNs

Because we now load the dns with get_parsed_dns_trusted we have
to manually explode them in the upgrade tests.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

replmd: parsed_dn_find() finds insertion point as well as exact hit

This will allow us to maintain the list of links in sorted order.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

binsearch: make BINARY_ARRAY_SEARCH_GTE compare against a pointer

This is in preparation for improvements in our handling of linked
attributes where we make changes to the pointer in the process of
comparing it (for caching purposes).

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

binsearch: clarify variable name in greater-than-or-equal search

The exact match variable was called "result" following the other
macros, which confused me for a moment.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

replmd: fix variable names in replmd_check_upgrade_links

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

replmd: replmd_check_upgrade_links() only checks the first DN

This assumes the links (on an object in the database) are either all in
the old format or all in the new.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

replmd: pass replmd_private down to replmd_add_backlink()

This is not much saving, but we are soon going to need replmd_private
in the intermediate layers (e.g. replmd_modify_la_add).

Pair-programmed-with: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

replmd: Fix some whitespace in repl_meta_data.c

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

replmd: check whether list is already sorted in get_parsed_dns()

If they are we can avoid the sort.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

selftest: Do not test for link ordering in tombstones_expunge test

By testing only for the DNs that are returned we do not change the strictness of
the test, because it is a test of the match rule which applies to the whole
object, not the returned values.

However, when this code asserted the returned order of the links, it prevents
us from changing this order. This order was not deterministic across DCs
but as this test ran against an offline DB, it was able to assume a
particular order.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

s4/linked_attribute tests: remove helper function unused parameter

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/linked_attribute tests: try adding linked attributes directly

Previously we have only added linked attributes using a modify.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-with: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/linked_attribute tests: test with the relax control

We had a theory this caused problems. It didn't, but the tests are
still worthwhile.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/linked_attribute tests: compare link lists in sorted order

This isn't functionally different[1] from the previous use of set(),
but it makes the error output easier to read.

[1] OK, it will also show duplicates, which we really don't expect and
would definitely want to see.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/linked_attribute tests: remove unused code

We don't test for sort order because we don't depend on it. So this
test was never used.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/linked_attribute tests: add multiple links and replace tests

Also a "delete all" test.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4/linked_attributes test: pep8 tidy-up, remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

torture/drs: expand test for DRSUAPI_DRS_GET_ANC

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Bob Campbell <bobcampbell@catalyst.net.nz>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Feb 9 03:16:09 CET 2017 on sn-devel-144

getncchanges: implement DRSUAPI_DRS_GET_ANC more correctly

The most important case is the combination of
DRSUAPI_DRS_CRITICAL_ONLY and DRSUAPI_DRS_GET_ANC.

With DRSUAPI_DRS_GET_ANC we need to make sure all ancestors
included even if they're not marked with
isCriticalSystemObject=TRUE.

I guess we still don't behave exactly as Windows, but it's much
better than before and fixes the initial replication if
someone moved the administrator account to an OU.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398

Pair-Programmed-With: Bob Campbell <bobcampbell@catalyst.net.nz>

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Bob Campbell <bobcampbell@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

getncchanges: calculate getnc_state->min_usn calculation based on the uptodateness vector

This should improve initial replication of a fresh destination dsa with
a zero highwatermark.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

getncchanges: improve get_nc_changes_add_links() by checking uSNChanged

This will make a difference once we handle DRSUAPI_DRS_GET_ANC correctly.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

getncchanges: improve get_nc_changes_build_object() by checking uSNChanged

This will make a difference once we handle DRSUAPI_DRS_GET_ANC correctly.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

getncchanges: fix highest_usn off by one calculation in get_nc_changes_add_links()

highest_usn is the the highest usn the destination dsa already knows about.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

getncchanges: remove unused c++ comments/code in getncchanges_collect_objects()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

getncchanges: do not replicate links for non critical objects if DRSUAPI_DRS_CRITICAL_ONLY is set

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398

Pair-programmed-with: Bob Campbell <bobcampbell@catalyst.net.nz>

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Bob Campbell <bobcampbell@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

getncchanges: don't process DRSUAPI_DRS_CRITICAL_ONLY for EXOPs

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

getncchanges: remember the ncRoot_guid on the getncchanges state

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

getncchanges: pass struct ldb_message as const

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

getncchanges: only set nc_{object,linked_attributes}_count with DRSUAPI_DRS_GET_NC_SIZE

The main change is that we return 0 values if DRSUAPI_DRS_GET_NC_SIZE is not
present in order to get the same result as a Windows server in that case.

If DRSUAPI_DRS_GET_NC_SIZE is return the number of links we found so far
during the cycle in addition the number of objects returned in this cycle.
Both values doesn't match what Windows returns, but doing that
correctly and efficient is a task for another day.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

torture/drs: remove pointless nc_object_count replication checks in test_link_utdv_hwm()

nc_object_count and nc_linked_attributes_count are only filled if
DRSUAPI_DRS_GET_NC_SIZE is requested. And they should contain
the total number. This is only useful for the initial replication.

Samba ignores DRSUAPI_DRS_GET_NC_SIZE currently but that will change in
the following commits.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/join: use DRSUAPI_DRS_GET_NC_SIZE for the initial replication

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

python/join: set common replica_flags in dc_join.__init__()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

drsuapi.idl: make drsuapi_DsGetNCChangesRequest10 [public]

This allows ndr_print to work.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

drsuapi.idl: add drsuapi_DrsMoreOptions with DRSUAPI_DRS_GET_TGT

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:libnet: s/highestCommitedUSN/highestCommittedUSN

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:dsdb/repl: s/highestCommitedUsn/highestCommittedUSN

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

ctdb-build: Install CTDB tests correctly from toplevel

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12547

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Thu Feb 2 08:25:57 CET 2017 on sn-devel-144

s3: VFS: vfs_streams_xattr.c: Make streams_xattr_open() store the same path as streams_xattr_recheck().

If the open is changing directories, fsp->fsp_name->base_name
will be the full path from the share root, whilst
smb_fname will be relative to the $cwd.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12546

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Feb 2 01:55:42 CET 2017 on sn-devel-144

smbd: Fix "map acl inherit" = yes

Brown-Paper-Bag bug in f85c2a6852a. The assignment contains a self-reference
in get_pai_flags which I missed.

Fix an uninitialized read.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=12551
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Wed Feb 1 22:06:50 CET 2017 on sn-devel-144

talloc/wscript: avoid passing pointless enabled=True to SAMBA_PYTHON()

This is the default and should not be passed explicitly.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Wed Feb 1 18:16:58 CET 2017 on sn-devel-144

ctdb-common: ioctl(.. FIONREAD ..) returns an int value

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12549

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Wed Feb 1 14:29:14 CET 2017 on sn-devel-144

util:charset: Return EILSEQ in smb_iconv() if newer libc is detected

This is the behaviour of glibc 2.24 and newer.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Feb 1 05:16:46 CET 2017 on sn-devel-144

auth: fix mem leak & use appropriate free function

coverity fix.

cli_credentials_set_principal does a strdup, we want to free 'name'
regardless of the result in 'ok'.

Signed-off-by: Aurelien Aptel <aaptel@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

ctdb-tests: Use replace headers instead of system headers

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12469

This ensures that PTHREAD_MUTEX_ROBUST, pthread_mutexattr_setrobust()
and pthread_mutex_consistent() are always defined.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Tue Jan 31 11:57:01 CET 2017 on sn-devel-144

ctdb-tests: Do not build mutex test if robust mutexes are not supported

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12469

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

libcli/smb: outbuf length is a IVAL ie a uint32_t

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Jan 31 04:34:29 CET 2017 on sn-devel-144

s3/rpc_server: shared rpc modules loading

The previous commit 58889e04bd545d7420d1193e134351bd0ccb8430 for this
bug was broken as it didn't move the goto into the "if (errno !=
ENOENT)" condition.

This updated fix folds the test "mod_init_fns == NULL" and the check for
the errno into one if condition.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=12184

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3: VFS: Don't allow symlink, link or rename on already converted paths.

Snapshot paths are a read-only filesystem.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12531

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Mon Jan 30 22:26:29 CET 2017 on sn-devel-144

s3: VFS: shadow_copy2: Fix usage of saved_errno to only set errno on error.

Rationale:

VFS calls must act like their POSIX equivalents, and the POSIX versions
*only* set errno on a failure. There is actually code in the upper smbd
layers that depends on errno being correct on a fail return from a VFS call.

For a compound VFS module like this, a common pattern is :

SMB_VFS_CALL_X()
{
      int ret;

      syscall1();
      ret = syscall2();
      syscall3();

      return ret;
}

Where if *any* of the contained syscallX()'s fail, they'll set errno.
However, the actual errno we should return is *only* the one returned
if syscall2() fails (the others are lstat's checking for existence etc.).

So what we should do to correctly return only the errno from syscall2() is:

SMB_VFS_CALL_X()
{
      int ret;
      int saved_errno = 0;

      syscall1()

      ret = syscall2();
      if (ret == -1) {
            saved_errno = errno;
      }
      syscall3()

      if (saved_errno != 0) {
           errno = saved_errno;
      }
      return ret;
}

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12531

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

s3: VFS: shadow_copy2: Fix a memory leak in the connectpath function.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12531

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

s3: VFS: shadow_copy2: Fix module to work with variable current working directory.

Completely cleans up the horrible shadow_copy2_strip_snapshot()
and adds an explaination of what it's actually trying to do.

* This function does two things.
*
* 1). Checks if an incoming filename is already a
* snapshot converted pathname.
*     If so, it returns the pathname truncated
*     at the snapshot point which will be used
*     as the connectpath, and then does an early return.
*
* 2). Checks if an incoming filename contains an
* SMB-layer @GMT- style timestamp.
*     If so, it strips the timestamp, and returns
*     both the timestamp and the stripped path
*     (making it cwd-relative).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12531

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

s3: VFS: Add utility function check_for_converted_path().

Detects an already converted path. Not yet used.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12531

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

s3: VFS: Ensure shadow:format cannot contain a / path separator.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12531

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

s3: VFS: Allow shadow_copy2_connectpath() to return the cached path derived from $cwd.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12531

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

s3: VFS: shadow_copy2: Fix chdir to store off the needed private variables.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12531

This is not yet used, the users of this will be added later.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

s3: VFS: shadow_copy2: Add two currently unused functions to make pathnames absolute or relative to $cwd.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12531

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

s3: VFS: shadow_copy2: Change a parameter name.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12531

Allows easy substitution later.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

s3: VFS: shadow_copy2: Add a wrapper function to call the original shadow_copy2_strip_snapshot().

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12531

Allows an extra (currently unused) parameter to be added.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

s3: VFS: shadow_copy2: Add two new variables to the private data. Not yet used.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12531

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

s3: VFS: shadow_copy2: Fix length comparison to ensure we don't overstep a length.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12531

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

s3: VFS: shadow_copy2: Ensure pathnames for parameters are correctly relative and terminated.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12531

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

s3: VFS: shadow_copy2: Correctly initialize timestamp and stripped variables.

Allow the called functions to be fixed to not touch them on error.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12531

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

s3: smbd: Make set_conn_connectpath() call canonicalize_absolute_path().

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12531

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

s3: smbtorture: Add new local test LOCAL-CANONICALIZE-PATH

Tests new canonicalize_absolute_path() function.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12531

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

s3: lib: Fix two old, old bugs in set_conn_connectpath(), now in canonicalize_absolute_path().

Canonicalizing a path of /foo/bar/../baz would return /foo/barbaz
as moving forward 3 characters would delete the / character.

Canonicalizing /foo/.. would end up as '\0'.

Test to follow.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12531

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

s3: lib: Add canonicalize_absolute_path().

Resolves any invalid path components (.) (..)
in an absolute POSIX path.

We will be re-using this in several places.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12531

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

s3: smbd: Correctly canonicalize any incoming shadow copy path.

Converts to:

@GMT-token/path/last_component

from all incoming path types. Allows shadow_copy modules
to work when current directory is changed after removing
last component.

Ultimately when the VFS ABI is changed to add a timestamp
to struct smb_filename, this is where the parsing will be
done.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12531

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>