smbd: Use asys_results

When multiple aio requests finish simultaneously, this saves a few syscalls

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Mar 27 08:05:46 CET 2014 on sn-devel-104

asys: Allow multiple results to be received

This makes use of C99 dynamic arrays. In this performance-sensitive code, I
would like to avoid malloc/free, and I think 15 years after the standard we
might be able to use this feature. Alternatively, we could use the "results"
memory area and store the jobids in the upper range, playing some cast-tricks.
Should work as well.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

pthreadpool: Allow multiple jobs to be received

This can avoid syscalls when multiple jobs are finished simultaneously

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

pthreadpool: Avoid a malloc/free per job

pthreadpool_add_job is in our hottest code path for r/w intensive workloads, so
we should avoid anything CPU-intensive. pthreadpool used to malloc each job and
free it in the worker thread. This patch adds a FIFO queue for jobs that helper
threads copy from, avoiding constant malloc/free. This cuts user space
CPU in the local-bench-pthreadpool benchmark by roughly 10% on my system.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

pthreadpool: Add a simple benchmark

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

auth/gensec/spnego: map SPNEGO_REJECT to NT_STATUS_LOGON_FAILURE

This is what NTLMSSP also gives.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Mar 27 02:34:36 CET 2014 on sn-devel-104

auth/gensec: remove tevent_context argument from gensec_update()

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:ntlm_auth: make use of gensec_update_ev()

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:rpc_server: make use of gensec_update_ev()

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:smb_server: make use of gensec_update_ev()

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:librpc: make use of gensec_update_ev()

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:libcli: make use of gensec_update_ev()

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:ldap_server: make use of gensec_update_ev()

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:kdc: make use of gensec_update_ev()

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:dns_server: make use of gensec_update_ev()

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

auth/gensec: make use of gensec_update_ev() in spnego.c

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

auth/gensec: add a gensec_update_ev() function

This is the current gensec_update() which takes an optional
tevent_context structure and allows semi-async code.

This is just a temporary solution on the way to kill
the semi-async code completely, by using gensec_update_send/recv.

By providing a gensec_update_ev(), we can remove the explicit
tevent_context from gensec_update() and fix all the sane callers.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:pygensec: don't pass an explicit tevent_context to gensec_update()

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

auth/gensec: fix gensec_update() with ev == NULL.

In future we should remove the tevent_context argument from
gensec_update() completely!

If we have sane backends we should also remove the
tevent_loop_allow_nesting() call again!
t
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

samba-tool dbcheck: handle missing objectClass

In several cases we have seen objects without the objectClass attribute.
Here the suggestion for a patch to find such objects in "samba-tool dbcheck"
with the option to delete them.

(patch improved by Andrew Bartlett to suggest DRS re-replication)

Signed-off-by: Felix Botner <botner@univention.de>
Change-Id: I8eb0d191a2089271a9af5884d6bfbf173a5c85c6
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

dsdb: Improve missing objectClass handling

This attempts to permit deletion of objects that have no objectClass
to allow dbcheck to clean up a corrupt database.  It is not complete,
the replmd_replPropertyMetaDataCtr1_sort_and_verify() call will still
fail, but this is as much as is safe to do without a way to replicate
the original issue.

Andrew Bartlett

Change-Id: If0b6c7f18e8aee587e6b3b4af878a0145f5eac37
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

dsdb: Improve errors and checks for missing objectClass values

Change-Id: I8c4ac679accc90748d20c9c86986b127c939fa75
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

dsdb: Clarify how the DSDB_REPL_FLAG_PRIORITISE_INCOMING flag works

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Change-Id: Ib9f2f4ba417dbf0ee24b6e7db02d78a9bfe8850c
Reviewed-by: Stefan Metzmacher <metze@samba.org>

dsdb: Do not update notify_uSN until the transaction is genuinely committed to the DB

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Change-Id: I734bc75ed348de8f0a5ff92e18e08de2340b8951
Reviewed-by: Stefan Metzmacher <metze@samba.org>

ctdb-tests: Add NAT gateway eventscript unit tests for static routes

Signed-off-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Wed Mar 26 06:24:01 CET 2014 on sn-devel-104

ctdb-eventscripts: CTDB_NATGW_STATIC_ROUTES can specify gateways

Extend CTDB_NATGW_STATIC_ROUTES so that each network can have an
optional gateway that overrides CTDB_NATGW_DEFAULT_GATEWAY.

Signed-off-by: Martin Schwenke <martin@meltin.net>

ctdb-eventscripts: New configuration variable CTDB_NATGW_STATIC_ROUTES

This can be used to create more specific NATGW routes than the usual
NATGW default route.

Signed-off-by: Martin Schwenke <martin@meltin.net>

ctdb-eventscripts: Clarify that CTDB_NATGW_DEFAULT_GATEWAY is optional

This has been implied since the command to add the route has had
errors redirected to /dev/null.  If infrastucture (e.g. ADS, DNS) is
on the same network as CTDB_NATGW_PUBLIC_IP then no route is
necessary.

Signed-off-by: Martin Schwenke <martin@meltin.net>

ctdb-eventscripts: Improve check in NATGW "startup" event

Although the dots in $CTDB_NATGW_PUBLIC_IP could probably only help
match an invalid public IP address, this is only executed once so do
as exact a check as possible.

Use CTDB_BASE instead of hardcoding /etc/ctdb.

Make the error message less redundant.

Signed-off-by: Martin Schwenke <martin@meltin.net>

ctdb-eventscripts: Reformat natgw_clear()

Signed-off-by: Martin Schwenke <martin@meltin.net>

ctdb-eventscripts: Rename some NAT gateway functions

delete_all() really needed renaming for clarity.  While doing this,
might as well rename some of the others that don't start with
"natgw_".

Signed-off-by: Martin Schwenke <martin@meltin.net>

ctdb-tests: Add a test for NAT gateway sanity checking

Signed-off-by: Martin Schwenke <martin@meltin.net>

ctdb-eventscripts: Sanity check NAT gateway configuration

NAT gateway really can't operate unless most of the configuration
variables are set.

A check in delete_all() can be removed - strange that this isn't also
done in the add case.

Signed-off-by: Martin Schwenke <martin@meltin.net>

ctdb-eventscripts: Improve readability of NAT gateway update code

Put the code into a couple of usefully named functions.

Signed-off-by: Martin Schwenke <martin@meltin.net>

ctdb-tests: Add some tests for 11.natgw eventscript

This includes adding support for:

* Configuring fake NATGW state in the eventscript unit tests

* "natgwlist" and "setnatgwstate" in ctdb command stub

* ip command stub to default to "main table" when no table specified,
  allow routes to be added without "dev" option (just add a default
  dev), support "metric" option

Signed-off-by: Martin Schwenke <martin@meltin.net>

ctdb-eventscripts: Use set_proc() to update /proc

In case we want to write some unit tests in the future.

Signed-off-by: Martin Schwenke <martin@meltin.net>

selftest: Remove print_smbtorture4_version and smbtorture4_possible check

smbtorture is now always the same version as the rest of Samba, and is strictly required.

Andrew Bartlett

Change-Id: I89d9c52275477177fa8a89050920ff8a2fec9288
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-By: Jelmer Vernooij <jelmer@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Mar 26 04:20:16 CET 2014 on sn-devel-104

dsdb: Do checks for invalid renames in samldb, before repl_meta_data

This ensures that conflict objects can be created in CN=System, and
that we do not stop replication just because some other DC allowed a
rename we do not like.

This is achived by doing the work in the samldb module, which is above
repl_meta_data in the stack.

Andrew Bartlett

Change-Id: I8c1a7d3e0fbd5a470cf1326cc055044ca885f7d9
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Guenter Kukkukk <kukks@samba.org>
Tested-by: Guenter Kukkukk <kukks@samba.org>

s3-nmbd: reset debug settings after reading config file (bug #10239)

Signed-off-by: Björn Baumbach <bb@sernet.de>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Mar 25 18:29:06 CET 2014 on sn-devel-104

s3: smbd: Factor out code that calls getgroups_unix_user() into a separate function.

This code needs to special-case the guest user, as
this token can have the token_sid[0] set to the Guest
SID, not the mapping of UNIX uid -> SID.

Other users that may have a well-known SID
set in token_sid[0] (like SYSTEM) are usually
not mappable to UNIX users and can be ignored
when adding local groups from /etc/group.

Found by <linux@kukkukk.com>.

Second part of the bugfix for:

https://bugzilla.samba.org/show_bug.cgi?id=10508

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:librpc/rpc: use dcerpc_binding_get_object() in order to pass the object to the epmapper

This way we'll be able to do epmapper lookups for the DFS-R (MS-FRS2) endpoint, by using
"5bc1ed07-f5f5-485f-9dfd-6fd0acf9a23c@ncacn_ip_tcp:hostname.exmple.com[krb5,seal]"
as binding.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Mar 25 02:43:39 CET 2014 on sn-devel-104

s4:librpc/tests: assert the the abstract syntax has the expected value (null)

This makes sure that it's not mixed with the object guid anymore.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

librpc/rpc: finally maintain only the object guid

This has nothing to do with ndr_syntax_id...

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

librpc/rpc: maintain "abstract_syntax" as string option of dcerpc_binding

This should not be mixed with the object guid! They are different things!

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:librpc/tests: reset the object on the binding created from the tower

The tower doesn't contain information about the object only about
the abstract syntax.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:torture/raw: fix debug message in torture_raw_qfileinfo_pipe()

We no longer use dcerpc_pipe_open_smb() there.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:torture/rpc: make use of dcerpc_binding_handle_auth_info() in backupkey.c

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:torture/rpc: make use of dcerpc_binding_handle_auth_info() in lsa.c

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:torture/rpc: fix altercontext test against windows

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:torture/rpc: remove bogus rpc.multibind test

We can later add a more useful test that tests
security context multiplexing correctly.

And another one that demonstrates that only DCERPC_BIND
must be the first (and only the first) PDU on a connection.
Otherwise DCERPC_ALTER_CONTEXT is used.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:selftest: don't run rpc.multibind anymore

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:dsdb/repl: make use of dcerpc_binding_handle_is_connected()

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:librpc/test: test ipv6 addresses in dcerpc_binding strings

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

librpc/rpc: handle ipv6 addresses without transport in dcerpc_parse_binding()

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

librpc/rpc: add "schannel" => DCERPC_SCHANNEL as ncacn_option

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:librpc/tests: add more no transport tests

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

lib/util: let is_ipaddress_v6() cope with "fe80::1234%3"

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:librpc/rpc: correctly map the fault code of alter context to NTSTATUS

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:librpc/rpc: remove unused dcecli_connection->binding_string

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:torture/rpc: avoid using dcecli_connection->binding_string

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

librpc/rpc: use dcerpc_binding_set_string_option(b, "endpoint", NULL) to reset the endpoint

We should always go through just one code path to [re]set a value.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

librpc/rpc: let dcerpc_binding_set_transport() also reset the assoc_group_id

This is transport/endpoint specific.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

libcli/smb: reuse tstream_smbXcli_np_disconnect_send/recv as helper

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

libcli/smb: add tstream_smbXcli_np_disconnect_cleanup() to handle talloc_free(req)

If the tevent_req of tstream_smbXcli_np_disconnect_* is explicitly or
implicitly free'ed, we need to make sure we still deliver the
close request to the server! Otherwise the SMB signing sequence gets out of
sync.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

libcli/smb: make TSTREAM_SMBXCLI_NP_MAX_BUF_SIZE public

This should be used to negotiate the may fragment size
of DCERPC connections.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

libcli/smb: keep references to smbXcli_{conn,session,tcon} in tstream_smbXcli_np

This fixes some valgrind errors when the smbXcli_tcon disappears before the
smbXcli_conn.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

s4:torture/rpc: fix error path in torture_leave_domain()

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

librpc: inline CHECK_SYNTAX macro logic

The CHECK_SYNTAX macro is currently used to compare ndr_syntax_ids and
return false on mismatch. Macros affecting control flow are evil!

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Mon Mar 24 21:46:39 CET 2014 on sn-devel-104

librpc: inline VT CHECK macro logic

The CHECK macro is currently used to dump error and return false on
VT condition check failure. Macros affecting control flow are evil!

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

librpc: split type specific VT checks into helpers

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

librpc: dump correct ndr_syntax_id on VT check failure

Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

s3:build: remove global CLUSTER_SUPPORT define

All ctdb specific code is isolated in samba-cluster-support.so now.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Autobuild-User(master): Michael Adam <obnox@samba.org>
Autobuild-Date(master): Mon Mar 24 19:08:44 CET 2014 on sn-devel-104

s3:wscript: only build ctdb_dummy.c if we have no cluster support

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>

s3:wscript: only fill in CTDB_CFLAGS instead of config.h

Only files in 'samba3-ctdb-client' will see HAVE_CTDB* defines.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>

s3:wscript: don't add ctdb include dir globally

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>

s3:serverid: use ctdb_serverids_exist_supported() runtime check

This is better than a compile time check for
HAVE_CTDB_CONTROL_CHECK_SRVIDS_DECL.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>

s3:ctdb_conn: only use ctdb_protocol.h or ctdb_private.h in ctdb_conn.c

We don't need them in ctdb_conn.h.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>

wafsamba: use cluster_support_features in build_options.c

This way 'smbd -b' returns the cluster features of the currently used
libsamba-cluster-support.so.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>

s3:lib: add cluster_support_[available|features]()

This allows runtime selection of libsamba-cluster-support.so

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>

s3:param: move lp_ctdbd_socket() to ctdbd_conn.c

This allows us to use CTDB_PATH only in ctdbd_conn.c

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>

s3:lib: move all ctdb related dummy functions to ctdb_dummy.c

This gets all dummy functions for the build without CLUSTER_SUPPORT.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>

s3:ctdbd_conn: always provide ctdb_serverids_exist()

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>

s3:ctdbd_conn: add ctdb_serverids_exist_supported()

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>

s3:wscript: create a libsamba-cluster-support.so

This collects the ctdb version dependent files,
which allows vendors to provide multiple versions
of libsamba-cluster-support.so each compiled against different
ctdb versions.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>

wafsamba: add CONFIG_RESET()

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>

lsa.idl: define lsa.ForestTrustCollisionInfo and ForestTrustCollisionRecord as public structs

For Python bindings PIDL wants the struct to be defined as public if we
want to use __ndr_print/pack/unpack.

Define the [public] attribute to ForestTrustCollisionInfo/Record to
allow easier debugging when there is collision in establishing a trust.

This change does not affect C code as NDR functions are always
generated, only not exposed in Python bindings.

https://bugzilla.samba.org/show_bug.cgi?id=10504

Signed-off-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Jelmer Vernooij <jelmer@samba.org>
Autobuild-User(master): Alexander Bokovoy <ab@samba.org>
Autobuild-Date(master): Mon Mar 24 12:44:50 CET 2014 on sn-devel-104

ctdb/daemon: Optimise deletion of IPs

Previous commits maintained the ordering between
ctdb_remove_orphaned_ifaces() and ctdb_vnn_unassign_iface().  This
meant that ctdb_remove_orphaned_ifaces() needed to steal the orphaned
interfaces and they would be freed later.

Unassign the interface first and things get simpler.
ctdb_remove_orphaned_ifaces() is now self-contained.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Sun Mar 23 06:20:43 CET 2014 on sn-devel-104

ctdb/daemon: Make delete IP wait until the IP is released

reloadips really expects deleted IPs to be released before completing.
Otherwise the recovery daemon starts failing the local IP check.  The
races that follow can cause a node to be banned.

To make the error handling simple, do the actual deletion in
release_ip_callback().

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Improve tickle tests

It is hard to diagnose failures in the NFS tickle test because there's
no way of telling if the test node doesn't have the tickle or if it
didn't get propagated.

Factor out check_tickles() into local.bash and give it some
parameters.

Have the NFS test call it first to ensure the tickle has been
registered.  Then use new function check_tickles_all() to ensure the
tickle has been propagated to all nodes.  Give this a bit of extra
time (double the timeout) just in case we're racing with the update.

Add a useful comment to the CIFS test so that I stop asking myself how
the test could ever have worked reliably.  :-)

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>

ctdb-eventscripts: Fix regression in IP add/delete functions

Commit 176ae6c704528c021fcc34a41878584f43a00119 caused these functions
to exit on failure.  This is incorrect and broke NAT gateway.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-daemon: Do not disable monitoring when running eventscripts

This is racy and cbffbb7c2f406fc1d8ebad3c531cc2757232690e makes it
unnecessary.

The eventscript code still knows that monitor events are special
compared to other events.  However, the general concept of monitoring
is no longer tangled up with running scripts.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-eventscripts: Attach to persistent ctdb.tdb in "startup" event

"statd-callout notify" currently complains until an add-client or
del-client is done.

Given that we might use ctdb.tdb for something else in the future it
makes sense attach to it in the "startup" event.  This could be done
in the background but it should be so lightweight that a timeout will
indicate serious problems.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-daemon: Fix tickle updates to recently started nodes

Commit 0723fedcedd4a97870f7b1224945f1587363c9bf added a cheap
implemention of ctdb_control_startup() that simply flags the recipient
node as needing to send updates for each IP when the tickle update
loop next fires.  Commit 026996550d726836091ff5ebd1ebf925bf237bb0
ensures that a node only sends tickle updates once being flagged to do
so.

CTDB_CONTROL_STARTUP is broadcast to all nodes, so this is a good
start.  However, the tickle updates are only broadcast to connected
nodes.  A recently started node may not yet be considered to be
connected because the keepalive monitoring loop may not yet have
marked the node as connected.  This means that the tickle update loop
races with the keepalive monitoring loop.  If the tickle update loop
wins then updates will not be sent to the recently started node.

The simplest improvement is to stop the tickle update from depending
on whether a node is connected or not.  So instead of broadcasting
tickle updates to connected nodes, they are broadcast to all nodes.
Since no reply is expected, this should work just fine.

While looking at this code, ctdb_ctrl_set_tcp_tickles() is named like
a client function.  It isn't a client function.  Also, 2 of the
arguments are ignored.  So rename this function to
ctdb_send_set_tcp_tickles_for_ip() and remove the ignored arguments.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>

ctdb-tools-ctdb: Parse IP addresses when reading a list from a file

This way this logic is centralised.  It also means that the IP address
comparisons in the NAT gateway code are IPv6 safe.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tools-ctdb: Remove redundant filtering of trailing empty lines

There is a check for empty lines in the loop just below.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tools-ctdb: Use DLIST_ADD_END() to avoid reversing the list

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tools-ctdb: Factor out function read_pnn_node_file()

Factor it from read_nodes_file().  Use it there and in
read_natgw_nodes_file().

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tests: Add "ctdb listnodes" and "ctdb xpnn" stub tests

Tests for xpnn need to implement a stub for ctdb_sys_have_ip().  The
cheapest way of doing this is to read a fake nodemap using the
existing code and check if the IP of the "current" node is the one
being asked about.  However, the fake state initialisation isn't
currently available to without_daemon commands because it is meant to
represent daemon state.  However, it can be made available by moving
the relevant code into a new stub for tevent_context_init().  The stub
still needs to initialise a tevent context - this can be done by
calling a lower level function.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tools-ctdb: Read NAT gateway nodes from a separate function

Now it gets easier to refactor.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

ctdb-tools-ctdb: Add and use function filter_nodemap_by_natgw_nodes()

Add another filter function, like the ones for capabilities and flags
to, for filtering by NAT gateway nodes.  This makes the main
natgw_list function more readable.

Note that this drops the early filtering of disconnected nodes, so
they will now be listed in a NAT gateway group.  This makes sense.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>