metze/ctdb/wip.git
8 years agotdb: fix non-WAF build, commit 1.2.6 ABI file.
Rusty Russell [Mon, 27 Sep 2010 01:36:51 +0000 (11:06 +0930)]
tdb: fix non-WAF build, commit 1.2.6 ABI file.

Sorry Jeremy.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
8 years agotdb: TDB_INCOMPATIBLE_HASH, to allow safe changing of default hash.
Rusty Russell [Fri, 24 Sep 2010 06:15:11 +0000 (15:45 +0930)]
tdb: TDB_INCOMPATIBLE_HASH, to allow safe changing of default hash.

This flag to tdb_open/tdb_open_ex effects creation of a new database:
1) Uses the Jenkins lookup3 hash instead of the old gdbm hash if none is
   specified,
2) Places a non-zero field in header->rwlocks, so older versions of TDB will
   refuse to open it.

This means that the caller (ie Samba) can set this flag to safely
change the hash function.  Versions of TDB from this one on will either
use the correct hash or refuse to open (if a different hash is specified).
Older TDB versions will see the nonzero rwlocks field and refuse to open
it under any conditions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
8 years agotdb: automatically identify Jenkins hash tdbs
Rusty Russell [Fri, 24 Sep 2010 06:09:43 +0000 (15:39 +0930)]
tdb: automatically identify Jenkins hash tdbs

If the caller to tdb_open_ex() doesn't specify a hash, and tdb_old_hash
doesn't match, try tdb_jenkins_hash.

This was Metze's idea: it makes life simpler, especially with the upcoming
TDB_INCOMPATIBLE_HASH flag.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
8 years agotdb: add Bob Jenkins lookup3 hash as helper hash.
Rusty Russell [Fri, 24 Sep 2010 06:04:06 +0000 (15:34 +0930)]
tdb: add Bob Jenkins lookup3 hash as helper hash.

This is a better hash than the default: shipping it with tdb makes it easy
for callers to use it as the hash by passing it to tdb_open_ex().

This version taken from CCAN and modified, which took it from
http://www.burtleburtle.net/bob/c/lookup3.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
8 years agotdb: add restore
Volker Lendecke [Sat, 18 Sep 2010 06:56:10 +0000 (10:56 +0400)]
tdb: add restore

Based on an idea by Simon McVittie, largely rewritten

8 years agolib/tdb: fix c++ build warning in tdb_header_hash().
Günther Deschner [Mon, 20 Sep 2010 23:01:51 +0000 (16:01 -0700)]
lib/tdb: fix c++ build warning in tdb_header_hash().

Guenther

8 years agopytdb: Make filename argument optional.
Jelmer Vernooij [Sun, 19 Sep 2010 17:42:29 +0000 (10:42 -0700)]
pytdb: Make filename argument optional.

8 years agopytdb: Add support for tdb_freelist_size()
Kirill Smelkov [Sun, 19 Sep 2010 09:53:29 +0000 (13:53 +0400)]
pytdb: Add support for tdb_freelist_size()

Cc: 597386@bugs.debian.org
Signed-off-by: Kirill Smelkov <kirr@landau.phys.spbu.ru>
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
8 years agopytdb: Add support for tdb_transaction_prepare_commit()
Kirill Smelkov [Sun, 19 Sep 2010 09:53:32 +0000 (13:53 +0400)]
pytdb: Add support for tdb_transaction_prepare_commit()

Cc: 597386@bugs.debian.org
Signed-off-by: Kirill Smelkov <kirr@landau.phys.spbu.ru>
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
8 years agopytdb: Add support for tdb_enable_seqnum, tdb_get_seqnum and tdb_increment_seqnum_non...
Kirill Smelkov [Sun, 19 Sep 2010 16:34:33 +0000 (09:34 -0700)]
pytdb: Add support for tdb_enable_seqnum, tdb_get_seqnum and tdb_increment_seqnum_nonblock

Cc: 597386@bugs.debian.org
Signed-off-by: Kirill Smelkov <kirr@landau.phys.spbu.ru>
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
8 years agopytdb: Update open flags to match those for tdb_open() in tdb.h
Kirill Smelkov [Sun, 19 Sep 2010 09:53:19 +0000 (13:53 +0400)]
pytdb: Update open flags to match those for tdb_open() in tdb.h

Namely TDB_NOSYNC, TDB_SEQNUM, TDB_VOLATILE, TDB_ALLOW_NESTING and
TDB_DISALLOW_NESTING were missing.

Cc: 597386@bugs.debian.org
Signed-off-by: Kirill Smelkov <kirr@mns.spb.ru>
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
8 years agopytdb: Fix repr segfault for internal db
Kirill Smelkov [Sun, 19 Sep 2010 09:53:21 +0000 (13:53 +0400)]
pytdb: Fix repr segfault for internal db

The problem was tdb->name is NULL for TDB_INTERNAL databases, and
so it was crashing ...

    #0  0xb76944f3 in strlen () from /lib/i686/cmov/libc.so.6
    #1  0x0809862b in PyString_FromFormatV (format=0xb72b6a26 "Tdb('%s')", vargs=0xbfc26a94 "")
        at ../Objects/stringobject.c:211
    #2  0x08098888 in PyString_FromFormat (format=0xb72b6a26 "Tdb('%s')") at ../Objects/stringobject.c:358
    #3  0xb72b65f2 in tdb_object_repr (self=0xb759e060) at ./pytdb.c:439

Cc: 597089@bugs.debian.org
Signed-off-by: Kirill Smelkov <kirr@mns.spb.ru>
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
8 years agopytdb: Add support for tdb_add_flags() & tdb_remove_flags()
Kirill Smelkov [Sun, 19 Sep 2010 09:53:20 +0000 (13:53 +0400)]
pytdb: Add support for tdb_add_flags() & tdb_remove_flags()

Note, unlike tdb_open where flags is `int', tdb_{add,remove}_flags want
flags as `unsigned', so instead of "i" I used "I" in PyArg_ParseTuple.

Cc: 597386@bugs.debian.org
Signed-off-by: Kirill Smelkov <kirr@mns.spb.ru>
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
8 years agotdb: added TDB_NO_FSYNC env variable
Andrew Tridgell [Thu, 16 Sep 2010 10:06:44 +0000 (20:06 +1000)]
tdb: added TDB_NO_FSYNC env variable

this might help reduce test times and load on test machines

8 years agotdb: increment version to 1.2.4
Rusty Russell [Thu, 7 Oct 2010 04:37:22 +0000 (15:07 +1030)]
tdb: increment version to 1.2.4

8 years agotdb: put example hashes into header, so we notice incorrect hash_fn.
Rusty Russell [Mon, 13 Sep 2010 10:35:59 +0000 (20:05 +0930)]
tdb: put example hashes into header, so we notice incorrect hash_fn.

This is Stefan Metzmacher <metze@samba.org>'s patch with minor changes:
1) Use the TDB_MAGIC constant so both hashes aren't of strings.
2) Check the hash in tdb_check (paranoia, really).
3) Additional check in the (unlikely!) case where both examples hash to 0.
4) Cosmetic changes to var names and complaint message.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
8 years agotdb: fix tdb_check() on other-endian tdbs.
Rusty Russell [Mon, 13 Sep 2010 10:29:18 +0000 (19:59 +0930)]
tdb: fix tdb_check() on other-endian tdbs.

We must not endian-convert the magic string, just the rest.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
8 years agotdb: fix tdb_check() on read-only TDBs to actually work.
Rusty Russell [Mon, 13 Sep 2010 10:28:23 +0000 (19:58 +0930)]
tdb: fix tdb_check() on read-only TDBs to actually work.

Commit bc1c82ea137 "Fix tdb_check() to work with read-only tdb databases."
claimed to do this, but tdb_lockall_read() fails on read-only databases.

Also make sure we can still do tdb_check() inside a transaction (weird,
but we previously allowed it so don't break the API).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
8 years agotdb: make check more robust against recovery failures.
Rusty Russell [Mon, 13 Sep 2010 10:25:26 +0000 (19:55 +0930)]
tdb: make check more robust against recovery failures.

We can end up with dead areas when we die during transaction commit;
tdb_check() fails on such a (valid) database.

This is particularly noticable now we no longer truncate on recovery;
if the recovery area was at the end of the file we used to remove it
that way.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
8 years agoadd a machinereadable version of ctdb stats/statistics
Ronnie Sahlberg [Thu, 30 Sep 2010 04:59:59 +0000 (14:59 +1000)]
add a machinereadable version of ctdb stats/statistics

8 years agoCreate a tunable for how often to collect rolling statistics and initialize it to...
Ronnie Sahlberg [Thu, 30 Sep 2010 04:39:54 +0000 (14:39 +1000)]
Create a tunable for how often to collect rolling statistics and initialize it to 1 second

8 years agoAdd rolling statistics that are collected across 10 second intervals.
Ronnie Sahlberg [Wed, 29 Sep 2010 02:13:05 +0000 (12:13 +1000)]
Add rolling statistics that are collected across 10 second intervals.

Add a new command "ctdb stats [num]" that prints the [num] most recent statistics intervals collected.

8 years agoAdd a new statistics structure to keep the current running statistics
Ronnie Sahlberg [Wed, 29 Sep 2010 00:58:18 +0000 (10:58 +1000)]
Add a new statistics structure to keep the current running statistics

8 years agoCreate macros to update the statistics counters and use these macros
Ronnie Sahlberg [Wed, 29 Sep 2010 00:38:41 +0000 (10:38 +1000)]
Create macros to update the statistics counters and use these macros
everywhere instead of manipulating the coutenrs directly.

8 years agoAdd back monitoring for time skips, forward as well as backward.
Ronnie Sahlberg [Mon, 27 Sep 2010 22:58:03 +0000 (08:58 +1000)]
Add back monitoring for time skips, forward as well as backward.
This serviceability tool was lost during the migration from the old eventsystem to the tevent system.

8 years agoupdate/improve the log message related to rerecovery timeouts
Ronnie Sahlberg [Mon, 27 Sep 2010 22:46:12 +0000 (08:46 +1000)]
update/improve the log message related to rerecovery timeouts

8 years agoset up a handler to catch and log debug messages from the tevent layer
Ronnie Sahlberg [Wed, 22 Sep 2010 00:59:01 +0000 (10:59 +1000)]
set up a handler to catch and log debug messages from the tevent layer

8 years agoadda GETPUBLICIPS control to libctdb and use this in the test example
Ronnie Sahlberg [Wed, 15 Sep 2010 04:56:57 +0000 (14:56 +1000)]
adda GETPUBLICIPS control to libctdb and use this in the test example

enhance the test example to show the new releaseip/takeip messages

8 years agoserver/banning: also release all ips if we're banning ourself
Stefan Metzmacher [Tue, 31 Aug 2010 07:28:34 +0000 (09:28 +0200)]
server/banning: also release all ips if we're banning ourself

metze

8 years agoserver/recoverd: if we can't get the recovery lock, ban ourself
Stefan Metzmacher [Mon, 30 Aug 2010 16:25:28 +0000 (18:25 +0200)]
server/recoverd: if we can't get the recovery lock, ban ourself

metze

8 years agoserver/recoverd: do takeover_run after verifying the reclock file
Stefan Metzmacher [Tue, 31 Aug 2010 06:42:32 +0000 (08:42 +0200)]
server/recoverd: do takeover_run after verifying the reclock file

metze

8 years agoserver/monitor: ask for a takeoverrun after propagating our new flags
Stefan Metzmacher [Tue, 24 Aug 2010 07:22:49 +0000 (09:22 +0200)]
server/monitor: ask for a takeoverrun after propagating our new flags

metze

8 years agoadd a new serverid to send a message everytime an ip address is taken on the local...
Ronnie Sahlberg [Mon, 13 Sep 2010 05:42:00 +0000 (15:42 +1000)]
add a new serverid to send a message everytime an ip address is taken on the local node

8 years agoUpdate the comment for the range reserved for SAMBA and
Ronnie Sahlberg [Mon, 13 Sep 2010 05:08:36 +0000 (15:08 +1000)]
Update the comment for the range reserved for SAMBA and
define a new symbol to represent this range similarly to NFSD and ISCSID

Keep the old symbol name to be backward compatible with software using
these headers.

8 years agodefine and reserve a range of ctdb message ports for use by nfs and iscsi servers
Ronnie Sahlberg [Mon, 13 Sep 2010 05:06:43 +0000 (15:06 +1000)]
define and reserve a range of ctdb message ports for use by nfs and iscsi servers

8 years agoAdd two new server types to the server_id structure.
Ronnie Sahlberg [Mon, 13 Sep 2010 05:01:47 +0000 (15:01 +1000)]
Add two new server types to the server_id structure.
NFSD and ISCSID for now.

8 years agoImplement a new function GETNODEMAP in libctdb.
Ronnie Sahlberg [Mon, 13 Sep 2010 04:28:11 +0000 (14:28 +1000)]
Implement a new function GETNODEMAP in libctdb.
This function returns a pointer to a nodemap structure.

The returned structure must later be freed by calling ctdb_free_nodemap().

Move the definition of ctdb_sock_addr from ctdb_client.h to ctdb_protocol.h

Move the definition of the node flags, ctdb_node_and_flags and ctdb_node_map from ctdb_private.h to ctdb_protocol.h

Add both sync and async example for ctdb_getnodemap to the test application libctdb/tst.c

8 years agoremove an unused variable
Ronnie Sahlberg [Mon, 13 Sep 2010 03:12:41 +0000 (13:12 +1000)]
remove an unused variable

8 years agoDont try to read the nodemap from the daemon for "ctdb listnodes"
Ronnie Sahlberg [Wed, 8 Sep 2010 21:35:10 +0000 (07:35 +1000)]
Dont try to read the nodemap from the daemon for "ctdb listnodes"

Always read it from the /etc/ctdb/nodes file

8 years agoChange how NATGW is configured to allow special nodes that do not have
Ronnie Sahlberg [Tue, 7 Sep 2010 23:16:42 +0000 (09:16 +1000)]
Change how NATGW is configured to allow special nodes that do not have
network connectivity outside of the cluster to still be able to
participate in a natgw group.
These nodes can not become natgw master since they lack external network
connectivity.

These nodes are configured just the same way as for any other node with
NATGW, with the following two exceptions :
* we do NOT set CTDB_NATGW_PUBLIC_IFACE at all on these nodes.
  since these ndoes lack external network we should not check the interface
  for link.
* we must set CTDB_NATGW_SLAVE_ONLY=yes to flag that this is a node that
  can not become natgw master.

8 years agoDont store temporary runtime data in $CTDB_BASE/state
Ronnie Sahlberg [Fri, 3 Sep 2010 02:35:25 +0000 (12:35 +1000)]
Dont store temporary runtime data in $CTDB_BASE/state
since that will usually be /etc/ctdb/state and storing this under /etc is just
wrong.

Add a new variable CTDB_VARDIR that defaults to /var/ctdb and store the data there instead.

8 years agoWhen memory allocations for recovery fails,
Ronnie Sahlberg [Fri, 3 Sep 2010 01:58:27 +0000 (11:58 +1000)]
When memory allocations for recovery fails,
dont dereference a null pointer while trying to print the log message for the failure.

also shutdown ctdb with ctdb_fatal()

8 years agoAIX bind wants the correct addrsize
Harald Klatte [Mon, 30 Aug 2010 08:40:43 +0000 (10:40 +0200)]
AIX bind wants the correct addrsize

8 years agomake sure all statd state directories exist before we try to reference them
Ronnie Sahlberg [Wed, 1 Sep 2010 05:48:55 +0000 (15:48 +1000)]
make sure all statd state directories exist before we try to reference them
or else tar and friends will throw an error in the log

8 years agodont print a lot of log information about shutting down vsftpd
Ronnie Sahlberg [Wed, 1 Sep 2010 03:28:25 +0000 (13:28 +1000)]
dont print a lot of log information about shutting down vsftpd

8 years agoouch, remove a dummy debug printout that snuck in there somehow
Ronnie Sahlberg [Mon, 30 Aug 2010 09:47:50 +0000 (19:47 +1000)]
ouch, remove a dummy debug printout that snuck in there somehow

8 years agoouch, the ordering of the constants and the strings must be kept in sync
Ronnie Sahlberg [Mon, 30 Aug 2010 09:42:30 +0000 (19:42 +1000)]
ouch,   the ordering of the constants and the strings must be kept in sync
manually   and ther eis no check for errors.     should fix this later

8 years agoremove 61.nfstickles from the makefile
Ronnie Sahlberg [Mon, 30 Aug 2010 08:29:18 +0000 (18:29 +1000)]
remove 61.nfstickles from the makefile

8 years agowe no longer have a 61.nfstickle script
Ronnie Sahlberg [Mon, 30 Aug 2010 08:22:28 +0000 (18:22 +1000)]
we no longer have a 61.nfstickle script

8 years agoMerge commit 'martins/master'
Ronnie Sahlberg [Mon, 30 Aug 2010 08:22:05 +0000 (18:22 +1000)]
Merge commit 'martins/master'

8 years agoremove the mention of a tickle and statd directory in shared storage now that we...
Ronnie Sahlberg [Mon, 30 Aug 2010 08:15:41 +0000 (18:15 +1000)]
remove the mention of a tickle and statd directory in shared storage now that we are removing these and migrating to store the data inside ctdbd or persistent databases

8 years agoRemove the dependency on the underlying cluster filesystem for handling
Ronnie Sahlberg [Mon, 30 Aug 2010 08:13:28 +0000 (18:13 +1000)]
Remove the dependency on the underlying cluster filesystem for handling
the clusterwide persistent data associated with the lock manager and
statd notifications.

Use persistent databases to store this data instead of a shared directory.

8 years ago Add a new event "ipreallocated"
Ronnie Sahlberg [Mon, 30 Aug 2010 08:08:38 +0000 (18:08 +1000)]
Add a new event "ipreallocated"
    This is called everytime a reallocation is performed.

    While STARTRECOVERY/RECOVERED events are only called when
    we do ipreallocation as part of a full database/cluster recovery,
    this new event can be used to trigger on when we just do a light
    failover due to a node becomming unhealthy.

    I.e. situations where we do a failover but we do not perform a full
    cluster recovery.

    Use this to trigger for natgw so we select a new natgw master node
    when failover happens and not just when cluster rebuilds happen.

8 years agoTest suite: Make NFS tickle test more flexible.
Martin Schwenke [Fri, 27 Aug 2010 01:43:50 +0000 (11:43 +1000)]
Test suite: Make NFS tickle test more flexible.

Use onnode any where possible rather than a fixed node.

Signed-off-by: Martin Schwenke <martin@meltin.net>
8 years agoTest suite: Fix NFS tickle test.
Martin Schwenke [Fri, 27 Aug 2010 01:40:44 +0000 (11:40 +1000)]
Test suite: Fix NFS tickle test.

We now kill ctdbd on the test node instead of disabling it.  This
ensures that the only tickles we see will come from the takeover node.

We also sleep for TickleUpdateInterval before checking for asking ctdb
about the tickles.

Signed-off-by: Martin Schwenke <martin@meltin.net>
8 years agoTest suite: Tweak NFS tickle test.
Martin Schwenke [Thu, 26 Aug 2010 07:56:50 +0000 (17:56 +1000)]
Test suite: Tweak NFS tickle test.

Signed-off-by: Martin Schwenke <martin@meltin.net>
8 years agoTest suite: Fix typos in NFS tickle test.
Martin Schwenke [Thu, 26 Aug 2010 05:50:35 +0000 (15:50 +1000)]
Test suite: Fix typos in NFS tickle test.

Signed-off-by: Martin Schwenke <martin@meltin.net>
8 years agoTest suite: NFS tickle test uses gettickles if events.d/61.nfstickle missing.
Martin Schwenke [Thu, 26 Aug 2010 05:28:19 +0000 (15:28 +1000)]
Test suite: NFS tickle test uses gettickles if events.d/61.nfstickle missing.

Signed-off-by: Martin Schwenke <martin@meltin.net>
8 years agoNFS tickles: use addtickle/deltickle instead of shared tickle directory.
Martin Schwenke [Thu, 26 Aug 2010 04:59:59 +0000 (14:59 +1000)]
NFS tickles: use addtickle/deltickle instead of shared tickle directory.

This adds a new function update_tickles() that tracks tickles for a
given port using the new ctdb addtickle/deltickle commands.  This
function is used in events.d/60.nfs to handle NFS tickles.

events.d/61.nfstickle is removed.  The
/proc/sys/net/ipv4/tcp_tw_recycle setup is also moved to
events.d/60.nfs.

Signed-off-by: Martin Schwenke <martin@meltin.net>
8 years agoTest suite: in the test eventscript, run "ctdb" not "$CTDB".
Martin Schwenke [Thu, 26 Aug 2010 04:04:03 +0000 (14:04 +1000)]
Test suite: in the test eventscript, run "ctdb" not "$CTDB".

It is too hard to do anything else...

Signed-off-by: Martin Schwenke <martin@meltin.net>
8 years agoMerge branch 'master' of git://git.samba.org/sahlberg/ctdb
Martin Schwenke [Thu, 26 Aug 2010 01:06:57 +0000 (11:06 +1000)]
Merge branch 'master' of git://git.samba.org/sahlberg/ctdb

8 years agoAdd a configuration database, implemented as a persistent database.
Ronnie Sahlberg [Wed, 25 Aug 2010 01:37:32 +0000 (11:37 +1000)]
Add a configuration database, implemented as a persistent database.

This database can be used, as an option, to store
the public address assignment instead of editing the /etc/ctdb/public-addresses file manually.

This configuration is stored in one record per key, with a key-name of
public-addresses:node#<pnn>
where <pnn> is the node number.

The content of this record is the same syntax as the /etc/ctdb/public-addresses file.

When ctdbd starts, if this key exist and contains data. It is extracted from the database and compared with the normal file /etc/ctdb/public-addresses.

If the content differs, the config database "wins" and is used to overwrite/update the /etc/ctdb/public-addresses file, after which ctdbd is restarted.

The main benefit with this option is that it can be used to update the public address configuration for nodes that are offline/unreachable by updating their configuration in the persistent database.
Once the offline node is available again, it will resync its databases with the rest of the cluster, find out that the config has changed, apply the changes and restart ctdbd automatically.

The command to store the public address configuration for a node into the persistent database is :

ctdb pstore config.tdb public-addresses:node#<pnn> <filename>

where <pnn> is the node# we wish to update the config for, and <filename> is a file containing the new content for  that nodes public address configuration.

8 years agothe tfetch command can be used without the daemon running, so flag it as such.
Ronnie Sahlberg [Wed, 25 Aug 2010 01:10:08 +0000 (11:10 +1000)]
the tfetch command can be used without the daemon running, so flag it as such.

fix a couple of incorrect settings for "auto-all" for a few of the commands as well.

8 years agoadd a new command "ctdb tfetch" that can read a record straight out of the
Ronnie Sahlberg [Wed, 25 Aug 2010 00:53:54 +0000 (10:53 +1000)]
add a new command "ctdb tfetch" that can read a record straight out of the
tdb file.

the command automatically strips off the initial ctdb header off the record so it can only be used on ctdb managed tdb files, not on normal tdb files.

8 years agoWhen "ctdb pfetch" creates a new file, make sure we set some initial sane mode bits
Ronnie Sahlberg [Tue, 24 Aug 2010 23:54:37 +0000 (09:54 +1000)]
When "ctdb pfetch" creates a new file, make sure we set some initial sane mode bits

8 years agorun the "init" event before we freeze the databases
Ronnie Sahlberg [Tue, 24 Aug 2010 22:34:35 +0000 (08:34 +1000)]
run the "init" event before we freeze the databases
so that we can read from databases during this event

8 years agochange "ctdb pfetch" to take an optional third argument
Ronnie Sahlberg [Tue, 24 Aug 2010 22:07:03 +0000 (08:07 +1000)]
change "ctdb pfetch" to take an optional third argument
as a file to store the record in.

8 years agoadd a command to write a record to a persistent database
Ronnie Sahlberg [Tue, 24 Aug 2010 03:55:38 +0000 (13:55 +1000)]
add a command to write a record to a persistent database
"ctdb pstore <db> <key> <file containing possibly binary data>"

8 years agoget rid of two compiler warnings
Ronnie Sahlberg [Tue, 24 Aug 2010 03:35:33 +0000 (13:35 +1000)]
get rid of two compiler warnings

8 years agoAdd a command "ctdb pfetch <db> <record>" to read a record from
Ronnie Sahlberg [Tue, 24 Aug 2010 03:34:09 +0000 (13:34 +1000)]
Add a command "ctdb pfetch <db> <record>" to read a record from
a persistent database.

8 years agoMerge branch 'master' of git://git.samba.org/sahlberg/ctdb
Martin Schwenke [Tue, 24 Aug 2010 01:53:29 +0000 (11:53 +1000)]
Merge branch 'master' of git://git.samba.org/sahlberg/ctdb

8 years agomove the directives to build the devel file to the end of the specfile
Ronnie Sahlberg [Mon, 23 Aug 2010 06:00:19 +0000 (16:00 +1000)]
move the directives to build the devel file to the end of the specfile
so that the dependencies are right
or else the dependencies all end up in the devel package and not the main
ctdb package

9 years agoDont set next_interval to 0.
Ronnie Sahlberg [Fri, 20 Aug 2010 04:54:03 +0000 (14:54 +1000)]
Dont set next_interval to 0.
This can cause ctdbd to spin at 100% in the eventsystem,
creating a timed event that will immediately trigger again
and again.

On uniprocessors this cause the eventscript we are actually waiting for to
basically become cpu starved and never complete.

9 years agoctdb ip is very busy.
Ronnie Sahlberg [Fri, 20 Aug 2010 01:38:34 +0000 (11:38 +1000)]
ctdb ip is very busy.

revert the defauls case back to only showing the ip and node
and only display the extra info if -v verbose output is requested

9 years agoadd a new commandline flag -v to enable verbose output
Ronnie Sahlberg [Fri, 20 Aug 2010 01:28:24 +0000 (11:28 +1000)]
add a new commandline flag -v to enable verbose output

9 years agomake it possible to "ctdb gettickle" to only list tickles for a certain
Ronnie Sahlberg [Fri, 20 Aug 2010 01:25:12 +0000 (11:25 +1000)]
make it possible to "ctdb gettickle" to only list tickles for a certain
port.

Default is to continue to show all tickles, but if a second argument
is given, only tickles for that port will be shown.

9 years agoDont use the deprecated talloc_append_string()
Ronnie Sahlberg [Fri, 20 Aug 2010 01:03:17 +0000 (11:03 +1000)]
Dont use the deprecated talloc_append_string()
Use talloc_strdup_append() instead

9 years agoWe need the deprecated talloc_append_string() for now
Ronnie Sahlberg [Thu, 19 Aug 2010 04:48:19 +0000 (14:48 +1000)]
We need the deprecated talloc_append_string() for now
so set the TALLOC_DEPRECATED sympol to allow use of this call
from ctdb_client.c

9 years agoMerge commit 'rusty/ports-from-1.0.112' into foo
Ronnie Sahlberg [Thu, 19 Aug 2010 03:17:56 +0000 (13:17 +1000)]
Merge commit 'rusty/ports-from-1.0.112' into foo

9 years agoMerge commit 'rusty/vacuum-fix-master'
Ronnie Sahlberg [Thu, 19 Aug 2010 03:16:35 +0000 (13:16 +1000)]
Merge commit 'rusty/vacuum-fix-master'

9 years ago On RHEL, "service nfs stop;service nfs start" and "service nfs restart"
Ronnie Sahlberg [Wed, 18 Aug 2010 21:18:22 +0000 (07:18 +1000)]
On RHEL,    "service nfs stop;service nfs start"  and "service nfs restart"
    sometimes (very rarely) fails to restart the service.

    Add a function to restart NFSd on SLES and RHEL-like systems.

    If we detect the system is unhealthy due to kNFSd not running,
    try to restart the service again "service nfs restart" and
    hope for the best.

CQ1019372

9 years agoAdd machinereadable output for the "ctgdb gettickles <ip>" command
Ronnie Sahlberg [Wed, 18 Aug 2010 04:37:16 +0000 (14:37 +1000)]
Add machinereadable output for the "ctgdb gettickles <ip>" command

9 years agoRemove the structure ctdb_control_tcp_vnn since this is identical to the structure...
Ronnie Sahlberg [Wed, 18 Aug 2010 02:36:03 +0000 (12:36 +1000)]
Remove the structure ctdb_control_tcp_vnn since this is identical to the structure ctdb_tcp_connection.

Add a new "ctdb deltickle" command to delete tickles from the database.
This can ONLY be used for tickles created by "ctdb addtickle".

Push any "addtickle/deltickle" updates to other nodes every TickleUpdateInterval seconds'

9 years agologging: give a unique logging name to each forked child.
Rusty Russell [Mon, 19 Jul 2010 09:59:09 +0000 (19:29 +0930)]
logging: give a unique logging name to each forked child.

This means we can distinguish which child is logging, esp. via syslog where we have no pid.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
9 years agotakeover: prevent crash by avoiding free in traverse on RST timeout
Rusty Russell [Mon, 26 Jul 2010 04:28:48 +0000 (13:58 +0930)]
takeover: prevent crash by avoiding free in traverse on RST timeout

After 5 attempts to send a RST to a client without any response, we free
"con"; this is done during a traverse.  This frees the node we are walking
through (the node is made a child of "con" down in rb_tree.c's
trbt_create_node() (Valgrind would catch this, as Martin confirmed).

So, we create a temporary parent and reparent onto that; then we free
that parent after the traverse, thus deleting the unwanted nodes.

CQ:S1019041
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
9 years agoMove NAT gateway firewall rules to recovered|updatenatgw events.
Martin Schwenke [Tue, 6 Jul 2010 07:54:43 +0000 (17:54 +1000)]
Move NAT gateway firewall rules to recovered|updatenatgw events.

The existing code wasn't working as designed in the start event.  It
should work here.

BZ: 62613
Signed-off-by: Martin Schwenke <martin@meltin.net>
9 years agovacuum: disabling vacuuming during a freeze
Rusty Russell [Wed, 21 Jul 2010 02:58:04 +0000 (12:28 +0930)]
vacuum: disabling vacuuming during a freeze

We shouldn't even think about vacuuming when we've frozen the database
(which is earlier than when we set CTDB_RECOVERY_ACTIVE)

CQ:S1018154 & S1018349
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
9 years agovacuum: fix crash on vacuum abort
Rusty Russell [Mon, 26 Jul 2010 06:38:07 +0000 (16:08 +0930)]
vacuum: fix crash on vacuum abort

Martin Schwenke discovered that 517f05e42f17766b1e8db8f1f4789cbad968e304
("freeze: abort vacuuming when we're going to freeze.") used ctdb_db for
a logging message which is in fact uninitialized, causing a crash (even
if it wasn't actually logged).

Initialize it properly.  Also fix incorrect format in another logging
message introduced in that same change.

CQ:S1019093
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
9 years agoTest suite: loosen the getmonmode test.
Martin Schwenke [Wed, 18 Aug 2010 01:25:44 +0000 (11:25 +1000)]
Test suite: loosen the getmonmode test.

Monitoring could be off at the beginning of the test.

Signed-off-by: Martin Schwenke <martin@meltin.net>
9 years agofreeze: abort vacuuming when we're going to freeze.
Rusty Russell [Wed, 21 Jul 2010 02:59:55 +0000 (12:29 +0930)]
freeze: abort vacuuming when we're going to freeze.

There are some reports of freeze timeouts, and it looks like vacuuming might
be the culprit.  So we add code to tell them to abort when a freeze is
going on.

(This is based on the 1.0.112 branch version 517f05e42f, but far
 simpler since tdb is now robust against processes being killed during
 transaction commit)

CQ:S1018154 & S1018349
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
9 years agoAdd a new "ctdb addtickle" command to manually add tickles to ctdbd
Ronnie Sahlberg [Wed, 18 Aug 2010 01:09:32 +0000 (11:09 +1000)]
Add a new "ctdb addtickle" command to manually add tickles to ctdbd

This can be used to set ctdbd up to generate a tickle for non-samba
services.
(samba contains code to set tickles up automatically)

9 years agoupdate the example for the new signature of
Ronnie Sahlberg [Wed, 18 Aug 2010 00:18:35 +0000 (10:18 +1000)]
update the example for the new signature of
ctdb_set_message_handler_send()

9 years agoWe use eventloop nesting in a couple of places, notably the sync
Ronnie Sahlberg [Wed, 18 Aug 2010 00:11:59 +0000 (10:11 +1000)]
We use eventloop nesting in a couple of places, notably the sync
parts of the recovery daemon.

Initialize all event contexts to allow nesting

9 years agoMerge commit 'rusty/libctdb-new' into foo
Ronnie Sahlberg [Tue, 17 Aug 2010 23:53:52 +0000 (09:53 +1000)]
Merge commit 'rusty/libctdb-new' into foo

9 years agoevent: Update events to latest Samba version 0.9.8
Rusty Russell [Tue, 17 Aug 2010 23:46:31 +0000 (09:16 +0930)]
event: Update events to latest Samba version 0.9.8

In Samba this is now called "tevent", and while we use the backwards
compatibility wrappers they don't offer EVENT_FD_AUTOCLOSE: that is now
a separate tevent_fd_set_auto_close() function.

This is based on Samba version 7f29f817fa939ef1bbb740584f09e76e2ecd5b06.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
9 years agotalloc: update to 2.0.3 version from SAMBA
Rusty Russell [Tue, 17 Aug 2010 23:41:58 +0000 (09:11 +0930)]
talloc: update to 2.0.3 version from SAMBA

This is based on SAMBA as at revision 2de63aa2801a907905b3e05557074af5b896d486.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
9 years agoTest suite: Add more timestamping of debugging information.
Martin Schwenke [Mon, 16 Aug 2010 23:55:48 +0000 (09:55 +1000)]
Test suite: Add more timestamping of debugging information.

Signed-off-by: Martin Schwenke <martin@meltin.net>
9 years agoTest suite: print date/time at test completion.
Martin Schwenke [Mon, 16 Aug 2010 23:52:15 +0000 (09:52 +1000)]
Test suite: print date/time at test completion.

This should help with log cross-checking.

Signed-off-by: Martin Schwenke <martin@meltin.net>
9 years agoCorrectly set docdir
Volker Lendecke [Fri, 6 Aug 2010 08:12:13 +0000 (10:12 +0200)]
Correctly set docdir

9 years agotdb: workaround starvation problem in locking entire database.
Rusty Russell [Mon, 16 Aug 2010 00:52:21 +0000 (10:22 +0930)]
tdb: workaround starvation problem in locking entire database.

(Imported from SAMBA 11ab43084b10cf53b530cdc3a6036c898b79ca38)

We saw tdb_lockall() take 71 seconds under heavy load; this is because Linux
(at least) doesn't prevent new small locks being obtained while we're waiting
for a big log.

The workaround is to do divide and conquer using non-blocking chainlocks: if
we get down to a single chain we block.  Using a simple test program where
children did "hold lock for 100ms, sleep for 1 second" the time to do
tdb_lockall() dropped signifiantly.  There are ln(hashsize) locks taken in
the contended case, but that's slow anyway.

More analysis is given in my blog at http://rusty.ozlabs.org/?p=120

This may also help transactions, though in that case it's the initial
read lock which uses this gradual locking routine; the update-to-write-lock
code is separate and still tries to update in one go.

Even though ABI doesn't change, minor version bumped so behavior change
can be easily detected.

CQ:S1018154
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>