Andrew Tridgell [Sun, 13 Jul 2008 23:19:22 +0000 (09:19 +1000)]
fixed up exit status for onnode
Andrew Tridgell [Fri, 11 Jul 2008 09:21:39 +0000 (19:21 +1000)]
Merge commit 'ronnie/master'
Ronnie Sahlberg [Fri, 11 Jul 2008 01:48:41 +0000 (11:48 +1000)]
new version 1.0.47
Signed-off-by: Ronnie Sahlberg <ronniesahlberg@gmail.com>
Ronnie Sahlberg [Fri, 11 Jul 2008 00:33:46 +0000 (10:33 +1000)]
Fix a very subtle race where we could get a double free of a talloced
memory if ctdb_run_eventscript() would be called
during processing of ctdb_event_script_timeout() for
user unvoked eventscripts. (eventsccripts invoked by "ctdb eventscript ...")
Signed-off-by: Ronnie Sahlberg <ronniesahlberg@gmail.com>
Martin Schwenke [Thu, 10 Jul 2008 23:24:21 +0000 (09:24 +1000)]
Signed-off-by: Martin Schwenke <martin@meltin.net>
Update packaging/RPM/ctdb.spec to reflect onnode changes.
Ronnie Sahlberg [Thu, 10 Jul 2008 07:06:52 +0000 (17:06 +1000)]
Revert "Yip yip yip!"
This reverts commit
f7bdf96843a7e4ad61ad378786922d6281de9d93.
Martin Schwenke [Thu, 10 Jul 2008 06:56:30 +0000 (16:56 +1000)]
Yip yip yip!
Martin Schwenke [Thu, 10 Jul 2008 04:19:52 +0000 (14:19 +1000)]
When in verbose mode with -p, each line is prefixed with the node
address/name. To implement this stderr has redirected to stdout -
this doesn't need to be done but is the simplest implementation.
Remove -t option since it doesn't seem to accomplish much but causes
spurious messages to be displayed by ssh. Add explicit -h and --help
options. Make style of usage message consistent with documentation.
Document new features in doc/onnode.1.xml.
Martin Schwenke [Wed, 9 Jul 2008 04:23:02 +0000 (14:23 +1000)]
Update Makefile.in for new version of onnode.
Martin Schwenke [Wed, 9 Jul 2008 04:18:15 +0000 (14:18 +1000)]
Complete rewrite of tools/onnode. Remove old tools/onnode.ssh,
tools/onnode.rsh.
Ronnie Sahlberg [Thu, 10 Jul 2008 03:40:00 +0000 (13:40 +1000)]
explain why you have to have a real ip address as well as the "virtual"
ip address for lvs
Ronnie Sahlberg [Thu, 10 Jul 2008 03:00:50 +0000 (13:00 +1000)]
new version 10.0.46
Ronnie Sahlberg [Thu, 10 Jul 2008 02:50:16 +0000 (12:50 +1000)]
add documentation for both LVS:single-ip and CAPABILITIES:wan-accelerator
Ronnie Sahlberg [Thu, 10 Jul 2008 01:42:37 +0000 (11:42 +1000)]
Update to the LVS eventscript.
Do not assume all nodes are members of LVS so always deciding the recmaster will be lvsmaster wont work.
Instead,
Create the set of active LVS nodes as those nodes that are LVS capable and
also HEALTHY.
Except if ALL LVS capable nodes are unhealthy in which case we allow the unhealthy
nodes to be part of the active set.
In the active set, pick one of the active nodes as being the lvsmaster
which will receive all incoming traffic and distribute it across
the active lvs nodes in the cluster.
Ronnie Sahlberg [Thu, 10 Jul 2008 01:12:58 +0000 (11:12 +1000)]
Add three mode commands to the CTDB tool.
lvs: which shows which nodes are active LVS servers
lvsmaster: which shows which node is the lvs master multiplex node
pnn: which prints the pnn of the local node
Ronnie Sahlberg [Thu, 10 Jul 2008 00:37:22 +0000 (10:37 +1000)]
make LVS a capability so that we can see which nodes are configured with
LVS and which are not using LVS.
"ctdb getcapabilities"
Ronnie Sahlberg [Wed, 9 Jul 2008 22:56:33 +0000 (08:56 +1000)]
add an option to skip checking that all the samba shares are ok
when monitoring the node health.
this might be useful to skip for environments with thousands of shares
Ronnie Sahlberg [Wed, 9 Jul 2008 22:05:34 +0000 (08:05 +1000)]
remove the attempts to restart NFS.
nfs should never stop spontaneously so trying to restart it is
just counterproductive and at best a workaround to
hide real bugs.
Ronnie Sahlberg [Wed, 9 Jul 2008 05:17:27 +0000 (15:17 +1000)]
if we have enabled LVS but we dont have all the required packages
just log it to the messages
dont stop ctdb from starting
Ronnie Sahlberg [Wed, 9 Jul 2008 04:02:54 +0000 (14:02 +1000)]
proper waitpid() fix.
remove all waitpid() calls and use the event system to trap sigchld
Ronnie Sahlberg [Wed, 9 Jul 2008 03:14:47 +0000 (13:14 +1000)]
Revert "pull the development files out into their own package"
This reverts commit
36be210bbc5e0af75c5fd6e57863272bfa0e942e.
Ronnie Sahlberg [Wed, 9 Jul 2008 03:14:34 +0000 (13:14 +1000)]
Revert "add spec file for development rpm"
This reverts commit
bd7b254b81dda4d9d62516abf32f93f2503eb9bb.
Ronnie Sahlberg [Wed, 9 Jul 2008 03:14:07 +0000 (13:14 +1000)]
Revert "copy ctdb-dev to the spec directory"
This reverts commit
8814997c1b9623397058088dd0e1775cecfe371b.
Ronnie Sahlberg [Wed, 9 Jul 2008 03:07:17 +0000 (13:07 +1000)]
copy ctdb-dev to the spec directory
Ronnie Sahlberg [Wed, 9 Jul 2008 01:37:02 +0000 (11:37 +1000)]
add spec file for development rpm
Ronnie Sahlberg [Wed, 9 Jul 2008 01:32:19 +0000 (11:32 +1000)]
pull the development files out into their own package
Ronnie Sahlberg [Wed, 9 Jul 2008 01:08:44 +0000 (11:08 +1000)]
install the readme in /usr/share/doc/ctdb/ instead of under /etc
Ronnie Sahlberg [Wed, 9 Jul 2008 00:24:19 +0000 (10:24 +1000)]
mark /etc/ctdb/functions as a config file to keep rpmlint happy
Ronnie Sahlberg [Wed, 9 Jul 2008 00:17:39 +0000 (10:17 +1000)]
From Chris Cowan, patch to make aix compile again
Ronnie Sahlberg [Wed, 9 Jul 2008 00:03:21 +0000 (10:03 +1000)]
Replace \s with [[:space:]] in our regexps we use for egrep.
Kevin Collins noticed that RHEL5 grep-2.5.1-54.2.el5 built for
x86 does not handle \s while the exact same RHEL5 package for amd64
does!
[[:space:]] is more portable. Even across the same package version ( different architecture ) from the same vendor :-)
Ronnie Sahlberg [Tue, 8 Jul 2008 07:41:31 +0000 (17:41 +1000)]
Revert "waitpid() can block if it takes a long time before the child terminates"
This reverts commit
bfba5c7249eff8a10a43b53c1b89dd44b625fd10.
revert the waitpid changes. we need to waitpid for some childredn so should
refactor the approach completely
Ronnie Sahlberg [Tue, 8 Jul 2008 07:40:53 +0000 (17:40 +1000)]
Revert "set sigchild to SIG_IGN instead of SIG_DFL"
This reverts commit
b1f1e80d3ad50280a300f2ed021513cf0a6f3a76.
Ronnie Sahlberg [Tue, 8 Jul 2008 06:31:23 +0000 (16:31 +1000)]
set sigchild to SIG_IGN instead of SIG_DFL
Ronnie Sahlberg [Tue, 8 Jul 2008 00:03:57 +0000 (10:03 +1000)]
new version 1.0.45
Ronnie Sahlberg [Mon, 7 Jul 2008 23:58:10 +0000 (09:58 +1000)]
update the monitor event for nfs to track how many times in a row it has failed
to "ping" the local nfs daemon.
Once it has failed more than 3 times in a row it will attempt to restart the nfs service.
Ronnie Sahlberg [Mon, 7 Jul 2008 17:48:11 +0000 (03:48 +1000)]
waitpid() can block if it takes a long time before the child terminates
so we should not call it from the main daemon.
1, set SIGCHLD to SIG_DFL to make sure we ignore this signal
2, get rid of all waitpid() calls
3, change reporting of event script status code from _exit()/waitpid() to write()/read() one byte across the pipe.
Ronnie Sahlberg [Mon, 7 Jul 2008 10:38:59 +0000 (20:38 +1000)]
use more libral handling of event scripts timing out.
If the event script that timed out was for the "monitor" event, then
even if it timed out we still return SUCCESS back to the guy invoking the eventscript.
Only consider the eventscript for "monitor" to have failed with an error
IFF it actually terminated with an error, or if it timed out 5 times in a row and hung.
Ronnie Sahlberg [Sun, 6 Jul 2008 23:07:49 +0000 (09:07 +1000)]
new version .44
Ronnie Sahlberg [Sun, 6 Jul 2008 22:53:22 +0000 (08:53 +1000)]
zero out the sockaddr_in structure before we store the ipv4 data in it to make sure that all data is initialized. Othervise valgrind will complain about uninitialized data when we write this structure out on the wire
Ronnie Sahlberg [Sun, 6 Jul 2008 22:52:04 +0000 (08:52 +1000)]
we need a 'case x:' in our ugly 'encode the control opcode as a linenumber in valgrind output' hack to make it work
Ronnie Sahlberg [Sun, 6 Jul 2008 22:51:05 +0000 (08:51 +1000)]
If a transaction commit fails. Log this error and cancel all pending transactions to the
databases instead of calling ctdb_fatal()
Ronnie Sahlberg [Sun, 6 Jul 2008 22:50:12 +0000 (08:50 +1000)]
in the destructor for the lock-wait child, make sure that we cancel any pending
transactions.
Andrew Tridgell [Fri, 4 Jul 2008 08:03:24 +0000 (18:03 +1000)]
fixed a case statement
Andrew Tridgell [Fri, 4 Jul 2008 08:00:24 +0000 (18:00 +1000)]
an extraordinarily ugly patch!
This is a hack to allow backtraces under valgrind to show what opcode
is getting uninitialised bytes
Andrew Tridgell [Fri, 4 Jul 2008 07:40:25 +0000 (17:40 +1000)]
ensure pad bytes in the ltdb_header are initialised
Andrew Tridgell [Fri, 4 Jul 2008 07:32:21 +0000 (17:32 +1000)]
don't use mmap in tdb if --nosetsched is set. That makes valgrind
happier (it doesn't like the mmap/msync calls in tdb)
Andrew Tridgell [Fri, 4 Jul 2008 07:15:06 +0000 (17:15 +1000)]
prevent valgrind errors where we print unitialised values on control errors
Andrew Tridgell [Fri, 4 Jul 2008 07:04:37 +0000 (17:04 +1000)]
fixed a warning
Andrew Tridgell [Fri, 4 Jul 2008 07:04:26 +0000 (17:04 +1000)]
fixed some incorrect CTDB_NO_MEMORY*() calls found after fixing the
_VOID varient
Andrew Tridgell [Fri, 4 Jul 2008 06:58:29 +0000 (16:58 +1000)]
CTDB_NO_MEMORY_VOID() needs to return on error
Andrew Tridgell [Fri, 4 Jul 2008 06:58:14 +0000 (16:58 +1000)]
added option to start ctdb under valgrind
Just add CTDB_VALGRIND=yes in /etc/sysconfig/ctdb, and look at the
logs in /var/log/ctdb_valgrind.*
Andrew Tridgell [Fri, 4 Jul 2008 06:05:04 +0000 (16:05 +1000)]
zero out the ctdb->freeze_handle when we free it
This prevents heap corruption when a freeze child dies
Ronnie Sahlberg [Thu, 3 Jul 2008 02:46:09 +0000 (12:46 +1000)]
we dont need to explicitely thaw the databases from the recovery daemon
since this is already done implicitely when we changed recovery mode
back to normal
Ronnie Sahlberg [Wed, 2 Jul 2008 03:55:59 +0000 (13:55 +1000)]
track both when we last started and ended a recovery.
make ctdb uptime print how long the recovery took
in the recovery daemon when we check that the public ip address
allocation on the local node is correct (we have the ips we should have
and we dont have any we shouldnt have) use ctdb uptime and check the
recovery start/stop times and make sure we dont check for ip allocation
inconsistencies during a recovery where the ip address allocation is in flux.
Ronnie Sahlberg [Wed, 2 Jul 2008 02:21:53 +0000 (12:21 +1000)]
print the opcode when an async callback detects an error
Ronnie Sahlberg [Wed, 2 Jul 2008 02:01:19 +0000 (12:01 +1000)]
update a comment to reflect that this is not always a real recovery
it can also be printed when we just do an ip reallocation
Ronnie Sahlberg [Mon, 30 Jun 2008 23:34:43 +0000 (09:34 +1000)]
new version
Ronnie Sahlberg [Thu, 26 Jun 2008 23:31:18 +0000 (09:31 +1000)]
initdit/ctdb is not a config file
Ronnie Sahlberg [Thu, 26 Jun 2008 23:29:38 +0000 (09:29 +1000)]
make /etc/ctdb/functions executable and add a hashbang to it so
rpmlint wont complain
Ronnie Sahlberg [Thu, 26 Jun 2008 04:14:37 +0000 (14:14 +1000)]
test
Ronnie Sahlberg [Thu, 26 Jun 2008 04:00:36 +0000 (14:00 +1000)]
Ronnie Sahlberg [Thu, 26 Jun 2008 03:51:18 +0000 (13:51 +1000)]
test
Ronnie Sahlberg [Thu, 26 Jun 2008 03:15:41 +0000 (13:15 +1000)]
reduce loglevel of the info message we are updating the flags on all nodes
Ronnie Sahlberg [Thu, 26 Jun 2008 03:08:37 +0000 (13:08 +1000)]
force an update of the flags from the recmaster after each monitoring run
Ronnie Sahlberg [Thu, 26 Jun 2008 02:43:30 +0000 (12:43 +1000)]
/etc/ctdb/functions should not be executable
Ronnie Sahlberg [Thu, 26 Jun 2008 01:52:26 +0000 (11:52 +1000)]
third attempt for fixing a freeze child writing to the socket
Ronnie Sahlberg [Thu, 26 Jun 2008 01:08:09 +0000 (11:08 +1000)]
verify that the recmaster has the correct flags for us and if not tell the recmaster what the flags should be
Ronnie Sahlberg [Thu, 26 Jun 2008 01:02:08 +0000 (11:02 +1000)]
only loop over the write it the write failed
Ronnie Sahlberg [Wed, 25 Jun 2008 23:54:27 +0000 (09:54 +1000)]
the write() from the freeze child process can fail
try writing many times and log an error if the write failed
Ronnie Sahlberg [Fri, 13 Jun 2008 03:53:05 +0000 (13:53 +1000)]
it is 2008 not 2008 right now :-)
Ronnie Sahlberg [Fri, 13 Jun 2008 03:50:28 +0000 (13:50 +1000)]
update to 1.0.42
Ronnie Sahlberg [Fri, 13 Jun 2008 03:45:23 +0000 (13:45 +1000)]
ban the node after 3 failed scripts by default
Ronnie Sahlberg [Fri, 13 Jun 2008 03:18:06 +0000 (13:18 +1000)]
if the event scripts hangs EventScriptsBanCount consecutive times in a row
the node will ban itself for the default recovery ban period
Ronnie Sahlberg [Fri, 13 Jun 2008 02:18:00 +0000 (12:18 +1000)]
when a eventscript has timed out, log the event options (i.e. "monitor" "takeip 1.2..." etc)
to the log
Ronnie Sahlberg [Fri, 13 Jun 2008 01:47:42 +0000 (11:47 +1000)]
make it possible to re-start a recovery without marking the current node as
the culprit.
Ronnie Sahlberg [Thu, 12 Jun 2008 06:53:36 +0000 (16:53 +1000)]
add a callback for failed nodes to the async control helper.
this callback is called for every node where the control failed (or timed out)
when we issue the start recovery control from recovery master,
set any node that fails as a culprit so it will eventually be banned
Ronnie Sahlberg [Wed, 4 Jun 2008 07:12:57 +0000 (17:12 +1000)]
first cut to convert takeover_callback_state{}
to use ctdb_sock_addr instead of sockaddr_in
Ronnie Sahlberg [Wed, 4 Jun 2008 05:23:06 +0000 (15:23 +1000)]
fix a comment
note that we dont actually send the ipv6 "gratious arp" on the wire just yet.
(since ipv6 doesnt use arp)
but all the infrastructure is there when we implement sending raw neig.disc. packets
Ronnie Sahlberg [Wed, 4 Jun 2008 05:13:00 +0000 (15:13 +1000)]
convert handling of gratious arps and their controls and helpers to
use the ctdb_sock_addr structure so tehy work for both ipv4 and ipv6
Ronnie Sahlberg [Wed, 4 Jun 2008 00:46:20 +0000 (10:46 +1000)]
add a parameter for the tdb-flags to the client function
ctdb_attach() so that we can pass TDB_NOSYNC when we attach to
a persistent database and want fast unsafe writes instead of
slow but safe tdb_transaction writes.
enhance the ctdb_persistent test suite to test both safe and unsafe writes
Ronnie Sahlberg [Tue, 3 Jun 2008 08:19:48 +0000 (18:19 +1000)]
run the persistent write test with 4 nodes by default
use the timelimit argument to the persistent writer to run the test for
30 seconds by default
Ronnie Sahlberg [Tue, 3 Jun 2008 08:18:28 +0000 (18:18 +1000)]
redesign the test of persistent writes
so that we have n persistent writers on n nodes,
all writers writing persistently to the same record.
each writer on a node has its own "counter" in this record that is incremented by one in each iteration.
the persistent writer on node 0 also checks that all the counters in the record are increasing monotonically and if they are not, flagging it as an ERROR.
Ronnie Sahlberg [Tue, 3 Jun 2008 08:14:54 +0000 (18:14 +1000)]
create the nodes file in a 'test' subdirectory and not the current directory
delete all persistent databases when the test starts
(the tests only uses test databases in a special test directory)
do not set up any public addresses in the tests
wait until there are no disconnected or unhealthy nodes when starting the
test daemons instead of waiting for the recovery mode to change.
we do want to wait until the system has recovered and ALL nodes are ok.
Ronnie Sahlberg [Wed, 28 May 2008 22:19:35 +0000 (08:19 +1000)]
debugleves can now be negative so print their value using %d instead of %u
Ronnie Sahlberg [Wed, 28 May 2008 04:51:46 +0000 (14:51 +1000)]
update to .41
Ronnie Sahlberg [Wed, 28 May 2008 03:40:12 +0000 (13:40 +1000)]
dont bother casting to a void* private_data pointer,
just pass it as 'state' structure
Ronnie Sahlberg [Wed, 28 May 2008 03:31:58 +0000 (13:31 +1000)]
remove another field we dont need in the childwrite_handle structure
Ronnie Sahlberg [Wed, 28 May 2008 03:30:22 +0000 (13:30 +1000)]
remote a comment that is no longer relevant
remove a field in the childwrite_handle structure we dont need
Ronnie Sahlberg [Wed, 28 May 2008 03:04:25 +0000 (13:04 +1000)]
do persistent writes in a child process
Ronnie Sahlberg [Mon, 26 May 2008 22:23:46 +0000 (08:23 +1000)]
update to .40
Ronnie Sahlberg [Mon, 26 May 2008 22:21:18 +0000 (08:21 +1000)]
read the samba sysconfig from the samba eventscript
Ronnie Sahlberg [Thu, 22 May 2008 08:33:54 +0000 (18:33 +1000)]
disable transactions for now, there are more situations where there are conflicting locks and the "net" command is not prepared that the persistent store can fail.
Ronnie Sahlberg [Thu, 22 May 2008 06:33:36 +0000 (16:33 +1000)]
restore a timeout value to the default settings instead of the hardcoded 3 second test value
Ronnie Sahlberg [Thu, 22 May 2008 06:29:46 +0000 (16:29 +1000)]
fix some memory hierarchy bugs in allocation of the state structure for persistent writes.
since these two controls (UPDATE_RECORD and PERSISTENT_STORE) can respond
asynchronously to the control, we can not allocate the state variable as a child off ctdb_req_control instead we must allocate state as a child off ctdb itself
and steal ctdb_req_control so it becomes a child of state.
othervise both ctdb_req_control and also state will be released immediately after we have finished setting up the async reply and returned.
Ronnie Sahlberg [Thu, 22 May 2008 03:12:53 +0000 (13:12 +1000)]
cleanup of the previous patch.
With these patches, ctdbd will enforce and (by default) always use
tdb_transactions when updating/writing records to a persistent database.
This might come with a small performance degratation since transactions
are slower than no transactions at all.
If a client, such as samba wants to use a persistent database but does NOT
want to pay the performance penalty, it can specify TDB_NOSYNC as the
srvid parameter in the ctdb_control() for CTDB_CONTROL_DB_ATTACH_PERSISTENT.
In this case CTDBD will remember that "this database is not that important"
so I can use unsafe (no transaction) tdb_stores to write the updates.
It will be faster than the default (always use transaction) but less crash safe.
Ronnie Sahlberg [Thu, 22 May 2008 02:47:33 +0000 (12:47 +1000)]
second try for safe transaction stores into persistend tdb databases
for stores into persistent databases, ALWAYS use a lockwait child take out the lock for the record and never the daemon itself.
Ronnie Sahlberg [Wed, 21 May 2008 20:22:25 +0000 (06:22 +1000)]
update version to .39
Ronnie Sahlberg [Wed, 21 May 2008 20:08:38 +0000 (06:08 +1000)]
move CTDB_MANAGES_NFS from /etc/sysconfig/nfs to /etc/sysconfig/ctdb
Ronnie Sahlberg [Wed, 21 May 2008 20:04:36 +0000 (06:04 +1000)]
move the CTDB_MANAGES_ISCSI setting from /etc/sysconfig/iscsi to /etc/sysconfig/ctdb
Ronnie Sahlberg [Wed, 21 May 2008 20:01:17 +0000 (06:01 +1000)]
move the config optoin CTDB_MANAGES_VSFTPD from /etc/sysconfig/vsftpd to /etc/sysconfig/ctdb