Amitay Isaacs [Wed, 6 Feb 2013 03:33:51 +0000 (14:33 +1100)]
New version 1.2.57
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Tue, 5 Feb 2013 05:40:39 +0000 (16:40 +1100)]
initscript: export CTDB_DEBUG_LOCKS
Signed-off-by: Martin Schwenke <martin@meltin.net>
Martin Schwenke [Tue, 5 Feb 2013 02:16:46 +0000 (13:16 +1100)]
initscript: export CTDB_EXTERNAL_TRACE
This means it can be set like any other configuration option in the
configuration file, without needing to export it there.
Cherry-pick-from:
a0ef73e197dc9147f7718e0813fe803ff0b3d54d
Signed-off-by: Martin Schwenke <martin@meltin.net>
Martin Schwenke [Thu, 17 May 2012 00:17:51 +0000 (10:17 +1000)]
ctdbd: Backport use of external script to debug hung eventscript
This is a cherry-pick from
6e68797af67bee36f2bad045f94806e7e98f27e9,
combined with several recent fixes:
8507303b525d20c74e8ec4e7c4f5f275945cd3b6
scripts: debug-hung-script.sh doesn't need functions/loadconfig
501461cc3e132d4adee9e91b5d4513a26bae2846
ctdbd: Remove debug_hung_script_ctx
0581f9a84e58764d194f4e04064c2c5b393c348b
ctdbd: Remove command-line option --debug-hung-script
3400b2ed34b6eb9496eb55f1aab6f89d2952060d
ctdbd: Complain loudly if CTDB_DEBUG_HUNG_SCRIPT script isn't executable
9b0d56b16775aa16f33bdfdf831256e085fa3339
ctdbd: Don't use a fixed length buffer for the hung script command
Signed-off-by: Martin Schwenke <martin@meltin.net>
Amitay Isaacs [Tue, 5 Feb 2013 01:59:53 +0000 (12:59 +1100)]
doc: Rebuild all documentation
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Ronnie Sahlberg [Tue, 20 Mar 2012 00:38:20 +0000 (11:38 +1100)]
DOC document the FetchCollapse tunable
Cherry-pick-from:
c37aa6f3738693653f64c2fa015ace061da38b5a
Ronnie Sahlberg [Tue, 20 Mar 2012 00:31:59 +0000 (11:31 +1100)]
FETCH COLLAPSE : Change the fetch-lock collapse to collapse ALL fetches, including fetch-locks into a single command in flight per record. Also add a tunable to enable/disable this optimization for hot records
Conflicts:
server/ctdb_tunables.c
Cherry-pick-from:
eafd7bbaaa5931546a96c8beae3cf9a39a49c925
Ronnie Sahlberg [Mon, 7 Nov 2011 19:55:46 +0000 (06:55 +1100)]
Record Fetch Collapse: Collapse multiple fetch request into one single request.
When multiple clients fetch the same record concurrently, send only one single
fetch across the network and deferr all other fetches locally.
This improves performance for hot records and reduces cpu load on ctdb.
Conflicts:
server/ctdb_ltdb_server.c
Cherry-pick-from:
82d6946ad8b3348e8b9d3d971f24925ade02d1be
Amitay Isaacs [Wed, 9 Jan 2013 00:03:18 +0000 (11:03 +1100)]
scripts: Fix the variable name for sed expressions
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Amitay Isaacs [Wed, 2 Jan 2013 05:17:58 +0000 (16:17 +1100)]
New version 1.2.56
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Amitay Isaacs [Wed, 2 Jan 2013 04:49:39 +0000 (15:49 +1100)]
daemon: Change the default recovery method for persistent databases
Use sequence numbers to do the recovery for persistent databases instead
of RSNs. This fixes the problem of registry corrpution during recovery.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Ronnie Sahlberg [Tue, 29 Nov 2011 21:59:03 +0000 (08:59 +1100)]
DOC: describe the RecoverPDBBySeqNum tunable
Conflicts:
doc/ctdbd.1
doc/ctdbd.1.html
Cherry-pick-from:
86d956170d4806065f1470fc44710c085c57f17a
Ronnie Sahlberg [Mon, 28 Nov 2011 02:56:30 +0000 (13:56 +1100)]
Recover Persistent database DB by DB and not record by record
Add a new tunable that changes the mode how persistent databases are recovered.
RecoveryPDBBySeqNum
When set to 1, persistent databases will be recovered in whole from the node which
has the highest "__db_sequence_number__" record.
This record is managed by samba for those databases where we do persistent writes and have
inter-record relations.
For these databases we do not want the usual "blend records from all nodes based
on individual record RSN" but instead a mode where we pick one instance of the persistent database.
If no node was found with a "__db_sequence_number__" record at all, we fail back to the original "recover records independently based on record RSN".
Some persistent databases do not contain record interrelations and as such does not
contain this special record at all.
Conflicts:
include/ctdb_private.h
server/ctdb_tunables.c
Cherry-pick-from:
502150c764298a9fa8c4d8aa445bf7d85d4ee9dc
Ronnie Sahlberg [Mon, 28 Nov 2011 05:30:46 +0000 (16:30 +1100)]
LibCTDB: add get persistent db seqnum control
Conflicts:
tools/ctdb.c
Cherry-pick-from:
6e96a62494bbb2c7b0682ebf0c2115dd2f44f7af
Ronnie Sahlberg [Sun, 27 Nov 2011 23:41:17 +0000 (10:41 +1100)]
DB Seqnum: must provide a ctdb_ltdb_header when calling ctdb_ltdb_fetch()
Cherry-pick-from:
1fea9ef55a6a9d201ad1b49583451ac3e6b1c66d
Amitay Isaacs [Wed, 5 Dec 2012 00:38:42 +0000 (11:38 +1100)]
scripts: Add helper script to log locking information using /proc/locks
This finds any processes locking tdb databases used by CTDB and logs
stack trace for each process.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Amitay Isaacs [Wed, 5 Dec 2012 00:37:26 +0000 (11:37 +1100)]
daemon: Run an external script if freeze locks were not obtained during recovery
If the freeze child is already created in ctdb_start_freeze(), then it indicates
that the child process has not yet obtained the locks. This may be because
another process has locked the databases and has not yet released the locks.
In this case, invoke a helper script defined by environmental variable
CTDB_DEBUG_LOCKS, to log information about locks.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Amitay Isaacs [Tue, 27 Nov 2012 04:50:54 +0000 (15:50 +1100)]
New version 1.2.55
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Amitay Isaacs [Thu, 22 Nov 2012 03:37:45 +0000 (14:37 +1100)]
Revert "when creating/adding a public ip, set the initial interface to be the first interface specified"
This reverts commit
4308935ba48ac7a29e7523315acf580019715f0f.
When IP is added to a node on a new interface for the first time,
vnn->iface gets set to the first interface defined for that IP. This
actually causes problem in ctdb_vnn_assign_iface(). Since vnn->iface
is set it takes an early exit without updating vnn->pnn. This results
in IP being hosted on the node, but CTDB still thinks it's unassigned.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Fri, 16 Nov 2012 09:21:15 +0000 (20:21 +1100)]
Eventscripts: 10.interface should list configured interfaces
The current code lists available interfaces. If IPs are configured in
some other way than the public addresses file (e.g. ctdb addip) and their
interfaces default to being marked down then, since down interfaces are
not available, these interfaces can never be marked up.
The configured interfaces should be listed instead.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Cherry-pick-from:
d8f010355b715e49709836e057a5d0f110919897
Conflicts:
config/events.d/10.interface
Martin Schwenke [Fri, 16 Nov 2012 08:43:14 +0000 (19:43 +1100)]
ctdbd: Make the link status of new interfaces more flexible
Neither up nor down is a good default value for the link status of a
new interface. Up means that IPs can be assigned to interfaces before
the true state is known and they can move away quickly if the interface
is actually down. Down means that IPs can't be assigned to an interface
for a variable amount of time - until a monitor cycle occurs - and this
can result in imbalanced IPs.
This is a neat compromise. Before the startup event completes, IPs
can't be assigned to interfaces because all interfaces begin in a down
state. As soon as the startup event completes, IPs can be allocated
to any interface that has been marked up by the eventscript. Later,
during normal operation, newly added IPs can be assigned to new
interfaces immediately. The IPs will still move away if an interface
is noticed to be down in the next monitor cycle, but that is the
exception rather than the rule.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Cherry-pick-from:
9275a69a414482f1053ae14528d5972575b9214e
Amitay Isaacs [Tue, 6 Nov 2012 06:06:54 +0000 (17:06 +1100)]
tools/ctdb: Do not use function return value as pnn
This fixes the wrong code where same variable 'ret' is used to track the pnn
and the return value of a function call.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Cherry-pick-from:
718233c445cd6627ab3962b6565c2655f1f8efd0
Amitay Isaacs [Tue, 23 Oct 2012 05:23:12 +0000 (16:23 +1100)]
recoverd: Track the nodes that fail takeover run and set culprit count
If any of the nodes fail takeover run (either due to timeout or failure
to complete within takeover_timeout interval) from main loop, recovery
master will give up trying takeover run with following message:
"Unable to setup public takeover addresses. Try again later"
And as a side-effect the monitoring is disabled on all the nodes. Before
ctdb_takeover_run() is called from main loop, monitoring get disabled via
startrecovery event. Since ctdb_takeover_run() fails, it never runs
recovered event and monitoring does not get re-enabled.
In main_loop, ctdb_takeover_run() is called with a takeover_fail_callback.
This callback will get called if any of the nodes fail in handling
takeip/releaseip/ipreallocated events in ctdb_takeover_run().
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Cherry-pick-from:
a5c6bb1fffb8dc3960af113957a1fd080cc7c245
Conflicts:
include/ctdb_private.h
server/ctdb_takeover.c
Amitay Isaacs [Tue, 23 Oct 2012 04:06:33 +0000 (15:06 +1100)]
daemon: Do not ignore timed out monitor events
If an eventscript times out for monitor event, it is considered successful
and the remaining eventscripts are not run. This can make a node prematurely
healthy, cause healthy node to fail over IPs to this node and this node will
not be able to host those IPs. Thus causing loss of access and in case of NAT-GW
configuration, loss of a default route.
Copy-code-from:
6e68797af67bee36f2bad045f94806e7e98f27e9
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Amitay Isaacs [Tue, 30 Oct 2012 01:39:00 +0000 (12:39 +1100)]
New Version 1.2.54
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Amitay Isaacs [Mon, 3 Sep 2012 02:39:36 +0000 (12:39 +1000)]
scripts: Remove duplicate code from init script to set tunables
The tunable variables defined in CTDB configuration file are currently
set up from init script as well as part of "setup" event in 00.ctdb
eventscript. Remove the duplication of this code and set tunable
variables only from setup event. During the "setup" event, it's possible
that ctdb tool commands can timeout if CTDB daemon is not ready. To guard
against such eventuality, wait till "ctdb ping" command succeeds before
executing any other ctdb tool commands.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Cherry-picked-from:
632c1b9c1cc2e242376358ce49fd2022b3f27aa2
Conflicts:
config/events.d/00.ctdb
Amitay Isaacs [Mon, 29 Oct 2012 03:56:10 +0000 (14:56 +1100)]
daemon: Protect against double free of callback state while shutting down
When CTDB is shut down and monitoring has been stopped, monitor_context
gets freed and all the callback states hanging off it. This includes
callback state for current_monitor, if the current monitor event has
not yet finished. As a result, when the shutdown event is called,
current_monitor->callback state is not NULL, but it's actually freed
and it's a dangling reference.
So before executing callback function and freeing callback state check
if ctdb->monitor->monitor_context is not NULL.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Amitay Isaacs [Fri, 26 Oct 2012 05:19:35 +0000 (16:19 +1100)]
New version 1.2.53
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Wed, 28 Mar 2012 03:50:36 +0000 (14:50 +1100)]
Initscript - add backup of corrupt non-persistent databases
Corrupt non-persistent databases never get analysed because ctdbd
zeroes them at startup.
Modify the initscript so that corrupt non-persistent databases are
moved aside to a backup. If the number of backups for a particular
database exceeds $CTDB_MAX_CORRUPT_DB_BACKUPS (default 10) then the
oldest excess backups are garbage collected.
Abstracts from and cleans up the code for checking persistent
databases.
Logging of related messages is done to syslog or a log file as
specified.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Cherry-picked-from:
00cd75595685dae829758abf1a4cb644af7ed50e
Conflicts:
config/ctdb.init
Martin Schwenke [Fri, 5 Oct 2012 02:05:19 +0000 (12:05 +1000)]
New version 1.2.52
Signed-off-by: Martin Schwenke <martin@meltin.net>
Martin Schwenke [Tue, 2 Oct 2012 01:51:24 +0000 (11:51 +1000)]
util: ctdb_fork() closes all sockets opened by the main daemon
Do some other housekeeping including stopping tevent.
Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>
Signed-off-by: Martin Schwenke <martin@meltin.net>
Martin Schwenke [Tue, 2 Oct 2012 01:54:00 +0000 (11:54 +1000)]
Revert "logging: Close unix socket /tmp/ctdb.socket in syslogd process"
This reverts commit
450bedccbee3f89aba3b33777a4ae8841c456a65.
This will be fixed in ctdb_fork() for all children. Won't somebody
PLEASE think of the children?!?
Amitay Isaacs [Tue, 2 Oct 2012 02:45:10 +0000 (12:45 +1000)]
New version 1.2.51
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Thu, 27 Sep 2012 23:39:12 +0000 (09:39 +1000)]
Logging: Map TEVENT_DEBUG_FATAL to DEBUG_CRIT
This is currently mapped to DEBUG_EMERG. CTDB really has no business
logging anything at EMERG level since the whole system is not about to
abort or catch fire. EMERG causes the message to appear on the
console and on every terminal. That's a bit overzealous!
There would be very few situations where logs are being filtered at
level below ERROR, so CRIT should certainly suffice.
The trigger for this was curious messages saying "No event for <n>
seconds!" logged in a user's terminal.
Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>
Signed-off-by: Martin Schwenke <martin@meltin.net>
Martin Schwenke [Wed, 26 Sep 2012 04:37:49 +0000 (14:37 +1000)]
Eventscripts: "recovered" event should not fail on NATGW failure
The recovery process has no protection against the "recovered" event
failing, so this can cause a recovery loop.
Instead of failing the "recovered" event, add a "monitor" event and
fail that instead. In this case the failure semantics are well
defined.
A separate patch should ban nodes if the "recovered" event fails for
an unknown reason.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Amitay Isaacs [Wed, 12 Sep 2012 05:02:30 +0000 (15:02 +1000)]
New version 1.2.50
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Thu, 6 Sep 2012 10:22:38 +0000 (20:22 +1000)]
common: Debug ctdb_addr_to_str() using new function ctdb_external_trace()
We've seen this function report "Unknown family, 0" and then CTDB
disappeared without a trace. If we can reproduce it then this might
help us to debug it.
The idea is that you do something like the following in /etc/sysconfig/ctdb:
export CTDB_EXTERNAL_TRACE="/etc/ctdb/config/gcore_trace.sh"
When we hit this error than we call out to gcore to get a core file so
we can do forensics. This might block CTDB for a few seconds.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Martin Schwenke [Tue, 21 Aug 2012 04:35:35 +0000 (14:35 +1000)]
New version 1.2.49
Signed-off-by: Martin Schwenke <martin@meltin.net>
Martin Schwenke [Tue, 21 Aug 2012 04:28:37 +0000 (14:28 +1000)]
Initscript: Kill any existing ctdbd processes if the ping succeeds
Initialising a new ctdbd will destroy the Unix domain socket so
existing processes will be useless anyway.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Amitay Isaacs [Tue, 14 Aug 2012 05:42:12 +0000 (15:42 +1000)]
logging: Close unix socket /tmp/ctdb.socket in syslogd process
Since the unix socket is opened before syslogd process is forked, syslogd
process also keeps listening to it. If main ctdbd process dies and has any
child processes that are blocked waiting for locks, these child processes
keep connecting to unix socket and thus syslogd cannot exit.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Thu, 9 Aug 2012 09:47:08 +0000 (19:47 +1000)]
New version 1.2.48
Signed-off-by: Martin Schwenke <martin@meltin.net>
Martin Schwenke [Tue, 28 Jun 2011 04:54:33 +0000 (14:54 +1000)]
Eventscripts: new functions set_proc() and get_proc().
These provide a thin layer around writing and reading files in /proc.
They can be easily replaced by stubs for unit testing.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Amitay Isaacs [Thu, 9 Aug 2012 06:57:15 +0000 (16:57 +1000)]
New version 1.2.47
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Fri, 3 Aug 2012 00:54:30 +0000 (10:54 +1000)]
Eventscripts: Add "reconfigure" pseudo-event for policy routing
This rebuilds all policy routes and can be used if the configuration
changes.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Martin Schwenke [Tue, 24 Jul 2012 01:26:32 +0000 (11:26 +1000)]
New version 1.2.46
Signed-off-by: Martin Schwenke <martin@meltin.net>
Martin Schwenke [Fri, 20 Jul 2012 06:43:39 +0000 (16:43 +1000)]
Eventscripts: Default route on NAT gateway should have a metric of 10
At the moment routes from 11.routing can fail to be added because they
conflict with the default route added by 11.natgw.
NAT gateway is meant to be a last resort, so routes from 11.routing
should override it.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Martin Schwenke [Thu, 12 Jul 2012 04:03:58 +0000 (14:03 +1000)]
New version 1.2.45
Signed-off-by: Martin Schwenke <martin@meltin.net>
Ronnie Sahlberg [Wed, 20 Jun 2012 05:10:05 +0000 (15:10 +1000)]
When we find an ip we shouldnt host, just release it
Dont call a full blown clusterwide ipreallocation, just release it locally
Ronnie Sahlberg [Wed, 20 Jun 2012 00:08:11 +0000 (10:08 +1000)]
When we release an ip, get the interface name from the kernel
instead of using the interface where ctdb thinks the ip is hosted at.
The difference is that this now allows us to handle cases where we want to release an ip but ctdbd does not know which interface the ip is assigned on.
(user has used 'ip addr add...' and manually assigned an ip to the wrong interface)
Ronnie Sahlberg [Wed, 20 Jun 2012 03:32:02 +0000 (13:32 +1000)]
Add new command to find which interface is located on
Ronnie Sahlberg [Fri, 29 Jun 2012 02:31:13 +0000 (12:31 +1000)]
New version 1.2.44
Martin Schwenke [Thu, 21 Jun 2012 04:18:35 +0000 (14:18 +1000)]
eventscripts: 13.per_ip_routing - flock should have a timeout
... and flock failure should be fatal.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Martin Schwenke [Thu, 21 Jun 2012 04:17:06 +0000 (14:17 +1000)]
eventscripts: 13.per_ip_routing - readability improvement
Signed-off-by: Martin Schwenke <martin@meltin.net>
Martin Schwenke [Fri, 15 Jun 2012 07:22:02 +0000 (17:22 +1000)]
eventscripts: 13.per_ip_routing should remove bogus routes on ipreallocated
Signed-off-by: Martin Schwenke <martin@meltin.net>
Martin Schwenke [Wed, 13 Jun 2012 03:49:49 +0000 (13:49 +1000)]
eventscripts: Print a warning on failure to delete a routing rule
del_routing_for_ip() currently fails silently, which could hide real
errors.
In add_routing_for_ip() we don't want to see any error when calling
del_routing_for_ip(), since we don't expect the rule to be there.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Martin Schwenke [Tue, 24 Apr 2012 07:17:07 +0000 (17:17 +1000)]
Eventscripts: fix basename -> dirname typo
I fixed one of these previously but didn't notice this one... :-(
Signed-off-by: Martin Schwenke <martin@meltin.net>
Martin Schwenke [Wed, 18 Apr 2012 06:49:48 +0000 (16:49 +1000)]
Eventscripts - Fix typo in 13.per_ip_routing support for __auto_link_local__
Signed-off-by: Martin Schwenke <martin@meltin.net>
Martin Schwenke [Tue, 20 Mar 2012 05:23:24 +0000 (16:23 +1100)]
Eventscripts - make 13.per_ip_routing fail gracefully if config is missing
Currently it spews out random messages about the file being missing.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Martin Schwenke [Tue, 20 Mar 2012 05:21:09 +0000 (16:21 +1100)]
Eventscripts - make 13.per_ip_routing try harder to find public_addresses
Signed-off-by: Martin Schwenke <martin@meltin.net>
Martin Schwenke [Tue, 20 Mar 2012 05:20:10 +0000 (16:20 +1100)]
Eventscripts - use set_proc() rather than accessing /proc directly
Signed-off-by: Martin Schwenke <martin@meltin.net>
Martin Schwenke [Tue, 20 Mar 2012 05:18:49 +0000 (16:18 +1100)]
Eventscripts - 13.per_ip_routing should use dirname not basename for mkdir
Signed-off-by: Martin Schwenke <martin@meltin.net>
Ronnie Sahlberg [Wed, 28 Mar 2012 23:10:04 +0000 (10:10 +1100)]
New version 1.2.43
Volker Lendecke [Mon, 31 Oct 2011 12:29:13 +0000 (13:29 +0100)]
Add CTDB_CONTROL_CHECK_SRVID
Ronnie Sahlberg [Mon, 12 Mar 2012 02:05:07 +0000 (13:05 +1100)]
new version 1.2.42
Ronnie Sahlberg [Mon, 12 Mar 2012 02:03:19 +0000 (13:03 +1100)]
ReadOnly: Only restrict the readonly sanity/paranoia check that only allows
UPDATE_RECORD to create a record to only apply to normal databases, not
persistent databases or else the net registry command breaks
and other persistent database creation/updates .
S1035666
Ronnie Sahlberg [Tue, 6 Mar 2012 05:03:18 +0000 (16:03 +1100)]
new version 1.2.41
Martin Schwenke [Tue, 23 Aug 2011 06:32:34 +0000 (16:32 +1000)]
Eventscripts - new function ctdb_check_args()
Pass this "$@" to do common eventscript argument checking.
For regular use putting this in 00.ctdb would be enough. However, for
developer testing it can be useful to call this in other eventscripts.
For example, 10.interfaces and 13.per_ip_routing currently check these
by hand.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Martin Schwenke [Tue, 6 Mar 2012 00:21:41 +0000 (11:21 +1100)]
Eventscript functions - remove functions only used by 13.per_ip_routing
The relevant functions are now in that script.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Martin Schwenke [Thu, 1 Mar 2012 04:23:53 +0000 (15:23 +1100)]
Eventscripts - redesign and rewrite 13.per_ip_routing
The current version is quite difficult to read. This one is hopefully
clearer.
Major changes:
* The configuration file has a more forgiving syntax. Items can be
separated by arbitrary whitespace.
* Mappings between IP addresses and table IDs are no longer stored in
files in a state directory. Instead they are stored in
/etc/iproute2/rt_tables as mappings between table IDs and labels, as
allowed by the ip command. The current structure of the labels is
ctdb.<source-ip>. This means that once the labels are setup the
routing tables can be referenced by just knowing the source IP. As
with the old state directory, mappings in this file owned by CTDB
are deleted when CTDB shuts down.
* There are no release or re-add scripts.
- Release scripts are not necessary as an optimisation because of
the previous improvement (i.e. use of rt_tables). No lookup is
necessary to delete rules or flush tables.
- Re-add scripts are no longer used. Routes can still go missing
when removal of a primary IP from an interfaces (or similar)
causes removal of all other addresses (i.e. secondaries) and also
all associated routes. However, any missing routes are now
re-added in the "ipreallocated" event. This happens shortly after
takeip/releaseip/updateip and means that the routes will only be
re-added once. The window for missing routes is slightly bigger
but is not expected to be significant.
* The magic "__auto_link_local__" configuration value no longer causes
a dynamic configuration file to be maintained in a state directory.
The link local configuration is now generated when needed from the
public_addresses file. This greatly simplifies the code. This
approach is slightly less efficient but should not be significant.
The above changes mean that, apart from maintaining mappings in the
rt_tables file, there are no state files kept anymore.
Some utility functions only used by this script have been rewritten
and moved into this script. They will be removed from the functions
file by a future commit.
The route re-add code will also be removed from interface_modify.sh by
a future commit. It is currently harmless.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Ronnie Sahlberg [Mon, 5 Mar 2012 23:51:00 +0000 (10:51 +1100)]
readonly, dont allow any kind of fast vacuum for ro delegated records
Ronnie Sahlberg [Fri, 2 Mar 2012 03:48:15 +0000 (14:48 +1100)]
Revert "READONLY: readonly fetch collapse. Make sure we only keep one single readonly fetch for a record in flight at a time."
This reverts commit
9bde066f6eb46124168e5686fc41a323e67401e8.
Ronnie Sahlberg [Wed, 8 Feb 2012 02:42:30 +0000 (13:42 +1100)]
STATISTICS: add total counts for number of delegations and number of revokes
Everytime we give a delegation to another node we count this as one delegation.
If the same record is delegated to several nodes we count one for each node.
Everytime a record has all its delegations revoked we count this as one revoke.
Ronnie Sahlberg [Fri, 2 Mar 2012 03:12:37 +0000 (14:12 +1100)]
READONLY: readonly fetch collapse. Make sure we only keep one single readonly fetch for a record in flight at a time.
Ronnie Sahlberg [Thu, 1 Mar 2012 23:52:00 +0000 (10:52 +1100)]
READONLY: only send a control to schedule fast-vacuuming from child context iff we have a connection open to the main daemon
there are some child processes where we do not create a connection to the main daemon (switch_from_server_to_client()) because it is expensive to set up and we normally might not need to talk to the daemon at all via a domainsocket.
but we might want to still call to ctdb_ltdb_store() from such chil processes.
Ronnie Sahlberg [Thu, 1 Mar 2012 22:43:39 +0000 (09:43 +1100)]
READONLY: when updating a remote node to revoke a delegation, make sure
we dont create thje record if it doesnt exist
Ronnie Sahlberg [Sun, 12 Feb 2012 23:27:59 +0000 (10:27 +1100)]
READONLY: allow specifying the db name for setdbreadonly instead of just the hash
Ronnie Sahlberg [Thu, 1 Mar 2012 22:04:39 +0000 (09:04 +1100)]
Niceify the readonlyrecord API. Dont force clients to be exposed to the fetch_with_header function
We dont strictly need to force clients to use CTDB_FETCH_WITH_HEADER instead of CTDB_FETCH when they ask for readonly records.
Have ctdbd internally remap this internally to FETCH_WITH_HEADER and map the reply back to CTDB_FETCH_FUNC or CTDB_FETCH_WITH_HEADER_FUNC based on what the client initially asked for.
This removes the need for the client to know about the CTDB_FETCH_WITH_HEADER_FUNC function and simplifies the client code.
Clients that do not care what the header after the request is can just continue using the old CTDB_FETCH_FUNC call and ctdbd will do all the difficult stuff.
Ronnie Sahlberg [Thu, 1 Mar 2012 21:53:16 +0000 (08:53 +1100)]
READONLY: skip vacuuming or deleting records with readonly delegations.
these records are hot. wait until they have been revoked before we recall them.
Ronnie Sahlberg [Wed, 29 Feb 2012 01:25:41 +0000 (12:25 +1100)]
Vacuuming: change default timeout to 120 seconds
S1035431
Ronnie Sahlberg [Mon, 27 Feb 2012 19:44:08 +0000 (06:44 +1100)]
Add a tunable variable to control how long we defer after a ctdb addip until we force a rebalance and try to failback addresses onto this node
Ronnie Sahlberg [Mon, 27 Feb 2012 08:16:08 +0000 (19:16 +1100)]
When adding ips to nodes, set up a deferred rebalance for the whole node to trigger after 60 seconds in case the normal ipreallocated is not sufficient to trigger rebalance.
S1035306
Ronnie Sahlberg [Sun, 26 Feb 2012 20:18:19 +0000 (07:18 +1100)]
Make KILLTCP structure a child of VNN so that it is freed at the same time
the referenced VNN structure is.
Also, remove the circular reference between the two objects KIPPCTP and VNN
Ronnie Sahlberg [Wed, 22 Feb 2012 06:43:04 +0000 (17:43 +1100)]
make tdb valgrind-happy
Ronnie Sahlberg [Wed, 22 Feb 2012 06:38:12 +0000 (17:38 +1100)]
Eventscripts: remove the horrible horrible circular reference between state and callback since these two structures do not even share the same parent talloc context.
Instead, tie them together via referencing a permanent linked list hung off the ctdb structure.
Ronnie Sahlberg [Mon, 20 Feb 2012 01:12:56 +0000 (12:12 +1100)]
ReadOnly: Do not force all normal database to READONLY.
Leave ro disabled by default and activate later by explicit calls to ctdb setdbreadonly
Ronnie Sahlberg [Sun, 5 Feb 2012 22:22:37 +0000 (09:22 +1100)]
New version 1.2.40
Ronnie Sahlberg [Tue, 31 Jan 2012 22:32:02 +0000 (09:32 +1100)]
ReadOnly: update the loop test tool to print number of fetches per second
Ronnie Sahlberg [Fri, 28 Oct 2011 02:38:32 +0000 (13:38 +1100)]
libctdb: dont allow ctdb_writerecord() for readonly records
Ronnie Sahlberg [Fri, 28 Oct 2011 01:41:27 +0000 (12:41 +1100)]
ReadOnly: If record does not exist, upgrade to write-lock
If we attempt a readonly lock request for a record that do not exist (yet)
in the local TDB, then upgrade the request to ask for a write lock and force a
request for migrate the record onto the local node.
This allows the "only request record on second local request for known contended records"
heuristics to try to avoid creating readonly delegations unless we have good reason to
assume it is a contended record.
Ronnie Sahlberg [Tue, 31 Jan 2012 23:26:41 +0000 (10:26 +1100)]
ReadOnly: add readonly record lock requests to libctdb
Initial readonly record support in libctdb.
New records are not yet created by the library but extising records will be
This needs a bit more tests before we can drop the "old style" implementatio
code in client/ctdb_client.c
Ronnie Sahlberg [Fri, 28 Oct 2011 00:44:19 +0000 (11:44 +1100)]
ReadOnly: fix bug writing incorrect amount of data in delegated record
Fix bug when ctdbd updates the local copy of a delegated record to write the correct
amount of data to the record.
Ronnie Sahlberg [Mon, 24 Oct 2011 02:19:30 +0000 (13:19 +1100)]
ReadOnly DOCS: update the docs for readonly delegations to remove the passage that records are written/updated by the client
Ronnie Sahlberg [Mon, 24 Oct 2011 02:14:26 +0000 (13:14 +1100)]
ReadOnly: Dont update the record header from the calling client. While it is convenient since it avoids having to create a child process from the main dameon for writing the updated record it makes the cleitn more complex.
Remove the code in the example client code that writes the record to the local tdb.
Add code to the local ctdbd processing of replies to check if this reply contain a ro delegation and if so, spawn a child process to lock the tdb and then write the data.
Ronnie Sahlberg [Tue, 13 Sep 2011 08:47:18 +0000 (18:47 +1000)]
ReadOnly: revokechild_active is a list, not a context.
Dont reset the pointer to NULL after deleting the first entry, loop deleting one entry
at a time until they are all gone or we will leak some memory and possibly a process.
Ronnie Sahlberg [Tue, 13 Sep 2011 08:41:34 +0000 (18:41 +1000)]
fix some compiler warnings for the test tools
Ronnie Sahlberg [Tue, 13 Sep 2011 08:38:20 +0000 (18:38 +1000)]
ReadOnly: Rename the function ctdb_ltdb_fetch_readonly() to ctdb_ltdb_fetch_with_header() since this is what it actually does.
Ronnie Sahlberg [Thu, 1 Sep 2011 01:40:51 +0000 (11:40 +1000)]
ReadOnly: update the documentation about readonly locks
Ronnie Sahlberg [Thu, 1 Sep 2011 01:08:18 +0000 (11:08 +1000)]
ReadOnly: add a new control to activate readonly lock capability for a database.
let all databases default to not support this until enabled through this control
Ronnie Sahlberg [Thu, 1 Sep 2011 00:28:15 +0000 (10:28 +1000)]
ReadOnly: add a readonly flag to the getdbmap control and show the readonly setting in ctdb getdbmap output
Ronnie Sahlberg [Thu, 1 Sep 2011 00:21:55 +0000 (10:21 +1000)]
ReadOnly: Change the ctdb_db structure to keep a uint8_t for flags instead of a boolean for
the persistent flag.
This is the same size as the original boolean but allows ut to add additional flags for the database