tdb: Fix bug 11381, deadlock

This fixes a deadlock in tdb that is a bad interaction between tdb_lockall
and tdb_traverse. This deadlock condition has been around even before
tdb mutexes, it's just that the kernel fcntl EDEADLK detection protected
us from this ABBA lock condition to become a real deadlock stalling
processes. With tdb mutexes, this deadlock protection is gone, so we do
lock dead.

This patch glosses over this particular ABBA condition, making tdb with
mutexes behave the same as tdb without mutexes. Admittedly this is no
real fix, but it works around a real user's problem.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=11381
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

diff --git a/lib/tdb/common/traverse.c b/lib/tdb/common/traverse.c
index 618670fd1cbe392874590e69065e7cee10ccfef8..e18e3c34fd30263562009e7d115111d11096bbba 100644 (file)
--- a/lib/tdb/common/traverse.c
+++ b/lib/tdb/common/traverse.c
@@ -245,13 +245,25 @@ _PUBLIC_ int tdb_traverse(struct tdb_context *tdb,
                 tdb_traverse_func fn, void *private_data)
 {
        struct tdb_traverse_lock tl = { NULL, 0, 0, F_WRLCK };
+       enum tdb_lock_flags lock_flags;
        int ret;
 
        if (tdb->read_only || tdb->traverse_read) {
                return tdb_traverse_read(tdb, fn, private_data);
        }
 
-       if (tdb_transaction_lock(tdb, F_WRLCK, TDB_LOCK_WAIT)) {
+       lock_flags = TDB_LOCK_WAIT;
+
+       if (tdb->allrecord_lock.count != 0) {
+               /*
+                * This avoids a deadlock between tdb_lockall() and
+                * tdb_traverse(). See
+                * https://bugzilla.samba.org/show_bug.cgi?id=11381
+                */
+               lock_flags = TDB_LOCK_NOWAIT;
+       }
+
+       if (tdb_transaction_lock(tdb, F_WRLCK, lock_flags)) {
                return -1;
        }