amitay/samba.git
10 days agoctdb-recovery: Refactor banning a node into separate computation master
Amitay Isaacs [Mon, 2 Mar 2020 05:16:26 +0000 (16:16 +1100)]
ctdb-recovery: Refactor banning a node into separate computation

If a node is marked for banning, confirm that it's not become inactive
during the recovery.  If yes, then don't ban the node.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
10 days agoctdb-recovery: Don't trust nodemap obtained from local node
Amitay Isaacs [Tue, 18 Feb 2020 05:17:00 +0000 (16:17 +1100)]
ctdb-recovery: Don't trust nodemap obtained from local node

It's possible to have a node stopped, but recovery master not yet
updated flags on the local ctdb daemon when recovery is started.  So do
not trust the list of active nodes obtained from the local node.  Query
the connected nodes to calculate the list of active nodes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
10 days agoctdb-recovery: Consolidate node state
Amitay Isaacs [Mon, 2 Mar 2020 04:07:21 +0000 (15:07 +1100)]
ctdb-recovery: Consolidate node state

This avoids passing multiple arguments to async computation.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
10 days agoctdb-recovery: Fetched vnnmap is never used, so don't fetch it
Amitay Isaacs [Mon, 2 Mar 2020 02:59:42 +0000 (13:59 +1100)]
ctdb-recovery: Fetched vnnmap is never used, so don't fetch it

New vnnmap is constructed using the information from all the connected
nodes.  So there is no need to fetch the vnnmap from recovery master.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
10 days agosmbd: enforce AIO requests draining
Ralph Boehme [Tue, 17 Mar 2020 10:17:46 +0000 (11:17 +0100)]
smbd: enforce AIO requests draining

Now we wait for all aio to finish on all SHUTDOWN_CLOSE
cases, this is no longer needed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Mar 18 19:40:00 UTC 2020 on sn-devel-184

10 days agoRevert "s3: VFS: vfs_default: Add tevent_req pointer to state struct in vfswrap_pread...
Jeremy Allison [Fri, 13 Mar 2020 17:40:50 +0000 (10:40 -0700)]
Revert "s3: VFS: vfs_default: Add tevent_req pointer to state struct in vfswrap_pread_state."

This reverts commit 594a435b33e8447625ca83b50daec2d08cf66d64.

Now we wait for all aio to finish on all SHUTDOWN_CLOSE
cases, this is no longer needed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agoRevert "s3: VFS: vfs_default. Pass in struct vfswrap_pread_state as the callback...
Jeremy Allison [Fri, 13 Mar 2020 17:40:33 +0000 (10:40 -0700)]
Revert "s3: VFS: vfs_default. Pass in struct vfswrap_pread_state as the callback data to the subreq."

This reverts commit e102908f112866d657b8c0cd6a5b217d070210c8.

Now we wait for all aio to finish on all SHUTDOWN_CLOSE
cases, this is no longer needed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agoRevert "s3: VFS: vfs_default. Protect vfs_pread_done() from accessing a freed req...
Jeremy Allison [Fri, 13 Mar 2020 17:40:19 +0000 (10:40 -0700)]
Revert "s3: VFS: vfs_default. Protect vfs_pread_done() from accessing a freed req pointer."

This reverts commit b9ad06079fe362385cc4c77f8e8d54f5f74d6db6.

Now we wait for all aio to finish on all SHUTDOWN_CLOSE
cases, this is no longer needed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agoRevert "s3: VFS: vfs_default: Add tevent_req pointer to state struct in vfswrap_pwrit...
Jeremy Allison [Fri, 13 Mar 2020 17:40:02 +0000 (10:40 -0700)]
Revert "s3: VFS: vfs_default: Add tevent_req pointer to state struct in vfswrap_pwrite_state."

This reverts commit 86cc7439501ab9b9eb018a18dbbef9567eb9b6f9.

Now we wait for all aio to finish on all SHUTDOWN_CLOSE
cases, this is no longer needed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agoRevert "s3: VFS: vfs_default. Pass in struct vfswrap_pwrite_state as the callback...
Jeremy Allison [Fri, 13 Mar 2020 17:39:46 +0000 (10:39 -0700)]
Revert "s3: VFS: vfs_default. Pass in struct vfswrap_pwrite_state as the callback data to the subreq."

This reverts commit 13e25d68385aa951115e0e063ec6a9a281fea4a4.

Now we wait for all aio to finish on all SHUTDOWN_CLOSE
cases, this is no longer needed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agoRevert "s3: VFS: vfs_default. Protect vfs_pwrite_done() from accessing a freed req...
Jeremy Allison [Fri, 13 Mar 2020 17:39:31 +0000 (10:39 -0700)]
Revert "s3: VFS: vfs_default. Protect vfs_pwrite_done() from accessing a freed req pointer."

This reverts commit c8cd93dd54cb9f78665928d4bc8fcc3baf084b6f.

Now we wait for all aio to finish on all SHUTDOWN_CLOSE
cases, this is no longer needed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agoRevert "s3: VFS: vfs_default: Add tevent_req pointer to state struct in vfswrap_fsync...
Jeremy Allison [Fri, 13 Mar 2020 17:39:07 +0000 (10:39 -0700)]
Revert "s3: VFS: vfs_default: Add tevent_req pointer to state struct in vfswrap_fsync_state."

This reverts commit 4adde71b99d4ab09914072458329d5f1008b77e3.

Now we wait for all aio to finish on all SHUTDOWN_CLOSE
cases, this is no longer needed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agoRevert "s3: VFS: vfs_default. Pass in struct vfswrap_fsync_state as the callback...
Jeremy Allison [Fri, 13 Mar 2020 17:38:18 +0000 (10:38 -0700)]
Revert "s3: VFS: vfs_default. Pass in struct vfswrap_fsync_state as the callback data to the subreq."

This reverts commit d623779913e0d4a46d7e299dc41b5c83cb127872.

Now we wait for all aio to finish on all SHUTDOWN_CLOSE
cases, this is no longer needed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agoRevert "s3: VFS: vfs_default. Protect vfs_fsync_done() from accessing a freed req...
Jeremy Allison [Fri, 13 Mar 2020 17:38:03 +0000 (10:38 -0700)]
Revert "s3: VFS: vfs_default. Protect vfs_fsync_done() from accessing a freed req pointer."

This reverts commit 18671534e42f66b904e51c3fbe887e998ff79493.

Now we wait for all aio to finish on all SHUTDOWN_CLOSE
cases, this is no longer needed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agoRevert "s3: VFS: vfs_glusterfs: Add tevent_req pointer to state struct in vfs_gluster...
Jeremy Allison [Fri, 13 Mar 2020 17:37:48 +0000 (10:37 -0700)]
Revert "s3: VFS: vfs_glusterfs: Add tevent_req pointer to state struct in vfs_gluster_pread_state."

This reverts commit 0e3dc0078ebd6aa79553bf2afa8e72945e23dfb0.

Now we wait for all aio to finish on all SHUTDOWN_CLOSE
cases, this is no longer needed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agoRevert "s3: VFS: vfs_glusterfs. Pass in struct vfs_gluster_pread_state as the callbac...
Jeremy Allison [Fri, 13 Mar 2020 17:37:30 +0000 (10:37 -0700)]
Revert "s3: VFS: vfs_glusterfs. Pass in struct vfs_gluster_pread_state as the callback data to the subreq."

This reverts commit c6c4e2de22cd3d84f45f5c21e6b09b09274f7f7b.

Now we wait for all aio to finish on all SHUTDOWN_CLOSE
cases, this is no longer needed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agoRevert "s3: VFS: vfs_glusterfs. Protect vfs_gluster_pread_done() from accessing a...
Jeremy Allison [Fri, 13 Mar 2020 17:37:16 +0000 (10:37 -0700)]
Revert "s3: VFS: vfs_glusterfs. Protect vfs_gluster_pread_done() from accessing a freed req pointer."

This reverts commit 99283871c5230e640a8102943ebed685459ed9af.

Now we wait for all aio to finish on all SHUTDOWN_CLOSE
cases, this is no longer needed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agoRevert "s3: VFS: vfs_glusterfs: Add tevent_req pointer to state struct in vfs_gluster...
Jeremy Allison [Fri, 13 Mar 2020 17:36:59 +0000 (10:36 -0700)]
Revert "s3: VFS: vfs_glusterfs: Add tevent_req pointer to state struct in vfs_gluster_pwrite_state."

This reverts commit 058a7effd00b47e4778f7d680cc9c2a7d40d5fa8.

Now we wait for all aio to finish on all SHUTDOWN_CLOSE
cases, this is no longer needed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agoRevert "s3: VFS: vfs_glusterfs. Pass in struct vfs_gluster_pwrite_state as the callba...
Jeremy Allison [Fri, 13 Mar 2020 17:36:39 +0000 (10:36 -0700)]
Revert "s3: VFS: vfs_glusterfs. Pass in struct vfs_gluster_pwrite_state as the callback data to the subreq."

This reverts commit 3357a77d0823eddc1b0db68cfa251a0d54058c88.

Now we wait for all aio to finish on all SHUTDOWN_CLOSE
cases, this is no longer needed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agoRevert "s3: VFS: vfs_glusterfs. Protect vfs_gluster_pwrite_done() from accessing...
Jeremy Allison [Fri, 13 Mar 2020 17:36:19 +0000 (10:36 -0700)]
Revert "s3: VFS: vfs_glusterfs. Protect vfs_gluster_pwrite_done() from accessing a freed req pointer."

This reverts commit 67910c751c9f5ce8cdd1e57b34e51e5b7163838b.

Now we wait for all aio to finish on all SHUTDOWN_CLOSE
cases, this is no longer needed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agoRevert "s3: VFS: vfs_glusterfs: Add tevent_req pointer to state struct in vfs_gluster...
Jeremy Allison [Fri, 13 Mar 2020 17:36:03 +0000 (10:36 -0700)]
Revert "s3: VFS: vfs_glusterfs: Add tevent_req pointer to state struct in vfs_gluster_fsync_state."

This reverts commit c0c088b1b786f0ba248960114191277e91bbba2f.

Now we wait for all aio to finish on all SHUTDOWN_CLOSE
cases, this is no longer needed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agoRevert "s3: VFS: vfs_glusterfs. Pass in struct vfs_gluster_fsync_state as the callbac...
Jeremy Allison [Fri, 13 Mar 2020 17:35:44 +0000 (10:35 -0700)]
Revert "s3: VFS: vfs_glusterfs. Pass in struct vfs_gluster_fsync_state as the callback data to the subreq."

This reverts commit cdde55a69d0dacd2f9939c2f00cd356c0186f791.

Now we wait for all aio to finish on all SHUTDOWN_CLOSE
cases, this is no longer needed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agoRevert "s3: VFS: vfs_glusterfs. Protect vfs_gluster_fsync_done() from accessing a...
Jeremy Allison [Fri, 13 Mar 2020 17:35:21 +0000 (10:35 -0700)]
Revert "s3: VFS: vfs_glusterfs. Protect vfs_gluster_fsync_done() from accessing a freed req pointer."

This reverts commit 9ecbda263f102a24257fd47142b7c24d1f429d8d.

Now we wait for all aio to finish on all SHUTDOWN_CLOSE
cases, this is no longer needed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agoRevert "vfs_default: pass in state as the callback data to the subreq"
Jeremy Allison [Fri, 13 Mar 2020 17:35:04 +0000 (10:35 -0700)]
Revert "vfs_default: pass in state as the callback data to the subreq"

This reverts commit 0e894f3e48285415f72cf7a68e26f1802fe8045d.

Now we wait for all aio to finish on all SHUTDOWN_CLOSE
cases, this is no longer needed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agoRevert "vfs_default: Protect vfs_getxattrat_done() from accessing a freed req pointer"
Jeremy Allison [Fri, 13 Mar 2020 17:34:46 +0000 (10:34 -0700)]
Revert "vfs_default: Protect vfs_getxattrat_done() from accessing a freed req pointer"

This reverts commit 95cfcda13fe9a70b9955a7c44173d619eacb34c1.

Now we wait for all aio to finish on all SHUTDOWN_CLOSE
cases, this is no longer needed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agos3: smbd: Remove file_close_pid().
Jeremy Allison [Thu, 12 Mar 2020 18:02:19 +0000 (11:02 -0700)]
s3: smbd: Remove file_close_pid().

The old synchronous reply_exit() was the only user.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agos3: smbd: Remove old synchronous SMB1 reply_exit().
Jeremy Allison [Thu, 12 Mar 2020 18:01:05 +0000 (11:01 -0700)]
s3: smbd: Remove old synchronous SMB1 reply_exit().

SMB1 exit is now fully async.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agos3: smbd: Add async internals of reply_exit().
Jeremy Allison [Thu, 12 Mar 2020 17:59:16 +0000 (10:59 -0700)]
s3: smbd: Add async internals of reply_exit().

Waits until all aio requests on all fsp's owned by this
vuid are finished before returning to the client.

Charges the profile time in the done function. Not strictly
correct but better than the other SMB1 async code that
double-charges profiling in both send and done at the
moment.

Done this way (commented out) so it is a clean
diff and it's clear what is being added.

A later commit will remove the old synchronous version.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agos3: smbd: Remove old synchronous SMB1 reply_ulogoffX().
Jeremy Allison [Thu, 12 Mar 2020 17:19:31 +0000 (10:19 -0700)]
s3: smbd: Remove old synchronous SMB1 reply_ulogoffX().

SMB1 user logoff is now fully async.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agos3: smbd: reply_ulogoffX() Update to modern coding standards.
Jeremy Allison [Thu, 12 Mar 2020 17:16:42 +0000 (10:16 -0700)]
s3: smbd: reply_ulogoffX() Update to modern coding standards.

Minimizes the diff in the later commits.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agos3: smbd: In reply_ulogoffX(), replace req -> smb1req.
Jeremy Allison [Thu, 12 Mar 2020 17:15:10 +0000 (10:15 -0700)]
s3: smbd: In reply_ulogoffX(), replace req -> smb1req.

Minimises the diff in later commits.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agos3: smbd: Add async internals of reply_ulogoffX.
Jeremy Allison [Thu, 12 Mar 2020 17:11:14 +0000 (10:11 -0700)]
s3: smbd: Add async internals of reply_ulogoffX.

Waits until all aio requests on all fsp's owned by this
vuid are finished before returning to the client.

Charges the profile time in the done function. Not strictly
correct but better than the other SMB1 async code that
double-charges profiling in both send and done at the
moment.

Done this way (commented out) so it is a clean
diff and it's clear what is being added.

A later commit will remove the old synchronous version.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agos3: smbd: Remove old synchronous SMB1 reply_tdis().
Jeremy Allison [Thu, 12 Mar 2020 17:03:27 +0000 (10:03 -0700)]
s3: smbd: Remove old synchronous SMB1 reply_tdis().

SMB1 tree disconnect is now fully async.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agos3: smbd: reply_tdis() Update to modern coding standards.
Jeremy Allison [Thu, 12 Mar 2020 16:59:47 +0000 (09:59 -0700)]
s3: smbd: reply_tdis() Update to modern coding standards.

Minimizes the diff in the next commit.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agos3: smbd: In reply_tdis(), replace req -> smb1req.
Jeremy Allison [Thu, 12 Mar 2020 16:55:30 +0000 (09:55 -0700)]
s3: smbd: In reply_tdis(), replace req -> smb1req.

Minimises the diff in the next commit.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agos3: smbd: Add async internals of reply_tdis().
Jeremy Allison [Thu, 12 Mar 2020 16:51:53 +0000 (09:51 -0700)]
s3: smbd: Add async internals of reply_tdis().

Waits until all aio requests on all fsp's under this
conn struct are finished before returning to the client.

Charges the profile time in the done function. Not strictly
correct but better than the other SMB1 async code that
double-charges profiling in both send and done at the
moment.

Done this way (commented out) so it is a clean
diff and it's clear what is being added.

A later commit will remove the old synchronous version.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agos3: smbd: Replace synchronous conn_force_tdis() with the async version.
Jeremy Allison [Thu, 12 Mar 2020 16:47:26 +0000 (09:47 -0700)]
s3: smbd: Replace synchronous conn_force_tdis() with the async version.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agos3: smbd: Add async internals of conn_force_tdis().
Jeremy Allison [Thu, 12 Mar 2020 16:28:21 +0000 (09:28 -0700)]
s3: smbd: Add async internals of conn_force_tdis().

Commented out so it can be seen complete as
a diff. The next commit will replace the old
synchronous conn_force_tdis() code with the
new async code.

Uses a wait_queue to cause the force close
requests to stay pending until all outstanding
aio is finished on all file handles opened
on the connection.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agos3: smbd: Don't allow force disconnect of a connection already being disconnected.
Jeremy Allison [Tue, 10 Mar 2020 19:25:58 +0000 (12:25 -0700)]
s3: smbd: Don't allow force disconnect of a connection already being disconnected.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agos3: smbd: Every place we check fsp->deferred_close, also check for fsp->closing.
Jeremy Allison [Wed, 11 Mar 2020 22:16:35 +0000 (15:16 -0700)]
s3: smbd: Every place we check fsp->deferred_close, also check for fsp->closing.

Eventually this will allow us to remove fsp->deferred_close
from the fsp struct (and also source3/lib/tevent_wait.[ch]).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agos3: smbd: In async SMB1 reply_close() set fsp->closing = true, as we already do in...
Jeremy Allison [Thu, 12 Mar 2020 00:25:59 +0000 (17:25 -0700)]
s3: smbd: In async SMB1 reply_close() set fsp->closing = true, as we already do in SMB2 async close.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agos3: smbd: Now we free fsp->aio_requests when it gets zero entries, talloc in chunks...
Jeremy Allison [Wed, 11 Mar 2020 21:47:50 +0000 (14:47 -0700)]
s3: smbd: Now we free fsp->aio_requests when it gets zero entries, talloc in chunks of 10 instead of 1.

Prevents incremental +1 tallocs, and the original
idea of this array was that it wasn't freed for
io efficiency reasons. Add paranoia integer wrap
protection also.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agos3: smbd: In aio_del_req_from_fsp() talloc_free(fsp->aio_requests[]) when fsp->num_ai...
Jeremy Allison [Tue, 10 Mar 2020 17:55:22 +0000 (10:55 -0700)]
s3: smbd: In aio_del_req_from_fsp() talloc_free(fsp->aio_requests[]) when fsp->num_aio_requests reaches zero.

The add code in aio_add_req_to_fsp() re-tallocs
this array on demand, and talloc freeing it here
allows it to be used as the parent for a tevent
wait queue, so callers can get notified when
all outstanding aio on an fsp is finished.

We'll deal with any performance issues in
the next commit.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
10 days agos3: tests: Slight tweak to the force-close share test.
Jeremy Allison [Fri, 13 Mar 2020 22:45:16 +0000 (15:45 -0700)]
s3: tests: Slight tweak to the force-close share test.

Turns out on a fast desktop machine 10MB is too small,
and by the time we've done the 'sleep 1' to make sure
the smbclient got scheduled and started processing
the 'put' command, it's already done.

Tweak the file size to be 20MB from 10MB. 10MB
seems to work reliably on gitlab-ci and on
sn-devel, but making the put size 20MB makes
sure it's still in flight when we force-close
the share, even on a fast desktop box. 20MB
shouldn't be too burdonsome even on ci VM's.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2 weeks agoctdb-tcp: Do not stop outbound connection in ctdb_tcp_node_connect()
Martin Schwenke [Fri, 6 Mar 2020 05:11:23 +0000 (16:11 +1100)]
ctdb-tcp: Do not stop outbound connection in ctdb_tcp_node_connect()

The only place the outgoing connection needs to be stopped is when
there is a timeout when waiting for the connection to become writable.
Add a new function ctdb_tcp_node_connect_timeout() to handle this
case.

All of the other cases are attempts to establish a new outgoing
connection (initial attempt, retry after an error or disconnect, ...)
so drop stopping the connection in those cases.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14295

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Signed-off-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Thu Mar 12 05:29:20 UTC 2020 on sn-devel-184

2 weeks agoctdb-tcp: Factor out function ctdb_tcp_start_outgoing()
Martin Schwenke [Fri, 6 Mar 2020 04:59:32 +0000 (15:59 +1100)]
ctdb-tcp: Factor out function ctdb_tcp_start_outgoing()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14295

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Signed-off-by: Martin Schwenke <martin@meltin.net>
2 weeks agoctdb-tcp: add ctdb_tcp_stop_incoming()
Ralph Boehme [Sat, 29 Feb 2020 10:54:51 +0000 (11:54 +0100)]
ctdb-tcp: add ctdb_tcp_stop_incoming()

No change in behaviour.  This makes the code self-documenting.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14295

Signed-off-by: Ralph Boehme <slow@samba.org>
Signed-off-by: Martin Schwenke <martin@meltin.net>
2 weeks agoctdb-tcp: rename ctdb_tcp_stop_connection() to ctdb_tcp_stop_outgoing()
Ralph Boehme [Fri, 28 Feb 2020 10:36:00 +0000 (11:36 +0100)]
ctdb-tcp: rename ctdb_tcp_stop_connection() to ctdb_tcp_stop_outgoing()

No change in behaviour.  This makes the code self-documenting.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14295

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2 weeks agoctdb-tcp: Remove redundant restart in ctdb_tcp_tnode_cb()
Ralph Boehme [Sat, 29 Feb 2020 11:28:20 +0000 (12:28 +0100)]
ctdb-tcp: Remove redundant restart in ctdb_tcp_tnode_cb()

The node dead upcall has already restarted the outgoing connection.
There's no need to repeat it.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14295

Signed-off-by: Ralph Boehme <slow@samba.org>
Signed-off-by: Martin Schwenke <martin@meltin.net>
2 weeks agoctdb-tcp: always call node_dead() upcall in ctdb_tcp_tnode_cb()
Ralph Boehme [Sat, 29 Feb 2020 11:13:12 +0000 (12:13 +0100)]
ctdb-tcp: always call node_dead() upcall in ctdb_tcp_tnode_cb()

ctdb_tcp_tnode_cb() is called when we receive data on the outgoing connection.

This can happen when we get an EOF on the connection because the other side as
closed. In this case data will be NULL.

It would also be called if we received data from the peer. In this case data
will not be NULL.

The latter case is a fatal error though and we already call
ctdb_tcp_stop_connection() for this case as well, which means even though the
node is not fully connected anymore, by not calling the node_dead() upcall
NODE_FLAGS_DISCONNECTED will not be set.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14295

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2 weeks agoctdb-tcp: move free of inbound queue to TCP restart
Noel Power [Sat, 29 Feb 2020 15:49:28 +0000 (15:49 +0000)]
ctdb-tcp: move free of inbound queue to TCP restart

Since commit 77deaadca8e8dbc3c92ea16893099c72f6dc874e, a nodeA which
had previously accepted a connection from nodeB (where nodeB dies
e.g. as as result of fencing) when nodeB attempts to connect again
after restarting is always rejected with

 ctdb_listen_event: Incoming queue active, rejecting connection from w.x.y.z

messages.

Consolidate dead node handling in the TCP restart handling.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14295

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2 weeks agoctdb-daemon: more logical whitespace, debug modernisation
Martin Schwenke [Sun, 1 Mar 2020 05:40:41 +0000 (16:40 +1100)]
ctdb-daemon: more logical whitespace, debug modernisation

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14295

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Ralph Boehme <slow@samba.org>
2 weeks agoctdb-daemon: ensure restart() callback is called in half-connected state
Ralph Boehme [Sat, 29 Feb 2020 11:26:19 +0000 (12:26 +0100)]
ctdb-daemon: ensure restart() callback is called in half-connected state

If NODE_FLAGS_DISCONNECTED is set the node can be in half-connected state. With
this change we ensure to restart the transport for this case.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14295

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2 weeks agoselftest: Add test for rpcclient LSA lookup calls
Christof Schmitt [Mon, 9 Mar 2020 23:25:00 +0000 (16:25 -0700)]
selftest: Add test for rpcclient LSA lookup calls

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Mar 11 09:52:44 UTC 2020 on sn-devel-184

2 weeks agorpcclient: Ask for minimal permissions for SID and name lookups
Christof Schmitt [Mon, 9 Mar 2020 21:21:41 +0000 (14:21 -0700)]
rpcclient: Ask for minimal permissions for SID and name lookups

The RPC calls to lookup SIDS and names only require the
POLICY_LOOKUP_NAMES permission. Only ask for that instead of the
MAXIMUM_ALLOWED flag. This allows these calls to work against a NetApp
that does not accept MAXIMUM_ALLOWED (see bugzilla 11105).

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2 weeks agolibcli: Align integer types
Volker Lendecke [Sun, 8 Mar 2020 15:50:59 +0000 (16:50 +0100)]
libcli: Align integer types

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Mar 10 23:08:20 UTC 2020 on sn-devel-184

2 weeks agolib: Fix a typo
Volker Lendecke [Mon, 9 Mar 2020 14:28:14 +0000 (15:28 +0100)]
lib: Fix a typo

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2 weeks agosmbd: Fix a comment, "flags" expanded to 2 bytes
Volker Lendecke [Mon, 9 Mar 2020 11:36:31 +0000 (12:36 +0100)]
smbd: Fix a comment, "flags" expanded to 2 bytes

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2 weeks agolibsmbclient: Put it back to a known, well-working state
Volker Lendecke [Tue, 10 Mar 2020 18:51:09 +0000 (11:51 -0700)]
libsmbclient: Put it back to a known, well-working state

For adapting unix extensions in our client libraries, we need a fresh start
with additional APIs. We can't change existing application behaviour.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2 weeks agomit-kdc: Explicitly reject S4U requests
Isaac Boukris [Fri, 31 Jan 2020 21:34:21 +0000 (22:34 +0100)]
mit-kdc: Explicitly reject S4U requests

Signed-off-by: Isaac Boukris <iboukris@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Isaac Boukris <iboukris@samba.org>
Autobuild-Date(master): Tue Mar 10 14:46:04 UTC 2020 on sn-devel-184

2 weeks agoselftest: Set KRB5RCACHETYPE to none for selftest
Andreas Schneider [Fri, 6 Mar 2020 16:15:45 +0000 (17:15 +0100)]
selftest: Set KRB5RCACHETYPE to none for selftest

This is required that out tests work with MIT KRB5 1.18.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Isaac Boukris <iboukris@samba.org>
2 weeks agoSign and verify PAC with ticket principal instead of canon principal
Isaac Boukris [Thu, 16 Jan 2020 21:00:21 +0000 (22:00 +0100)]
Sign and verify PAC with ticket principal instead of canon principal

With MIT library 1.18 the KDC no longer set
KRB5_KDB_FLAG_CANONICALIZE for enterprise principals which allows
us to not canonicalize them (like in Windows / Heimdal).

However, it now breaks the PAC signature verification as it was
wrongly done using canonical client rather than ticket client name.

Signed-off-by: Isaac Boukris <iboukris@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
2 weeks agoFix uxsuccess test with new MIT krb5 library 1.18
Isaac Boukris [Thu, 16 Jan 2020 20:36:24 +0000 (21:36 +0100)]
Fix uxsuccess test with new MIT krb5 library 1.18

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14155

Signed-off-by: Isaac Boukris <iboukris@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
2 weeks agoAdapt sign_authdata in our KDB module for krb5 v1.18
Isaac Boukris [Mon, 16 Sep 2019 13:40:12 +0000 (16:40 +0300)]
Adapt sign_authdata in our KDB module for krb5 v1.18

Signed-off-by: Isaac Boukris <iboukris@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2 weeks agoctdb-tests: Use built-in hexdump() in system socket tests
Martin Schwenke [Wed, 4 Mar 2020 03:38:15 +0000 (14:38 +1100)]
ctdb-tests: Use built-in hexdump() in system socket tests

Better compatibility, since od output isn't consistent on FreeBSD.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Tue Mar 10 09:17:12 UTC 2020 on sn-devel-184

2 weeks agoctdb-tests: Split system socket test
Martin Schwenke [Tue, 10 Mar 2020 06:32:52 +0000 (17:32 +1100)]
ctdb-tests: Split system socket test

One test for each of types, TCP, ARP.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2 weeks agoctdb-tests: Skip "ctdb process-exists" tests when not on Linux
Martin Schwenke [Wed, 4 Mar 2020 03:08:12 +0000 (14:08 +1100)]
ctdb-tests: Skip "ctdb process-exists" tests when not on Linux

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2 weeks agoctdb-tests: Add function ctdb_test_check_supported_OS
Martin Schwenke [Wed, 4 Mar 2020 03:05:02 +0000 (14:05 +1100)]
ctdb-tests: Add function ctdb_test_check_supported_OS

Skips test if not on one of the supported OSes.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2 weeks agoctdb-tests: Use ctdb_test_skip() when initscript can not be found
Martin Schwenke [Mon, 9 Mar 2020 09:03:09 +0000 (20:03 +1100)]
ctdb-tests: Use ctdb_test_skip() when initscript can not be found

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2 weeks agoctdb-tests: Use ctdb_test_skip() when shellcheck is not installed
Martin Schwenke [Tue, 3 Mar 2020 06:48:40 +0000 (17:48 +1100)]
ctdb-tests: Use ctdb_test_skip() when shellcheck is not installed

When the tests are run interactively this will make it more noticeable
that shellcheck is not installed because the test summary will
indicate missing tests.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2 weeks agoctdb-tests: Skipped tests should not cause failure
Martin Schwenke [Thu, 5 Mar 2020 00:40:56 +0000 (11:40 +1100)]
ctdb-tests: Skipped tests should not cause failure

Skipped tests return a status that indicates failure.  In combination
with the -e option this results in an exit with failure on the first
skipped test.

Convert skipped test status to success.  The skip has already been
counted.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2 weeks agoautobuild: Run the none env in the samba-o3 build
Andreas Schneider [Fri, 22 Nov 2019 14:23:35 +0000 (15:23 +0100)]
autobuild: Run the none env in the samba-o3 build

This includes tests which should make sure that certain code is not
optimized away, like memset_s().

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Mon Mar  9 23:42:26 UTC 2020 on sn-devel-184

2 weeks agoRequire Python 3.6 for Samba 4.13
Andrew Bartlett [Wed, 4 Mar 2020 00:51:23 +0000 (13:51 +1300)]
Require Python 3.6 for Samba 4.13

This allows Samba to use formatted string literals, which
are quite handy.

REF: https://docs.python.org/3/whatsnew/3.6.html#whatsnew36-pep498

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2 weeks ago.gitlab-ci.yml: Do not build Samba for Ubuntu 16.04 or Debian 9 any longer
Andrew Bartlett [Wed, 4 Mar 2020 00:55:27 +0000 (13:55 +1300)]
.gitlab-ci.yml: Do not build Samba for Ubuntu 16.04 or Debian 9 any longer

These only have Python 3.5 and we want to increase the minimum to Python 3.6.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2 weeks agobootstrap: Remove long-unsupported OS versions
Andrew Bartlett [Wed, 4 Mar 2020 00:58:48 +0000 (13:58 +1300)]
bootstrap: Remove long-unsupported OS versions

Samba has not built on these versions for quite some time due to
the need for Python 3.5 and GnuTLS 3.4.7

These were always marked as broken, but given the requirements
these are never likely to come back.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2 weeks agosmbd: reuse close_free_pending_aio() in close_directory()
Ralph Boehme [Mon, 9 Mar 2020 10:18:23 +0000 (11:18 +0100)]
smbd: reuse close_free_pending_aio() in close_directory()

A directory fsp can have outstanding aio requests as well.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Mon Mar  9 19:34:27 UTC 2020 on sn-devel-184

2 weeks agosmbd: call tevent_req_nterror() for directories when cleaning up pending aio
Ralph Boehme [Mon, 9 Mar 2020 10:31:00 +0000 (11:31 +0100)]
smbd: call tevent_req_nterror() for directories when cleaning up pending aio

smbd_smb2_query_directory_recv() calls tevent_req_is_nterror() which requires a
NTSTATUS error code.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2 weeks agosmbd: move pending aio cleanup to a helper function
Ralph Boehme [Mon, 9 Mar 2020 10:16:37 +0000 (11:16 +0100)]
smbd: move pending aio cleanup to a helper function

We'll be reusing this from close_directory() in the next commit.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2 weeks agovfs_default: Protect vfs_getxattrat_done() from accessing a freed req pointer
Ralph Boehme [Mon, 9 Mar 2020 10:54:37 +0000 (11:54 +0100)]
vfs_default: Protect vfs_getxattrat_done() from accessing a freed req pointer

If the fsp is forced closed by a SHUTDOWN_CLOSE whilst the request is in
flight (share forced closed by smbcontrol), then we set state->req = NULL in the
state destructor.

The existing state destructor prevents the state memory from being freed, so
when the thread completes and calls vfs_getxattrat_done(), just throw away the result
if state->req == NULL.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2 weeks agovfs_default: pass in state as the callback data to the subreq
Ralph Boehme [Mon, 9 Mar 2020 10:54:28 +0000 (11:54 +0100)]
vfs_default: pass in state as the callback data to the subreq

Find the req we're finishing off by looking inside the state.  In a shutdown
close the caller calls talloc_free(req), so we can't access it directly as
callback data.

The next commit will NULL out the state->req pointer when a caller calls
talloc_free(req), and the request is still in flight.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2 weeks agowinexe: add configure option to control whether to build it (default: auto)
Günther Deschner [Wed, 4 Mar 2020 17:51:01 +0000 (18:51 +0100)]
winexe: add configure option to control whether to build it (default: auto)

Guenther

Signed-off-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Mon Mar  9 16:27:21 UTC 2020 on sn-devel-184

2 weeks agolibrpc: fix IDL for svcctl_ChangeServiceConfigW
Günther Deschner [Wed, 4 Mar 2020 14:23:43 +0000 (15:23 +0100)]
librpc: fix IDL for svcctl_ChangeServiceConfigW

Found while trying to run winexe against Windows Server 2019.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14313

Guenther

Signed-off-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2 weeks agos4-torture: add ndr svcctl testsuite
Günther Deschner [Thu, 5 Mar 2020 19:42:21 +0000 (20:42 +0100)]
s4-torture: add ndr svcctl testsuite

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14313

Guenther

Signed-off-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2 weeks agos4-torture: add rpc test for ChangeServiceConfigW
Günther Deschner [Thu, 5 Mar 2020 21:45:48 +0000 (22:45 +0100)]
s4-torture: add rpc test for ChangeServiceConfigW

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14313

Guenther

Signed-off-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2 weeks agovfs_recycle: prevent flooding the log if we're called on non-existant paths
Ralph Boehme [Fri, 6 Mar 2020 11:22:25 +0000 (12:22 +0100)]
vfs_recycle: prevent flooding the log if we're called on non-existant paths

vfs_recycle is assuming that any path passed to unlink must exist, otherwise it
logs this error. Turn this into a DEBUG level message.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14316
See also: https://bugzilla.redhat.com/show_bug.cgi?id=1780802

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Isaac Boukris <iboukris@samba.org>
Autobuild-User(master): Isaac Boukris <iboukris@samba.org>
Autobuild-Date(master): Mon Mar  9 14:15:06 UTC 2020 on sn-devel-184

2 weeks agoctdb-docs: Improve recovery lock documentation
Martin Schwenke [Tue, 18 Feb 2020 07:32:21 +0000 (18:32 +1100)]
ctdb-docs: Improve recovery lock documentation

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Christof Schmitt <cs@samba.org>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Mon Mar  9 02:27:18 UTC 2020 on sn-devel-184

2 weeks agos3: tests: Add samba3.blackbox.force-close-share
Jeremy Allison [Tue, 3 Mar 2020 21:31:18 +0000 (13:31 -0800)]
s3: tests: Add samba3.blackbox.force-close-share

Checks server stays up whilst writing to a force closed share.
Uses existing aio_delay_inject share to delay writes while
we force close the share.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sun Mar  8 19:34:14 UTC 2020 on sn-devel-184

2 weeks agos3: VFS: vfs_aio_pthread: Make aio opens safe against connection teardown.
Jeremy Allison [Thu, 5 Mar 2020 18:22:00 +0000 (10:22 -0800)]
s3: VFS: vfs_aio_pthread: Make aio opens safe against connection teardown.

Allocate state off fsp->conn, not NULL, and add a destructor
that catches deallocation of conn which happens
on connection shutdown or force close.

Note - We don't allocate off fsp as the passed in
fsp will get freed once we return EINPROGRESS/NT_STATUS_MORE_PROCESSING_REQUIRED.
A new fsp pointer gets allocated on every re-run of the
open code path.

The destructor allows us to NULL out the saved conn struct pointer
when conn is deallocated so we know not to access deallocated memory.
This matches the async teardown code changes for bug #14301
in pread/pwrite/fsync vfs_default.c and vfs_glusterfs.c

state is still correctly deallocated in all code
paths so no memory leaks.

This allows us to safely complete when the openat()
returns and then return the error NT_STATUS_NETWORK_NAME_DELETED
to the client open request.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2 weeks agos3: VFS: vfs_aio_pthread: Add a talloc context parameter to create_private_open_data().
Jeremy Allison [Fri, 6 Mar 2020 17:30:26 +0000 (09:30 -0800)]
s3: VFS: vfs_aio_pthread: Add a talloc context parameter to create_private_open_data().

Pass in NULL for now so no behavior change.
We will be changing this from NULL to fsp->conn in a later commit.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2 weeks agos3: VFS: vfs_aio_pthread. Move xconn into state struct (opd).
Jeremy Allison [Thu, 5 Mar 2020 00:39:39 +0000 (16:39 -0800)]
s3: VFS: vfs_aio_pthread. Move xconn into state struct (opd).

We will need this in future to cause a pending open to
be rescheduled after the connection struct we're using
has been shut down with an aio open in flight. This will
allow a correct error reply to an awaiting client.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2 weeks agos3: VFS: vfs_aio_pthread: Replace state destructor with explicitly called teardown...
Jeremy Allison [Wed, 4 Mar 2020 21:47:13 +0000 (13:47 -0800)]
s3: VFS: vfs_aio_pthread: Replace state destructor with explicitly called teardown function.

This will allow repurposing a real destructor to allow
connections structs to be freed whilst the aio open
request is in flight.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2 weeks agos3: VFS: vfs_aio_pthread. Fix leak of state struct on error.
Jeremy Allison [Wed, 4 Mar 2020 21:29:08 +0000 (13:29 -0800)]
s3: VFS: vfs_aio_pthread. Fix leak of state struct on error.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2 weeks agos3: smbd: Make sure we correctly reply to outstanding aio requests with an error...
Jeremy Allison [Mon, 2 Mar 2020 21:11:06 +0000 (13:11 -0800)]
s3: smbd: Make sure we correctly reply to outstanding aio requests with an error on SHUTDOWN_CLOSE.

SHUTDOWN_CLOSE can be called when smbcontrol close-share
is used to terminate active connections.

Previously we just called talloc_free()
on the outstanding request, but this
caused crashes (before the async callback
functions were fixed not to reference req
directly) and also leaves the SMB2 request
outstanding on the processing queue.

Using tevent_req_error() instead
causes the outstanding SMB1/2/3 request to
return with NT_STATUS_INVALID_HANDLE
and removes it from the processing queue.

The callback function called from this
calls talloc_free(req). The destructor will remove
itself from the fsp and the aio_requests array.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2 weeks agos3: VFS: vfs_glusterfs. Protect vfs_gluster_fsync_done() from accessing a freed req...
Jeremy Allison [Sat, 29 Feb 2020 00:01:11 +0000 (16:01 -0800)]
s3: VFS: vfs_glusterfs. Protect vfs_gluster_fsync_done() from accessing a freed req pointer.

If the fsp is forced closed by a SHUTDOWN_CLOSE whilst the
request is in flight (share forced closed by smbcontrol),
then we set state->req = NULL in the state destructor.

The existing state destructor prevents the state memory
from being freed, so when the thread completes and calls
vfs_gluster_fsync_done(), just throw away the result if
state->req == NULL.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2 weeks agos3: VFS: vfs_glusterfs. Pass in struct vfs_gluster_fsync_state as the callback data...
Jeremy Allison [Fri, 28 Feb 2020 23:59:16 +0000 (15:59 -0800)]
s3: VFS: vfs_glusterfs. Pass in struct vfs_gluster_fsync_state as the callback data to the subreq.

Find the req we're finishing off by looking inside vfs_gluster_fsync_state.
In a shutdown close the caller calls talloc_free(req), so we can't
access it directly as callback data.

The next commit will NULL out the vfs_gluster_fsync_state->req pointer
when a caller calls talloc_free(req), and the request is still in
flight.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2 weeks agos3: VFS: vfs_glusterfs: Add tevent_req pointer to state struct in vfs_gluster_fsync_s...
Jeremy Allison [Fri, 28 Feb 2020 23:57:20 +0000 (15:57 -0800)]
s3: VFS: vfs_glusterfs: Add tevent_req pointer to state struct in vfs_gluster_fsync_state.

We will need this to detect when this request is outstanding but
has been destroyed in a SHUTDOWN_CLOSE on this file.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2 weeks agos3: VFS: vfs_glusterfs. Protect vfs_gluster_pwrite_done() from accessing a freed...
Jeremy Allison [Fri, 28 Feb 2020 23:55:36 +0000 (15:55 -0800)]
s3: VFS: vfs_glusterfs. Protect vfs_gluster_pwrite_done() from accessing a freed req pointer.

If the fsp is forced closed by a SHUTDOWN_CLOSE whilst the
request is in flight (share forced closed by smbcontrol),
then we set state->req = NULL in the state destructor.

The existing state destructor prevents the state memory
from being freed, so when the thread completes and calls
vfs_gluster_pwrite_done(), just throw away the result if
state->req == NULL.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2 weeks agos3: VFS: vfs_glusterfs. Pass in struct vfs_gluster_pwrite_state as the callback data...
Jeremy Allison [Fri, 28 Feb 2020 23:53:19 +0000 (15:53 -0800)]
s3: VFS: vfs_glusterfs. Pass in struct vfs_gluster_pwrite_state as the callback data to the subreq.

Find the req we're finishing off by looking inside vfs_gluster_pwrite_state.
In a shutdown close the caller calls talloc_free(req), so we can't
access it directly as callback data.

The next commit will NULL out the vfs_gluster_pwrite_state->req pointer
when a caller calls talloc_free(req), and the request is still in
flight.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2 weeks agos3: VFS: vfs_glusterfs: Add tevent_req pointer to state struct in vfs_gluster_pwrite_...
Jeremy Allison [Fri, 28 Feb 2020 23:47:52 +0000 (15:47 -0800)]
s3: VFS: vfs_glusterfs: Add tevent_req pointer to state struct in vfs_gluster_pwrite_state.

We will need this to detect when this request is outstanding but
has been destroyed in a SHUTDOWN_CLOSE on this file.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2 weeks agos3: VFS: vfs_glusterfs. Protect vfs_gluster_pread_done() from accessing a freed req...
Jeremy Allison [Fri, 28 Feb 2020 23:38:04 +0000 (15:38 -0800)]
s3: VFS: vfs_glusterfs. Protect vfs_gluster_pread_done() from accessing a freed req pointer.

If the fsp is forced closed by a SHUTDOWN_CLOSE whilst the
request is in flight (share forced closed by smbcontrol),
then we set state->req = NULL in the state destructor.

The existing state destructor prevents the state memory
from being freed, so when the thread completes and calls
vfs_gluster_pread_done(), just throw away the result if
state->req == NULL.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14301

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>