Volker Lendecke [Thu, 19 Mar 2020 16:00:59 +0000 (17:00 +0100)]
lib: Try nonblocking writes in writev_send()
All callers now use nonblocking sockets, so that we can optimize by
doing early writes
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Wed Mar 25 10:41:29 UTC 2020 on sn-devel-184
Volker Lendecke [Thu, 12 Mar 2020 15:20:25 +0000 (16:20 +0100)]
lib: Use optimize_empty in writev_send()
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Volker Lendecke [Fri, 20 Mar 2020 11:17:08 +0000 (12:17 +0100)]
winbind: Set the parent->child sock nonblocking
The parent goes through wb_simple_trans_send(), which deals with short
writes fine.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Volker Lendecke [Fri, 20 Mar 2020 10:46:43 +0000 (11:46 +0100)]
libsmb: Make sure that the TCP socket is non-blocking
All traffic goes through smbXcli_base.c, and that is prepared to deal
with short writes via the conn->outgoing queue. Instead of making sure
that all callers properly set the socket nonblocking, do it here, so
that we can later optimize sending out data to the server.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Volker Lendecke [Fri, 20 Mar 2020 10:31:15 +0000 (11:31 +0100)]
lib: Remove unused SOCKET_FLAG_BLOCK
Nobody in the code set this flag, so remove it
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Volker Lendecke [Thu, 12 Mar 2020 10:42:41 +0000 (11:42 +0100)]
lib: Simplify writev_cancel()
We can only reasonably cancel a writev request that is still
queued. Once writing has started, cancel is pointless. Simplify the
if-conditions.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Stefan Metzmacher [Wed, 18 Mar 2020 13:51:12 +0000 (14:51 +0100)]
third_party: Update resolv_wrapper to version 1.1.6
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Wed Mar 25 08:33:17 UTC 2020 on sn-devel-184
Stefan Metzmacher [Tue, 10 Mar 2020 12:27:27 +0000 (13:27 +0100)]
third_party: Update socket_wrapper to version 1.2.4
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Andreas Schneider [Fri, 20 Mar 2020 14:25:47 +0000 (15:25 +0100)]
third_party: Update pam_wrapper to version 1.1.2
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Ralph Boehme [Mon, 23 Mar 2020 09:44:08 +0000 (10:44 +0100)]
vfs: remove SMB_VFS_OPENDIR()
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Mar 24 21:23:43 UTC 2020 on sn-devel-184
Ralph Boehme [Wed, 18 Mar 2020 15:01:17 +0000 (16:01 +0100)]
smbd: use open_internal_dirfsp_at() and OpenDir_fsp() in OpenDir()
This allows consolidating symlink safe processing of pathnames to the low-level
function non_widelink_open() used in the file open codepath via fd_open() and
the new utility function open_internal_dirfsp_at().
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Mon, 16 Mar 2020 17:53:15 +0000 (18:53 +0100)]
smbd: add open_internal_dirfsp_at()
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Mon, 23 Mar 2020 12:56:49 +0000 (13:56 +0100)]
smbd: reformat OpenDir() function definition
Gives a nicer diff in the next commit. No change in behaviour.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Wed, 18 Mar 2020 17:25:33 +0000 (18:25 +0100)]
smbd: update smb_Dir_destructor() to cope with fsp->dptr not being set
Currently the only caller of OpenDir_fsp() is dptr_create() which means
fsp->dptr will always be set by dptr_create().
A subsequent commit will add another caller so that fsp->dptr will end up being
NULL.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Wed, 18 Mar 2020 14:59:11 +0000 (15:59 +0100)]
smbd: remove fdopendir() fallback
In order to get rid of SMB_VFS_OPENDIR, we have to require
fdopendir(). Everybody but macOS seems to have it, so RIP opendir() fallback.
This also prepares for a subsequent commit where we're going to start calling
OpenDir_fsp() from inside OpenDir(). Without removing the fallback this would
result in a recursion.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Tue, 17 Mar 2020 16:35:44 +0000 (17:35 +0100)]
smbd: simplify non_widelink_open()
As fsp->is_directory is already correctly populated by our callers, we can drop
the complicated and possibly broken logic that relies on O_DIRECTORY being
defined.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Sun, 22 Mar 2020 11:21:10 +0000 (12:21 +0100)]
vfs_shadow_copy2: use create_internal_dirfsp_at() and SMB_VFS_FDOPENDIR()
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Mon, 23 Mar 2020 10:36:42 +0000 (11:36 +0100)]
smbd: add create_internal_dirfsp_at()
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Thu, 19 Mar 2020 11:03:27 +0000 (12:03 +0100)]
vfs_shadow_copy: use OpenDir() in shadow_copy_get_shadow_copy_data()
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Thu, 19 Mar 2020 11:03:27 +0000 (12:03 +0100)]
vfs_streams_depot: use OpenDir() in walk_streams()
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Thu, 19 Mar 2020 11:03:27 +0000 (12:03 +0100)]
vfs_fruit: use OpenDir() in fruit_disk_free()
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Thu, 19 Mar 2020 11:42:45 +0000 (12:42 +0100)]
vfs_fruit: pass e->d_name directly to fruit_tmsize_do_dirent()
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Thu, 19 Mar 2020 11:39:44 +0000 (12:39 +0100)]
vfs_fruit: let fruit_get_num_bands() take bundle as const
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Thu, 19 Mar 2020 11:03:27 +0000 (12:03 +0100)]
vfs_fruit: use OpenDir() instead of SMB_VFS_OPENDIR() in fruit_get_num_bands()
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Thu, 19 Mar 2020 11:03:27 +0000 (12:03 +0100)]
vfs_ceph_snapshots: use OpenDir() in ceph_snap_gmt_convert_dir()
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Thu, 19 Mar 2020 11:03:27 +0000 (12:03 +0100)]
vfs_ceph_snapshots: use OpenDir() in ceph_snap_enum_snapdir()
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Wed, 18 Mar 2020 16:40:29 +0000 (17:40 +0100)]
vfstest: use OpenDir()
Note that as ReadDirName() returns translated names (in Windows "encoding"), in
cmd_translate_name() test we have to translate back to UNIX "encoding" to check
if the filename matches the user requested path which is also in UNIX
"encoding".
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Thu, 19 Mar 2020 10:19:48 +0000 (11:19 +0100)]
smbd: use canonicalize_connect_path() in create_conn_struct_as_root()
This fix ensures create_conn_struct_as_root() works the same way as
make_connection_snum() used in smbd for SMB sessions. Without this any caller of
create_conn_struct_as_root() will fail to use work on shares if the sharepath
contains a symlink.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Thu, 19 Mar 2020 10:19:34 +0000 (11:19 +0100)]
smbd: make canonicalize_connect_path() public
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Wed, 18 Mar 2020 15:12:09 +0000 (16:12 +0100)]
smbd: use OpenDir() in form_junctions()
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Wed, 18 Mar 2020 11:05:06 +0000 (12:05 +0100)]
smbd: use OpenDir() in count_dfs_links()
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Thu, 19 Mar 2020 13:07:23 +0000 (14:07 +0100)]
vfs_fruit: move fruit_unlink_internal() back into fruit_unlinkat()
I think this makes fruit_unlinkat() easier to read. No change in behaviour.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ralph Boehme [Tue, 17 Mar 2020 18:03:53 +0000 (19:03 +0100)]
vfs_fruit: remove internal directory cleanup logic
Currently fruit does a cleanup run on a directory when it is being removed: it
lists the directory and does an explicit unlink on any found file that is a
genuine AppleDouble file.
This is not realy needed: the ._ AppleDouble sidecar files are either vetoed by
default, so cleanup can be turned on by the built-in "delete veto files"
options. Or, if the user sets "fruit:veto_appledouble", the client will see the
file in a directory list as ordinary files, so the client is responsible for
deleting them first before removing the parent directory.
tl;dr
One caller of SMB_VFS_OPENDIR() less. :)
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Andreas Schneider [Fri, 13 Mar 2020 14:15:53 +0000 (15:15 +0100)]
selftest: Fix string compare in DnsHandler() of dns_hub.py
dns_hub.py:115: SyntaxWarning: "is" with a literal. Did you mean "=="?
if forwarder is 'ignore':
dns_hub.py:117: SyntaxWarning: "is" with a literal. Did you mean "=="?
elif forwarder is 'fail':
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Tue Mar 24 14:31:25 UTC 2020 on sn-devel-184
Martin Schwenke [Sun, 22 Mar 2020 02:46:46 +0000 (13:46 +1100)]
ctdb-recoverd: Avoid dereferencing NULL rec->nodemap
Inside the nested event loop in ctdb_ctrl_getnodemap(), various
asynchronous handlers may dereference rec->nodemap, which will be
NULL.
One example is lost_reclock_handler(), which causes rec->nodemap to be
unconditionally dereferenced in list_of_nodes() via this call chain:
list_of_nodes()
list_of_active_nodes()
set_recovery_mode()
force_election()
lost_reclock_handler()
Instead of attempting to trace all of the cases, just avoid leaving
rec->nodemap set to NULL. Attempting to use an old value is generally
harmless, especially since it will be the same as the new value in
most cases.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14324
Reported-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Tue Mar 24 01:22:45 UTC 2020 on sn-devel-184
Martin Schwenke [Tue, 25 Feb 2020 06:32:56 +0000 (17:32 +1100)]
ctdb-daemon: Don't allow attach from recovery if recovery is not active
Neither the recovery daemon nor the recovery helper should attach
databases outside of the recovery process.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Mon, 24 Feb 2020 19:20:32 +0000 (06:20 +1100)]
ctdb-daemon: Remove more unused old client database functions
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Mon, 24 Feb 2020 08:51:19 +0000 (19:51 +1100)]
ctdb-recovery: Remove old code for creating missing databases
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Mon, 24 Feb 2020 00:31:33 +0000 (11:31 +1100)]
ctdb-recovery: Create database on nodes where it is missing
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Sun, 23 Feb 2020 23:26:34 +0000 (10:26 +1100)]
ctdb-recovery: Fetch database name from all nodes where it is attached
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Fri, 21 Feb 2020 05:51:10 +0000 (16:51 +1100)]
ctdb-recovery: Pass db structure for each database recovery
Instead of db_id and db_flags.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Fri, 21 Feb 2020 05:10:05 +0000 (16:10 +1100)]
ctdb-recovery: GET_DBMAP from all nodes
This builds a complete list of databases across the cluster so it can
be used to create databases on the nodes where they are missing.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Fri, 21 Feb 2020 01:24:39 +0000 (12:24 +1100)]
ctdb-recovery: Replace use of ctdb_dbid_map with local db_list
This will be used to build a merged list of databases from all nodes,
allowing the recovery helper to create missing databases.
It would be possible to also include the db_name field in this
structure but that would cause a lot of churn. This field is used
locally in the recovery of each database so can continue to live in
the relevant state structure(s).
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Wed, 26 Feb 2020 00:50:09 +0000 (11:50 +1100)]
ctdb-daemon: Respect CTDB_CTRL_FLAG_ATTACH_RECOVERY when attaching databases
This is currently only set by the recovery daemon when it attaches
missing databases, so there is no obvious behaviour change. However,
attaching missing databases can now be moved to the recovery helper as
long as it sets this flag.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Fri, 21 Feb 2020 00:13:05 +0000 (11:13 +1100)]
ctdb-recovery: Use CTDB_CTRL_FLAG_ATTACH_RECOVERY to attach during recovery
ctdb_ctrl_createdb() is only called by the recovery daemon, so this is
a safe, temporary change. This is temporary because
ctdb_ctrl_createdb(), create_missing_remote_databases() and
create_missing_local_databases() will all go away soon.
Note that this doesn't cause a change in behaviour. The main daemon
will still only defer attaches from non-recoverd processes during
recovery.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Fri, 21 Feb 2020 00:04:14 +0000 (11:04 +1100)]
ctdb-protocol: Add control flag CTDB_CTRL_FLAG_ATTACH_RECOVERY
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Wed, 26 Feb 2020 06:03:49 +0000 (17:03 +1100)]
ctdb-daemon: Remove unused old client database functions
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Martin Schwenke [Thu, 20 Feb 2020 02:48:13 +0000 (13:48 +1100)]
ctdb-daemon: Fix database attach deferral logic
Commit
3cc230b5eeca749ab68d19cfda969f72c269f1f6 says:
Dont allow clients to connect to databases untile we are well past
and through the initial recovery phase
It is unclear what this commit was attempting to do. The commit
message implies that more attaches should be deferred but the code
change adds a conjunction that causes less attaches to be deferred.
In particular, no attaches will be deferred after startup is complete.
This seems wrong.
To implement what seems to be stated in the commit message an "or"
needs to be used so that non-recovery daemon attaches are deferred
either when in recovery or before startup is complete. Making this
change highlights that attaches need to be allowed during the
"startup" event because this is when smbd is started.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Amitay Isaacs [Mon, 2 Mar 2020 05:16:26 +0000 (16:16 +1100)]
ctdb-recovery: Refactor banning a node into separate computation
If a node is marked for banning, confirm that it's not become inactive
during the recovery. If yes, then don't ban the node.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Amitay Isaacs [Tue, 18 Feb 2020 05:17:00 +0000 (16:17 +1100)]
ctdb-recovery: Don't trust nodemap obtained from local node
It's possible to have a node stopped, but recovery master not yet
updated flags on the local ctdb daemon when recovery is started. So do
not trust the list of active nodes obtained from the local node. Query
the connected nodes to calculate the list of active nodes.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Amitay Isaacs [Mon, 2 Mar 2020 04:07:21 +0000 (15:07 +1100)]
ctdb-recovery: Consolidate node state
This avoids passing multiple arguments to async computation.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Amitay Isaacs [Mon, 2 Mar 2020 02:59:42 +0000 (13:59 +1100)]
ctdb-recovery: Fetched vnnmap is never used, so don't fetch it
New vnnmap is constructed using the information from all the connected
nodes. So there is no need to fetch the vnnmap from recovery master.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14294
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Aurelien Aptel [Fri, 13 Mar 2020 12:42:53 +0000 (13:42 +0100)]
s3: libsmbclient.h: add missing time.h include
A recent change added a struct with timespec members in the public
libsmbclient header. This type is defined in time.h which was not
included thus making users of libsmbclient not build properly.
/.../libsmbclient.h:158:18: error: field 'btime_ts' has incomplete type
struct timespec btime_ts;
Fixes: bf13fe0f222 ("s3: libsmbclient: Add internal/external structures needed for readdirplus.")
Signed-off-by: Aurelien Aptel <aaptel@suse.com>
Reviewed-by: Noel Power <npower@samba.org>
Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Mon Mar 23 20:47:44 UTC 2020 on sn-devel-184
Andrew Bartlett [Sat, 14 Mar 2020 21:47:32 +0000 (10:47 +1300)]
s4-librpc: Simplify bytes or unicode input checking in python GUID() bindings
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
Andrew Bartlett [Sat, 14 Mar 2020 21:43:16 +0000 (10:43 +1300)]
py3: Remove #define IsPy3BytesOrString(pystr)
This was
(PyUnicode_Check(pystr) || PyBytes_Check(pystr))
This allows us to end the use of Python 2/3 compatability macros.
The one caller will be simplified in the next commit
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <nopower@samba.org>
Andrew Bartlett [Sat, 14 Mar 2020 21:40:46 +0000 (10:40 +1300)]
py3: Remove #define IsPy3Bytes PyBytes_Check
This allows us to end the use of Python 2/3 compatability macros.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
Andrew Bartlett [Sat, 14 Mar 2020 21:36:59 +0000 (10:36 +1300)]
py3: Remove #define PyInt_FromLong PyLong_FromLong
This allows us to end the use of Python 2/3 compatability macros.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power
Andrew Bartlett [Sat, 14 Mar 2020 21:32:11 +0000 (10:32 +1300)]
py3: Remove #define PyInt_AsLong PyLong_AsLong
This allows us to end the use of Python 2/3 compatability macros.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
Andrew Bartlett [Sat, 14 Mar 2020 21:26:22 +0000 (10:26 +1300)]
py3: Remove #define PyInt_Type PyLong_Type
This allows us to end the use of Python 2/3 compatability macros.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
Andrew Bartlett [Sat, 14 Mar 2020 21:15:15 +0000 (10:15 +1300)]
pidl: Remove reference to PyInt_Type from error string when we wanted a sensible Long
PyInt_Type is no longer a thing in Python3, we will remove the compatability
reference shortly.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
Andrew Bartlett [Sat, 14 Mar 2020 21:10:24 +0000 (10:10 +1300)]
pidl: Remove duplicate "if (PyLong_Check($cvar)" clauses
Since we moved to Python3, these have been dead code, and it is clearer
now that we have removed the compatability define.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
Andrew Bartlett [Sat, 14 Mar 2020 21:04:52 +0000 (10:04 +1300)]
py3: Remove #define PyInt_Check PyLong_Check
This will allow us to remove some unused code in the PIDL-generated
python bindings.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
Andrew Bartlett [Sat, 21 Mar 2020 19:15:30 +0000 (08:15 +1300)]
autobuild: Merge samba-simpleserver into samba-nt4
This saves CI resources (20mins build time) at the expense of making
the samba-nt4 job take 45min (5 min longer).
The new maximum job time is 1:15 by samba-o3 on the different hosts.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Mon Mar 23 14:45:59 UTC 2020 on sn-devel-184
Andrew Bartlett [Sat, 21 Mar 2020 19:29:33 +0000 (08:29 +1300)]
selftest: Run smb2.compound_find against filesrever only, not nt4_dc and ad_dc
This is a slow test of SMB2 protocol features, it needs not to be repeated. It only
got run against both by accident at the original merge and then extended to
run on the [compount_find] share (with smbd:find async delay usec = 10000)
but really does not need to be run three times.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Andrew Bartlett [Thu, 12 Mar 2020 03:07:01 +0000 (16:07 +1300)]
autobuild: Merge the samba-ktest-heimdal and samba-fileserver jobs
This avoids a full compile of Samba just to test Kerberos with a system Heimdal
while still providing an environment to test other fileserver features.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Andrew Bartlett [Thu, 12 Mar 2020 00:43:45 +0000 (13:43 +1300)]
.gitlab-ci.yml: Move the short samba-xc job into "others"
This job is over in 8mins total and much of that is spent setting up.
It can safely be combined with the others in "others"
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Andrew Bartlett [Wed, 24 Jul 2019 04:22:12 +0000 (16:22 +1200)]
build: Require --without-ad-dc for --without-ads
Building an AD DC while setting --without-ads makes no sense and just wastes compile time on our build hosts.
To allow samba-nt4 to build --without-ad-dc we set rpc.spoolss.notify
(which is built on the NTVFS fileserver for the callbacks) to run in
the ad_member environment rather than nt4_dc and ad_dc.
This is also just more realistic in any case.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Jeremy Allison [Wed, 18 Mar 2020 23:09:50 +0000 (16:09 -0700)]
s3: Remove tevent_wait code. The last user was fsp->deferred_close.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Mon Mar 23 12:06:45 UTC 2020 on sn-devel-184
Jeremy Allison [Wed, 18 Mar 2020 23:07:56 +0000 (16:07 -0700)]
s3: VFS: Remove deferred_close from struct files_struct.
Bump up the VFS number to 43. Samba 4.13 will ship with that.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Jeremy Allison [Wed, 18 Mar 2020 23:01:18 +0000 (16:01 -0700)]
s3: smbd: Remove all references to fsp->deferred_close.
We are now free to remove it from struct files_struct.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Jeremy Allison [Wed, 18 Mar 2020 22:59:03 +0000 (15:59 -0700)]
s3: smbd: Convert async SMB2 close to use the wait_queue idiom.
Use the same mechanism to wait for aio on a handle used by
SMB1 reply_tdis(), reply_ulogoffX(), reply_exit(), reply_close()
and SMB2 tree disconnect.
This removes the last user of fsp->deferred_close, allowing
us to remove it.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Jeremy Allison [Wed, 18 Mar 2020 22:21:43 +0000 (15:21 -0700)]
s3: smbd: Remove old async versions of SMB1 reply_close().
Use the common pattern to wait for aio.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Jeremy Allison [Wed, 18 Mar 2020 22:09:51 +0000 (15:09 -0700)]
s3: smbd: Add async internals of reply_close().
Waits until all aio requests on the closing fsps
before returning to the client.
Slightly modified version of the existing async
reply_close code, updated to use the wait_queue
pattern standardized in reply_tdis, reply_ulogoffX
and reply_exit.
Done this way (commented out) so it is a clean
diff and it's clear what is being added.
The next commit will remove the old version.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Jeremy Allison [Wed, 18 Mar 2020 21:32:30 +0000 (14:32 -0700)]
s3: smbd: In asnyc do_smb1_close() use the utility function meant for async SMB1 replies.
Don't use the raw call.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Jeremy Allison [Wed, 18 Mar 2020 21:29:30 +0000 (14:29 -0700)]
s3: smbd: Update reply_close() to modern DBG_ coding style.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Jeremy Allison [Wed, 18 Mar 2020 21:06:38 +0000 (14:06 -0700)]
s3: smbd: In SMB1 reply_close() rename all uses of 'struct smb_request' to smb1req.
Will make further changes easier to see, as we now use
req for tevent_req structs.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Douglas Bagnall [Mon, 9 Mar 2020 22:31:12 +0000 (11:31 +1300)]
idl/drsblobs: do not overwrite number of schedules == 1
If the struct has zero or two schedules, that is what it has, and we
should let that be.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Sun Mar 22 06:19:51 UTC 2020 on sn-devel-184
Andrew Bartlett [Mon, 2 Mar 2020 05:01:29 +0000 (18:01 +1300)]
dsdb: Allow delete (directly and over DRS) of an object with a link to itself
Previously this would fail with Unsupported critical extension 1.3.6.1.4.1.7165.4.3.2
Reported by Alexander Harm. Many thanks for helping make Samba better
and for your patience with patches and providing debugging information.
REF: https://lists.samba.org/archive/samba/2020-February/228153.html
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14306
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Andrew Bartlett [Mon, 2 Mar 2020 04:44:10 +0000 (17:44 +1300)]
dsdb: Add test for the case of a link pointing back at its own object
This type of object was not possible to delete in Samba without first removing
the link.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14306
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Andrew Bartlett [Tue, 3 Mar 2020 02:51:22 +0000 (15:51 +1300)]
selftest: Add test for dangling backlink to ourself, a missing and a real object
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14306
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Andrew Bartlett [Mon, 2 Mar 2020 23:36:42 +0000 (12:36 +1300)]
selftest: Add test for dangling backlinks to objects that do not exist
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14306
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Gary Lockyer [Mon, 16 Mar 2020 21:12:49 +0000 (10:12 +1300)]
ldb build: Remove some PEP8 warnings from wscript
Fix indentation of list members and fix lines > 79 characters to remove
PEP8 warnings.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Gary Lockyer [Mon, 16 Mar 2020 02:18:12 +0000 (15:18 +1300)]
ldb tests: Confirm lmdb free list handling
Add cmocka tests to confirm lmdb's handling of the free list.
As a result of lmdb's MVCC (Multiversion Concurrency Control) long
running read transactions or stale readers (read transactions where the
process exited without ending the transaction) can cause the database to
run out of space.
Items in the free list are only reused when they would not be visible in
a read transaction. So long running read transactions prevent entries
in the free list being reused, and the database can run out of space.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Samuel Cabrero [Tue, 28 May 2019 15:15:19 +0000 (17:15 +0200)]
s3:rpc_server: Remove dead code
Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Samuel Cabrero <scabrero@samba.org>
Autobuild-Date(master): Fri Mar 20 17:11:28 UTC 2020 on sn-devel-184
Samuel Cabrero [Tue, 28 May 2019 14:54:52 +0000 (16:54 +0200)]
s3:rpc_server: Remove unused RPC module init and shutdown callbacks
The setup function registers the endpoint server and RPC core routines
initialize and shutdown it.
Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Samuel Cabrero [Tue, 28 May 2019 15:01:36 +0000 (17:01 +0200)]
s3:rpc_server: Do not generate and build s3 RPC server code
Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Samuel Cabrero [Tue, 28 May 2019 14:12:51 +0000 (16:12 +0200)]
s3:rpc_server: Do not include s3 autogenerated headers
Prototype is generated by the server compat parser.
Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Samuel Cabrero [Tue, 28 May 2019 14:32:39 +0000 (16:32 +0200)]
s3:rpc_server: Remove api_struct
Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Samuel Cabrero [Thu, 7 Mar 2019 10:37:48 +0000 (11:37 +0100)]
s3:rpc_server: Remove dead code and unused struct members
Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Samuel Cabrero [Thu, 28 Feb 2019 13:04:08 +0000 (14:04 +0100)]
s3:rpc_server: Remove s3 rpc server loop
Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Samuel Cabrero [Mon, 9 Sep 2019 16:22:43 +0000 (18:22 +0200)]
s3:rpc_server: Remove call to s3 rpc services shutdown code
The rpc_<service>_shutdown function unregisters the legacy api_struct
from the local dispatch table, which is not longer used as local
dispatching is done through dcesrv_interface and will be removed in
following commits.
The dcesrv_shutdown_registered_ep_servers will unregister endpoint servers.
Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Samuel Cabrero [Thu, 5 Dec 2019 10:45:54 +0000 (11:45 +0100)]
s3:rpc_server: Switch to core dcerpc server loop
This commit finally switches the RPC server implementation.
At the same we have to do other related changes to keep code compiling
and test environments running.
First avoid moving the session_info into the allocated pipes_struct memory
context as it is owned now by the core RPC server, and the s3compat pidl
compiler will update the pipes_struct session_info before dispatching
the call with dcesrv_call->auth_state->session_info.
Also, fix a segfault in the endpoint mapper daemon when it tries to delete
the endpoints previously registered over a NCALRPC connection.
If we have:
rpc_server : epmapper = external
rpc_server : lsarpc = external
rpc_daemon : epmd = fork
rpc_daemon : lsasd = fork
The sequence is:
* The endpoint mapper starts (start_epmd in source3/smbd/server.c)
* The lsarpc daemon starts (start_lsasd in source3/smbd/server.c)
* The lsarpc daemon creates the sockets and registers its endpoints
(rpc_ep_register in source3/rpc_server/lsasd.c)
* The endpoint registration code opens a NCALRPC connection to the
endpoint mapper daemon (ep_register in source3/librpc/rpc/dcerpc_ep.c)
and keeps it open to re-register if the endpoint mapper daemon dies
(rpc_ep_register_loop in source3/rpc_server/rpc_ep_register.c)
* When the endpoint mapper daemon accepts a NCALRPC connection it sets a
termination function (srv_epmapper_delete_endpoints)
* Suppose the lsarpc daemon exits. The NCALRPC connection termination
function is called.
* The termination function tries to delete all endpoints registered by that
connection by calling _epm_Delete
* _epm_Delete calls is_privileged_pipe which access to
pipes_struct->session_info.
As the call to _epm_Delete occurs outside of the PIDL generated code,
the pipes_stuct->session_info is NULL. This commit also sets
pipes_struct->session_info from the dcerpc_connection before calling
_epm_Delete. As the core rpc server supports security context multiplexing we
need to pass the dcesrv_connection to the termination function and let the
implementation pick a auth context. In the case of the endpoint mapper
the termination function has to pick one of type NCALRPC_AS_SYSTEM to
check if the connection is privileged and delete the endpoints
registered by the connection being closed.
Finally, the samba.tests.dcerpc.raw_protocol testsuite passes against
the ad_member environment.
Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Samuel Cabrero [Thu, 28 Feb 2019 12:04:02 +0000 (13:04 +0100)]
s3:rpc_server: Update winspool generated server stub
Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Samuel Cabrero [Tue, 22 Oct 2019 16:28:44 +0000 (18:28 +0200)]
s3:winbindd: Dispatch RPC calls through interface local handler
Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Samuel Cabrero [Tue, 28 May 2019 10:05:45 +0000 (12:05 +0200)]
s3:rpc_server: Dispatch local calls through interfaces local handler
Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Samuel Cabrero [Tue, 28 May 2019 10:02:09 +0000 (12:02 +0200)]
pidl:NDR/ServerCompat: Generate local dispatching function
To be used in rpcint_binding_handle, prepare to remove legacy api_struct
and S3 pidl generated code.
Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Samuel Cabrero [Mon, 11 Mar 2019 13:10:28 +0000 (14:10 +0100)]
pidl:NDR/ServerCompat: Initialize and allocate out vars
Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Samuel Cabrero [Thu, 28 Feb 2019 12:03:33 +0000 (13:03 +0100)]
pidl:NDR/ServerCompat: Add dispatch, reply, pull and push functions to header
Will be used by winspool to forward selected opnums to spoolss.
Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Samuel Cabrero [Wed, 27 Feb 2019 22:28:25 +0000 (23:28 +0100)]
pidl:NDR/ServerCompat: Retrieve and setup pipes struct before dispatch
Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Samuel Cabrero [Wed, 27 Feb 2019 22:44:16 +0000 (23:44 +0100)]
s3:rpc_server: Store dcesrv context and endpoint in ncacn_conn
Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>