17 <<<point-cut-1280.jpg,height=.8\textheight>>>
20 ==== Multi-Channel - General ====
23 <[block]{multiple transport connections in one SMB(3) session}
24 * '''channel''': transport connection bound to a session
25 * client decides which connections to bind and to use
26 * session is valid as long as at least one channel is intact
31 <[block]{two purposes}
32 # increase throughput:
33 #* use multiple connections of same type
34 # improve fault tolerance:
35 #* channel failure: replay/retry detection
36 %%#* session is valid as long as one channel is still intact
40 ==== Multi-Channel - General ====
42 <[block]{use case: channels of different type/quality}
43 * use only the channels of best quality
44 * fall back to inferior channels if superior ones fail
45 * e.g.: laptop switching between WiFi and LAN (?)
48 ==== Multi-Channel - Windows/Protocol ====
50 # establish initial session on TCP connection
51 # find interfaces with interface discovery: \\ %
52 @FSCTL\_QUERY\_NETWORK\_INTERFACE\_INFO@
53 # bind additional TCP (or later RDMA) connection (channel) to established SMB3 session (''session bind'')
54 # Windows: uses connections of same (and best) quality
55 # Windows: binds only to a single node
56 # replay / retry mechanisms, sequence numbers
62 <<<samba-kisses-better-selection.jpg,height=.8\textheight>>>
65 ==== Multi-Channel $\in$ Samba ====
67 <[block]{samba/smbd: multi-process}
68 * '''Originally:''' process $\Leftrightarrow$ TCP connection
69 *<0> '''Idea:''' transfer new TCP connection to existing smbd
70 *<0> '''How?''' ==> use fd-passing (sendmsg/recvmsg)
72 ** ''Natural choice'': at SessionSetup (Bind)
73 ** !Idea!: as early as possible, based on ClientGUID \\ %
74 ==> per ClientGUID single process model
75 %%*<0> !But!: This may not work! \frownie
78 ==== Multi-Channel $\in$ Samba ====
81 <<<smb3-mc-daemons-n1.png,width=.9\textwidth>>>
84 ==== Multi-Channel $\in$ Samba ====
86 <[block]{samba/smbd: multi-process}
87 * '''Originally:''' process $\Leftrightarrow$ TCP connection
88 * '''Idea:''' transfer new TCP connection to existing smbd
89 *<0> '''How?''' ==> use fd-passing (sendmsg/recvmsg)
91 ** ''Natural choice'': at SessionSetup (Bind)
92 ** !Idea!: as early as possible, based on ClientGUID \\ %
93 ==> per ClientGUID single process model
94 %%*<0> !But!: There may be problems! ... \frownie
98 ==== Multi-Channel $\in$ Samba ====
102 <<<smb3-mc-daemons-n1a.png,width=.9\textwidth>>>
108 <<<smb3-mc-daemons-n2.png,width=.9\textwidth>>>
114 <<<smb3-mc-daemons-n3.png,width=.9\textwidth>>>
119 ==== Multi-Channel $\in$ Samba ====
121 <[block]{samba/smbd: multi-process}
122 * '''Originally:''' process $\Leftrightarrow$ TCP connection
123 * '''Idea:''' transfer new TCP connection to existing smbd
124 * '''How?''' ==> use fd-passing (sendmsg/recvmsg)
126 ** ''Natural choice'': at SessionSetup (Bind)
127 ** !Idea!: as early as possible, based on ClientGUID \\ %
128 ==> per ClientGUID single process model
129 %%*<0> !But!: There may be problems! ... \frownie
133 ==== Multi-Channel $\in$ Samba : pass by ClientGUID ====
136 %%<<<smb3-mc-samba_exp.png,height=.9\textheight>>>
137 <<<smb3-mc-samba.png,height=.9\textheight>>>
141 ==== Multi-Channel $\in$ Samba : pass by ClientGUID ====
143 <[block]{Wait a minute - what about performance?}
145 * But we use short-lived worker-pthreads for I/O ops! \\ %
146 ==> using multiple CPUs
147 * Benchmarks and tunings in progress
150 %%%% ==== Multi-Channel $\in$ Samba ====
152 %%%% <[block]{samba/smbd: multi-process}
153 %%%% * '''Orignally:''' process $\Leftrightarrow$ TCP connection
154 %%%% * '''Idea:''' transfer new TCP connection to existing smbd
155 %%%% * '''How?''' ==> use fd-passing (sendmsg/recvmsg)
157 %%%% ** ''Natural choice'': at SessionSetup (Bind)
158 %%%% ** ''Idea'': as early as possible, based on ClientGUID \\ %
159 %%%% ==> per ClientGUID single process model
160 %%%% * !But!: There may be problems! ... \frownie
164 %%%% ==== The Relevance of the ClientGUID ====
167 %%%% <[block]{Assumption was:}
168 %%%% * All channels in a session have the same ClientGUID
169 %%%% * The server enforces this
174 %%%% <[block]{Evidence from [MS-SMB2]:}
175 %%%% * 3.3.5.9 Receiving an SMB2 CREATE Request:
176 %%%% ** sets Open.ClientGuid to Connection.ClientGuid
177 %%%% ** replay detection checks \\ %
178 %%%% Open.ClientGuid == Connection.ClientGuid
179 %%%% * 3.3.5.9.7/12 Durable (v2) Reconnect Create Context:
180 %%%% ** check Open.ClientGuid == Connection.ClientGuid
185 %%%% <[block]{The truth is...}
186 %%%% The Windows server does not enforce it!
191 %%%% ==== The Relevance of the ClientGUID ====
193 %%%% <[block]{Windows behaviour according to MS}
194 %%%% * The server does NOT enforce same ClientGUID in a session.
195 %%%% * But clients can be expected to do it.
196 %%%% * But it is not explicitly documented like this.
200 %%%% <[block]{The good news:}
201 %%%% There will be documentation notes:
202 %%%% * Things will not work as expected when clients behave differently.
203 %%%% * It is OK for a server to enforce equality of ClientGUID within session.
207 %%%% ==== More digression on ClientGUID : Leases ====
209 %%%% <[block]{According to [MS-SMB2]}
210 %%%% * Server Global data Structures:
211 %%%% ** GlobalLeaseTableList indexed by ClientGuid (3.3.1.5)
212 %%%% *** LeaseTable indexed by LeaseKey (3.3.1.11)
213 %%%% * Requesting a lease (3.3.1.4):
214 %%%% ** Object store takes an abstract ClientLeaseId
215 %%%% *** Win7: combination of ClientGuid and LeaseKey
216 %%%% *** Win8+: LeaseKey
217 %%%% * Object store indicates a lease break (3.3.4.7):
218 %%%% ** smb server uses ClientGuid and LeaseKey given by ObjectStore
220 %%%% ==> !Inconsistent\! -- What to do?!
224 %%%% ==== Multi-Channel $\in$ Samba : pass by SessionID (plan B) ====
227 %%%% <<<smb3-mc-samba-v2.png,height=.9\textheight>>>
231 ==== Multi-Channel $\in$ Samba : Status ====
234 # messaging rewrite using unix dgm sockets with sendmsg [DONE,4.2]
235 # add fd-passing to messaging [DONE,4.2]
236 # preparations in internal structures [DONE,4.2--4.4]
237 # prepare code to cope with multiple channels [DONE,4.4]
238 # implement smbd message to pass a tcp socket [DONE,4.4]
239 # transfer connection in Negotiate (by ClientGUID) [DONE,4.4]
240 # implement session bind [DONE,4.4]
241 # implement channel sequence numbers [DONE,4.4]
242 # implement interface discovery [DONE(linux/conf),4.4]
243 # implement test cases [WIP(isn't it always?... $\smiley$)]
244 # implement fd-passing in socket-wrapper [WIP]
245 # implement lease break replay [TODO]
248 ==== Multi-Channel $\in$ Samba : How we got there ====
250 * Based on preparations in 4.2 and earlier (200+ patches)
251 ** Patches by Stefan Metzmacher, Michael Adam, Volker Lendecke, Anubhav Rakshit
253 ** Polishing of large parts of massively WIP branch
254 ** Added new code (create replay, interface detection)
255 ** Result merged in units. Overall some ~130 patches.
258 *** Stefan Metzmacher
262 * Just made it as experimental feature into Samba 4.4
265 %%==== Multi-Channel $\in$ Samba : Details from @smbXsrv.idl@ ====
266 %%%%==== @MSG\_SMBXSRV\_CONNECTION\_PASS@ ====
268 %%<[block]{for @MSG\_SMBXSRV\_CONNECTION\_PASS@}
271 %% NTTIME initial_connect_time;
274 %% DATA_BLOB negotiate_request;
275 %%} smbXsrv_connection_pass0;
279 %%%%==== Internal Structures (@smbXsrv.idl@) ====
280 %%==== Multi-Channel $\in$ Samba : Details from @smbXsrv.idl@ ====
283 %%<[block]{layering before}
286 %% ->smbXsrv_connection
290 %%<[block]{layering now}
294 %% ->smbXsrv_connections
299 %%% ==== Multi-Channel $\in$ Samba : Status ====
301 %%% <[block]{WIP code}
302 %%% * @git://git.samba.org/obnox/samba/samba-obnox.git@
303 %%% * branch: @master-multi-channel-obnox@
309 <<<instructions-1-1280.jpg,height=.8\textwidth>>>
312 ==== Multi-Channel $\in$ Samba : How to enable it ====
318 server multi channel support = yes
324 ==== Multi-Channel $\in$ Samba: TODOs ====
326 *<2-> teach socket\_wrapper fd-passing ( ==> selftest...)
327 *<3-> Replay lease breaks upon channel failure (server $\rightarrow$ client) \\ %
329 *<4-> clustering integration (CTDB) \\ %
332 ==== Multi-Channel $\in$ Samba : Clustering/CTDB ====
335 <[block]{Special considerations}
336 * channels of one session only to one node !
337 * do not bind connections to CTDB public IPs (can move)!
338 * problem: CTDB clustering transparent to SMB clients...
345 <<<instructions-2-cut-1280.jpg,height=.8\textheight>>>
349 ==== Multi-Channel $\in$ Samba : Clustering/CTDB ====
352 <[block]{Plan for integration}
353 * establish blacklist of addresses (e.g. CTDB public IPs)
354 * add static IPs to public interfaces
355 * optionally establish whitelist (interfaces ...)
356 * ==> list of allowed addresses
357 * only publish allowed addresses in interfaces info ioctl
358 * only give more than one address in interface info when asked via an allowed address
359 * deny session bind on non-allowed address