1 THIS IS INCOMPLETE! I'M ONLY COMMITING IT IN ORDER TO SOLICIT COMMENTS
2 FROM A FEW PEOPLE. DON'T TAKE THIS AS THE FINAL VERSION YET.
7 Samba4 Programming Guide
8 ------------------------
10 The internals of Samba4 are quite different from previous versions of
11 Samba, so even if you are an experienced Samba developer please take
12 the time to read through this document.
14 This document will explain both the broad structure of Samba4, and
15 some of the common coding elements such as memory management and
22 In past versions of Samba we have basically let each programmer choose
23 their own programming style. Unfortunately the result has often been
24 that code that other members of the team find difficult to read. For
25 Samba version 4 I would like to standardise on a common coding style
26 to make the whole tree more readable. For those of you who are
27 horrified at the idea of having to learn a new style, I can assure you
28 that it isn't as painful as you might think. I was forced to adopt a
29 new style when I started working on the Linux kernel, and after some
30 initial pain found it quite easy.
32 That said, I don't want to invent a new style, instead I would like to
33 adopt the style used by the Linux kernel. It is a widely used style
34 with plenty of support tools available. See Documentation/CodingStyle
35 in the Linux source tree. This is the style that I have used to write
36 all of the core infrastructure for Samba4 and I think that we should
37 continue with that style.
39 I also think that we should most definately *not* adopt an automatic
40 reformatting system in cvs (or whatever other source code system we
41 end up using in the future). Such automatic formatters are, in my
42 experience, incredibly error prone and don't understand the necessary
43 exceptions. I don't mind if people use automated tools to reformat
44 their own code before they commit it, but please do not run such
45 automated tools on large slabs of existing code without being willing
46 to spend a *lot* of time hand checking the results.
48 Finally, I think that for code that is parsing or formatting protocol
49 packets the code layout should strongly reflect the packet
50 format. That means ordring the code so that it parses in the same
51 order as the packet is stored on the wire (where possible) and using
52 white space to align packet offsets so that a reader can immediately
53 map any line of the code to the corresponding place in the packet.
56 Static and Global Data
57 ----------------------
59 The basic rule is "avoid static and global data like the plague". What
60 do I mean by static data? The way to tell if you have static data in a
61 file is to use the "size" utility in Linux. For example if we run:
65 in Samba4 then you get the following:
67 text data bss dec hex filename
68 2015 0 0 2015 7df libcli/raw/clikrb5.o
69 202 0 0 202 ca libcli/raw/clioplock.o
70 35 0 0 35 23 libcli/raw/clirewrite.o
71 3891 0 0 3891 f33 libcli/raw/clisession.o
72 869 0 0 869 365 libcli/raw/clisocket.o
73 4962 0 0 4962 1362 libcli/raw/clispnego.o
74 1223 0 0 1223 4c7 libcli/raw/clitransport.o
75 2294 0 0 2294 8f6 libcli/raw/clitree.o
76 1081 0 0 1081 439 libcli/raw/raweas.o
77 6765 0 0 6765 1a6d libcli/raw/rawfile.o
78 6824 0 0 6824 1aa8 libcli/raw/rawfileinfo.o
79 2944 0 0 2944 b80 libcli/raw/rawfsinfo.o
80 541 0 0 541 21d libcli/raw/rawioctl.o
81 1728 0 0 1728 6c0 libcli/raw/rawnegotiate.o
82 723 0 0 723 2d3 libcli/raw/rawnotify.o
83 3779 0 0 3779 ec3 libcli/raw/rawreadwrite.o
84 6597 0 0 6597 19c5 libcli/raw/rawrequest.o
85 5580 0 0 5580 15cc libcli/raw/rawsearch.o
86 3034 0 0 3034 bda libcli/raw/rawsetfileinfo.o
87 5187 0 0 5187 1443 libcli/raw/rawtrans.o
88 2033 0 0 2033 7f1 libcli/raw/smb_signing.o
90 notice that the "data" and "bss" columns are all zero? That is
91 good. If there are any non-zero values in data or bss then that
92 indicates static data and is bad (as a rule of thumb).
94 Lets compare that result to the equivalent in Samba3:
96 text data bss dec hex filename
97 3978 0 0 3978 f8a libsmb/asn1.o
98 18963 0 288 19251 4b33 libsmb/cliconnect.o
99 2815 0 1024 3839 eff libsmb/clidgram.o
100 4038 0 0 4038 fc6 libsmb/clientgen.o
101 3337 664 256 4257 10a1 libsmb/clierror.o
102 10043 0 0 10043 273b libsmb/clifile.o
103 332 0 0 332 14c libsmb/clifsinfo.o
104 166 0 0 166 a6 libsmb/clikrb5.o
105 5212 0 0 5212 145c libsmb/clilist.o
106 1367 0 0 1367 557 libsmb/climessage.o
107 259 0 0 259 103 libsmb/clioplock.o
108 1584 0 0 1584 630 libsmb/cliprint.o
109 7565 0 256 7821 1e8d libsmb/cliquota.o
110 7694 0 0 7694 1e0e libsmb/clirap.o
111 27440 0 0 27440 6b30 libsmb/clirap2.o
112 2905 0 0 2905 b59 libsmb/clireadwrite.o
113 1698 0 0 1698 6a2 libsmb/clisecdesc.o
114 5517 0 0 5517 158d libsmb/clispnego.o
115 485 0 0 485 1e5 libsmb/clistr.o
116 8449 0 0 8449 2101 libsmb/clitrans.o
117 2053 0 4 2057 809 libsmb/conncache.o
118 3041 0 256 3297 ce1 libsmb/credentials.o
119 1261 0 1024 2285 8ed libsmb/doserr.o
120 14560 0 0 14560 38e0 libsmb/errormap.o
121 3645 0 0 3645 e3d libsmb/namecache.o
122 16815 0 8 16823 41b7 libsmb/namequery.o
123 1626 0 0 1626 65a libsmb/namequery_dc.o
124 14301 0 1076 15377 3c11 libsmb/nmblib.o
125 24516 0 2048 26564 67c4 libsmb/nterr.o
126 8661 0 8 8669 21dd libsmb/ntlmssp.o
127 3188 0 0 3188 c74 libsmb/ntlmssp_parse.o
128 4945 0 0 4945 1351 libsmb/ntlmssp_sign.o
129 1303 0 0 1303 517 libsmb/passchange.o
130 1221 0 0 1221 4c5 libsmb/pwd_cache.o
131 2475 0 4 2479 9af libsmb/samlogon_cache.o
132 10768 32 0 10800 2a30 libsmb/smb_signing.o
133 4524 0 16 4540 11bc libsmb/smbdes.o
134 5708 0 0 5708 164c libsmb/smbencrypt.o
135 7049 0 3072 10121 2789 libsmb/smberr.o
136 2995 0 0 2995 bb3 libsmb/spnego.o
137 3186 0 0 3186 c72 libsmb/trustdom_cache.o
138 1742 0 0 1742 6ce libsmb/trusts_util.o
139 918 0 28 946 3b2 libsmb/unexpected.o
141 notice all of the non-zero data and bss elements? Every bit of that
142 data is a bug waiting to happen.
144 Static data is evil as it has the following consequences:
145 - it makes code much less likely to be thread-safe
146 - it makes code much less likely to be recursion-safe
147 - it leads to subtle side effects when the same code is called from
150 Static data is particularly evil in library code (such as our internal
151 smb and rpc libraries). If you can get rid of all static data in
152 libraries then you can make some fairly strong guarantees about the
153 behaviour of functions in that library, which really helps.
155 Of course, it is possible to write code that uses static data and is
156 safe, it's just much harder to do that than just avoid static data in
157 the first place. We have been tripped up countless times by subtle
158 bugs in Samba due to the use of static data, so I think it is time to
159 start avoiding it in new code. Much of the core infrastructure of
160 Samba4 was specifically written to avoid static data, so I'm going to
161 be really annoyed if everyone starts adding lots of static data back
164 So, how do we avoid static data? The basic method is to use context
165 pointers. When reading the Samba4 code you will notice that just about
166 every function takes a pointer to a context structure as its first
167 argument. Any data that the function needs that isn't an explicit
168 argument to the function can be found by traversing that context.
170 Note that this includes all of the little caches that we have lying
171 all over the code in Samba3. I'm referring to the ones that generally
172 have a "static int initialised" and then some static string or integer
173 that remembers the last return value of the function. Get rid of them!
174 If you are *REALLY* absolutely completely certain that your personal
175 favourite mini-cache is needed then you should do it properly by
176 putting it into the appropriate context rather than doing it the lazy
177 way by putting it inside the target function. I would suggest however
178 that the vast majority of those little caches are useless - don't
179 stick it in unless you have really firm benchmarking results that show
180 that it is needed and helps by a significant amount.
182 Note that Samba4 is not yet completely clean of static data like
183 this. I've gotten the smbd/ directory down to 24 bytes of static data,
184 and libcli/raw/ down to zero. I've also gotten the ntvfs layer and all
185 backends down to just 8 bytes in ntvfs_base.c. The rest still needs
188 Also note that truly constant data is OK, and will not in fact show up
189 in the data and bss columns in "size" anyway (it will be included in
190 "text"). So you can have constant tables of protocol data.
196 We introduced the talloc() system for memory contexts during the 2.2
197 development cycle and it has been a great success. It has greatly
198 simplified a lot of our code, particularly with regard to error
201 In Samba4 we use talloc even more extensively to give us much finer
202 grained memory management. The really important thing to remember
203 about talloc in Samba4 is:
205 "don't just use the first talloc context that comes to hand - use
206 the RIGHT talloc context"
208 Just using the first talloc context that comes to hand is probably the
209 most common systematic bug I have seen so far from programmers that
210 have worked on the Samba4 code base. The reason this is vital is that
211 different talloc contexts have vastly different lifetimes, so if you
212 use a talloc context that has a long lifetime (such as one associated
213 with a tree connection) for data that is very short lived (such as
214 parsing an individual packet) then you have just introduced a huge
217 In fact, it is quite common that the correct thing to do is to create
218 a new talloc context for some little function and then destroy it when
219 you are done. That will give you a memory context that has exactly the
222 You should also go and look at a new talloc function in Samba4 called
223 talloc_steal(). By using talloc_steal() you can move a lump of memory
224 from one memory context to another without copying the data. This
225 should be used when a backend function (such as a packet parser)
226 produces a result as a lump of talloc memory and you need to keep it
227 around for a longer lifetime than the talloc context it is in. You
228 just "steal" the memory from the short-lived context, putting it into
229 your long lived context.
235 One of the biggest changes in Samba4 is the universal use of interface
236 structures. Go take a look through include/smb_interfaces.h now to get
237 an idea of what I am talking about.
239 In Samba3 many of the core wire structures in the SMB protocol were
240 never explicitly defined in Samba. Instead, our parse and generation
241 functions just worked directly with wire buffers. The biggest problem
242 with this is that is tied our parse code with out "business logic"
243 much too closely, which meant the code got extremely confusing to
246 In Samba4 we have explicitly defined interface structures for
247 everything in the protocol. When we receive a buffer we always parse
248 it completely into one of these structures, then we pass a pointer to
249 that structure to a backend handler. What we must *not* do is make any
250 decisions about the data inside the parse functions. That is critical
251 as different backends will need different portions of the data. This
252 leads to a golden rule for Samba4:
254 "don't design interfaces that lose information"
256 In Samba3 our backends often received "condensed" versions of the
257 information sent from clients, but this inevitably meant that some
258 backends could not get at the data they needed to do what they wanted,
259 so from now on we should expose the backends to all of the available
260 information and let them choose which bits they want.
262 Ok, so now some of you will be thinking "this sounds just like our
263 msrpc code from Samba3", and while to some extent this is true there
264 are extremely important differences in the approach that are worth
267 In the Samba3 msrpc code we used explicit parse strucrures for all
268 msrpc functions. The problem is that we didn't just put all of the
269 real variables in these structures, we also put in all the artifacts
270 as well. A good example is the security descriptor strucrure that
271 looks like this in Samba3:
273 typedef struct security_descriptor_info
278 uint32 off_owner_sid;
289 The problem with this structure is all the off_* variables. Those are
290 not part of the interface, and do not appear in any real descriptions
291 of Microsoft security descriptors. They are parsing artifacts
292 generated by the IDL compiler that Microsoft use. That doesn't mean
293 they aren't needed on the wire - indeed they are as they tell the
294 parser where to find the following four variables, but they should
295 *NOT* be in the interface structure.
297 In Samba3 there were unwritten rules about which variables in a
298 strucrure a high level caller has to fill in and which ones are filled
299 in by the marshalling code. In Samba4 those rules are gone, because
300 the redundent artifact variables are gone. The high level caller just
301 sets up the real variables and the marshalling code worries about
302 generating the right offsets.
304 The same rule applies to strings. In many places in the SMB and MSRPC
305 protocols complex strings are used on the wire, with complex rules
306 about padding, format, alighment, termination etc. None of that
307 information is useful to a high level calling routine or to a backend
308 - its all just so much wire fluff. So, in Samba4 these strings are
309 just "char *" and are always in our internal multi-byte format (which
310 is usually UTF8). It is up to the parse functions to worry about
311 translating the format and getting the padding right.
313 The one exception to this is the use of the WIRE_STRING type, but that
314 has a very good justification in terms of regression testing. Go and
315 read the comment in smb_interfaces.h about that now.
317 So, here is another rule to code by. When writing an interface
318 structure think carefully about what variables in the structure can be
319 left out as they are redundent. If some length is effectively defined
320 twice on the wire then only put it once in the packet. If a length can
321 be inferred from a null termination then do that and leave the length
322 out of the structure completely. Don't put redundent stuff in
329 Samba4 has an asynchronous design. That affects *lots* of the code,
330 and the implications of the asynchronous design needs to be considered
331 just about everywhere.
333 The first aspect of the async design to look at is the SMB client
334 library. Lets take a look at the following three functions in
335 libcli/raw/rawfile.c:
337 struct cli_request *smb_raw_seek_send(struct cli_tree *tree, struct smb_seek *parms);
338 NTSTATUS smb_raw_seek_recv(struct cli_request *req, struct smb_seek *parms);
339 NTSTATUS smb_raw_seek(struct cli_tree *tree, struct smb_seek *parms);
341 Go and read them now then come back.
343 Ok, first notice there there are 3 separate functions, whereas the
344 equivalent code in Samba3 had just one. Also note that the 3rd
345 function is extremely simple - its just a wrapper around calling the
348 The three separate functions are needed because we need to be able to
349 generate SMB calls asynchronously. The first call, which for smb calls
350 is always called smb_raw_XXXX_send(), constructs and sends a SMB
351 request and returns a "struct cli_request" which acts as a handle for
352 the request. The caller is then free to do lots of other calls if it
353 wants to, then when it is ready it can call the smb_raw_XXX_recv()
354 function to receive the reply.
356 If all you want is a synchronous call then call the 3rd interface, the
357 one called smb_raw_XXXX(). That just calls the first two in order, and
358 blocks waiting for the reply.
360 But what if you want to be called when the reply comes in? Yes, thats
361 possible. You can do things like this:
363 struct cli_request *req;
365 req = smb_raw_XXX_send(tree, params);
367 req->async.fn = my_callback;
368 req->async.private = my_private_data;
370 then in your callback function you can call the smb_raw_XXXX_recv()
371 function to receive the reply. Your callback will receive the "req"
372 pointer, which you can use to retrieve your private data from
375 Then all you need to do is ensure that the main loop in the client
376 library gets called. You can either do that by polling the connection
377 using cli_transport_pending() and cli_request_receive_next() or you
378 can use transport->idle.func to setup an idle function handler to call
379 back to your main code. Either way, you can build a fully async
382 In order to support all of this we have to make sure that when we
383 write a piece of library code (SMB, MSRPC etc) that we build the
384 separate _send() and _recv() functions. It really is worth the effort.
386 Now about async in smbd, a much more complex topic.
388 The SMB protocol is inherently async. Some functions (such as change
389 notify) often don't return for hours, while hundreds of other
390 functions pass through the socket. Take a look at the RAW-MUX test in
391 the Samba4 smbtorture to see some really extreme examples of the sort
392 of async operations that Windows supports. I particularly like the
393 open/open/close sequence where the 2nd open (which conflicts with the
394 first) succeeds because the subsequent close is answered out of order.
396 In Samba3 we handled this stuff very badly. We had awful "pending
397 request" queues that allocated full 128k packet buffers, and even with
398 all that crap we got the semantics wrong. In Samba4 I intend to make
399 sure we get this stuff right.
401 So, how do we do this? We now have an async interface between smbd and
402 the NTVFS backends. Whenever smbd calls into a backend the backend has
403 an option of answer the request in a synchronous fashion if it wants
404 to just like in Samba3, but it also has the option of answering the
405 request asynchronously. The only backend that currently does this is
406 the CIFS backend, but I hope the other backends will soon do this to.
408 To make this work you need to do things like this in the backend:
410 req->control_flags |= REQ_CONTROL_ASYNC;
412 that tells smbd that the backend has elected to reply later rather
413 than replying immediately. The backend must *only* do this if
414 req->async.send_fn is not NULL. If send_fn is NULL then it means that
415 the smbd front end cannot handle this function being replied to in an
418 If the backend does this then it is up to the backend to call
419 req->async.send_fn() when it is ready to reply. It the meantime smbd
420 puts the call on hold and goes back to answering other requests on the
423 Inside smbd you will find that there is code to support this. The most
424 obvious change is that smbd splits each SMB reply function into two
425 parts - just like the client library has a _send() and _recv()
426 function, so smbd has a _send() function and the parse function for
429 As an example go and have a look at reply_getatr_send() and
430 reply_getatr() in smbd/reply.c. Read them? Good.
432 Notice that reply_getatr() sets up the req->async structure to contain
433 the send function. Thats how the backend gets to do an async reply, it
434 calls this function when it is ready. Also notice that reply_getatr()
435 only does the parsing of the request, and does not do the reply
436 generation. That is done by the _send() function.
438 The only missing piece in the Samba4 right now that prevents it being
439 fully async is that it currently does the low level socket calls (read
440 and write on sockets) in a blocking fashion. It does use select() to
441 make it somewhat async, but if a client were to send a partial packet
442 then delay before sending the rest then smbd would be stuck waiting
443 for the second half of the packet.
445 To fix this I plan on making the socket calls async as well, which
446 luckily will not involve any API changes in the core of smbd or the
447 library. It just involves a little bit of extra code in clitransport.c
448 and smbd/request.c. As a side effect I hope that this will also reduce
449 the average number of system calls required to answer a request, so we
450 may see a performance improvement.
456 One of the most noticeable changes in Samba4 is the introduction of
457 the NTVFS layer. This provided the initial motivation for the design
458 of Samba4 and in many ways lies at the heart of the design.
460 In Samba3 the main file serving process (smbd) combined the handling
461 of the SMB protocol with the mapping to POSIX semantics in the same
462 code. If you look in smbd/reply.c in Samba3 you see numerous places
463 where POSIX assumptions are mixed tightly with SMB parsing code. We
464 did have a VFS layer in Samba3, but it was a POSIX-like VFS layer, so
465 no matter how you wrote a plugin you could not bypass the POSIX
466 mapping decisions that had already been made before the VFS layer was
469 In Samba4 things are quite different. All SMB parsing is performed in
470 the smbd front end, then fully parsed requests are passed to the NTVFS
471 backend. That backend makes any semantic mapping decisions and fills
472 in the 'out' portion of the request. The front end is then responsible
473 for putting those results into wire format and sending them to the
476 Lets have a look at one of those request structures. Go and read the
477 definition of "union smb_write" and "enum write_level" in
478 include/smb_interfaces.h. (no, don't just skip reading it, really go
479 and read it. Yes, that means you!).
481 Notice the union? That's how Samba4 allows a single NTVFS backend
482 interface to handle the several different ways of doing a write
483 operation in the SMB protocol. Now lets look at one section of that
486 /* SMBwriteX interface */
488 enum write_level level;
504 see the "in" and "out" sections? The "in" section is for parameters
505 that the SMB client sends on the wire as part of the request. The smbd
506 front end parse code parses the wire request and fills in all those
507 parameters. It then calls the NTVFS interface which looks like this:
509 NTSTATUS (*write)(struct request_context *req, union smb_write *io);
511 and the NTVFS backend does the write request. The backend then fills
512 in the "out" section of the writex structure and gives the union back
513 to the front end (either by returning, or if done in an async fashion
514 then by calling the async send function. See the async discussion
515 elsewhere in this document).
517 The NTVFS backend knows which particular function is being requested
518 by looking at io->generic.level. Notice that this enum is also
519 repeated inside each of the sub-structures in the union, so the
520 backend could just as easily look at io->writex.level and would get
523 Notice also that some levels (such as splwrite) don't have an "out"
524 section. This happens because there is no return value apart from a
525 status code from those SMB calls.
527 So what about status codes? The status code is returned directly by
528 the backend NTVFS interface when the call is performed
529 synchronously. When performed asynchronously then the status code is
530 put into req->async.status before the req->async.send_fn() callback is
533 Currently the most complete NTVFS backend is the CIFS backend. I don't
534 expect this backend will be used much in production, but it does
535 provide the ideal test case for our NTVFS design. As it offers the
536 full capabilities that are possible with a CIFS server we can be sure
537 that we don't have any gaping holes in our APIs, and that the front
538 end code is flexible enough to handle any advances in the NT style
539 feature sets of Unix filesystems that make come along.
545 In Samba3 we supported just one process model. It just so happens that
546 the process model that Samba3 supported is the "right" one for most
547 users, but there are situations where this model wasn't ideal.
549 In Samba4 you can choose the smbd process model on the smbd command
561 - command line handling
572 GMT vs TZ in printout of QFILEINFO timezones
574 put in full UNC path in tconx
576 test timezone handling by using a server in different zone from client
578 don't just use any old TALLOC_CTX, use the right one!
580 do {} while (0) system
582 NT_STATUS_IS_OK() is NOT the opposite of NT_STATUS_IS_ERR()
584 need to implement secondary parts of trans2 and nttrans in server and
587 add talloc_steal() to move a talloc ptr from one pool to another
589 document access_mask in openx reply
591 check all capabilities and flag1, flag2 fields (eg. EAs)
593 large files -> pass thru levels
595 setpathinfo is very fussy about null termination of the file name
597 the overwrite flag doesn't seem to work on setpathinfo RENAME_INFORMATION
599 END_OF_FILE_INFORMATION and ALLOCATION_INFORMATION don't seem to work
602 on w2k3 setpathinfo DISPOSITION_INFORMATION fails, but does have an
603 effect. It leaves the file with SHARING_VIOLATION.
605 on w2k3 trans2 setpathinfo with any invalid low numbered level causes
606 the file to get into a state where DELETE_PENDING is reported, and the
607 file cannot be deleted until you reboot
609 trans2 qpathinfo doesn't see the delete_pending flag correctly, but
612 get rid of pstring, fstring, strtok
614 add programming documentation note about lp_set_cmdline()
616 need to add a wct checking function in all client parsing code,
617 similar to REQ_CHECK_WCT()
619 need to make sure that NTTIME is a round number of seconds when
620 converted from time_t
622 not using a zero next offset in SMB_FILE_STREAM_INFORMATION for last
623 entry causes explorer exception under win2000
626 if the server sets the session key the same for a second SMB socket as
627 an initial socket then the client will not re-authenticate, it will go
628 straight to a tconx, skipping session setup and will use all the
629 existing parameters! This allows two sockets with the same keys!?
632 removed blocking lock code, we now queue the whole request the same as
633 we queue any other pending request. This allows for things like a
634 close() while a pending blocking lock is being processed to operate
637 disabled change notify code
647 client library and test code
648 ----------------------------
650 convert client library to new structure
651 get smbtorture working
652 get smbclient working
653 expand client library for all requests
654 write per-request test suite
655 gentest randomised test suite
656 separate client code as a library for non-Samba use
660 add remaining core SMB requests
664 fix auth models (share, server, rpc)
665 get net command working
666 connect CIFS backend to server level auth
669 reconnect printing code
670 restore removed smbd options
671 add smb.conf macro substitution code
672 add async backend notification
673 add generic timer event mechanism
679 new server models (break 1-1)
680 test clustered models
681 add fulcrum statistics gathering