From c4c427576c02b27d829ae4aaee31cbf893b4a2ad Mon Sep 17 00:00:00 2001 From: Andrew Tridgell Date: Tue, 28 Sep 2004 11:04:55 +0000 Subject: [PATCH] r2716: created a separate detailed talloc_guide.txt document, after volker complained it was all too confusing :-) I recommend that everyone who wants to work on Samba4 have a read of this. --- prog_guide.txt | 61 +------- talloc_guide.txt | 361 +++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 365 insertions(+), 57 deletions(-) create mode 100644 talloc_guide.txt diff --git a/prog_guide.txt b/prog_guide.txt index 4bf8671e600..31a9fb88616 100644 --- a/prog_guide.txt +++ b/prog_guide.txt @@ -191,54 +191,8 @@ in the data and bss columns in "size" anyway (it will be included in How to use talloc ----------------- -If you are used to talloc from Samba3 then please read this carefully, -as talloc has changed rather a lot. - -The new talloc is a hierarchical, reference counted memory pool system -with destructors. Quite a mounthful really, but not too bad once you -get used to it. - -Perhaps the biggest change from Samba3 is that there is no distinction -between a "talloc context" and a "talloc pointer". Any pointer -returned from talloc() is itself a valid talloc context. This means -you can do this: - - struct foo *a = talloc(mem_ctx, sizeof(*s)); - a->name = talloc(a, strlen("foo")+1); - -and the pointer a->name would be a "child" of the talloc context "a" -which is itself a child of mem_ctx. So if you do talloc_free(mem_ctx) -then it is all destroyed, whereas if you do talloc_free(a) then just a -and a->name are destroyed. - -If you think about this, then what this effectively gives you is an -n-ary tree, where you can free any part of the tree with -talloc_free(). - -The next big change with the new talloc is reference counts. A talloc -pointer starts with a reference count of 1. You can call -talloc_increase_ref_count() on any talloc pointer and that increases -the reference count by 1. If you then call talloc_free() on a pointer -that has a reference count greater than 1, then the reference count is -decreased, but the memory is not released. - -Finally, talloc now has destructors. You can set a destructor on any -talloc pointer using talloc_set_destructor(). Your destructor will -then be called before the memory is released. An interesting feature -of these destructors is that they can return a error. If the -destructor returns -1 then that is interpreted as a refusal to release -the memory, and the talloc_free() will return. It will also prevent -the release of all memory "below" that memory in the tree. - -You should also go and look at a new talloc function in Samba4 called -talloc_steal(). By using talloc_steal() you can move a lump of memory -from one memory context to another without copying the data. This -should be used when a backend function (such as a packet parser) -produces a result as a lump of talloc memory and you need to keep it -around for a longer lifetime than the talloc context it is in. You -just "steal" the memory from the short-lived context, putting it into -your long lived context. - +Please see the separate document, talloc_guide.txt in this +directory. You _must_ read this if you want to program in Samba4. Interface Structures -------------------- @@ -583,9 +537,11 @@ other recognised flags are: sign : enable ntlmssp signing seal : enable ntlmssp sealing + connect : enable rpc connect level auth (auth, but no sign or seal) validate: enable the NDR validator print: enable debugging of the packets bigendian: use bigendian RPC + padcheck: check reply data for non-zero pad bytes For example, these all connect to the samr pipe: @@ -645,8 +601,6 @@ MSRPC - msrpc -- use _p talloc varients - don't zero structures! avoid ZERO_STRUCT() and talloc_zero() @@ -656,8 +610,6 @@ put in full UNC path in tconx test timezone handling by using a server in different zone from client -don't just use any old TALLOC_CTX, use the right one! - do {} while (0) system NT_STATUS_IS_OK() is NOT the opposite of NT_STATUS_IS_ERR() @@ -665,8 +617,6 @@ NT_STATUS_IS_OK() is NOT the opposite of NT_STATUS_IS_ERR() need to implement secondary parts of trans2 and nttrans in server and client -add talloc_steal() to move a talloc ptr from one pool to another - document access_mask in openx reply check all capabilities and flag1, flag2 fields (eg. EAs) @@ -803,7 +753,6 @@ Ideas BUGS: - non-signed non-sealed RPC (level == 2 == "connect") add a test case for last_entry_offset in trans2 find interfaces conn refused connect -> errno @@ -814,5 +763,3 @@ BUGS: trans2 and other calls handle servers that don't have the setattre call in torture add max file coponent length test and max path len test - - diff --git a/talloc_guide.txt b/talloc_guide.txt new file mode 100644 index 00000000000..925f485997e --- /dev/null +++ b/talloc_guide.txt @@ -0,0 +1,361 @@ +Using talloc in Samba4 +---------------------- + +Andrew Tridgell +September 2004 + +The most current version of this document is available at + http://samba.org/ftp/unpacked/samba4/talloc_guide.txt + +If you are used to talloc from Samba3 then please read this carefully, +as talloc has changed a lot. + +The new talloc is a hierarchical, reference counted memory pool system +with destructors. Quite a mounthful really, but not too bad once you +get used to it. + +Perhaps the biggest change from Samba3 is that there is no distinction +between a "talloc context" and a "talloc pointer". Any pointer +returned from talloc() is itself a valid talloc context. This means +you can do this: + + struct foo *X = talloc_p(mem_ctx, struct foo); + X->name = talloc_strdup(X, "foo"); + +and the pointer X->name would be a "child" of the talloc context "X" +which is itself a child of mem_ctx. So if you do talloc_free(mem_ctx) +then it is all destroyed, whereas if you do talloc_free(X) then just X +and X->name are destroyed, and if you do talloc_free(X->name) then +just the name element of X is destroyed. + +If you think about this, then what this effectively gives you is an +n-ary tree, where you can free any part of the tree with +talloc_free(). + +If you find this confusing, then I suggest you run the LOCAL-TALLOC +smbtorture test with the --leak-report-full option to watch talloc in +action. You may also like to add your own tests to +source/torture/local/talloc.c to clarify how some particular situation +is handled. + + +talloc API +---------- + +The following is a complete guide to the talloc API. Read it all at +least twice. + + +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- +void *talloc(const void *context, size_t size); + +The talloc() function is the core of the talloc library. It takes a +memory context, and returns a pointer to a new area of memory of the +given size. + +The returned pointer is itself a talloc context, so you can use it as +the context argument to more calls to talloc if you wish. + +The returned pointer is a "child" of the supplied context. This means +that if you talloc_free() the context then the new child disappears as +well. Alternatively you can free just the child. + +The context argument to talloc() can be NULL, in which case a new top +level context is created. + + +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- +int talloc_free(void *ptr); + +The talloc_free() function frees a piece of talloc memory, and all its +children. You can call talloc_free() on any pointer returned by +talloc(). + +The return value of talloc_free() indicates success or failure, with 0 +returned for success and -1 for failure. The only possible failure +condition is if the pointer had a destructor attached to it and the +destructor returned -1. See talloc_set_destructor() for details on +destructors. + +If this pointer has an additional reference when talloc_free() is +called then the memory is not actually released, but instead the +reference is destroyed and the memory becomes a child of the +referrer. See talloc_reference() for details on establishing +additional references. + +talloc_free() operates recursively on its children. + + +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- +void *talloc_reference(const void *context, const void *ptr); + +The talloc_reference() function returns an additional reference to +"ptr", and makes this additional reference a child of "context". + +The return value of talloc_reference() is always the original pointer +"ptr", unless talloc ran out of memory in creating the reference in +which case it will return NULL (each additional reference consumes +around 48 bytes of memory on intel x86 platforms). + +After creating a reference you can free it in one of the following +ways: + + - you can talloc_free() a parent of the original pointer. That will + destroy the reference and make the pointer a child of "context". + + - you can talloc_free() the pointer itself. That will destroy the + reference and make the pointer a child of "context". + + - you can talloc_free() the context where you placed the + reference. That will destroy the reference, and leave the pointer + where it is. + + +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- +void talloc_set_destructor(const void *ptr, int (*destructor)(void *)); + +The function talloc_set_destructor() sets the "destructor" for the +pointer "ptr". A destructor is a function that is called when the +memory used by a pointer is about to be released. The destructor +receives the pointer as an argument, and should return 0 for success +and -1 for failure. + +The destructor can do anything it wants to, including freeing other +pieces of memory. A common use for destructors is to clean up +operating system resources (such as open file descriptors) contained +in the structure the destructor is placed on. + +You can only place one destructor on a pointer. If you need more than +one destructor then you can create a zero-length child of the pointer +and place an additional destructor on that. + +To remove a destructor call talloc_set_destructor() with NULL for the +destructor. + +If your destructor attempts to talloc_free() the pointer that it is +the destructor for then talloc_free() will return -1 and the free will +be ignored. This would be a pointless operation anyway, as the +destructor is only called when the memory is just about to go away. + + +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- +void talloc_increase_ref_count(const void *ptr); + +The talloc_increase_ref_count(ptr) function is exactly equivalent to: + + talloc_reference(NULL, ptr); + +You can use either syntax, depending on which you think is clearer in +your code. + + +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- +void talloc_set_name(const void *ptr, const char *fmt, ...); + +Each talloc pointer has a "name". The name is used principally for +debugging purposes, although it is also possible to set and get the +name on a pointer in as a way of "marking" pointers in your code. + +The main use for names on pointer is for "talloc reports". See +talloc_report() and talloc_report_full() for details. Also see +talloc_enable_leak_report() and talloc_enable_leak_report_full(). + + +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- +void talloc_set_name_const(const void *ptr, const char *name); + +The function talloc_set_name_const() is just like talloc_set_name(), +but it takes a string constant, and is much faster. It is extensively +used by the "auto naming" macros, such as talloc_p(). + + +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- +void *talloc_named(const void *context, size_t size, const char *fmt, ...); + +The talloc_named() function creates a named talloc pointer. It is +equivalent to: + + ptr = talloc(context, size); + talloc_set_name(ptr, fmt, ....); + + +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- +void *talloc_named_const(const void *context, size_t size, const char *name); + +This is equivalent to: + + ptr = talloc(context, size); + talloc_set_name_const(ptr, name); + + +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- +const char *talloc_get_name(const void *ptr); + +This returns the current name for the given talloc pointer. See +talloc_set_name() for details. + + +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- +void *talloc_init(const char *fmt, ...); + +This function creates a zero length named talloc context as a top +level context. It is equivalent to: + + talloc_named(NULL, 0, fmt, ...); + + +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- +void *talloc_realloc(const void *context, void *ptr, size_t size); + +The talloc_realloc() function changes the size of a talloc +pointer. It has the following equivalences: + + talloc_realloc(context, NULL, size) ==> talloc(context, size); + talloc_realloc(context, ptr, 0) ==> talloc_free(ptr); + +The "context" argument is only used if "ptr" is not NULL, otherwise it +is ignored. + +talloc_realloc() returns the new pointer, or NULL on failure. The call +will fail either due to a lack of memory, or because the pointer has +an reference (see talloc_reference()). + + +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- +void *talloc_steal(const void *new_ctx, const void *ptr); + +The talloc_steal() function changes the parent context of a talloc +pointer. It is typically used when the context that the pointer is +currently a child of is going to be freed and you wish to keep the +memory for a longer time. + +The talloc_steal() function returns the pointer that you pass it. It +does not have any failure modes. + +NOTE: It is possible to produce loops in the parent/child relationship +if you are not careful with talloc_steal(). No guarantees are provided +as to your sanity or the safety of your data if you do this. + + +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- +off_t talloc_total_size(const void *ptr); + +The talloc_total_size() function returns the total size in bytes used +by this pointer and all child pointers. Mostly useful for debugging. + + +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- +void talloc_report(const void *ptr, FILE *f); + +The talloc_report() function prints a summary report of all memory +used by ptr. One line of report is printed for each immediate child of +ptr, showing the total memory and number of blocks used by that child. + +You can pass NULL for the pointer, in which case a report is printed +for the top level memory context. + + +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- +void talloc_report_full(const void *ptr, FILE *f); + +This provides a more detailed report than talloc_report(). It will +recursively print the ensire tree of memory referenced by the +pointer. References in the tree are shown by giving the name of the +pointer that is referenced. + +You can pass NULL for the pointer, in which case a report is printed +for the top level memory context. + + +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- +void talloc_enable_leak_report(void); + +This enables calling of talloc_report(NULL, stderr) when the program +exits. In Samba4 this is enabled by using the --leak-report command +line option. + + +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- +void talloc_enable_leak_report_full(void); + +This enables calling of talloc_report_full(NULL, stderr) when the +program exits. In Samba4 this is enabled by using the +--leak-report-full command line option. + + +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- +void *talloc_zero(const void *ctx, size_t size); + +The talloc_zero() function is equivalent to: + + ptr = talloc(ctx, size); + if (ptr) memset(ptr, 0, size); + + +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- +void *talloc_memdup(const void *ctx, const void *p, size_t size); + +The talloc_memdup() function is equivalent to: + + ptr = talloc(ctx, size); + if (ptr) memcpy(ptr, p, size); + + +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- +char *talloc_strdup(const void *ctx, const char *p); + +The talloc_strdup() function is equivalent to: + + ptr = talloc(ctx, strlen(p)+1); + if (ptr) memcpy(ptr, p, strlen(p)+1); + + +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- +char *talloc_strndup(const void *t, const char *p, size_t n); + +The talloc_strndup() function is the talloc equivalent of the C +library function strndup() + + +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- +char *talloc_vasprintf(const void *t, const char *fmt, va_list ap); + +The talloc_vasprintf() function is the talloc equivalent of the C +library function vasprintf() + + +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- +char *talloc_asprintf(const void *t, const char *fmt, ...); + +The talloc_asprintf() function is the talloc equivalent of the C +library function asprintf() + + +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- +char *talloc_asprintf_append(char *s, const char *fmt, ...); + +The talloc_asprintf_append() function appends the given formatted +string to the given string. + + +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- +void *talloc_array_p(const void *ctx, type, uint_t count); + +The talloc_array_p() macro is equivalent to: + + (type *)talloc(ctx, sizeof(type) * count); + +except that it provides integer overflow protection for the multiply, +returning NULL if the multiply overflows. + + +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- +void *talloc_realloc_p(const void *ctx, void *ptr, type, uint_t count); + +The talloc_realloc_p() macro is equivalent to: + + (type *)talloc_realloc(ctx, ptr, sizeof(type) * count); + +except that it provides integer overflow protection for the multiply, +returning NULL if the multiply overflows. + -- 2.34.1