5 NB: Wmem still does not provide all of the functionality of emem
6 (see README.malloc), although it should provide most of it. New code
7 may still need to use emem for the time being.
9 The 'emem' memory manager (described in README.malloc) has been a part of
10 Wireshark since 2005 and has served us well, but is starting to show its age.
11 The framework has become increasingly difficult to maintain, and limitations
12 in the API have blocked progress on other long-term goals such as multi-
13 threading, and opening multiple files at once.
15 The 'wmem' memory manager is an attempt to write a new memory management
16 framework to replace emem. It provides a significantly updated API, a more
17 modular design, and it isn't all jammed into one 2500-line file.
19 Wmem was originally conceived in this email to the wireshark-dev mailing list:
20 https://www.wireshark.org/lists/wireshark-dev/201210/msg00178.html
22 The wmem code can now be found in epan/wmem/ in the Wireshark source tree.
24 2. Usage for Consumers
26 If you're writing a dissector, or other "userspace" code, then using wmem
27 should be very similar to using emem. All you need to do is include the header
28 (epan/wmem/wmem.h) and get a handle to a memory pool (if you want to *create*
29 a memory pool, see the section "3. Usage for Producers" below).
31 A memory pool is an opaque pointer to an object of type wmem_allocator_t, and
32 it is the very first parameter passed to almost every call you make to wmem.
33 Other than that parameter (and the fact that functions are prefixed wmem_
34 instead of ep_ or se_) usage is exactly like that of emem. For example:
36 wmem_alloc(myPool, 20);
38 allocates 20 bytes in the pool pointed to by myPool.
42 2.1.1 (Sort Of) Global Pools
44 Dissectors that include the wmem header file will have three pools available
45 to them automatically: wmem_packet_scope(), wmem_file_scope() and
48 The packet pool is scoped to the dissection of each packet, replacing
49 emem's ep_ allocators. The file pool is scoped to the dissection of each file,
50 replacing emem's se_ allocators. For example:
53 se_malloc(sizeof(guint));
55 could be replaced with
57 wmem_alloc(wmem_packet_scope(), 32);
58 wmem_alloc(wmem_file_scope(), sizeof(guint));
60 NB: Using these pools outside of the appropriate scope (eg using the packet
61 pool when there isn't a packet being dissected) will throw an assertion.
62 See the comment in epan/wmem/wmem_scopes.c for details.
64 The epan pool is scoped to the library's lifetime - memory allocated in it is
65 not freed until epan_cleanup() is called, which is typically at the end of the
70 Certain places (such as AT_STRINGZ address allocations) need their memory to
71 stay around a little longer than the usual packet scope - basically until the
72 next packet is dissected. This is effectively the scope of Wireshark's pinfo
73 structure, so the pinfo struct has a 'pool' member which is a wmem pool scoped
74 to the lifetime of the pinfo struct.
100 2.5 Singly-Linked List
106 - wmem_slist_frame_next
107 - wmem_slist_frame_data
113 - wmem_strbuf_sized_new
115 - wmem_strbuf_append_printf
116 - wmem_strbuf_get_str
117 - wmem_strbuf_get_len
119 3. Usage for Producers
121 NB: If you're just writing a dissector, you probably don't need to read
124 One of the problems with the old emem framework was that there were basically
125 two allocator backends (glib and mmap) that were all mixed together in a mess
126 of if statements, environment variables and #ifdefs. In wmem the different
127 allocator backends are cleanly separated out, and it's up to the owner of the
130 3.1 Available Allocator Back-Ends
132 Each available allocator type has a corresponding entry in the
133 wmem_allocator_type_t enumeration defined in wmem_core.h.
135 The currently available allocators are:
136 - WMEM_ALLOCATOR_SIMPLE (wmem_allocator_simple.*)
137 A trivial allocator that g_allocs requested memory and tracks
138 allocations via a GHashTable.
139 - WMEM_ALLOCATOR_BLOCK (wmem_allocator_block.*)
140 A block allocator that grabs large chunks of memory at a time
141 (8 MB currently) and serves allocations out of those chunks.
142 - WMEM_ALLOCATOR_STRICT (wmem_allocator_strict.*)
143 An allocator that does its best to find invalid memory usage via
144 things like canaries and scrubbing freed memory. Valgrind is the
145 better choice on platforms that support it.
149 To create a pool, include the regular wmem header and call the
150 wmem_allocator_new() function with the appropriate type value.
153 #include "wmem/wmem.h"
155 wmem_allocator_t *myPool;
156 myPool = wmem_allocator_new(WMEM_ALLOCATOR_SIMPLE);
158 From here on in, you don't need to remember which type of allocator you used
159 (although allocator authors are welcome to expose additional allocator-specific
160 helper functions in their headers). The "myPool" variable can be passed around
161 and used as normal in allocation requests as described in section 2 of this
164 Note that the type of pool you request won't always be the type you get - the
165 WIRESHARK_DEBUG_WMEM_OVERRIDE environment variable (if set) can be used to
166 force all calls to wmem_allocator_new() to returns a specific type for debugging
167 purposes. It will always be safe to call allocator-specific helpers though, they
168 will simply be no-ops if the type doesn't match.
170 3.3 Destroying a Pool
172 Regardless of which allocator you used to create a pool, it can be destroyed
173 with a call to the function wmem_destroy_allocator(). For example:
175 #include "wmem/wmem.h"
177 wmem_allocator_t *myPool;
179 myPool = wmem_allocator_new(WMEM_ALLOCATOR_SIMPLE);
181 /* Allocate some memory in myPool ... */
183 wmem_destroy_allocator(myPool);
185 Destroying a pool will free all the memory allocated in it.
189 It is possible to free all the memory in a pool without destroying it,
190 allowing it to be reused later. Depending on the type of allocator, doing this
191 (by calling wmem_free_all()) can be significantly cheaper than fully destroying
192 and recreating the pool. This method is therefore recommended, especially when
193 the pool would otherwise be scoped to a single iteration of a loop. For example:
195 #include "wmem/wmem.h"
197 wmem_allocator_t *myPool;
199 myPool = wmem_allocator_new(WMEM_ALLOCATOR_SIMPLE);
202 /* Allocate some memory in myPool ... */
204 /* Free the memory, faster than destroying and recreating
205 the pool each time through the loop. */
206 wmem_free_all(myPool);
208 wmem_destroy_allocator(myPool);
212 Despite being written in Wireshark's standard C90, wmem follows a fairly
213 object-oriented design pattern. Although efficiency is always a concern, the
214 primary goals in writing wmem were maintainability and preventing memory
217 4.1 struct _wmem_allocator_t
219 The heart of wmem is the _wmem_allocator_t structure defined in the
220 wmem_allocator.h header file. This structure uses C function pointers to
221 implement a common object-oriented design pattern known as an interface (also
222 known as an abstract class to those who are more familiar with C++).
224 Different allocator implementations can provide exactly the same interface by
225 assigning their own functions to the members of an instance of the structure.
226 The structure has eight members in three groups.
228 4.1.1 Implementation Details
233 The private_data pointer is a void pointer that the allocator implementation can
234 use to store whatever internal structures it needs. A pointer to private_data is
235 passed to almost all of the other functions that the allocator implementation
238 The type field is an enumeration of type wmem_allocator_type_t (see
239 section 3.1). Its value is set by the wmem_allocator_new() function, not
240 by the implementation-specific constructor. This field should be considered
241 read-only by the allocator implementation.
243 4.1.2 Consumer Functions
249 These function pointers should be set to functions with semantics obviously
250 similar to their standard-library namesakes. Each one takes an extra parameter
251 that is a copy of the allocator's private_data pointer.
253 Note that realloc() and free() are not expected to be called directly by user
254 code in most cases - they are primarily optimisations for use by data
255 structures that wmem might want to implement (it's hard, for example, to
256 implement a dynamically sized array without some form of realloc).
258 Also note that allocators do not have to handle NULL pointers or 0-length
259 requests in any way - those checks are done in an allocator-agnostic way
260 higher up in wmem. Allocator authors can assume that all incoming pointers
261 (to realloc and free) are non-NULL, and that all incoming lengths (to malloc
262 and realloc) are non-0.
264 4.1.3 Producer/Manager Functions
270 The free_all() function takes the private_data pointer and should free all the
271 memory currently allocated in the pool. Note that this is not necessarilly
272 exactly the same as calling free() on all the allocated blocks - free_all() is
273 allowed to do additional cleanup or to make use of optimizations not available
274 when freeing one block at a time.
276 The gc() function takes the private_data pointer and should do whatever it can
277 to reduce excess memory usage in the dissector by returning unused blocks to
278 the OS, optimizing internal data structures, etc.
280 The destroy() function does NOT take the private_data pointer - it instead takes
281 a pointer to the allocator structure as a whole, since that structure may also
282 need freeing. This function can assume that free_all() has been called
283 immediately before it (though it can make no assumptions about whether or not
284 gc() has ever been called).
286 4.2 Pool-Agnostic API
288 One of the issues with emem was that the API (including the public data
289 structures) required wrapper functions for each scope implemented. Even
290 if there was a stack implementation in emem, it wasn't necessarily available
291 for use with file-scope memory unless someone took the time to write se_stack_
292 wrapper functions for the interface.
294 In wmem, all public APIs take the pool as the first argument, so that they can
295 be written once and used with any available memory pool. Data structures like
296 wmem's stack implementation only take the pool when created - the provided
297 pointer is stored internally with the data structure, and subsequent calls
298 (like push and pop) will take the stack itself instead of the pool.
302 The following is a list of things that wmem provides but are incomplete
303 (i.e. missing common operations):
308 The following is an incomplete list of things that emem provides but wmem has
314 The following is a list of things that emem doesn't provide but that it might
315 be nice if wmem did provide them:
322 * Editor modelines - http://www.wireshark.org/tools/modelines.html
327 * indent-tabs-mode: nil
330 * vi: set shiftwidth=4 tabstop=8 expandtab:
331 * :indentSize=4:tabSize=8:noTabs=true: