5 NB: Wmem still does not provide all of the functionality of emem
6 (see README.malloc), although it should provide most of it. New code
7 may still need to use emem for the time being.
9 The 'emem' memory manager (described in README.malloc) has been a part of
10 Wireshark since 2005 and has served us well, but is starting to show its age.
11 The framework has become increasingly difficult to maintain, and limitations
12 in the API have blocked progress on other long-term goals such as multi-
13 threading, and opening multiple files at once.
15 The 'wmem' memory manager is an attempt to write a new memory management
16 framework to replace emem. It provides a significantly updated API, a more
17 modular design, and it isn't all jammed into one 2500-line file.
19 Wmem was originally conceived in this email to the wireshark-dev mailing list:
20 https://www.wireshark.org/lists/wireshark-dev/201210/msg00178.html
22 The wmem code can now be found in epan/wmem/ in the Wireshark source tree.
24 2. Usage for Consumers
26 If you're writing a dissector, or other "userspace" code, then using wmem
27 should be very similar to using emem. All you need to do is include the header
28 (epan/wmem/wmem.h) and get a handle to a memory pool (if you want to *create*
29 a memory pool, see the section "3. Usage for Producers" below).
31 A memory pool is an opaque pointer to an object of type wmem_allocator_t, and
32 it is the very first parameter passed to almost every call you make to wmem.
33 Other than that parameter (and the fact that functions are prefixed wmem_
34 instead of ep_ or se_) usage is exactly like that of emem. For example:
36 wmem_alloc(myPool, 20);
38 allocates 20 bytes in the pool pointed to by myPool.
42 2.1.1 (Sort Of) Global Pools
44 Dissectors that include the wmem header file will have three pools available
45 to them automatically: wmem_packet_scope(), wmem_file_scope() and
48 The packet pool is scoped to the dissection of each packet, replacing
49 emem's ep_ allocators. The file pool is scoped to the dissection of each file,
50 replacing emem's se_ allocators. For example:
53 se_malloc(sizeof(guint));
55 could be replaced with
57 wmem_alloc(wmem_packet_scope(), 32);
58 wmem_alloc(wmem_file_scope(), sizeof(guint));
60 NB: Using these pools outside of the appropriate scope (eg using the packet
61 pool when there isn't a packet being dissected) will throw an assertion.
62 See the comment in epan/wmem/wmem_scopes.c for details.
64 The epan pool is scoped to the library's lifetime - memory allocated in it is
65 not freed until epan_cleanup() is called, which is typically at the end of the
70 Certain places (such as AT_STRINGZ address allocations) need their memory to
71 stay around a little longer than the usual packet scope - basically until the
72 next packet is dissected. This is effectively the scope of Wireshark's pinfo
73 structure, so the pinfo struct has a 'pool' member which is a wmem pool scoped
74 to the lifetime of the pinfo struct.
98 2.5 Singly-Linked List
104 - wmem_slist_frame_next
105 - wmem_slist_frame_data
117 - wmem_strbuf_sized_new
119 - wmem_strbuf_append_printf
120 - wmem_strbuf_get_str
121 - wmem_strbuf_get_len
123 3. Usage for Producers
125 NB: If you're just writing a dissector, you probably don't need to read
128 One of the problems with the old emem framework was that there were basically
129 two allocator backends (glib and mmap) that were all mixed together in a mess
130 of if statements, environment variables and #ifdefs. In wmem the different
131 allocator backends are cleanly separated out, and it's up to the owner of the
134 3.1 Available Allocator Back-Ends
136 Each available allocator type has a corresponding entry in the
137 wmem_allocator_type_t enumeration defined in wmem_core.h.
139 The currently available allocators are:
140 - WMEM_ALLOCATOR_SIMPLE (wmem_allocator_simple.*)
141 A trivial allocator that g_allocs requested memory and tracks
142 allocations via a simple linked list.
143 - WMEM_ALLOCATOR_BLOCK (wmem_allocator_block.*)
144 A block allocator that grabs large chunks of memory at a time
145 (8 MB currently) and serves allocations out of those chunks.
146 - WMEM_ALLOCATOR_STRICT (wmem_allocator_strict.*)
147 An allocator that does its best to find invalid memory usage via
148 things like canaries and scrubbing freed memory. Valgrind is the
149 better choice on platforms that support it.
153 To create a pool, include the regular wmem header and call the
154 wmem_allocator_new() function with the appropriate type value.
157 #include "wmem/wmem.h"
159 wmem_allocator_t *myPool;
160 myPool = wmem_allocator_new(WMEM_ALLOCATOR_SIMPLE);
162 From here on in, you don't need to remember which type of allocator you used
163 (although allocator authors are welcome to expose additional allocator-specific
164 helper functions in their headers). The "myPool" variable can be passed around
165 and used as normal in allocation requests as described in section 2 of this
168 Note that the type of pool you request won't always be the type you get - the
169 WIRESHARK_DEBUG_WMEM_OVERRIDE environment variable (if set) can be used to
170 force all calls to wmem_allocator_new() to returns a specific type for debugging
171 purposes. It will always be safe to call allocator-specific helpers though, they
172 will simply be no-ops if the type doesn't match.
174 3.3 Destroying a Pool
176 Regardless of which allocator you used to create a pool, it can be destroyed
177 with a call to the function wmem_destroy_allocator(). For example:
179 #include "wmem/wmem.h"
181 wmem_allocator_t *myPool;
183 myPool = wmem_allocator_new(WMEM_ALLOCATOR_SIMPLE);
185 /* Allocate some memory in myPool ... */
187 wmem_destroy_allocator(myPool);
189 Destroying a pool will free all the memory allocated in it.
193 It is possible to free all the memory in a pool without destroying it,
194 allowing it to be reused later. Depending on the type of allocator, doing this
195 (by calling wmem_free_all()) can be significantly cheaper than fully destroying
196 and recreating the pool. This method is therefore recommended, especially when
197 the pool would otherwise be scoped to a single iteration of a loop. For example:
199 #include "wmem/wmem.h"
201 wmem_allocator_t *myPool;
203 myPool = wmem_allocator_new(WMEM_ALLOCATOR_SIMPLE);
206 /* Allocate some memory in myPool ... */
208 /* Free the memory, faster than destroying and recreating
209 the pool each time through the loop. */
210 wmem_free_all(myPool);
212 wmem_destroy_allocator(myPool);
216 Despite being written in Wireshark's standard C90, wmem follows a fairly
217 object-oriented design pattern. Although efficiency is always a concern, the
218 primary goals in writing wmem were maintainability, and preventing memory
221 4.1 struct _wmem_allocator_t
223 The heart of wmem is the _wmem_allocator_t structure defined in the
224 wmem_allocator.h header file. This structure uses C function pointers to
225 implement a common object-oriented pattern known as an interface (AKA 'abstract
226 class' to those of you who are more familiar with C++).
228 Different allocator implementations can provide exactly the same interface by
229 assigning their own functions (of the appropriate signature) to the members of
230 an instance of the structure. The structure currently has five values:
238 The private_data pointer is a void pointer that the implementation can use to
239 store whatever internal structures it needs. The type field is an enumeration of
240 type wmem_allocator_type_t (see section 3.1) set by the wmem_allocator_new()
241 function and not by the implementation. The three function pointers should
242 be fairly obvious - each one is called with a pointer to the private_data in
243 addition to any other arguments.
245 4.2 Pool-Agnostic API
247 One of the issues with emem was that the API (including the public data
248 structures) required wrapper functions for each scope implemented. Even
249 if there was a stack implementation in emem, it wasn't necessarily available
250 for use with file-scope memory unless someone took the time to write se_stack_
251 wrapper functions for the interface.
253 In wmem, all public APIs take the pool as the first argument, so that they can
254 be written once and used with any available memory pool. Data structures like
255 wmem's stack implementation only take the pool when created - the provided
256 pointer is stored internally with the data structure, and subsequent calls
257 (like push and pop) will take the stack itself instead of the pool.
261 The following is a list of things that wmem provides but are incomplete
262 (i.e. missing common operations):
267 The following is an incomplete list of things that emem provides but wmem has
273 The following is a list of things that emem doesn't provide but that it might
274 be nice if wmem did provide them:
281 * Editor modelines - http://www.wireshark.org/tools/modelines.html
286 * indent-tabs-mode: nil
289 * vi: set shiftwidth=4 tabstop=8 expandtab:
290 * :indentSize=4:tabSize=8:noTabs=true: