Memory Allocators

malloc and Related Functions

The C library interfaces for memory allocation are provided by malloc, free and realloc, and the calloc function. In addition to these generic functions, there are derived functions such as strdup which perform allocation using malloc internally, but do not return untyped heap memory (which could be used for any object).

The C compiler knows about these functions and can use their expected behavior for optimizations. For instance, the compiler assumes that an existing pointer (or a pointer derived from an existing pointer by arithmetic) will not point into the memory area returned by malloc.

If the allocation fails, realloc does not free the old pointer. Therefore, the idiom ptr = realloc(ptr, size); is wrong because the memory pointed to by ptr leaks in case of an error.

Use-after-free errors

After free, the pointer is invalid. Further pointer dereferences are not allowed (and are usually detected by valgrind). Less obvious is that any use of the old pointer value is not allowed, either. In particular, comparisons with any other pointer (or the null pointer) are undefined according to the C standard.

The same rules apply to realloc if the memory area cannot be enlarged in-place. For instance, the compiler may assume that a comparison between the old and new pointer will always return false, so it is impossible to detect movement this way.

On a related note, realloc frees the memory area if the new size is zero. If the size unintentionally becomes zero, as a result of unsigned integer wrap-around for instance, the following idiom causes a double-free.

new_size = size + x; /* 'x' is a very large value and the result wraps around to zero */
new_ptr = realloc(ptr, new_size);
if (!new_ptr) {
	free(ptr);
}

Handling Memory Allocation Errors

Recovering from out-of-memory errors is often difficult or even impossible. In these cases, malloc and other allocation functions return a null pointer. Dereferencing this pointer lead to a crash. Such dereferences can even be exploitable for code execution if the dereference is combined with an array subscript.

In general, if you cannot check all allocation calls and handle failure, you should abort the program on allocation failure, and not rely on the null pointer dereference to terminate the process. See [sect-Defensive_Coding-Tasks-Serialization-Decoders] for related memory allocation concerns.

alloca and Other Forms of Stack-based Allocation

Allocation on the stack is risky because stack overflow checking is implicit. There is a guard page at the end of the memory area reserved for the stack. If the program attempts to read from or write to this guard page, a SIGSEGV signal is generated and the program typically terminates.

This is sufficient for detecting typical stack overflow situations such as unbounded recursion, but it fails when the stack grows in increments larger than the size of the guard page. In this case, it is possible that the stack pointer ends up pointing into a memory area which has been allocated for a different purposes. Such misbehavior can be exploitable.

A common source for large stack growth are calls to alloca and related functions such as strdupa. These functions should be avoided because of the lack of error checking. (They can be used safely if the allocated size is less than the page size (typically, 4096 bytes), but this case is relatively rare.) Additionally, relying on alloca makes it more difficult to reorganize the code because it is not allowed to use the pointer after the function calling alloca has returned, even if this function has been inlined into its caller.

Similar concerns apply to variable-length arrays (VLAs), a feature of the C99 standard which started as a GNU extension. For large objects exceeding the page size, there is no error checking, either.

In both cases, negative or very large sizes can trigger a stack-pointer wraparound, and the stack pointer and end up pointing into caller stack frames, which is fatal and can be exploitable.

If you want to use alloca or VLAs for performance reasons, consider using a small on-stack array (less than the page size, large enough to fulfill most requests). If the requested size is small enough, use the on-stack array. Otherwise, call malloc. When exiting the function, check if malloc had been called, and free the buffer as needed.

If portability is not important in your program, an alternative way of automatic memory management is to leverage the cleanup attribute supported by the recent versions of GCC and Clang. If a local variable is declared with the attribute, the specified cleanup function will be called when the variable goes out of scope.

static inline void freep(void *p) {
        free(*(void**) p);
}

void somefunction(const char *param) {
	if (strcmp(param, "do_something_complex") == 0) {
		__attribute__((cleanup(freep))) char *ptr = NULL;

		/* Allocate a temporary buffer */
		ptr = malloc(size);

		/* Do something on it, but do not need to manually call free() */
	}
}

Array Allocation

When allocating arrays, it is important to check for overflows. The calloc function performs such checks.

If malloc or realloc is used, the size check must be written manually. For instance, to allocate an array of n elements of type T, check that the requested size is not greater than ((size_t) -1) / sizeof(T). See [sect-Defensive_Coding-C-Arithmetic].

GNU libc provides a dedicated function reallocarray that allocates an array with those checks performed internally. However, care must be taken if portability is important: while the interface originated in OpenBSD and has been adopted in many other platforms, NetBSD exposes an incompatible behavior with the same interface.

Custom Memory Allocators

Custom memory allocates come in two forms: replacements for malloc, and completely different interfaces for memory management. Both approaches can reduce the effectiveness of valgrind and similar tools, and the heap corruption detection provided by GNU libc, so they should be avoided.

Memory allocators are difficult to write and contain many performance and security pitfalls.

  • When computing array sizes or rounding up allocation requests (to the next allocation granularity, or for alignment purposes), checks for arithmetic overflow are required.

  • Size computations for array allocations need overflow checking. See Array Allocation.

  • It can be difficult to beat well-tuned general-purpose allocators. In micro benchmarks, pool allocators can show huge wins, and size-specific pools can reduce internal fragmentation. But often, utilization of individual pools is poor, and external fragmentation increases the overall memory usage.

Conservative Garbage Collection

Garbage collection can be an alternative to explicit memory management using malloc and free. The Boehm-Dehmers-Weiser allocator can be used from C programs, with minimal type annotations. Performance is competitive with malloc on 64-bit architectures, especially for multi-threaded programs. The stop-the-world pauses may be problematic for some real-time applications, though.

However, using a conservative garbage collector may reduce opportunities for code reduce because once one library in a program uses garbage collection, the whole process memory needs to be subject to it, so that no pointers are missed. The Boehm-Dehmers-Weiser collector also reserves certain signals for internal use, so it is not fully transparent to the rest of the program.