The C Programming Language

Some C constructs are defined to be undefined by the C standard. This does not only mean that the standard does not describe what happens when the construct is executed. It also allows optimizing compilers such as GCC to assume that this particular construct is never reached. In some cases, this has caused GCC to optimize security checks away. (This is not a flaw in GCC or the C language. But C certainly has some areas which are more difficult to use than others.)

Common sources of undefined behavior are:

Always keep track of the size of the array you are working with. Often, code is more obviously correct when you keep a pointer past the last element of the array, and calculate the number of remaining elements by subtracting the current position from that pointer. The alternative, updating a separate variable every time when the position is advanced, is usually less obviously correct.

Array processing in C shows how to extract Pascal-style strings from a character buffer. The two pointers kept for length checks are inend and outend. inp and outp are the respective positions. The number of input bytes is checked using the expression len > (size_t)(inend - inp). The cast silences a compiler warning; inend is always larger than inp.

It is important that the length checks always have the form len > (size_t)(inend - inp), where len is a variable of type size_t which denotes the total number of bytes which are about to be read or written next. In general, it is not safe to fold multiple such checks into one, as in len1 + len2 > (size_t)(inend - inp), because the expression on the left can overflow or wrap around (see Recommendations for Integer Arithmetic), and it no longer reflects the number of bytes to be processed.

Overflow in signed integer arithmetic is undefined. This means that it is not possible to check for overflow after it happened, see Incorrect overflow detection in C.

The following approaches can be used to check for overflow, without actually causing it.

Basic arithmetic operations are commutative, so for bounds checks, there are two different but mathematically equivalent expressions. Sometimes, one of the expressions results in better code because parts of it can be reduced to a constant. This applies to overflow checks for multiplication a * b involving a constant a, where the expression is reduced to b > C for some constant C determined at compile time. The other expression, b && a > ((unsigned)-1) / b, is more difficult to optimize at compile time.

When a value is converted to a signed integer, GCC always chooses the result based on 2’s complement arithmetic. This GCC extension (which is also implemented by other compilers) helps a lot when implementing overflow checks.

Sometimes, it is necessary to compare unsigned and signed integer variables. This results in a compiler warning, comparison between signed and unsigned integer expressions, because the comparison often gives unexpected results for negative values. When adding a cast, make sure that negative values are covered properly. If the bound is unsigned and the checked quantity is signed, you should cast the checked quantity to an unsigned type as least as wide as either operand type. As a result, negative values will fail the bounds check. (You can still check for negative values separately for clarity, and the compiler will optimize away this redundant check.)

Legacy code should be compiled with the -fwrapv GCC option. As a result, GCC will provide 2’s complement semantics for integer arithmetic, including defined behavior on integer overflow.

Global variables should be avoided because they usually lead to thread safety hazards. In any case, they should be declared static, so that access is restricted to a single translation unit.

Global constants are not a problem, but declaring them can be tricky. Declaring a constant array of constant strings shows how to declare a constant array of constant strings. The second const is needed to make the array constant, and not just the strings. It must be placed after the *, and not before it.

Sometimes, static variables local to functions are used as a replacement for proper memory management. Unlike non-static local variables, it is possible to return a pointer to static local variables to the caller. But such variables are well-hidden, but effectively global (just as static variables at file scope). It is difficult to add thread safety afterwards if such interfaces are used. Merely dropping the static keyword in such cases leads to undefined behavior.

Another source for static local variables is a desire to reduce stack space usage on embedded platforms, where the stack may span only a few hundred bytes. If this is the only reason why the static keyword is used, it can just be dropped, unless the object is very large (larger than 128 kilobytes on 32-bit platforms). In the latter case, it is recommended to allocate the object using malloc, to obtain proper array checking, for the same reasons outlined in alloca and Other Forms of Stack-based Allocation.

The functions listed below must not be used because they are almost always unsafe. Use the indicated replacements instead.

The constants listed below must not be used, either. Instead, code must allocate memory dynamically and use interfaces with length checking.

The following structure members must not be used.

The following string manipulation functions can be used securely in principle, but their use should be avoided because they are difficult to use correctly. Calls to these functions can be replaced with asprintf or vasprintf. (For non-GNU targets, these functions are available from Gnulib.) In some cases, the snprintf function might be a suitable replacement, see String Functions with Explicit Length Arguments.

Use the indicated replacements for the functions below.

The C run-time library provides string manipulation functions which not just look for NUL characters for string termination, but also honor explicit lengths provided by the caller. However, these functions evolved over a long period of time, and the lengths mean different things depending on the function.

The C library interfaces for memory allocation are provided by malloc, free and realloc, and the calloc function. In addition to these generic functions, there are derived functions such as strdup which perform allocation using malloc internally, but do not return untyped heap memory (which could be used for any object).

The C compiler knows about these functions and can use their expected behavior for optimizations. For instance, the compiler assumes that an existing pointer (or a pointer derived from an existing pointer by arithmetic) will not point into the memory area returned by malloc.

If the allocation fails, realloc does not free the old pointer. Therefore, the idiom ptr = realloc(ptr, size); is wrong because the memory pointed to by ptr leaks in case of an error.

Allocation on the stack is risky because stack overflow checking is implicit. There is a guard page at the end of the memory area reserved for the stack. If the program attempts to read from or write to this guard page, a SIGSEGV signal is generated and the program typically terminates.

This is sufficient for detecting typical stack overflow situations such as unbounded recursion, but it fails when the stack grows in increments larger than the size of the guard page. In this case, it is possible that the stack pointer ends up pointing into a memory area which has been allocated for a different purposes. Such misbehavior can be exploitable.

A common source for large stack growth are calls to alloca and related functions such as strdupa. These functions should be avoided because of the lack of error checking. (They can be used safely if the allocated size is less than the page size (typically, 4096 bytes), but this case is relatively rare.) Additionally, relying on alloca makes it more difficult to reorganize the code because it is not allowed to use the pointer after the function calling alloca has returned, even if this function has been inlined into its caller.

Similar concerns apply to variable-length arrays (VLAs), a feature of the C99 standard which started as a GNU extension. For large objects exceeding the page size, there is no error checking, either.

In both cases, negative or very large sizes can trigger a stack-pointer wraparound, and the stack pointer and end up pointing into caller stack frames, which is fatal and can be exploitable.

If you want to use alloca or VLAs for performance reasons, consider using a small on-stack array (less than the page size, large enough to fulfill most requests). If the requested size is small enough, use the on-stack array. Otherwise, call malloc. When exiting the function, check if malloc had been called, and free the buffer as needed.

If portability is not important in your program, an alternative way of automatic memory management is to leverage the cleanup attribute supported by the recent versions of GCC and Clang. If a local variable is declared with the attribute, the specified cleanup function will be called when the variable goes out of scope.

When allocating arrays, it is important to check for overflows. The calloc function performs such checks.

If malloc or realloc is used, the size check must be written manually. For instance, to allocate an array of n elements of type T, check that the requested size is not greater than ((size_t) -1) / sizeof(T). See Recommendations for Integer Arithmetic.

GNU libc provides a dedicated function reallocarray that allocates an array with those checks performed internally. However, care must be taken if portability is important: while the interface originated in OpenBSD and has been adopted in many other platforms, NetBSD exposes an incompatible behavior with the same interface.

Custom memory allocates come in two forms: replacements for malloc, and completely different interfaces for memory management. Both approaches can reduce the effectiveness of valgrind and similar tools, and the heap corruption detection provided by GNU libc, so they should be avoided.

Memory allocators are difficult to write and contain many performance and security pitfalls.

Garbage collection can be an alternative to explicit memory management using malloc and free. The Boehm-Dehmers-Weiser allocator can be used from C programs, with minimal type annotations. Performance is competitive with malloc on 64-bit architectures, especially for multi-threaded programs. The stop-the-world pauses may be problematic for some real-time applications, though.

However, using a conservative garbage collector may reduce opportunities for code reduce because once one library in a program uses garbage collection, the whole process memory needs to be subject to it, so that no pointers are missed. The Boehm-Dehmers-Weiser collector also reserves certain signals for internal use, so it is not fully transparent to the rest of the program.

Some libraries provide wrappers for standard library functions. Common cases include allocation functions such as xmalloc which abort the process on allocation failure (instead of returning a NULL pointer), or alternatives to relatively recent library additions such as snprintf (along with implementations for systems which lack them).

In general, such wrappers are a bad idea, particularly if they are not implemented as inline functions or preprocessor macros. The compiler lacks knowledge of such wrappers outside the translation unit which defines them, which means that some optimizations and security checks are not performed. Adding attribute annotations to function declarations can remedy this to some extent, but these annotations have to be maintained carefully for feature parity with the standard implementation.

At the minimum, you should apply these attributes:

For other attributes (such as malloc), careful analysis and comparison with the compiler documentation is required to check if propagating the attribute is appropriate. Incorrectly applied attributes can result in undesired behavioral changes in the compiled code.