A buffer overflow attack is perhaps the most common system compromise in the history of computing security. When a programmer writes software that expects input, that expected input is assigned a buffer in the system. The size of that buffer is dependent upon the expected size of the data. If the programmer has not put checks in place to validate the size of the input data, a malicious user can send more data than can be allocated in the buffer, causing an overflow. In the best case, this will cause memory corruption in the program and it will fail. In the worst case, specific data can be sent in the overflow portion of the input data, which is then executed with the privileges of the owner of the program. In the case of a system service running as root, a remote buffer overflow attack can allow an attacker to run arbitrary code as the root user on a system. The real defense against a buffer overflow attack is to prevent them from ever happening. This can be accomplished through a complete code audit, ensuring that every data input point in an application does proper bounds checking to this is often difficult to do, even with the available code profiling tools available.

A common practice is to treat this kind of attack like any other in the networking security realm - assume it will happen and make plans to identify it and reduce the damage. Identification is normally handled by examining the logs of a system, looking for strange log entries from a system service. If a system has been compromised, file system integrity tools will tell you what files on the system have been modified. A “chroot jail” is commonly used to minimize the amount of damage that is possible from an attack of this nature. This has become a common enough solution that some popular server programs have built-in chroot functionality (sendmail is an example).

In Linux, there are kernel modules available to assist in the defense against buffer overflow attacks. These patches go about this in different ways, but most
focus on preventing arbitrary code from being inserted onto the stack and executed, making traditional buffer overflow attacks much more difficult to execute.

See Also

• Homework Questions and Answers #1
• Homework Questions and Answers #2

Linux Home: Linux System Administration Hints and Tips