Buffer Overflows, ASLR, and Stack Canaries

By Jimmy Trimer

Introduction

Buffer overflow exploits are one of the most common and classic security vulnerabilities in computer programs. A buffer overflow exploit sends a buffer more data than is expected with a goal of smashing the stack, overwriting the instruction pointer, and redirecting program execution to a malicious code of the attacker’s choice. To prevent buffer overflow attacks, there are various defense mechanisms that are built into most computer systems. A more in-depth look into buffer overflows, how they affect the stack, buffer overflow defense mechanisms, and how attackers can use this vulnerability to exploit a program will allow for a better understanding.

After a thorough explanation of buffer overflows and their functionality, there are two main defense mechanisms that will be discussed in this blog: a stack canary and an Address Space Layout Randomization (ASLR). This blog will take a deeper look into these defense mechanisms, which are used to prevent exploitation from occurring on a user’s computer. Gaining a better understanding of these defense mechanisms and how they function to prevent attacks will allow the reader to learn more about how buffer overflows can successfully exploit a program.

What is a buffer?

Let’s start with the basics. A buffer is a location in the computer’s memory that stores data of a certain length. In this example, the buffer is allocated with a size of 32 bytes. The string “Jimmy’s buffer overflow example” is 31 bytes long and strings in example1.C need to be null-terminated so we must leave space for the null byte at the end of the string.

 

Example1.C is a very simple example; there are no security issues here. The buffer is a statically allocated size and the data is the correct size for the buffer. There may be some instances where you, as the programmer, do not control the input to the buffer. In the following example, the programmer will statically assign the size of the buffer, but the user will provide the input.

On the flip side, example2.C contains serious security issues. The user in this program provides data that is more than 32 bytes, which causes severe vulnerabilities. If the user provides data that is longer than 32 bytes, that extra data will overflow onto the stack and could potentially overwrite local variables or even worse, the programs instruction pointer, which causes the program to crash.

What is the Stack?

The stack is a data structure in the computer’s memory that the executable program uses to keep track of the current function, its local variables, parameter values, and the return address to the previous function call. Every time a function is called, a new stack frame is created for that function to provide an organizational that keeps the variables and return addresses in the correct place. The stack pointer points to the top of the current stack frame and the base pointer points to the end of the current stack frame. As the stack grows and shrinks, the stack pointer and the base pointer get adjusted accordingly, as they will always point to the top and bottom of the current stack frame. Most security problems occur when there is a programming mistake that allows the attacker to overwrite other variables or return addresses on the stack to values of the attacker’s choosing. This technique is called “smashing the stack” and it is commonly used to exploit buffer overflow vulnerabilities.

 

What is Smashing the Stack?

“Smashing the stack” is a term used to describe when the process in which an attacker overflows a buffer with so much data that the stack becomes completely overwritten with attacker-controlled data. Once an attacker is able to smash the stack, they should be able to completely control the program flow and successfully exploit the program to execute their malicious payload instead of the program’s designed functionality.

It is also possible to only use a buffer overflow vulnerability to change the flow of the program. In the example above, example3.C, the program checks if the password inputted by the user is the same as the password the program expects. If the passwords match, the program will print out the user’s message, otherwise it will print “Wrong!”. The major security flaw within this program is that if the user’s message is longer than 32 bytes, it is possible to overwrite the value of result in order to bypass the password check.

The attacker will run the program with the argument “badpass” for userPass. Using python on the command line to create the message, the command ‘python –c print(”A”*60 + “\x00\x00\x00\x00”)’ will create the string for msg. This msg string has 60 A’s to be used as filler on the stack until we get to the variable we want to overwrite. Then the \x00\x00\x00\x00 will set the value equal to zero in order to bypass the password check. As you can see, we are able to bypass the password check without providing the correct password. Now, let’s take a look at the stack to see how this is possible.

The blue outlines the stack pointer (rsp) and the base pointer (rbp). The green outlines the buffer we are overflowing and all of the data we input to the buffer. The red outlines the value we are exploiting. The top of picture of the stack is right before the ‘gets’ call at line 12, and the bottom picture is immediately after the ‘gets’ call. As you can see, the input of 60 A’s fills up the stack exactly to where the value of result is stored. Since we are able to overflow the buffer, msg, we can continue to write and overwrite the stored value of result to zero in order to bypass the password check without knowing the correct password.

How can an attacker exploit a buffer overflow?

There are several ways in which an attacker can exploit a program with a buffer overflow vulnerability. The simplest way is for an attacker to write a shellcode to the stack and then redirect program flow to their malicious code. When an attacker is able to successfully smash the stack, they will have full control of the instruction pointer (rip) and it will be very easy for them to execute their shellcode. The code below suffers from a buffer overflow vulnerability. Let’s go through how to exploit this program to get a shell on the system.

The Python script, shown above, is used to exploit the buffer overflow in the example4.c program. The exploit is simple. The ‘FILL’ variable is used to smash the stack and overwrite everything except the return address. The ‘addr’ variable is used to overwrite the return address to the address we want to jump to, where our shellcode will start executing. The ‘NOP’ variable is the NOP sled that will be placed before the shellcode. A NOP sled is a section of no operation instructions that are used to ensure that our shellcode will be executed. The address we want to jump to in order to execute the shellcode is not an exact science. By putting a NOP sled before the shellcode, as long as we jump to any spot in the NOP sled, the shellcode will execute without any problems. Without a NOP sled, we would need to jump exactly to the beginning of the shellcode. If we are even one byte off, the exploit will fail. The final part of the exploit is the shellcode. The shellcode was created using msfvenom with the command “msfvenom -p linux/x64/exec CMD=/bin/sh -f python”. This command creates shellcode for 64-bit linux that executes the command /bin/sh and outputs the shellcode in Python format. Here, we just copy and paste it into the exploit script.

When we run the program with gdb, we will see something similar to what is noted above. This is what the stack looks like right before the exploit is executed. The blue outlines the stack pointer (rsp) and the base pointer (rbp). The green outlines the beginning of buffer we are overflowing, which is holding the ‘FILL’ variable. The orange is the address of our shellcode on the stack, and what we are overwriting the return address with in order for the program to return to the shellcode. The purple is the NOP sled that gives us extra confidence that the exploit will run correctly. The red outlines the beginning of the shellcode. When the exploit runs, we will return to the address 0x7fffffffe460, which is in the middle of the NOP sled, and then it will continue and execute the shellcode. As you can see, after the shellcode is executed, gdb says a new process /bin/dash is now executing, confirming the shellcode was executed correctly. Now, let’s go over how this could have been prevented.

What is a Stack Canary?

One of the main defense mechanisms used against buffer overflows is a stack canary. A stack canary stores a dynamically calculated value when the function begins on the stack before the return address. When the function is ready to return, the stack canary will be popped off the stack, recomputed, and compared to the stored canary that was placed before the return address. The stack canary is then checked to determine if it has been overwritten. If it has been tampered with, the program automatically exits with an error code stating: “the stack has been smashed.” This aborts the command and prevents any malicious activity from occurring. On the other hand, if the values match, the function was executed correctly and no stack smashing occurred, meaning that no malicious activity has occurred.

 

Looking above at the same example4 program in gdb, the only difference is that the program now has a stack canary, outlined in red. As shown in the disassembly, there is now a function called __stack_chk_fail right before the main function returns. This is the stack canary check. If it fails, the program will exit and the exploit will fail. The blue outlines the stack pointer (rsp) and the base pointer (rbp).

The images above demonstrate what the stack looks like after the buffer has been filled with the input. The blue outlines the stack pointer (rsp) and the base pointer (rbp). The red outlines the stack canary. The green outlines the buffer with our input on the stack. Since the data from the buffer overwrote the stack canary, it will fail the check at the end of the main function. In the picture below, the program says “stack smashing detected” and the program exited instead of being exploited.

What is NX (Non-Executable Stack)?

In the example shown below, the blue outlines the stack pointer (rsp) and the base pointer (rbp). The green outlines the beginning of buffer we are overflowing. The orange is the address of our shellcode on the stack, and what we are overwriting the return address with in order for the program to return to the shellcode. The purple is the NOP sled that gives us extra confidence the exploit will run correctly. The red outlines where the program has a segmentation fault because the stack is not executable. In this example, the stack is marked as a non-executable region of memory, meaning that code cannot be executed from an address on the stack. It is apparent that the buffer is overflowed just like before, and the shellcode is on the stack. When the program jumps to the address of the shellcode at 0x7fffffffe460, which is located on the stack, the program has a segmentation fault because we tried to execute instructions from the stack, which is not allowed. This successfully prevents the buffer overflow from occurring since we can no longer run our shellcode from the stack. However, this does not mean that it is impossible to exploit this program; it simply means that we will have to get more creative and find another place to put the shellcode.

What is Address Space Layout Randomization (ASLR)?

A more advanced technique to prevent buffer overflows is known as Address Space Layout Randomization (ASLR). ASLR randomizes the memory address space, making it difficult to predict the memory address layout of the program by randomizing the entry point of the program, the address of the stack, heap, and other areas of the executable. Doing so prevents the attacker from being able to predict the address they would likely redirect code flow to, thus preventing exploitation.

In the picture above, is it shown that every time the program is restarted, the stack pointer (rsp) is at a different address: first, 0x7ffd522d1b90, then 0x7fffea84f890, and then 0x7ffd78982260. The different addresses stops the exploit due to the fact that the exploit relies on a hard coded address on the stack. Since the stack address is changing every time, it is impossible to predict the addresses. As you can see, running the same example4 program with the same exploit as above does not work like before. The program simply seg faults and exits.

Conclusion

In conclusion, the overall goal of this blog is to give readers a complete understanding of how buffer overflows can successfully exploit a program and how the defensive mechanisms work to prevent exploitation. To enhance understanding, the code examples provide a visualization of the process of exploitation. Using these examples will allow readers to develop a stronger understanding of program flow, the stack, and how an attacker can abuse the program to execute malicious code. Understanding buffer overflows, ASLR, and stack canaries allows readers to learn more about security vulnerabilities and how to prevent them from occurring in the future.