Modern Linux Rootkits

By Kyle Carretto

What is a Linux Rootkit?

A rootkit is essentially a method for an attacker to maintain stealthy access to a victim machine. The methods for doing so are described in detail later in this article, however it is important to note that rootkits have the same basic goals. Their primary intention is to provide host-based stealth, and this is generally achieved through hiding files, processes, and network connections. Rootkits may also provide additional functionality, such as keylogging, granting root permissions to unauthorized users, or even spawning shells, however these are not the main purpose of the rootkit, and so this article with omit these extra features and focus in on the core goals and mechanisms of rootkits.

User Space and Kernel Space

An important concept to understand when first learning about rootkits is the difference between user space and kernel space. User space (Also referred to as user land) is the code that runs outside of the operating system’s kernel. Each process (i.e. ‘ls’) in user space is given it’s own virtual memory segment, separated from the other processes running in user space. The diagram below details the separation between user space and kernel space.


Figure 1 – An overview diagram of user space and kernel space.

Kernel space on the other hand, is the portion of memory where the operating system’s kernel is run, and where Linux kernel modules are located. This is also where system calls are located in memory.

System Calls

The system call interface is provided to user land processes to give them a method of requesting services that only the kernel has the ability to do. Generally, a process is not allowed to access kernel memory and it cannot call kernel functions. The main mechanism of process to kernel communication is therefore through system calls, which are kernel functions that user space processes are explicitly allowed to call. When a process executes a system call, execution jumps from the process to a point in kernel memory known as ‘system_call’, which then performs logic to see which service is being requested by the process. It then locates the address of the appropriate system call in the system call table, and executes that function. The system call table is essentially just an array in kernel memory that contains pointers to system calls (See Figure 3). System calls are used to perform low level functions such as opening files and listing files in a directory. A good way of understanding this is through analyzing the userland process ‘ls’. We can actually see what system calls this process uses using the command ‘strace’.


Figure 2 – A summary of the output of the command ‘strace ls’.

The output above shows relevant system calls being made by the userland process ‘ls’. As you can see, the program is using the system call ‘open’ to open the current directory, and then the system call ‘getdents’ to enumerate all entries within the directory. The below diagram shows how a system call like ‘getdents’ is normally accessed.


          Figure 3 – An example of ‘ls’ utilizing a system call.

System Call Hooking

Rootkits generally operate by modifying the functionality of these system calls. The idea is to replace the address of a given system call in the system call table with the address of a malicious function. This will cause the kernel to execute our malicious function instead of the function that is normally used, which will perform logic based on how we’d like the system call to act. In the case of ‘ls’, we could hook the ‘getdents’ system call to only return files that are not hidden by our rootkit. The below diagram shows an example of hooking the ‘getdents’ system call.


Figure 4 – An example of how a hooked system call table might look.

In the above diagram, we have changed the address of a system call in the system call table to point to our malicious function instead of the original system call. This gives us the ability to determine whether or not we should perform normal functionality based on the properties we have access to (PID, filename, etc.). This concept is the basis for how rootkits generally work, by hooking system calls in order to perform desired actions such as hiding a file.

Modern Protections

In older Linux kernel versions, it was much easier to design and create rootkits. This was because the kernel exported the ‘sys_call_table’ symbol which referenced the address of the system call table in memory. This made it very trivial to modify the system call table to point to malicious functions, therefore enabling basic rootkit functionality. However, since version kernel version 2.7.6, the ‘sys_call_table’ symbol is no longer exported and the system call table is stored in read-only memory. This was done to prevent rootkits from being easily created, however the protections currently in play are quite flimsy.

These protections now create two issues, which we will talk about bypassing in this article. The first is that the ‘sys_call_table’ symbol is no longer exported, which means we need to find another way to locate the address of the system call table in memory. The second is that even after we’ve found this address, it’s stored in read only memory. We need to figure out how to get around this protection so that we can modify the system call table to contain the address of our malicious function. In order to bypass these protection mechanisms, we will need to create a Linux kernel module, so that our code can run with the permissions of the kernel. If you are unfamiliar with Linux kernel modules, please refer to this article (Also listed below under References).

Locating the System Call Table

This is by far the more difficult protection to get around, however there are multiple solutions at our disposal. We could, for example, attempt to find the correct address in the file. This involves obtaining the version of the currently running kernel, and then locating the address of the ‘sys_call_table’ in the correct file. This will not be the method we employ, but an example of searching through this file is given below.


Figure 5 – An example of the system call table address in the file.

We will instead scan through kernel memory and compare pointers until we find a match. Although the ‘sys_call_table’ symbol is not exported, some system calls (such as ‘sys_close’) are still exported to Linux kernel modules. The below example shows functional code for locating the system call table.


Figure 6 – Example c code to locate the address of the system call table.

In the above code, ‘PAGE_OFFSET’ represents the start of kernel memory, and ‘ULONG_MAX’ represents the maximum value for an unsigned long. We know that the system call table must be stored between these two values. We start at ‘PAGE_OFFSET’, and increment ‘i’ by the size of a pointer at the end of each iteration. In this code, ‘i’ represents the address we are currently testing. The symbol ‘__NR_close’ refers to the index in the system call table that the address of the ‘sys_close’ system call is stored. By dereferencing the current address plus the ‘sys_close’ offset (‘__NR_close’), we can conclude that if the value at that location in memory points to the ‘sys_close’ system call, our current address must be the system call table.

Modifying the System Call Table

After obtaining the address of the system call table, we still face the obstacle of modifying it because it’s in read only memory. The question becomes “Why? What is enforcing that?”, and it turns out that the CPU is actually enforcing it. On the x86 architecture, there is a control register titled ‘cr0’. The CPU checks to see if the 16th bit on that register is set, referred to as the ‘Write Protection’ bit. If that bit is enabled, the CPU will not write to memory that has been marked as read only. This means that if we could temporarily flip that single bit from a 1 to a 0, we would be able to modify the system call table, therefore hooking system calls and completing our rootkit. Normal user land processes would not have access to change this bit, however because Linux kernel modules operate with the full permissions of the kernel, we actually can flip this bit. Interestingly enough there is already a function to write to this register for us (located in <asm/paravirt.h>) called ‘write_cr0’. This allows us to define two functions, one that will disable ‘Write Protection’, and one that will restore the register to it’s original state, as you can see below.


Figure 7 – Functions to disable write protection, and then re-enable write protection.


The information above allows us to create a Linux kernel module that could act as a rootkit. By putting it all together, we have essentially developed the below process.

  1. Locate the system call table
  2. Store the address of the original system call
  3. Store the original state of the cr0 register
  4. Disable ‘Write Protection’
  5. Modify the system call table to point to our malicious function
  6. Restore the cr0 register to it’s original state


An interesting note is that while rootkits attempt to achieve host-based stealth, they have no control over hiding traffic on the network. This means that network based detection could potentially detect a rootkit on an infected system if it generated suspicious traffic.

Something else to consider is that if you can prevent a kernel module from being loaded into the kernel, this type of rootkit would not work. This could be done by requiring that any module being loaded into the kernel must be signed by a specific key. I have included an article that details this process more under the References section.

Looking Forward

Interesting methods of detection for this type of rootkit have since grown. A common mitigation of this type of rootkit is to check the system call addresses in the system call table. A normal system call table will contain only addresses that reside in the kernel’s memory, however one hooked through a kernel module will contain addresses that point to a location in kernel module memory instead. This has lead to the development of more advanced Linux rootkits, employing methods such as ‘inline function hooking’ to achieve the same result in a much stealthier fashion. Instead of hooking system calls, this method overwrites the actual instructions of any kernel function in memory, replacing them with instructions to jump to malicious code. This is more difficult to detect because the system call table is never being modified. A reference has been included to a great article on this method.


System Calls:

Linux Kernel Memory:

Linux Kernel Modules:

Control Register cr0:

Kernel Module Signing:

Looking Forward – Inline Function Hooking:


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s