Analysis of RedXOR Malware

By Jakob Zielinski


The prevalence of malware written for Linux is increasing with each year. 2020 alone saw a 40% increase in the number of Linux malware families, and with the emergence of the Windows subsystem for Linux, even Windows machines may be vulnerable to attack from such malware. And while Linux may not be very prevalent on desktop computers, a large portion of the server space runs on Linux. In fact, 96.3% of the world’s top million servers run Linux. In my prior coursework, I have noticed a heavy emphasis placed upon the reversing of Windows binaries, with little to no attention given to other operating systems. While not as common, malware still exists for Linux operating systems, and it is important to understand how they might be analyzed. In this blogpost, I would like to analyze a malicious binary for Linux (RedXOR) while highlighting some of the tools available for the platform.


IDA – Although IDA itself runs on windows, it still provides the capability of reading elf formatted binaries. Its graph view provides for easy visualization of assembly code, and makes advanced static analysis a breeze, no matter which platform the binary is written for. The biggest downside to IDA is that it is prohibitively expensive, making it inaccessible for most.

Figure 1: IDA in graph view

BinNavi/Cutter – It should be noted that there are several open source alternatives to IDA. Among these are BinNavi and Cutter. Each provides a graph view similar to that of IDA, with the benefit of being completely free and multiplatform. For my analysis, I elected to use Cutter, but keep in mind that neither of these programs are platform specific and are interchangeable in the analysis process.

Figure 2: Cutter in graph view

REMnux – REMnux is a Linux distribution based off of Ubuntu, and bills itself as “A Linux Toolkit for Malware Analysis.” It comes as a pre-built virtual appliance, and can be easily installed into virtualization software of your choice. While technically not itself a tool, REMnux provides easy access to many analysis tools, a selection of which will be detailed below. If you are interested in learning more about what REMnux has to offer, I recommend checking out their documentation here.

Ghidra – Ghidra fits into a similar niche as IDA – that of an advanced static analysis tool. Ghidra, however, provides the user with decompiled C code, making it significantly easier to parse through than assembly.

Figure 3: Ghidra with decompiled code

Objdump – Objdump is a lightweight command line tool that allows for the fast dumping of assembly code. Because it sends its output to stdout, it can be easily filtered into other tools such as grep or awk. While not a particularly advanced tool, it has a job and it does it well.

Figure 4: Output from objdump

WireShark/TCPdump – Wireshark and TCPdump both play the role of live network analysis, with the latter being a lightweight command line tool and the former having an easy to use graphical interface. Whichever you prefer, they both do their job well.

fakedns/ – These are two simple dynamic analysis tools that function similarly to a utility like apateDNS. They set up a fake dns server on your machine that allows it to intercept any DNS requests being made and report them back to you. These utilities make discovering any domains malware uses quite simple.


RedXOR – dba757c20fbc1d81566ef2877a9bfca9b3ddb84b9f04c0ca5ae668b7f40ea8c3

Right off the bat RedXOR pulls out some classic obfuscation techniques. First, the software forks and exits the parent process, leaving an orphaned child process that will be more difficult to detect.

Figure 5: RedXOR forks and exits

It then uses the dup2 system call to replace stdout and stderr with /dev/null. /dev/null eats anything that is input into it, meaning that no program output is easily visible to the victim.

Figure 6: RedXOR uses the dup2 system call to direct output to /dev/null

Once this is done, the program calls a function called “AddAuto”. This function gives the malware persistence by writing to directories such as init.d, which may be executed on launch. RedXOR places the script into these directories as “po1kitd-update”, or some similar string.

Figure 7: Portion of “AddAuto” function

The program then sends the kill signal to PID 1, presumably to restart the init process and apply whatever changes it put in init.d.

Figure 8: RedXOR restarts the init process

The program then decodes a series of key values using its doXor function. This includes the IP and password for the server, as well as the username and password for a proxy.

Figure 9: Several subsequent calls to “doXor” function

The notorious xor function for which this malware is named is actually a quite simple, single byte XOR. The caller passes an initial xor value(xor_val) as well as a modifier(xor_mod). During each round, the current byte is XORed with the current xor value. At the end of each round, the modifier is added to the xor value. Pseudo-code for this function is listed below.

doXor(xor_val, xor_mod, buffer, counter){
    for(int i=0;i++;i<counter){
        buffer[i]=xor_val ^ buffer[i];
        xor_val = xor_val+xor_mod;
    return 0;

RedXOR then carries out a main process indefinitely. The decoded server IP address and a password are also passed to this function, and are used to connect to the server.

Figure 10: A while loop cause the main process to be carried out indefinitely

The purpose of this main process is to listen for commands from a command and control server and send responses. It first establishes a connection to the server. This connection can be established either by a proxy or directly to the server.

The primary method of communication used by RedXOR is HTTP. This allows it to remain inconspicuous and blend in with other traffic. Pictured below is the http string from the “makeHTTPData” function. If you are looking to create IDS/IPS rules for RedXOR, I would recommend taking a look at this function, since the bulk of the traffic generated by the malware is created by this function.

Figure 11: Portion of “makeHTTPData” function

All data received from the server is encrypted and is decrypted using the doXor function. To parse commands from the server, RedXOR has a quite complicated flow of execution. To put it simply, RedXOR parses any data it receives for a set of codes and then carries out certain functions based on the received code. Pictured below is a simplified flow of execution.

Figure 12: Simplified flow of execution for parsing commands from server

0x000Fetch system information
0x3e8Send empty http reply
0x3f2install LKM
0x801Get filesystem listing
0x806send file
0x807Open a file
0x808Execute a shell command with system
0x80aRemove File
0x80cremove directory
0x80drename a file
0x80emake a directory
0x812write a file
0xbb8Launch an interactive shell
0xbf2execute a shell command
0xf9fclose tty
0xfa0get port mapping (portmap)
0xfa1kill portmap
Figure 13: Chart mapping codes sent by server to client functionality


As can be seen, RedXOR is an advanced piece of malware with a wide variety of functionality, such as the ability to launch a remote shell and to update remotely. It also employs a number of obfuscation techniques; most notably its XOR function, among others. Luckily, there are a number of tools available for Linux systems which greatly simplify the analysis process. For my static analysis, I chose to use an open-source analysis tool called Cutter, which provides very similar functionality to utilities such as Ghidra or IDA Pro. While it is a common myth that Linux does not get malware, the threat is still very real and it is important to understand how malware for the platform can be analyzed.



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s