Open Source Memory Analysis Tools for Instructional Use

By Wyatt Tauber

In RIT’s Computer System Forensics class, students learn basic incident response procedures as well as methods to uncover and investigate the activities of computer users. Students also learn to employ activities needed to gather and preserve evidence to be presented in court cases. Some of the concepts discussed in class are incident response reporting, forensic imaging, Linux and Windows file systems and steganography.

While the class is well-designed and the concepts presented are highly applicable, assignments and labs can grow a bit stale because the two professors that regularly teach the class are actively involved in research. Such was the case for Spring 2019’s memory acquisition and analysis lab, which instructs on how to use the Linux Memory Extractor (LiME) and the Volatility Framework to analyze memory images for system information and malware infections. Here, I sought to identify the errors in the lab instructions and produce an updated version of the lab that would work more reliably for future students. I also explored the use of other forensics tools that I felt would be a worthy addition to the lab such as Google’s Rekall, a former fork of Volatility that has now grown into a significantly advanced independent memory analysis solution.

I believe many of the problems stemming from the current lab instructions include instructions for obtaining an incompatible memory image, outdated or missing dependencies in the SANS SIFT Workstation that Computer System Forensics labs are conducted on, instructions on how to create a Volatility profile with a different kernel version than the one currently in use, and conflicting information between Volatility’s documentation available via the Volatility Foundation website and the project’s GitHub wiki.

Computer System Forensics’ Lab 5 on the Volatility Framework

Issues with the lab

The memory acquisition lab is conducted on SANS’ SIFT Workstation, an Ubuntu virtual machine for digital forensic examinations. Provided as an Open Virtualization Format (.ova) file, the VM can be easily set up on a hypervisor in a few minutes. Importantly, it includes many common forensic tools pre-installed — including Volatility. Volatility does have many built-in profiles of various Linux distros that could be used, but this lab also aims to teach how to create such profiles. This is where the problems occur.

The instructions

The instructions to create a Volatility profile were given as follows:

The steps to build Ubuntux64 profile on SIFT:

1. Install dwarfdump package and kernel headers
$sudo apt-get update
$sudo apt-get install dwarfdump linux-headers-generic –fix-missing

2. Download the Volatility repository from https://github.com/volatilityfoundation/volatility
cd into tools > linux folder of the downloaded repo
$cd ~/Downloads/volatility-master/tools/linux

3. Generate the module.dwarf file using make
$make

4. Find the System.map version
$ls /boot

5. Create the profile zip (sift.zip) and place it in volatility overlays/linux folder where volatility looks for all profiles, given the System.map file’s (from step 4) for the kernel version.
$sudo zip /usr/lib/python2.7/dist-packages/volatility/plugins/overlays/linux/Ubuntu.zip module.dwarf /boot/System.map-4.4.0-31-generic

6. Run vol.py –info | grep Profile to make sure the profile “LinuxUbuntux64” is in the profiles list.

7. Run vol.py -f ‘/home/sansforensics/Desktop/ yourusername_memory_dump.bin’ –profile=LinuxUbuntux64 VolatilityLinuxCommand (Note: replace VolatilityLinuxCommand with the volatility Linux commands from https://github.com/volatilityfoundation/volatility/wiki/Linux-Command-Reference)

When my section began this lab, we were cautioned by our professor and via the lab instructions that creating a valid Volatility profile for memory analysis can be difficult and yield inconsistent results. This isn’t the way it should be — Volatility profiles should be simple to generate consistently and validly. In addition, by following the lab instructions as-is, running Volatility commands often resulted in Python errors or an output of random characters.

Getting the CPU information from a memory image with the previous lab instructions

Indeed, class averages for Lab 5 were at least 10% lower than for each other lab completed last semester.

Class averages per lab. Note the score drop on Lab 5.

After realizing the problems with running Volatility, I sought to identify the errors and produce an updated version of the lab with corrected instructions. To do this, I first reached out to several other students in the class to understand their troubles with the lab. I found that the most commonly experienced problem was that they were unable to create a valid Volatility profile and configure the utility properly to progress to the data collection steps of the assignment. Many assignments were turned in with this section of the lab incomplete.

My friends and I discuss problems with the lab

Making Volatility run properly

I spent some time researching the lab instructions and identified several problems. SANS provides a version of Volatility in SIFT Workstation. Many of the issues stemmed from differences with this SIFT-provided version and the official Volatility release, such as the SIFT-provided version not containing the prerequisites to build Volatility profiles. Therefore, instructions were provided on how to clone and compile Volatility from GitHub in addition to the version already installed, resulting in students’ confusion as they inadvertently called different versions of the program depending on the current directory. There were also issues with dependencies, obtaining the kernel module version, command syntax, and memory imaging strategies. I was able to correct most of the problems and provide a more reliable version of the lab for future students.

Missing dependencies

I first checked the recommended dependencies listed on the Volatility project page and compared them against the lab instructions. There were many discrepancies. These are the packages that the lab instructions state to install:

1. Install dwarfdump package and kernel headers
$sudo apt-get update
$sudo apt-get install dwarfdump linux-headers-generic –fix-missing

However, per the issues on Volatility’s GitHub page, Volatility operates much more reliably with additional packages installed. Some (such as linux-headers, build-essential, python-dev, and python-pip) are already installed in SIFT. Others are needed for specific output formats and must be installed independently, but such a process would be lengthier and out of scope of this assignment. Still others are required depending on the features of Volatility that are enabled. After spending some time compiling a list of prerequisites needed for the commands in this lab, I was able to correct the install instructions. In addition, -y can be added so that a confirmation isn’t needed when installing the packages.

The corrected instructions are:

1. Install dwarfdump package and kernel headers
$sudo apt-get update
$sudo apt-get install dwarfdump pcregrep libpcre++-dev yara -y
$sudo -H pip install pycrypto Distorm3 OpenPyxl ujson pillow

Using old kernel versions

The lab instructions include a hardcoded kernel version (System.map-4.4.0-31-generic) in the command when building the profile. While this may have been accurate at the time the lab was written, the version number of this file can change depending on the release of SIFT.

4. Find the System.map version
$ls /boot

5. Create the profile zip (sift.zip) and place it in volatility overlays/linux folder where volatility looks for all profiles, given the System.map file’s (from step 4) for the kernel version.
$sudo zip /usr/lib/python2.7/dist-packages/volatility/plugins/overlays/linux/Ubuntu.zip module.dwarf /boot/System.map-4.4.0-31-generic

When a kernel version is referenced in a document, it is always best to provide a way to retrieve it via a script instead to ensure the System.map file retrieved is the version that matches the operating system release. There could also be many different kernel header file versions to choose from in /boot, so the most reliable method is to build the profile with $(uname -r) specified in place of the kernel version in the map file. Step 4 (and the likelihood of manually picking an incorrect kernel version) can now be eliminated! The file name in the description doesn’t match the command any longer either, so that can be updated.

4. Create the profile zip (Ubuntu.zip) and place it in volatility overlays/linux folder where volatility looks for all profiles, given the System.map file for the kernel version.
$sudo zip /usr/lib/python2.7/dist-packages/volatility/plugins/overlays/linux/Ubuntu.zip module.dwarf /boot/System.map-$(uname -r)

Command syntax

The command to run Volatility recommends that students use the backtick (‘) instead of the quotation mark (‘). Using a backtick to specify a path isn’t supported. There is also a space between Desktop/ and yourusername_memory_dump.bin which is most likely inadvertent but still makes the path incorrect. Since students are expected to copy and paste some of these commands, this can cause issues.

7. Run vol.py -f ‘/home/sansforensics/Desktop/ yourusername_memory_dump.bin’ –profile=LinuxUbuntux64 VolatilityLinuxCommand
(Note: replace  VolatilityLinuxCommand with the volatility Linux commands from https://github.com/volatilityfoundation/volatility/wiki/Linux-Command-Reference)

Both issues are easily fixed, and the command is renumbered due to the elimination of the previous step 4.

6. Run vol.py -f ‘/home/sansforensics/Desktop/yourusername_memory_dump.bin’ –profile=LinuxUbuntux64 VolatilityLinuxCommand (Note: replace  VolatilityLinuxCommand with the volatility Linux commands from https://github.com/volatilityfoundation/volatility/wiki/Linux-Command-Reference)

Memory image issues

The lab concludes by telling students to run Volatility on a memory dump captured in Lab 3. There are many tools that can accomplish this task, such as fmem and LiME. LiME is the tool covered in this course, so it will be used in this example.

The instructions for Lab 3 instruct students to create a memory image using LiME (the Linux Memory Extractor). LiME supports three formats: padded, lime, and raw, of which only lime is supported by Volatility. Unfortunately, the instructions in that lab tell students to create the memory dump in the padded format, and the instructions for Lab 5 fail to specify that the lime format should be used instead (for this lab and most other purposes). Following is a comparison of the three formats and explanation as to why only the lime format is compatible with Volatility.

  1. Padded: The padded format fills all non-system RAM ranges with 0s, erasing any data that might be there already.
  2. Lime: The lime format prepends each range with a fixed-size header containing address space information. This is required for Volatility’s modules to be able to analyze the image, although the issue is not well documented nor is the user warned about the situation.
  3. Raw: The raw format concatenates all system RAM ranges together without concern for the original positions of the dumped memory.

Suggestions to improve Lab 3 were passed on to the professors as well.

Running Volatility

Getting the CPU information from a memory image with the corrected lab instructions

Volatility now runs successfully across multiple systems and kernel versions! Modules previously requiring specific packages and the current kernel version now run properly as well. Here are the corrected lab instructions as a single document:

1. Install dwarfdump package and kernel headers
$sudo apt-get update
$sudo apt-get install dwarfdump pcregrep libpcre++-dev yara -y
$sudo -H pip install pycrypto distorm3 openpyxl ujson pillow
 
2. Download volatility repo:
$cd ~/Downloads$git clone https://github.com/volatilityfoundation/volatility.git$cd volatility/tools/linux
 
3. Generate the module.dwarf file using make
$make
 
4. Create the profile zip (sift.zip) and place it in volatility overlays/linux folder where volatility looks for all profiles, given the System.map file's for the kernel version.
$sudo zip /usr/lib/python2.7/dist-packages/volatility/plugins/overlays/linux/Ubuntu.zip module.dwarf /boot/System.map-$(uname -r)
 
5. Run vol.py --info | grep Profile to make sure the profile "LinuxUbuntux64" is in the profiles list.
 
6. Run vol.py -f /home/sansforensics/Desktop/yourusername_memory_dump.bin --profile=LinuxUbuntux64 VolatilityLinuxCommand (Note: replace  VolatilityLinuxCommand with the volatility Linux commands from https://github.com/volatilityfoundation/volatility/wiki/Linux-Command-Reference)

These instructions work great for using Volatility’s linux_check_modules function for finding a rootkit hidden in an Ubuntu virtual machine, such as in this RITSEC challenge.

Google’s Rekall forensic and incident response framework is another worthy tool to consider employing in this lab. Originally a fork of Volatility, Rekall describes itself as a “complete end-to-end memory analysis framework, encapsulating acquisition, analysis, and reporting…and the only memory analysis platform specifically designed to run on the same platform it is analyzing.” This type of tool is interesting as the general technique for memory forensics and analysis is to do the following:

  1. Obtain a memory image of the machine in question.
  2. Configure a separate machine with forensics tools.
  3. Perform all memory analysis on the separate, forensically-sound machine.

This three-step process is taken to ensure that the results will be reliable. Typically, it is not possible to trust the compromised system if Volatility is run on it because any malware present may interfere with the operation of such tools. However, with Rekall it may be plausible due to a modular structure for components and a lower profile. Rekall is also not as specific about dependencies as Volatility is, making for supposedly easier setup.

Viewing details of a demo system with Rekall

Listing processes of a demo system with Rekall

Conclusion

I’ve found the Volatility framework very interesting to work with since being introduced to it in this lab, having used it and many plugins in a few personal projects and CTFs to date. Although not the focus of this lab, Rekall is also an excellent program to consider implementing as part of the Computer System Forensics assignments. While similar in intent, it’s design and ability to be used even on a compromised system would save a forensic analyst time and decrease the chance of a memory image being corrupted when obtained or moved via systems.

I felt that improving the lab instructions would be helpful to my professors and make such handy programs less complex for students taking the class, and after speaking with the professors that teach the course it looks like the new lab will be in use next semester!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s