JPCERT/CC Eyes

Volatility 3 had long been a beta version, but finally its v.1.0.0 was released in February 2021. Since Volatility 2 is no longer supported [1], analysts who used Volatility 2 for memory image forensics should be using Volatility 3 already. In this blog post, I introduce a tip for Volatility 3: how to use Volatility 3 offline. This instruction focuses on analyzing Windows OS memory image.

What is the problem in using Volatility 3 offline?

Malware and forensic analysis are sometimes conducted in an offline environment to reduce risks of malware infection and data breach. However, Volatility 3 shows the following error message and cannot be used with its default configuration in an offline environment.

Volatility 3 Framework 1.2.0
WARNING  volatility3.framework.symbols.windows.pdbutil: Symbol file could not be downloaded from remote server
Progress:  100.00       PDB scanning finished
Unsatisfied requirement plugins.Info.nt_symbols: Windows kernel symbols

A symbol table requirement was not fulfilled.  Please verify that:
    You have the correct symbol file for the requirement
    The symbol file is under the correct directory or zip file
    The symbol file is named appropriately or contains the correct banner

Unable to validate the plugin requirements: ['plugins.Info.nt_symbols']

This error does not occur with Volatility 2 because its package contains the profiles for memory image analysis of each OS. Instead of the profiles, Volatility 3 uses Symbol Table [2]. It is not included in the package but automatically generated in every memory analysis. A Symbol file of NT kernel is necessary when creating a Symbol Table, and Volatility 3 downloads the Symbol file from Microsoft website. That is why Volatility 3 shows the above error message in an offline environment.

To solve this problem, you first need to manually create a Symbol Table. Here are the steps:

Steps to create and use a Symbol Table (for Windows OS)

1. Identify the Symbol file to download
2. Download the Symbol file and create a Symbol Table
3. Apply the Symbol Table on Volatility 3

1. Identify the Symbol file to download

You first need to identify the Symbol file of NT kernel required to create a Symbol Table. With -v option, scan the memory image you are investigating.

$ python3 vol.py -v -f test.mem windows.info
Volatility 3 Framework 1.2.0
INFO     volatility3.cli: Volatility plugins path: ['/volatility3/volatility3/plugins', '/volatility3/volatility3/framework/plugins']
INFO     volatility3.cli: Volatility symbols path: ['/volatility3/volatility3/symbols', '/volatility3/volatility3/framework/symbols']
INFO     volatility3.framework.automagic: Detected a windows category plugin
INFO     volatility3.framework.automagic: Running automagic: ConstructionMagic
INFO     volatility3.framework.automagic: Running automagic: LayerStacker
INFO     volatility3.framework.automagic: Running automagic: WinSwapLayers
INFO     volatility3.framework.automagic: Running automagic: WintelHelper
INFO     volatility3.framework.automagic: Running automagic: KernelPDBScanner
INFO     volatility3.framework.symbols.windows.pdbconv: Download PDB file...
WARNING  volatility3.framework.symbols.windows.pdbutil: Symbol file could not be downloaded from remote server
INFO     volatility3.framework.symbols.windows.pdbutil: The symbols can be downloaded later using pdbconv.py -p ntkrnlmp.pdb -g 15B12C74F0E177581B6B27DD4C5022C21
INFO     volatility3.framework.automagic: Running automagic: KernelModule

The above result says that the symbol file can be downloaded using pdbconv.py. -p refers to the PDB file name, and -g refers to the file's identifier, which consists of "[GUID][AGE]" (PDB file name can vary: ntkrnlmp.pdb, ntkrnlpa.pdb, ntkrpamp.pdb, ntoskrnl.pdb, etc.).
Microsoft manages Symbol files with these values, and thus you need to specify PDB, GUID, and AGE in the URL to download a specific Symbol file.

http://msdl.microsoft.com/download/symbols/ntkrnlmp.pdb/15B12C74F0E177581B6B27DD4C5022C21/ntkrnlmp.pdb

Since GUID and AGE are based on the certain values in NT kernel, you can get them using the following script.

import binascii
from pefile import PE

pe = PE("C:\\Windows\\System32\\ntoskrnl.exe")
debug = pe.DIRECTORY_ENTRY_DEBUG[0].entry        
guid = "{0:08x}{1:04x}{2:04x}{3}_{4}".format(debug.Signature_Data1,
                                             debug.Signature_Data2,
                                             debug.Signature_Data3,
                                             binascii.hexlify(debug.Signature_Data4).decode("utf-8"),
                                             debug.Age)
print(guid)

2. Download the Symbol file and create a Symbol Table

You can automatically download a specific Symbol file and create a Symbol Table by running the script contained in Volatility 3 in an environment where the application can access Microsoft website.

$ python3 volatility3/framework/symbols/windows/pdbconv.py -p ntkrnlmp.pdb -g 15B12C74F0E177581B6B27DD4C5022C21

After created, the Symbol Table is saved with the name similar to 15B12C74F0E177581B6B27DD4C5022C2-1.json.xz. This file is a XZ-compressed json file. By default, Volatility 3 loads a file named "[GUID]-[AGE].json.xz" as the Symbol Table to use.

If you have already downloaded a Symbol file from Microsoft website, you can specify it in this way instead to create a Symbol Table.

$ pdbconv.py -f BC8878BDD1A2849E42F82393A5053B88A42197AE4EC5F058F95C2208903785A100.blob -o 15B12C74F0E177581B6B27DD4C5022C2-1.json.xz

* Change the exported file's extension to ".xz" to compress the file in XZ format.

3. Apply the Symbol Table on Volatility 3

To use the created Symbol Table, save it in the following directory:

volatility3/volatility3/symbols/windows/ntkrnlmp.pdb/

You can also specify any other directory with -s option instead.

$ ls symbol-dir/windows/ntkrnlmp.pdb/
15B12C74F0E177581B6B27DD4C5022C2-1.json.xz
$ python3 vol.py -f test.mem -s symbols-dir windows.info

In closing

In this blog post, I introduced how to create Symbol Table for analyzing Windows OS image memory. Such method is only available for Windows OS, and thus you need to manually create Symbol Table for macOS, Linux, and other OS [3]. For these OS, you can create a Symbol Table using the tool called dwarf2json, which I will introduce in another time. The Symbol Table for Windows OS is available on our GitHub, and I hope it helps when you use Volatility 3 in an offline environment.

GitHub: JPCERTCC/Windows-Symbol-Tables
https://github.com/JPCERTCC/Windows-Symbol-Tables

Shusei Tomonaga
(Translated by Takumi Nakano)

References

[1] Volatility Labs: Announcing the Volatility 3 Public Beta!
https://volatility-labs.blogspot.com/2019/10/announcing-volatility-3-public-beta.html

[2] Volatility Docs: Changes between Volatility 2 and Volatility 3
https://volatility3.readthedocs.io/en/latest/vol2to3.html

[3] Volatility Docs: Creating New Symbol Tables
https://volatility3.readthedocs.io/en/latest/symbol-tables.html