Intuitive Detections Research With Graph Analytics and Neo4J

By Nik Seetharaman

TLDR

I explore using graph analytics tools to visualize simulated attacks during the course of detections research to gain an intuition of what an attack is doing across multiple axes. I also discuss streamlining detections research workflows by automating Sysmon setup, teardown, and log export.

Background

On the heels of DEFCON, Black Hat, and the overall 2018 security research season, there are a number of new offensive techniques I’m psyched to begin researching through the lens of the Blue Team, especially some of SpecterOps’ and Will Schroeder’s new C#-based tooling. 

One of the challenges in doing Blue Team and Detections research is being able to easily digest relationships among various entities on a system during the execution of a particular attack. Event Viewer isn’t sufficient to gain such contextual understanding, and tools like Splunk and ELK aren’t much better when trying to execute link based analysis. Additionally, it’s a pain to time-bound logs to the exact events you care about after running a simulated attack. Ideally this constraining would happen upstream of the analysis such that a high percentage of the events we receive are directly related to the attack we are attempting to research.

In this post I’ll walk through how to use graph analytics with Neo4J to visualize what happens during execution of an attack, as well as how I think about the overall workflow of:

  • Firing the attack while only capturing the events we want
  • Extracting objects and relationships out of the event logs
  • Loading the objects and relationships into Ne04J to visualize them

The ultimate goal is to be able to execute a desired simulated attack (i.e. one of the many in Red Canary’s Atomic Red Team) and then quickly and intuitively evaluate what the footprint of that attack is along various dimensions.

Tools

Developing the graph-based research workflow is reliant on the following:

1. A VM on which we will be running our atomic attack test.
2. An analysis machine separate from the victim VM.
3. Sysmon with a modified version of Swift on Security’s configuration file.
4. Batch scripts to automate Sysmon configuration, log clearing, and exporting of logs.
5. Python to parse the exported Sysmon logs and generate queries that we’ll need to use in Neo4J.
6. Neo4J Desktop and Neo4J Browser to visualize our resulting graphs.

Automating Sysmon Setup and Teardown

We can obtain a victim VM from Chris Long’s Detection Lab project and then install Sysmon with the config located here. 

We can also download additional configuration files for more tailored research from Olaf Hartong’s modular Sysmon project.

We can then utilize a batch script on the victim VM to automate the process of setting up Sysmon after specifying the configuration file that we want depending on our research goals. The modified Swift on Security configuration provided above is a good start. After we run the attack that we want to research and capture the relevant logs, another batch file can be used to tear down that configuration and return Sysmon to a baseline state.

The reason we want to do this is we may want to use several versions of a Sysmon configuration in sequence to evaluate the footprint of a given attack under that specific set of Sysmon rules. Some of those rulsets may cause high system load, and in order to preserve VM and host system performance, we’ll want to revert Sysmon to a base state of not collecting anything when we’re not actively running attacks.

That no-capture configuration will look like this:

The following batch commands can then be used for the pre-attack setup:

In the first command, we use the Windows Event command line utility to clear any existing logs. Line 2 sets up a variable to capture the first user specified command line argument for the Sysmon configuration file we want to use. Line 3 executes Sysmon with that configuration.

After we run the atomic attack that we intend to research, we use the next set of batch commands in order to export logs, tear down and revert Sysmon to a no-capture baseline:

Line 1 again uses the Windows Event command line utility to query Sysmon logs and output an XML file that we will use later on. Line 2 reverts Sysmon to a no-capture state by using a special configuration file called “nocapture.xml” and finally line 3 clears the logs.

(Note – it is feasible that instead of reverting to a no-capture state, we could simply uninstall Sysmon via running the command Sysmon -u. This however can take longer and induce latency into your workflow.)

We’ll now run the setup script, execute Casey Smith’s remote COM scriptlet execution attack (MITRE T1117) as our simulated attack, and then tear down Sysmon:

We now have the log data we want to visualize in the form of an XML file. The next step will be to extract entities and relationships between them in order to populate the graph.

Modeling Objects and Relationships

Departure from log analysis towards graph analysis requires that we take the event logs in the XML output and extract entities and relationships from it that can then be displayed on the graph in the form of nodes and edges (links) between them. To do this, we’ll use a Python script to turn each Sysmon event into a Neo4J Cypher command that expresses the relationships between entities in that Sysmon event.

Cypher is Neo4J’s graph query language that can create graph objects and describe relationships between those objects using ASCII art syntax. For example, to express that calc.exe is a child process of cmd.exe, we would say:

This would show up in the Neo4J Graph as:

To do this for a Sysmon Process Create event, we would write the following to translate from the XML to Cypher:

In this case, we are using the command MERGE instead of CREATE so that Neo4J will bind relationships to objects if they already exist, or create objects and then bind relationships to them if they do not.

Each Sysmon Event ID will require merging potentially new types of objects (files for example) and then modeling the relationship between the Process GUID and the resultant object.

Let’s write XML to Cypher translations for most major Sysmon event types that we’d care about:
– Process Creates
– Network Connections
– Image Loads
– Process Accesses
– File Creates
– Registry Value Sets

We’ll also have a small helper function called getFilename to parse out a display-friendly name from the full path shown by Sysmon.

 

Parsing the XML Logfile

After creating our Cypher translation functions, we’re ready to parse the XML logfile containing the event data for Casey Smith’s Squiblydoo attack.

In the above code, we create a dictionary-like object using the collections.defaultdict() class to hold the Sysmon Event Data we need to extract entities and links. We then select the proper Cypher translation function depending on the Event ID and in the case of Process Access events, suppress anything with MsMpEng.exe (a Windows Defender process) and because I’m using VirtualBox, VBoxService.exe (VirtualBox’s Guest Additions service).

Our resulting Cypher queries look like so:

 

Generating the Graph

After downloading Neo4J and opening the Desktop client, we are presented with something similar to the following:

Click on New Graph and select “Create Local Graph.” Enter a name for the graph and set a password. Once the graph card populates with the new graph, click “Start.” Then click “Manage” and in the resulting window “Open Browser.”

This will bring you to the meat of the operation, the Neo4J Graph Browser:

Cypher commands may be run from the command bar at the top, with results of any commands / queries sequentially appearing in the results area below. New results will show up at the top of the result stack. To clear the stack, you can type “:clear” in the command bar.

To generate our graph, we’ll keep it simple and copy / paste our generated Cypher code into the command bar, and then hit CTRL+Enter to execute it. Each query will then run in sequence.

To view the resulting graph, we can click on the button labeled “*(9)” on the left under “Node Labels.”

The above graph serves as an intuitive visual reference about the different interactions between the objects involved when we ran our Squiblydoo attack, namely various Process Accesses, cmd.exe spawning regsvr32.exe and regsvr32 subsequently spawning calc.exe. Finally we have the fairly obvious network connection from regsvr32.exe to the external IP address where it pulled down the remote scriptlet from Github.

The relationship type label on each link lets us know what the nature of the relationship between nodes is. In the case of this attack, we observe types of:

  • Accessed
  • Spawned
  • ConnectedTo

Note that you can constrain what is visualized on the graph by selecting Node Labels or a certain Relationship Type on the left. The result will be generated in a new panel. So if I only wanted to see Process Spawns, I would select the “Spawned” relationship type and only be presented with the cmd->regsvr32->calc flow.

To clear your work and start from scratch (i.e. to try a new cypher import), clear the current database by typing

Then type

It is left as an exercise to the reader to:

  • Implement the automation discussed
  • Run several other atomic red team tests and capture the event logs via Sysmon
  • Export the logs to XML and translate them to Cypher
  • Load the translated Cypher into Neo4J to visualize each attack

In the coming weeks I’ll be working on a follow on to this post focused on utilizing graph analysis to research the nuances of unmanaged Powershell.