In many security incidents, response times were long because threats were not detected early enough and adequately due to multiple factors. These include the improvement of the threat actor's ability to evade detection or the lack of detection of the monitoring tools because they did not have the configuration or information about that specific threat.
That is why it is so important to have a proactive approach in incident response, where it is assumed that, at some point, there will be security compromises that could not be detected by the Security Operation Center (SOC).
One way to do this is through rules to hunt threats in a more specific way to have a clear idea about what and where you need to look for specific information, rather than very generically, as this would be the equivalent of looking for a needle in a haystack.
In this chapter, you will learn about the following:
In case you haven't already, you need to download and install VMware Workstation Player from this link https://www.vmware.com/products/workstation-player/workstation-player-evaluation.html.
You'll also need to download the following from the book's official GitHub repository https://github.com/PacktPublishing/Incident-Response-with-Threat-Intelligence:
Before we start with the practical exercises of this chapter, we will need to prepare the work environment.
To begin, start up the virtual machines that we will use throughout the chapter. To do this, follow these steps:
In the next part, we will start the Elasticsearch, Logstash, and Kibana (ELK) stack services on the IR-Workstation virtual machine.
As in the previous chapter, we will use the ELK stack to hunt for malicious behavior. If you didn't install it, follow the instructions described in Chapter 12, Working with Analytics and Detection Engineering in Incident Response.
If you started your virtual machine and the services are not initialized, run the following commands in a terminal of the IR-Workstation virtual machine:
sudo systemctl status elasticsearch.service
sudo systemctl status logstash.service
sudo systemctl status kibana.service
sudo systemctl start elasticsearch.service
sudo systemctl start logstash.service
sudo systemctl start kibana.service
This is shown in the following screenshot:
Once we have our virtual machines ready, we will begin working on creating rules for threat hunting in incident response.
You learned in the previous chapter how to work with detection engineering to identify malicious behavior in incident response.
Sometimes, you can detect this behavior through monitoring, but other times, this will not be the case, and you must act proactively to identify this behavior. This will be a priority to contain threats.
Turning detection engineering into actionable security through detection rules is one of the most important skills to develop as an incident responder.
Detection rules are structured patterns of key information to search for specific indicators in the form of queries that you can run on different platforms such as ELK, Splunk, NetWitness, and CrowdStrike.
There are different kinds of rules; for example, there are rules to identify Indicators of Compromise (IoCs) or specific content as YARA rules, or rules to identify Indicators of Attack (IoAs) as Sigma rules. You will learn how to use both in this chapter.
Detection rules are mainly used for threat hunting, and these can be run centrally or directly on the devices that you want to investigate. You can also automate the process of hunting threats, or you can do it manually.
YARA is a powerful tool developed by Victor Manuel Alvarez (Twitter handle @plusvic) from VirusTotal. With YARA, you can search files with specific content, and this is used by security professionals and malware researchers to identify and classify malicious files, but you can also use it to find any kind of content on a disk.
This tool is particularly helpful in incident response; for example, when you are in the triage process and you identify a file as potentially malicious, it will probably help you find that file on other devices across the network to size the compromise.
The creation of YARA rules is based on the identification of specific patterns of file contents, so you need to first get the suspicious file to analyze it and identify unique patterns that you could use to create the rule.
In the next subsection, you will learn the principal parts of the structure of a YARA rule.
The structure of a YARA rule is quite simple, as is shown in the following figure:
These are the main parts of a YARA rule:
According to the official documentation, there are three types of strings in YARA:
Text strings are case-sensitive by default, but you can use the nocase modifier at the end of the line to ignore the case.
You can use the wide modifier to search 2-byte strings per character.
If you don't define the wide modifier, the string will be ASCII by default.
Not all the sections are mandatory, but the simplest YARA must include at least the name of the rule and a condition.
There are several tools that you can use to create YARA rules. In fact, you can use any text editor, although it is recommended to use tools that support and recognize the specific syntax and the structure of a rule. In this way, it will be easier to create and debug the rules.
In this case, we are going to use Visual Studio Code (VS Code), since it has support for YARA by means of an extension queue you can download directly from the application. Other tools that can also support YARA are Atom, Sublime and Far Manager.
To install the YARA extension, first, you need to open VS Code:
Once the YARA extension appears, click on the Install button. This will install the YARA extension, as shown in the following screenshot:
Once the installation is finished, you can review the details of how to use the extension and the included features. Now, you are ready to create and use your first YARA rule.
An essential part of creating YARA rules is the identification of patterns in the files we want to search. There are several tools that can help us with this; in this case, we will use a tool known as PeStudio.
PeStudio is a tool to analyze files with a portable executable (PE) structure, for instance, Windows files with the .exe, .dll, and .bin extensions. You can find more details about the PE structure at https://docs.microsoft.com/en-us/windows/win32/debug/pe-format.
To download and install PeStudio, follow these steps:
The preceding steps are shown in the following screenshot:
This tool does not require installation, so once the folder is unzipped, it will be ready to use.
As mentioned previously, PeStudio will allow us to analyze the contents of a file that we could consider suspicious or malicious, and, in this way, we can create our YARA rule to search for that same file on other computers.
In Chapter 11, Integrating SOAR Capabilities into Incident Response, we downloaded the ProcDump tool from the Windows Sysinternals suite, which is used to dump processes from a computer and can be misused by malicious actors to obtain sensitive information such as passwords.
If at the identification stage in incident response, you found this program on one of the compromised computers and it was not an authorized tool for use on that computer, this could be an indicator that the threat actor used it as part of the attack.
We will use PeStudio to analyze this program. To do so, follow these steps:
PeStudio begins to classify the contents of the file according to its PE structure and content so that it can be easily analyzed.
On the left side of the tool, you can see the different groups of information that are identified in the file, and on the right side, its contents. For this example, we're going to focus on the following:
In this way, if you select the 0 (size) option, you will see the list of string values recognized in the file, as you can see in the following screenshot:
The challenge here is to identify the specific patterns that make this sample unique so that they can be used in search of this file.
For example, in the preceding screenshot, strings with messages appear that are unlikely to exist in other programs, so, they are good candidates to be used in our YARA rule.
On the other hand, if you move to the sections (blacklist) part, you will find highlighted in red the column that is in position 4 (in this case, the initial position is counted from 0) with a value in the name of _RDATA, as seen in the following screenshot:
In this way, you can collect information for the construction of your rule.
Now, look for additional information that you consider unique in this file; it is important to mention that quantity does not necessarily mean quality.
Building a good rule does not always require many elements, but you must identify those that can be important.
To start using YARA, you can download the most recent version from https://github.com/VirusTotal/yara/releases:
Note
You can add this directory to the environment variables of Windows to avoid explicitly specifying the path of YARA every time you execute a search using yara.
Once you have downloaded YARA, you are ready to create your first detection rule.
Open VS Code, click on the File menu, and select New File.
Now, click again on the File menu and select Save As….
The preceding steps are shown in the following screenshot:
When saving the file as a YARA type, VS Code will recognize the structure and syntax of the code, so that it can be debugged in a simpler way.
Next, we can start creating the YARA rule providing the specific strings and conditions about what we want to detect.
There is no single way to create a YARA rule; the best rule is the one that works and is efficient. The best way to learn how to create rules is by practicing; your skills will be perfected as you gain experience in doing so.
To create the YARA rule, use the following information:
import "pe"
rule procdump_tool {
meta:
description = "Simple YARA rule to detect the presence of Sysinternals Procdump"
version = "1.0"
strings:
$s1 = "D:\a\1\s\x64\Release\ProcDump64.pdb" ascii
$s2 = "Process cloning requires Windows 7 or higher." wide ascii
$s3 = "ProcDump Timed" wide ascii
condition:
(uint16(0)) == 0x5A4D and (filesize < 500000) and (2 of ($s*))
and pe.sections[4].name=="_RDATA"
}
The rule to detect ProcDump in a device must look as in the following screenshot:
The following are the elements of the rule explained in detail:
Note
It is important to mention that we need to add the escape character () when we define a path of a file or folder or when there are special characters, as is the case in the $s1 and $s2 string variables.
We created a YARA rule that would help us detect the ProcDump tool, and now, we are ready to test it.
Now, to work more efficiently, we will use the terminal integrated into VS Code. In this way, we can test our rule while we interact with it:
This will open a PowerShell terminal at the bottom of VS Code (which is configured by default).
Now, you have a Windows console window from where you can test your YARA rule and you can modify or debug it.
cd C:UsersInvestigatorWorkspaceToolsyara
yara64.exe -s "C:UsersInvestigatorWorkspaceLabsChapter-13procdump_tool.yara" -r "C:UsersInvestigatorWorkspaceTools"
In the preceding command, the following happens:
https://gchq.github.io/CyberChef/#recipe=Unescape_string()&input=UFx4MDByXHgwMG9ceDAwY1x4MDBlXHgwMHNceDAwc1x4MDAgXHgwMGNceDAwbFx4MDBvXHgwMG5ceDAwaVx4MDBuXHgwMGdceDAwIFx4MDByXHgwMGVceDAwcVx4MDB1XHgwMGlceDAwclx4MDBlXHgwMHNceDAwIFx4MDBXXHgwMGlceDAwblx4MDBkXHgwMG9ceDAwd1x4MDBzXHgwMCBceDAwN1x4MDAgXHgwMG9ceDAwclx4MDAgXHgwMGhceDAwaVx4MDBnXHgwMGhceDAwZVx4MDByXHgwMC5ceDAw
P.r.o.c.e.s.s. .c.l.o.n.i.n.g. .r.e.q.u.i.r.e.s. .W.i.n.d.o.w.s. .7. .o.r. .h.i.g.h.e.r...
You will learn more about CyberChef in Chapter 14, Hunting and Investigating Security Incidents.
One way to test whether this rule does not generate false positives is to run it on different directories or computers, mainly performing searches against files of the operating system itself or known application files.
My colleagues from Kaspersky's Global Research and Analysis Team (GreAT) developed an open source tool that allows you to install and configure your own server to test your rules and improve the quality of detections. This project is called Klara, and you can download it from this repository: https://github.com/KasperskyLab/klara.
In this case, we designed a rule to focus on the detection of the procdump64.exe file specifically, but what happens to the other two files, procdump.exe and procdump4a.exe, which can also be used by a malicious actor? Why were they not detected?
We did not review the other two files to see whether they contained the same strings or conditions present in procdump64.exe.
When this scenario occurs, it is very important that our rule can also detect those variants of the program that could also represent a threat and not only specific versions. To do so, it is recommended to analyze all the files and discard those elements that may not be common to everyone, and perhaps include some other elements that we might have overlooked.
Take your time to analyze what you need to modify in your rule so that the other files can also be detected.
Again, there is no one-stop solution, and you might find different ways to do it. In this case, try different solutions, for instance, comment out the last condition and you will get a different result. In this example, you detected the other ProcDump files but not the first one, as is shown in the following screenshot:
As you can see, if only the last condition is excluded (where the name of the _RDATA section is described), only the other versions of the program will be detected. This is because the _RDATA section doesn't exist in those files.
Now, it's your turn; try to find what you need to change on this rule to detect the three versions of the ProcDump tool by yourself.
As I mentioned before, YARA is an essential tool for incident response and there are many sources of information, courses, and repositories created by and for the community to make the detection process more efficient.
In the next section, we will learn about Sigma rules and how we can use them to identify malicious behavior.
One of the challenges for organizations is the standardization and integration of different security tools and the normalization of the formats that these tools use to process and store the information.
Sigma was created under the idea of developing generic rules in a structured format that can be transformed into specific query formats for different Security Information and Event Management (SIEM) systems. With Sigma, you can create rules under specific criteria of detection engineering, regardless of the platform you are using to hunt threats, as you can see in the following figure:
The advantage of this approach is that researchers can create universal rules shared with the community and used by everyone.
The main creators of the Sigma project are Florian Roth (Twitter handle @cyb3rops) and Thomas Patzke (Twitter handle @blubbfiction).
You can use Sigma along with other threat hunting tools such as YARA to detect threats in incident response.
The structure of Sigma rules is based on the YAML format, so it's very important to be familiar with this markup language. You can learn the basics of YAML here: https://www.cloudbees.com/blog/yaml-tutorial-everything-you-need-get-started.
Also, I recommend that you take a look at the Sigma specification wiki to learn about the structure of Sigma rules, at https://github.com/SigmaHQ/sigma/wiki/Specification.
Many targets support queries created by Sigma rules, such as Splunk, ELK, Azure Sentinel, ArcSight, QRadar, RSA NetWitness, and PowerShell.
Also, there are multiple projects and products that use Sigma, such as MISP, SOC Prime – Sigma Rule Editor, Uncoder.IO, THOR, and Joe Sandbox.
You can find more information on the GitHub of the project at https://github.com/SigmaHQ/sigma.
In the next part, you will learn how to get Sigma tools and create your own rules.
To have access to the Sigma project and the community Sigma rules, you can clone the repository to your IR-Laptop virtual machine:
git clone https://github.com/SigmaHQ/sigma.git
This will clone the Sigma GitHub repository onto your IR-Laptop virtual machine, as shown in the following screenshot:
cd sigma
ls -l
You will see the different directories of the project, as shown in the following screenshot:
In this case, we will use the content of the tools and rules folders to do the exercises.
You can convert rules into multiple query formats; in this case, we are going to run our queries in the IR-Laptop virtual machine looking for IoAs, so we will convert a Sigma rule into a PowerShell query.
In the previous section, we emulated malicious behavior; we used a living-off-the-land tool (certutil.exe) to download the Procdump64.exe tool (the command-line utility that creates crash dumps and is used sometimes by malicious actors).
After that, we ran ProcDump to dump the lsass.exe process and get the passwords from Windows.
This behavior generated multiple log entries on Windows-Sysmon. (If you didn't do this exercise or install Sysmon, you can follow the steps described in Chapter 11, Integrating SOAR Capabilities into Incident Response.)
In this case, we are going to create a Sigma rule to detect this behavior, but first, we will use the concepts of detection engineering that you learned about in Chapter 12, Working with Analytics and Detection Engineering in Incident Response.
Before creating a Sigma rule, we must first familiarize ourselves with the malicious behavior we want to detect and the possible sources of information to look for the IoA. We will start searching for this information in MITRE Cyber Analytics Repository (CAR):
Review the details of these analytics and go to the Implementation section; you will see the information you need to focus on for creating a detection rule. In this case, we are going to focus on a process creation where the name is procdump*.exe and the command line contains the *lsass* string, as shown in the following screenshot:
Now, using the previous information, we are going to create a very simple Sigma rule using VS Code.
Note
You can also download and use the SigmaRuleTemplate.yml file from the Chapter-13 folder in the book's repository to create the Sigma rule.
Your rule should look like the following screenshot:
Something important that you should consider is that threat actors almost all the time will seek to evade detections, therefore, it is important to create rules that detect these evasion techniques. In this case, we are creating a rule that detects the execution of ProcDump in a simple and direct way to obtain the dump of the lsass.exe process.
For instance, an evasion technique could be to call the process ID of lsass.exe instead of the name of the process, and this will work combined with other techniques to avoid detection.
To learn more about creating Sigma rules, you can see the tutorial created by Florian Roth here: https://www.nextron-systems.com/2018/02/10/write-sigma-rules/.
As I mentioned at the beginning of this section, Sigma rules are designed to define a generic detection format that you can convert through the field mapping to specific searches to be used by multiple platforms, so we are going to need a converter to do this. We will use two Sigma rules converters:
In the next part, we are going to learn how to install and use both tools.
Sigmac is the converter tool that uses the Sigma library; this library can be used to integrate with other projects and tools. To start using Sigmac, you will need to first install Python's Preferred Installer Program (pip) and then sigmatools:
sudo apt update
sudo apt install python3-pip
sudo pip3 install sigmatools
sigmac -h
You will see examples of how to use this tool with the different parameters, as you can see in the following screenshot:
Now that you have installed Sigmac, let's see how you can use it to convert rules for different targets.
At the bottom, you'll be able to see the built-in PowerShell console. In this case, we will need a WSL terminal, so we will have to add a new console.
You will see the new WSL console start working with the Sigmac converter.
cd /home/investigator/sigma
sigmac -t es-qs -c winlogbeat /mnt/c/Users/Investigator/Workspace/Labs/Chapter-13/proc_creation_win_procdump_lsass.yml
In the preceding command, we performed the following actions:
You will see the converted Sigma rule as the output shown in the following screenshot:
In the next part, you will run this query on Kibana.
To run the query created from the Sigma rule:
Note
If you get the message Kibana server is not ready yet when accessing the Kibana website, you will need to review the status of the ELK stack services on the IR-Workstation virtual machine and, if necessary, start them.
(winlog.channel:"Microsoft-Windows-Sysmon/Operational" AND winlog.event_id:"1" AND winlog.event_data.Image.keyword:*procdump*.exe* AND winlog.event_data.CommandLine.keyword:*lsass*)
You will see the list of occurrences about the execution of ProcDump in the period defined.
Additionally, you can add more detection elements to the rule to make it more accurate; test it for yourself by applying the concepts learned and consulting the reference links.
In the next part, you will learn how to create Sigma rules using the Sigma CLI.
For this part, we are going to convert Sigma rules into queries using the Sigma CLI.
sudo pip install pysigma==0.4.5
sudo pip install sigma-cli==0.3.1
This is shown in the following screenshot:
Note
You can also install the most recent versions of pysigma and sigma-cli by running the following commands:
sudo pip install pysigma
sudo pip install sigma-cli
sigma convert --help
Once the Sigma CLI is installed, we are going to convert Splunk queries from Sysmon and CrowdStrike Falcon Data Replicator (FDR) logs.
sigma convert -s -t splunk -p sysmon rules/windows/process_creation -o splunk_queries.conf
In the preceding command, we performed the following actions:
You will see the conversion process, as shown in the following screenshot:
vim splunk_queries.conf
You will see the different Splunk queries converted from the rules files, as shown in the following screenshot:
In the same way that you ran the query on Kibana, you can run any of these queries on Splunk.
As you can see, you can create generic Sigma rules from detection engineering, and then you can convert these rules into queries that can be run in different threat hunting platforms or SIEM solutions.
Fortunately, the cybersecurity community is developing more tools that use rules to search for threat indicators. These are some of the most interesting and free tools:
There are situations in enterprise environments where scalability is an important factor; for that reason, you should also consider tools that offer support for use cases where it is required to execute the rules on multiple devices on the network.
In this chapter, you learned about the importance of creating rules from detection engineering to detect IoCs, malicious tools, and malicious behavior.
You also learned the basic concepts about YARA and Sigma to create rules, and the different tools that can be used to improve the detection capabilities in incident response.
In the last chapter, you will have the opportunity to apply the knowledge learned in the different chapters of this book in practical scenarios of a simulated cybersecurity incident.