What Is Threat Intelligence?

There are a number of characteristics that define threat intelligence, but there is a general industry consensus around the definition proposed by Gartner:¹

Threat intelligence is evidence-based knowledge, including context, mechanisms, indicators, implications and actionable advice, about an existing or emerging menace or hazard to assets that can be used to inform decisions regarding the subject’s response to that menace or hazard.

This definition covers all aspects of threat intelligence from collection, to processing, to the decision-making process. It also covers the many different types of information that should be collected as part of the intelligence process.

In short, threat intelligence is any information that can be correlated with additional information in a manner that allows an organization to improve their security in a tangible manner. For example, if a third-party vendor tells an organization that Anonymous has been launching attacks against other organizations in the same vertical, and provides a list of tools that Anonymous is using to launch these attacks, that is threat intelligence.

In order for data from outside sources to be considered threat intelligence, it must be:

• Relevant: It must impact the organization in some way.
• Actionable: Concrete steps can be taken by security teams to protect the organization.
• Contextual: There should be enough evidence included to enable an intelligence analyst to effectively rank the threat.

New threats spring up all the time, but not all of those threats are relevant to every organization. For example, a new vulnerability against the webmail application SquirrelMail is not relevant to an organization that is not running SquirrelMail, no matter how severe the vulnerability is.

Simply knowing that Anonymous has launched new attacks is not sufficient because there is no actionable information. That basic knowledge does not provide an organization with enough information to take steps to ensure it is protected against those attacks.

Finally, in order for information to be considered threat intelligence there must be context surrounding it. The SquirrelMail vulnerability mentioned above is a perfect example. While a vulnerability in a popular Internet-facing application could be a cause for concern, if an organization is properly managing and scanning its network assets, the security team should be able to quickly determine whether SquirrelMail is installed anywhere on the network. If SquirrelMail is not installed, then there is most likely no threat to the organization from the vulnerability. It is that additional context—knowledge of an organization’s network assets—that can turn information into threat intelligence.

Of course, context does not always have to originate from within an organization. In fact, context can be entirely external. For example, if a third party provides an organization with the email address [email protected] as an indicator associated with the infamous NotPetya ransomware attacks, that provides some level of context. If an organization matches against that address in their logs or in collected NetFlow data, they will know there is a good chance they have a NotPetya attack. The first part of the intelligence, that [email protected] is associated with the NotPetya ransomware attack, is not something that would be picked just reviewing internal data, unless the organization happened to fall victim to the NotPetya attack. Instead, the context around the email address would come from a third-party source that is collecting data from thousands of different networks.

Context can be deeper than just a single connection. Knowing that the email address is associated with the NotPetya attack is a baseline of useful information. But tying that email address to other indicators associated with the attack, such as IP addresses and domains used for command and control purposes or file hashes associated with the malware is even better. Providing information about attack atmospherics—such as what organizations are being targeted or the fact that the email address has been disabled, so if an organization is infected they should not try to pay the ransom—is the ideal level of context.

What Threat Intelligence Isn’t

Now that there is a consensus definition of threat intelligence, it is important to take a step back and explain what threat intelligence isn’t. Threat intelligence is not a list of IP addresses or domain names with no context. Threat intelligence is also not a proprietary platform that exists solely inside a security vendor’s tool. Finally, threat intelligence is not data that rests solely in a portal or in a report that is isolated from the rest of an organization’s network.

Each of the examples listed in the previous paragraph can help create threat intelligence, but they are not threat intelligence in and of themselves.

Examples of “not” threat intelligence that are seen most often tend to be around the context of the indicators. A third party might share that IP address 101.200.81.187 is malicious. That isn’t threat intelligence because there is no context surrounding it. Knowing that a third party classifies an IP address, domain, file hash, or some other indicator as malicious, without understanding how they came to that conclusion, is not threat intelligence. In fact, these types of context-free data feeds can make the security team’s job more difficult, as they don’t have any context around which to judge a threat.

Problems with context can even happen with internal intelligence, which is why documentation between teams is so important. For example, a firewall administrator may receive a report that several hosts on the network are attempting to call out to a single IP address several times a minute. Based on that report, the firewall administrator puts a rule into the firewall blocking traffic to that IP address and goes on to the next task.

With no documentation in place, when the firewall administrator goes home for the evening and the night administrator comes into work, there is no information about why the rule was created. In addition, there is no information about whether or not the original traffic was really a threat, or, if it was, what the associated malware was and whether it has been as successfully cleaned. Actions, even those seemingly benign as a firewall rule change, that pass between different security teams can become threat intelligence for the organization. But they require context to understand what the threat is and determine whether there are additional actions that need to be taken.

Data Feeds versus Threat Intelligence

Many organizations get their start with threat intelligence through data feeds. A data feed is a list of indicators provided by a third party that can be correlated against internal security systems to find matches that can be acted upon.

The appeal of data feeds is understandable—there is a lot of malicious activity happening at any given time on the Internet. No single organization can see all of that activity. By aggregating indicators from security incidents across the Internet and combining them into a single data feed, organizations can better protect themselves against coming attacks or use the indicators to find attacks they may have missed.

As most organizations quickly find out, data feeds rarely live up to their promise. Too often, data feeds contain outdated data and don’t provide context or give security teams enough information to make the data actionable. Even the purported purpose of data feed correlation, to provide relevance, doesn’t always work as expected.

Many organizations who start down the path of a threat intelligence program with data feeds are surprised to discover that these feeds often create more work than expected. Correlating data feeds against logs from other network devices in a Security Information and Event Manager (SIEM) seems like a natural fit—after all, it will help security teams identify security incidents that may have been missed otherwise. Unfortunately, what many security teams quickly find out is that, rather than making their lives easier, data feeds generate more work. By identifying even more security incidents and, often, many more false positives, data feeds can quickly become a burden to security teams.

That is the primary difference between a data feed and threat intelligence, even threat intelligence delivered in feed format. Threat intelligence helps security teams improve the security posture of the organization and makes the security team more effective and responsive to potential security incidents.

That doesn’t mean that data feeds don’t serve a purpose. In fact, the data feed concept is an important one for improving an organization’s security posture. One of the biggest challenges that organizations of all sizes face when it comes to network security is that for 30 years, the security industry, as a whole, has solved the latest security problem by “adding a box” to the mix. First, it was firewalls, then intrusion detection systems (IDS), proxies, endpoint protection, web application firewalls (WAF), data loss prevention (DLP), and so on, to the point that many organizations have 20 or more security systems in their network, none of which talk to each other.

This leads many security teams to suffer from “console fatigue.” They constantly have to jump from one security vendor’s console to another’s, and correlation is often done manually, as in “Oh wait, I think I saw that same indicator in an alert from my endpoint vendor. Let me go see if I can find it.”

Data feeds, delivered in an automated fashion, from one security system to another can help improve the security of an organization by having those systems communicate. That automation between systems, even cloud systems, allows the security team to have a better view of what is happening within their organization and allows them to make more effective and faster decisions when it comes to prioritizing incidents.

Even external data feeds, when they are processed correctly, can help an organization improve security. Some data feeds can be fed directly into a firewall, mail server, or even fed directly into a DNS server as a Response Policy Zone (RPZ). These tend to be specialty feeds that are well-vetted and serve a specific purpose. Most threat data feeds from reliable sources can contain valuable data that, when combined with other information, can eventually become threat intelligence.

Threat Intelligence from the Inside Out

Inevitably when an organization is serious about building out a threat intelligence program they start by looking externally. Whether the start involves looking for threat feeds, as described above, or reaching out to vendors who sell threat intelligence, there is a strong draw to rely on vendors to deliver threat intelligence.

The problem is that even the best threat intelligence providers can’t deliver effective threat intelligence unless the organization knows what their needs are. Remember, actual threat intelligence has to be relevant, provide context, and be actionable. In order for these requirements to be met there must be something against which to correlate the external collection.

To that end, the strongest threat intelligence programs start internally and work their way out. Often, this is the hardest part of building a threat intelligence program: getting the internal data from the network into the hands of the people who need to be able to analyze it.

Make no mistake, there is a lot of valuable security data inside an organization of any size that is owned by different teams and this data is very siloed. For example, the vulnerability management team gets threat intelligence from their vendors regarding new vulnerabilities and exploits that target those vulnerabilities. However, the vulnerability team is usually separate from the security team, so the security team doesn’t always learn about these new threats in a timely fashion. But it goes deeper than just getting data from one team to another—a successful threat intelligence program has to encompass the entire organization, and it needs to start from the top.

Summary

Threat intelligence is a complex topic that can be daunting to organizations that are just starting to build out a threat intelligence program. It doesn’t have to be though. Starting with the definition outlined in this chapter, it is possible to build out a threat intelligence program that uses relevant, actionable indicators to provide context to threat or a potential threat.

The ultimate purpose of threat intelligence and the threat intelligence team is to protect the core assets of an organization. In order to do that effectively, the threat intelligence team needs to understand the core mission of the organization and what the organization considers to be its most valuable data. All intelligence requirements should stem from that.

While third-party threat intelligence can be invaluable, it doesn’t help an organization improve its security unless it can be correlated against data collected within an organization. Some of that internally collected data will be log data, but internal intelligence is more than just log dat—it also involves understanding the processes of other groups within the organization.