APTs: Minimizing losses with early detection

Let’s travel back to 2006, the year the blockbuster, “The Departed,” came out. Matt Damon plays a young criminal who has infiltrated the state police as an informer for South Boston’s Irish Mob. Working his way up the ranks, he gathers sensitive information about the plans and counter-plans of the operations he has penetrated and leaks them to his organized crime cohorts. Eventually, police suspect that there’s a mole in their midst. Now, we all know how this ends – Damon is exposed and killed by Mark Wahlberg for his stint – but not before wreaking havoc throughout the department.

There are some solid parallels between Damon in the Departed and the recent spate of high profile data breaches in which access is gained via authorized credentials and the perpetrator remains undetected for an extended period of time. This situation, unfortunately, is so common for organizations with high-value information that many experts advise them to operate on the assumption that they’ve already been breached.

With a seemingly infinite number of security products on the market, one might ask how we find ourselves in this situation. One explanation is that human nature makes us susceptible to a whole host of social exploits and phishing schemes, and once hackers gain legitimate access, signature based perimeter defenses are largely ineffective. Once inside the network, it appears that cyber criminals have carte blanche to move laterally, secure their objectives and steal whatever they want. And given that the average breach goes undetected for months, they seem to be able to take their sweet time doing it.

Attackers try hard to mask their activities – but try as they might, in order to accomplish their goals, their behaviors are likely to be anomalous at some point in time. Quickly detecting these anomalies as they develop could make the difference between losing tens of millions of customer records and losing a few hundred – or none at all.

While it seems somewhat obvious that looking for “unusual” activity would be beneficial to early detection, it turns out that in practice, it’s not as easy as it might sound. In fact, in a late 2014 Analytics and Intelligence survey performed by the SANS Institute, respondents said that the Inability to understand and baseline “normal behavior” (in order to detect abnormal behavior) is one of the top impediments to effective attack detection and response.

This is where machine learning anomaly detection technology comes in. It can process millions of data points each minute, establishing, or learning a “normal” baseline, comparing data points to past behavior, and identifying anomalous differences in values over time, differences in rates over time, and population outliers. Using this technology, user transactions, server processes, internet traffic, IPS alerts and proxy logs can all be analyzed for unusual activities.

An anomaly in a single dimension, say access to a never before seen external URL, may be uninteresting, so we certainly wouldn’t want to generate more useless alerts for the incident response team to investigate. Instead, anomaly detection software analyzes multiple data relationships, increasing the anomaly level or “score” when an activity is anomalous in multiple dimensions. For example, a large HTTP POST access to a new URL, from an internal system that typically doesn’t use the POST method might be an indicator of some sort of data exfiltration. Or an unusual number of DNS requests with a very large number of unique subdomains might be an indicator of malware command and control communication (C2).

While these are simple examples, and you might argue that a skilled IT team could perform these analyses manually or with scripts, albeit much more slowly, the real power of machine learning anomaly detection comes from the automated learning of baselines across multiple sources of system log and event data, relationships that security pros thought were virtually impossible to analyze using existing technology that relies on manual searching or script-based analysis.

As practical implementations of this type of big data security analytics become available to security teams, you might be tempted to think that we’ll no longer read about major data breaches going undetected. Unfortunately, this will take some time. A survey from Lieberman Software suggests that despite the continued occurrence of massive data breaches, an alarming 65 percent of security professionals believe that perimeter security technologies, such as firewalls and anti-malware solutions, are sufficient in defending against APTs. Hopefully, given the news of late, they have awakened to the fact that there is no defense sufficient to prevent APTs and that the emphasis must be shifted to early detection.

In the battle between IT teams and cyber criminals, the only way of spotting the metaphorical Matt Damon in your network may be machine learning anomaly detection.