Big Data is big noise

Big Data was supposed to be the solution to all our security problems, but this spotlight on intruders turned out to be a mess of white noise. Hiding comfortably in that noise, however, are legitimate indicators that point to valid network threats, such as suspicious user behavior.

There is still an immense opportunity to use Big Data security analytics to fish out these attacks, but you can’t just attach an engine on top of every piece of data that’s collected and expect it to miraculously extract intelligence. For an engine to be effective, it needs to focus not on the biggest data, but the most valuable.

The challenge for businesses is understanding that not all data is created equal and determining which feeds are truly important. There are three steps businesses can take to begin optimizing Big Data for analytical purposes.

Step 1: Understand the threat landscape
Businesses are logging every piece of IT data they can for both regulatory and forensic purposes, as they should. If something happens, organizations need to make sure they have access to all evidence to identify and rectify their vulnerabilities. But to protect against breaches, companies need to apply proper context to Big Data in order for it to be valuable. Most of the data that’s logged comes firewalls and intrusion protection systems (IPS) – but much of it is useless in flagging underlying or inside threats since modern attacks don’t trip any firewall or IPS rules.

Data breaches are on the rise, and hackers are walking right through the front door undetected by using stolen employee credentials and impersonating users. The tools that hackers use, such as malware and social engineering tactics like phishing, are increasingly becoming more targeted and sophisticated to get around common cybersecurity defenses and give them access to valid credentials. Businesses are failing to detect this activity because hackers try a multitude of different methods until they’re eventually successful.

You won’t find the security intelligence you need to identify evolving threats associated with these tactics in your firewall or IPS logs. That’s not to say that data isn’t important for forensic purposes – it is – but it’s all noise from an analytics standpoint, triggering false positives that distract security teams from quickly identifying imposters on the network.

IT security has become more of a science in that you have to focus specifically on what matters by looking at the feeds where individuals access IT. And once hackers have used stolen credentials to breach the network, it’s game over.

Step 2: Block out the noise to focus on the feeds that matter
IT security teams need to ask themselves which threats are most concerning that won’t be caught by prevention tools. These are the kinds of advanced threats that exist outside what is typically stopped by Web application firewalls (WAF). These technologies are obviously important to have – businesses should try as hard as possible to prevent credential theft – but this is not where the focus should be in terms of analytics purposes as they just contribute to the noise. To stop breaches before they take hold, businesses need to focus on the data feeds that point to how users are maneuvering within the IT environment in real time.

Take, for example, the 2012 attack on the South Carolina office of the IRS, which resulted in the exposure of more than 3.6 million Social Security Numbers and 387,000 credit and debit cards as a result of an employee who fell victim to a phishing attack and had his credentials compromised.

Firewalls were not strong enough to stop the breach, but it could have been prevented had the feeds that users and systems access, like the virtual private network (VPN), been more closely monitored and analyzed in real time. By vetting the identities accessing the VPN, the IRS would have found the imposters logging in.

Step 3: Quantify the data
Once you’ve identified the feeds that matter and have a smaller pool of data to look at, this is where an engine will be effective in extracting intelligence. Not all anomalous activity in an IT network is a potential threat, but a pattern of suspicious user behavior is often a good indication.

Hackers will always leave fingerprints and a trail of information that will point to malicious intent, and security teams need to act before it’s too late. Having to separate valid from invalid threats takes away time that could be better spent responding to an attack underway.

To win the war against today’s hackers, businesses need to focus on valuable data, rather than Big Data. By having a better understanding of the threat landscape, businesses can block out the noise and focus only on the data feeds that matter, creating the spotlight they need more quickly identify modern day attacks.