New detection system spots zero-day malware

A group of researchers has created a new infection detection system that can help Internet service providers and large enterprises – or anyone running large-scale networks – spot malware attacks that antivirus and blacklisting solutions can’t.

The latter are reasonably successful when it comes to stopping known malware, but the fact that they rely on signatures and lists that are compiled after a piece of malicious software has been analyzed and identified as such means they aren’t adept at spotting zero-day malware.

Drive-by download attacks consist of three phases:

1. The exploitation phase, during which the attacker aims to run shellcode on the victim’s computer
2. The installation phase, during which the aforementioned shellcode fetches the actual malicious binary and runs it,
3. The control phase, during which the malware reaches out to its C&C server for instructions, additional malware, and to send information.

Nazca (as the researchers dubbed the tool) does not seek to detect drive-by exploits that lead to malware downloads, nor does it depend on the analysis and reputation of downloaded programs – it concentrates on spotting malware distribution infrastructures (i.e. on the second phase).

It does so by looking at the bigger picture – the combined traffic produced by a great number of users on the same network – and spotting telltale signs such as potentially malicious HTTP requests (most drive-by exploits use the web to download malware binaries) and suspicious web connections employing evasive techniques (domain fluxing, malware repackaging, etc.).

Finally, Nazca aggregates all these connections and searches for related malicious activity.

According to the researchers, the tool worked well when tested on nine days of data traffic provided by an unnamed ISP – it proved immune to content obfuscation, spotted never before seen malware, and turned out very few false positive.

For more details about their research, check out their paper. They will also be presenting their research at the upcoming Network and Distributed System Security Symposium.