Adaptive and Behavioral Approach to New Threats

A wake up call

It was early morning on a typical business day when an intrusion detection system generated an alert of unusual TCP activity at a customer’s Boston branch office. Someone was scanning the network’s internal subnets for a backdoor program that could be used to control remote systems.

Network security specialists quickly determined this activity as hostile and identified the intruder as a consultant working for the client. After a verbal warning the hacking attempt ceased, only to resume later that night as a more sophisticated and difficult-to-track User Datagram Protocol (UDP) hacking attempt.

The hacker had switched to another remote control program consisting of two components: a server component that uses a virus like stealth mode to distribute itself on a network, and a client component the intruder can then use to explore and control the infected network.

But good detective work and a mistake on the part of the intruder, led to his demise. Rather than confront the intruder immediately, security personnel scanned the hacker’s own computer and began capturing forensic data. That analysis revealed that the hacker had inadvertently installed both the client and server components of the remote control program on his own system – a flaw the security exploited to turn the hacker’s own tool against him.

A detailed examination of the consultant’s computer revealed sensitive information taken from the customer’s network, and by logging the intruder’s actions, the security team amassed absolute proof of the attempted theft.

Realizing he had been caught, the consultant worked desperately to delete both the pilfered files and his hacking tools, but thanks to good intrusion detection technology and sound security management, this hacker was shut down before he could do any serious damage.

Highlighting the trends

That early-morning threat can be seen as the exception that proves the rule of internet security. It was an exception, because unlike so many cyber attacks, the intruder was identified and thwarted before he could cause significant harm. Yet it proved the rule that in today’s online society, companies and agencies of all kinds are threatened by a rising tide of internet-based intrusion, crime and warfare.

“To protect themselves, organizations must understand who is trying to compromise their networks and the tools those intruders use,” says Mike Stute, co-founder and Chief Technology Officer of Global DataGuard. “They must also understand the technologies that are available to identify and fight those attacks.”

Experts who track online attacks say the perpetrators can be categorized into five broad groups: the archetypical hacker who takes perverse pleasure in successfully breaching a network or in the creation of a new worm or virus; an insider acting out of anger or greed; industrial spies opening online pathways to steal intellectual property; individual or organized criminals committing web-based fraud; and governments and non-government groups using cyber attacks to further their political objectives.

When internet security issues were first raised more than 15 years ago, most intrusions amounted to little more than the exploitation of passwords or other clear vulnerabilities.

In today’s far more complex world, the intrusion profile includes the exploitation of known flaws in protocols, source code and executable files, sniffer programs, IP source address spoofing, DoS attacks, automated scanning, distributed attacks, and the creation of command and control networks that use compromised computers to launch attacks.

A growing threat

In the early years of the internet, most attacks were launched against individual computer systems or networks. But with the rapid growth in home PCs, broadband access and the size and complexity of the internet itself, attacks today are characterized increasingly by the use of easily available exploitation scripts, by compromising large groups of computers for use as DoS weapons, and by leveraged attacks on the infrastructure itself.

The CERT Coordination Center is an internet security clearinghouse located at the Software Engineering Institute, a federally funded research and development center operated by Carnegie Mellon University.

According to CERT, in 1988 the organization received just six reports of internet security incidents, a number which swelled to 3734 a decade later, and by 2003 the number of internet security incidents reported to CERT numbered a staggering 137,529.

“The complexity of administration of computer and network infrastructures makes it difficult to properly manage the security of computer and network resources,” CERT noted in a 2003 report.

“As the number of internet users grows and intruder tools become more sophisticated and easy to use, more people can become ‘successful’ intruders.”

Countermeasures

As the threats to networks and systems have evolved, so too have the technologies deployed to meet those threats.

Most companies and agencies long ago installed firewalls, antivirus scanning software and user authentication systems. But to fully understand both the current status of their systems, and to detect and counteract developing threats, organizations are fortifying those static defense measures with more proactive and predictive security technologies.

To really understand what is going on in your network, you must do more than deploy security devices, you must also monitor your security situation on a constant basis. Intrusion detection monitoring is a major trend in the security industry.

One early form of intrusion detection, called knowledge-based monitoring systems, continuously scans strategic points in a network, and then compares current activity against a periodically updated database of known worms, viruses and other threats.

Knowledge-based intrusion detection, also known as signature based, provides a proven and cost-effective line of protection. But much like the more familiar virus scanning systems, knowledge based ID can only detect and defeat known threats. When a new worm is created or when someone tweaks the code of an existing threat – events which occur with unfortunate regularity – knowledge-based systems are vulnerable until that variant is identified and cataloged.

To provide the proactive security needed in today’s dynamic IT environment, a new and more powerful form of intrusion detection has now emerged.

A behavioral approach

This new approach, called behavioral intrusion detection, uses sensors placed at strategic points throughout an organization’s network – such as at the firewall, on internal servers, databases and other locations – to monitor and analyze potential security threats.

The first generation of behavioral intrusion detection systems employed an initial “learning mode’ period, during which the data collected by these sensors is evaluated and stored, and used to create a profile of network behavior under typical operating conditions. Once a profile has been established, the system is switched to monitoring mode, and current network activity is compared to the profile to identify and investigate potential security threats.

Behavioral ID represents a notable advance in security protection, but those first-generation systems suffer from a timebased limitation not unlike those associated with library-based scans. Once the learning mode is switched off, these early generation behavioral systems can identify only those threats contained in the established and increasingly obsolete profile.

Nor can those set-profile systems adjust quickly enough to effectively monitor the changing behavior of robust enterprise networks – networks that change constantly as organizations launch new business initiatives, consumer demand fluctuates and security threats emerge and mutate.

Adaptive profiles

To meet the dynamic needs of today’s networks, a new generation of technology has now added a sophisticated adaptive capability to the science of behavioral intrusion detection.

These adaptive solutions collect data from host and network ID devices on an ongoing basis, and then constantly analyze and correlate that information to create a continually evolving – and thus always current – behavioral profile of the network. This predictive and adaptive approach essentially creates a custom security system for every organization, and provides optimum protection in an environment where both network traffic and security threats are constantly changing.

By capturing and analyzing raw packet data, versus aggregated log files, over long periods of time, this preventive approach can identify previously unknown threats, covert channel attacks and sophisticated evasion techniques. The enormous volume of historical data is automatically correlated across all customer devices and internet-wide intrusion attempts.

In fact, a true behavioral analysis system handles data in volumes similar to those managed by security information management (SIM) systems, except that behavioral systems correlate and analyze that data based on continually changing learned normal behaviors, rather than through the use of business rules.

Adaptive behavioral ID catches things other systems simply cannot see. It’s a way for network managers to stay one step ahead of the bad guys.

In a true behavioral system, sensors should be deployed not just on the enterprise side of a firewall (where they would identify only those threats that have already breached the wall), but also outside the firewall where all intrusion attempts can be monitored.

You should think of the firewall as a cop, and if you stand behind the cop you see only the things that the cop misses. By putting sensors outside the firewall, you detect and analyze early-stage probes of the network. And those early detections can often thwart future attacks.

“Network security is best viewed as a process, and that process must be managed,” says Stute. “There are people out there who are constantly working to compromise your network, so you must work constantly to protect it.”

Don't miss