Rootkit Evolution

I saw my first rootkit in 2004, when I was still a rookie virus analyst. At that point I had some vague knowledge of UNIX-based rootkits. One day I stumbled on an executable for Windows that didn’t seem to do anything when I launched it. But I had a funny feeling about it and took a closer look-¦and saw a file in the list of loaded modules that weren’t present on disk. Obviously, I was lucky to be able to see this with the naked eye – the rootkit had errors in its code. Today I’d need a number of dedicated tools to achieve the same result, and even they might not be enough.

The rootkit I’d found was far from being the first Windows rootkit. However, it was new to me and served as a door into a new world; a world where programs played with the operating system and could break rules, miraculously disappearing from lists of processes and files. I spent an inordinate amount of time studying the drivers which the program used to hide itself in the system. Trojan-Dropper.Win32.SmallProxy was a program designed to target a specific system and deployed in specified locations – something relatively complex and unusual for that time.

This article focuses mainly on Windows rootkits – they are the most numerous, they are continuing to evolve, they pose a serious threat for users and because Windows is the most popular OS today, they are widely used by virus writers. I define rootkits as programs that evade or circumvent standard system mechanisms by using stealth techniques to hide system objects: files, processes, drivers, services, registry keys, open ports, connections and so on.

UNIX rootkits
In any discussion of rootkits, it is impossible to avoid mentioning the etymology of the term “rootkit’. In UNIX systems “root’ denotes an administrator with full privileges, while “kit’ is used to designate a set of tools. Thus the term “rootkit’ denotes a set of tools which can be used with malicious intent to gain access to the system unbeknownst to the real administrator. Such tools first appeared for UNIX in the early 90s. They still exist, but are not evolving in any significant way.

However, it’s important to remember that even though Windows rootkits have inherited the name “rootkits’ from the Unix world, Windows malware of this type is directly descended from DOS stealth viruses, not UNIX rootkits.

Stealth viruses
DOS stealth viruses appeared around 1990; at the same time, if not slightly earlier, than UNIX rootkits. Unlike UNIX rootkits, which were designed to enable the authors to gain access to the system and mask their presence within it, DOS stealth viruses simply hid themselves from the user and AV programs. This is exactly what modern Windows rootkits do. The techniques used by DOS stealth viruses are very similar to the ones used by Windows rootkits today. For instance, stealth viruses rely on techniques such as intercepting system calls and masking the malicious code by serving false data on disk or memory content. Windows rootkits also use these techniques extensively.

Windows rootkits appeared about 10 years later. Rather than naming these new programs stealth viruses, or some other more logical name, they became known as rootkits, thanks to Greg Hoglund. He was one of the first to build a tool designed to hide data in the system by combining various techniques for evading system protection features in Windows. He published his results in the e-magazine PHRACK (http://phrack.org/issues.html?issue=55&id=5#article) where the tool was named NT Rootkit. It went on to be used in many pieces of malware; in fact, NT Rootkit continues to intrigue and inspire both researchers and rootkit authors to this day.

Origins and popularization
Hoglund’s article is dated 1999, and based on research into the Windows kernel conducted the previous year and published on Usenet by a programmer from Sri Lanka. Back in 1995, Jeffrey Richter, a Windows programming guru, disclosed techniques for intercepting system calls in user mode in his famous book “Advanced Windows” and its fourth edition, which was called “Programming Applications for Microsoft Windows”. These techniques were later implemented in many rootkits, which even went as far as to copy source code directly from the book.

Techniques for intercepting system calls in kernel mode were disclosed publicly in two other classic programming manuals: Schreiber’s “Undocumented Windows 2000 Secrets”, published in 2001, and “Undocumented Windows NT” by P. Dabak et al, 1999.

The first Windows rootkits
Researchers continued to investigate Windows system protection, and soon after NTRootkit was released several other tools appeared, all designed to hide objects in the operating system:

• 2000 – he4hook, designed by a Russian programmer. The tool is not malicious, but it does hide files. It works in kernel mode. Interestingly, even the author doesn’t call this program a rootkit.
• 2002 – Hacker Defender (aka HacDef). This is also just a tool, but a more powerful one: it can be used to hide files, processes and registry keys with flexible settings in the configuration file. It also works in kernel mode.
• 2003 – Vanquish. This tool can be used to hide files, directories and registry keys. Moreover, it has a malicious payload – it log passwords. Vanquish works in user mode.

It is immediately clear that the thoughts of those researching such issues were changing course, with the focus moving from neutral tools to tools designed with ulterior motives, including malicious ones.

• 2003 – Haxdoor (aka A-311 Death and Nuclear Grabber, a modified variant of the same program). This is already more than a tool – it is a backdoor that uses rootkit techniques to conceal its presence in the system. It works primarily in user mode.
• 2004 – FU – a tool to conceal processes. This tool introduces a new technique based on modifying the system structure itself, rather than modifying access to the system. It works in kernel mode.

The list above is not exhaustive, but it includes rootkits which are key to understanding the evolution of Windows rootkits, especially HacDef, Haxdoor and FU. These 3 rootkits were commonly found in the wild in tandem with other malware. Rootkits from 2000 – 2004 fit neatly into the standard (outmoded) classification system: they can function both at user and at kernel level, by using Execution Path Modification or Direct Kernel Object Manipulation.

Rootkits – mass production
As rootkits evolved, they were also being embedded in malicious programs. In those days it was difficult to create stealth technologies independently, because little was written about this field: as a result, the small number of malicious rootkits could be divided into three categories:

• Trojans which used ready-made tools and libraries to hide themselves. The overwhelming majority of these Trojans used Hacker Defender and FU.
• Ready-made malicious rootkits which could be downloaded or purchased and which could be modified by the user. Haxdoor is one example. Like HacDef, Haxdoor was very popular in the fall of 2005; Kaspersky Lab was adding around ten new signatures daily to protect against new variants of Haxdoor.
• Custom rootkits developed for targeted attacks. AV vendors usually learned about these rootkits directly form customers, mostly large enterprises. Typically, virus analysts conducted on-site manual forensic investigations after network administrators couldn’t identify the cause of the problem. This group of rootkits was extremely small, but the samples showed a high level of technical sophistication.

By 2005, almost 80% of extant rootkits were variants of HacDef and Haxdoor. Rbot and SdBot were the first multi-functional backdoor Trojans to include built in rootkit technologies. The motive was clear; any technologies that improved the overall functionality of a commercial Trojan resulted directly in additional financial gain for the author and/or controller. Thus bot masters were the first to latch on to stealth/rootkit technologies.

By 2006 we saw rootkit technologies being built into common email worms such as Bagle; Trojan-Spy programs such as Goldun; and Mailbot programs, such as Rustock. This development proved to be a serious challenge for AV vendors. However, by the time using rootkit technologies in Trojans became standard there were a number of anti-rootkit tools, both standalone and in products. The balance of power was restored.

Rootkits and scandal
By 2005 the use of rootkit technologies in malware was so widespread that it fell under the gaze of the mass media and, naturally, security vendors. Microsoft representatives brought up the topic at RSA. The numerous scandals related to rootkit-like technologies found in various software and hardware products in 2006 clearly demonstrates how rootkits have become a public issue.

1. The Sony DRM copy protection on some CDs hid its files from users. Moreover, the technology was implemented in such a way as to create a serious vulnerability: anyone could name their own files in a certain way and the files would be hidden by the Sony DRM technology.

2. Symantec included a similar feature in their products: they used a directory that was hidden from users. This incident is rather amusing; Symantec’s ‘protected basket’ was documented by the company and was easy to disable. In fact, the concept of hiding files in this hidden folder is really not much more interesting than the concept of hiding files in the depths of the system directory tree, where no user ever looks.

3. The next to fall victim to rootkit related scandals was Kaspersky Anti-Virus itself; the product turned out to store certain data in file streams, i.e. in parts of the file system that are hidden from users. Although it’s not clear exactly what threat this posed, the use of the term “rootkit’ scared a lot of people.

Anti-rootkit hysteria
Another important aspect of the evolution of rootkits was the parallel anti-rootkit hysteria. By mid-2006 all major AV vendors had acknowledge it was necessary to react to the threat posed by rootkits. And every company reacted in its own way. Some vendors modified their products to access hidden objects during regular anti-virus scanning. Others released standalone anti-rootkit tools. Still others compromised by including anti-rootkit scanning in their products, making this function accessible via the product interface.

Truth be told, no one was particularly successful – it was simply a matter of locking the stable door after the horse had bolted. In the context of the escalating situation, F-Secure, whose anti-rootkit tool was released soon after Rootkit Revealer by Sysinternals, was one of the first major vendors to take action of note. The F-Secure tool only detected hidden processes, but was based on proof of concept technologies.

Vendor independent anti-rootkits
Vendor independent anti-rootkit tools appeared even earlier, around 2005. Unlike the solutions from AV vendors, who needed to make it obvious that they were protecting their users, the writers of free tools simply wanted to uncover as much hidden data as possible. Therefore, vendor independent tools were more professional, more powerful, and better able to react appropriately to the changing environment.

The first anti-rootkit tools were designed to reveal a single type of object e.g. hidden files. As time went on, they became increasingly multi-functional and used a systematic approach. Today, the most useful general anti-rootkit tools are GMER and Rootkit Unhooker, the latter is no longer supported. Both tools make it possible to conduct a rapid, superficial analysis from a number of angles of the condition of a system. They can also be used, if necessary, for deeper, more specialized analysis.

Today there are at least 20 free anti-rootkit tools, each based on a number of approaches to detecting rootkits. The battle rages on, however. One of the latest trends in rootkit evolution is the use of Bluepill technology (i.e. hardware virtualization) which can’t be detected using the technologies currently available. There simply aren’t any fully functional anti-rootkit tools that detect rootkits that use Bluepill technology. On the other hand, there are also no known fully-functional malicious programs which use this technique, only proof of concept code.

Nevertheless, there is work in progress on developing detection for these hypothetical threats. I am aware of two on-going projects investigating such rootkits – the Russian North Security Labs and the American Hypervista Technologies. To date, Hypervista has only released some theoretical information, whereas North Security Labs has posted a beta of their Hypersight Rootkit Detector from download from their site. Although the project is still in its early stages, it is interesting nevertheless.

Proof of concept rootkits
By 2006 most rootkits were detected by anti-rootkit tools and the rootkit boom started to tail off. Naturally, this led researchers, both blackhat and whitehat, to investigate new techniques which would be undetectable. Most researchers focused on the concept of using hardware virtualization (integrated into the new Intel and AMD processors) to gain control over the operating system. This method makes it possible to create rootkits which are undetectable by current anti-rootkit tools. Over 2006 three such proof-of-concept rootkits were publicized: SubVirt, BluePill and Vitriol. At the time of writing, detecting these rootkits is still an unresolved issue. However, they have not yet (as far as we are aware) appeared in the wild.

The second major area researchers are focusing on is rootkits that work in the boot sector, or bootkits as they are now called. This group of rootkits gains total control over the system while it is booting. This technique goes back to the boot viruses that reigned during the DOS era. The first proof of concept rootkit targeting the boot sector was eEye Bootroot, which appeared in 2005. Vbootkit, a similar proof of concept, appeared in 2007; it was presented as a piece of research which addressed the then hot topic of the day – Vista security.

Recent trends
After a relatively long period during which no new rootkits appeared, 2008 has brought new challenges for the antivirus industry. A new piece of malware appeared which purportedly infected the boot sector. It is known by a number of names: Sinowal, Mebroot, and StealthMBR. Most AV solutions still have trouble detecting it, much less disinfecting machines. This rootkit, or as it is usually referred to, bootkit because it runs during the boot sequence, is based on the eEye Bootroot code. Essentially, it’s not so much a separate piece of malware as a tool to hide Trojans – any Trojans. Consequently, it seems a reasonable conclusion that Sinowal is being shared (possibly for a fee) in certain circles and that we haven’t seen the last of it by any means.

Although it has had less attention paid to it, one other important recent technique is the use of CmRegisterCallback, functions created by Microsoft to help developers hide objects in the registry. A number of the more sophisticated malicious programs including Bulknet make use of this technique.

The mythical rootkit
At the end of 2006, security forums were rife with rumors about Rustock.c, supposedly a new and undetectable rootkit. This new rootkit was allegedly the latest development in the Rustock spam bot family. One and a half years later, researchers from Dr. Web, a Russian AV company, announced they had found a sample of Rustock.c and analyzed its propagation routine. Unfortunately, not all the details are available yet, so it is unclear to date as to how serious the threat really is. The data from Dr. Web seems to indicate that this rootkit was actually created in September of 2007, not late 2006. However, what is clear is that Rustock.c is not impossible to detect. The program does not use any cutting edge technologies that would make it impossible to detect. Although it does evade existing protection methods, this is not all that difficult – a number of less notorious programs also evade security measures.

In short, Rustock.c proved more interesting from the point of view of information flow than from a technical one. The rumors at the end of 2006 were unfounded, but they did prepare the ground for future events. When a real sample finally appeared everyone was excited to finally get a glimpse of the “undetectable’ rootkit – even though in reality Rustock.c is not technically of any particular interest. In my opinion, Rustock.c was significant for the reaction it provoked among the computing community rather than a serious technological leap forward. More details about the history of Rustock.c can be found at Virus List.

Non-Windows rootkits
From time to time virus labs get a sample of a targeted rootkits for less common UNIX systems such as Solaris. This is normally because a given organization hasn’t been monitoring its servers, which were configured a long time ago and which have been untouched ever since. It is easy to hack into such a system and plant a rootkit that is accidentally detected years later, only after significant amount of proprietary information or confidential data have been siphoned off, causing corresponding financial losses. There are several rootkits written for OS X (Macintosh) and there is even an anti-rootkit tool for this operating system: OS X Rootkit Hunter. Moreover, since OS X is Unix-based, some UNIX rootkits can be modified to work on OS X as well. On the whole, however, we are seeing neither a significant evolution of Mac rootkits, nor any serious threats in this area.

Operating systems for mobile devices, Windows Mobile, Symbian etc. have not yet been attacked by rootkits as far as I am aware. However, there is relatively little malware for devices running these operating systems, and few people use the AV solutions which are currently available, making the application of rootkit technology unnecessary.
Almost-rootkits

In this article I have chosen not to address certain proof of concept pieces of malware; they cannot be classified as rootkits and most researchers would not label them as such. If we state that a rootkit is a program that evades operating system security systems and hides its own activity, then the malware detailed below is certainly borderline.

First of all, there is malware that hides in the operating system, but in an ‘honest’ way: such programs neither modify the system nor evade built in security. For instance, there are malicious programs whose bodies are located in directory or file streams. Then there are programs that use documented system functions to install event hooks. These programs are not classified as rootkits, even though they do exhibit stealth behavior. Secondly, there are programs whose invisibility is achieved thanks to the program’s internal architecture e.g. the CodeRed worm, which neither creates files on disk, nor any processes in the memory. Thirdly, there are sophisticated techniques for tricking the operating system. These can be used by a malicious program not to hide traces of its activity but to burrow into the system or to evade AV solutions.

Rootkits – a few final thoughts
The borderline cases listed above lead to the conclusion that there is no such thing as a malicious rootkit, but rather a number of stealth techniques used to evade system protection and to hide programs in the system. Like any other powerful technology, these techniques can be used equally successfully for both good and ill. For instance, proactive anti-rootkits usually fight fire with fire, using the same approaches as rootkits such as hooking system functions etc. Any or all of these techniques can be classified as rootkits to a greater or lesser degree. In the final analysis, defining what exactly a rootkit is means either siding with the majority – the generally accepted definition – or the authoritative opinion of the minority, i.e. the experts.

So why is a precise definition necessary? Without it, we are in danger of slipping into verbal sniping over more or less borderline cases, such as the incidents where so-called rootkits are detected in products. It is crucial to analyze the code and techniques in each case in detail instead of simply labeling the technique as being a rootkit, thus scaring everyone and creating a lot of noise around the incident. For instance, the so-called rootkit which was identified in Kaspersky Anti-Virus. The technique used in the product which was seen as compromising entailed placing service information in file system streams, i.e. keeping the data in areas of the file system that are impossible to monitor directly.

Is this a technique for evading the operating system? No – streams are a documented operating system function. Is concealing data in these streams malicious activity? No, in this case only service information is concealed. Is this technique a vulnerability, i.e. can it be used for malicious purposes? No, a malicious user could only use streams to conceal malicious code (and this technique has already been implemented in a number of malicious programs), but this is an operating system vulnerability, not an application vulnerability. So, where is the rootkit? Not in Kaspersky Anti-Virus, or for that matter, not in any other product where a system vulnerability is confused with an application vulnerability.

The Sony DRM incident is slightly different. The techniques used in the copy-protection did result in a vulnerability in the sense that they allowed malicious users to name their malware in such a way as to make it invisible. Today, we are aware of several malicious programs that are exploiting this vulnerability, though to be fair, the malware appeared only after the vulnerability was disclosed and discussed.

Conclusion
Overall, rootkit evolution is following the same path as spyware. First, rootkits were identified as a separate class of malware. Then there was a lot of media hype which led to a large number of anti-rootkit tools and products together with a noticeable reaction from the antivirus industry. Today both rootkits and spyware have merged into the general malware stream and no longer cause any particular excitement. However, the concept of evading system features to hide something is obviously still valid and we are very likely to see new threats implementing stealth.