Interview with Gregory Conti, Assistant Professor of Computer Science at the United States Military Academy

Besides his work as a professor and Director of the Information and Technology and Operations research center, Gregory Conti is also the author of the excellent book Security Data Visualization and the RUMINT visualization tool. His work can be found at gregconti.com.

Introduce the concept of security data visualization to our readers.
Security data visualization seeks to create insightful graphical windows on security datasets, files, file systems, network communications, and logs. It excels at providing big picture context that is impossible using text and simple charting techniques. More importantly, security data visualization is inherently interactive, allowing analysts to take cumbersome data and iteratively study slices of activity and find new and interesting patterns, outliers, and anomalies. If done correctly, the process is fun and powerful, but it is important to avoid the common pitfall of creating just pretty pictures, which while beautiful, don’t provide useful insight.

Visualization is obviously of great significance for analyzing large amounts of data. Many also praise its usefulness when it comes to illustrating security problems to the management. What are the areas in which security visualization comes out as essential?
From my experience it is possible to use visualization to study ten to one hundred times more data than competing manual methods. It is probably possible to increase this gain to one thousand times or more with very well thought out visualization systems. There are number of places that I see visualization as very valuable. Visualization is at its best been you are dealing with the new, unfamiliar or when you don’t really know what you are looking for. It facilitates exploration of data whether in a static dataset or when faced with dynamically changing data, such as in network communications or protocols, particularly those that aren’t well documented.

In your opinion, how important is the visualization of security data in general?
I believe visualization is quite important if used properly. When I first began looking at security data visualization I imagined graphical intrusion detection systems. I’ve since backed away from that idea because it isn’t realistic to expect 100% attention from a human operator all the time. However, I’ve found data particularly useful when conducting forensic analysis. For example, a friend and I were looking at the network communications of a new gaming system. We captured network packets from the console back to the server and spent a good deal of time learning the unfamiliar protocol offline. A common problem in systems development is security through obscurity. Designers assume that no one will poke into the odd corners of systems. Anyone familiar with security analysis will tell you this is a bad design idea. Visualization helps lift the veil on systems designed using security through obscurity and shows data in ways that designers didn’t intend, with a great deal of success. Visual cryptanalysis is another area that I feel bears great promise. The right visualization systems can help identify flaws in cryptographic implementations that are difficult to detect using traditional manual analysis and machine processing techniques. Anytime you hear the phrase “art and science” that is an indicator that visualization may be helpful. The trick, and the fun, is designing the right graphical windows

What are the best security visualization tools available at the moment?
This is a tricky question, because security visualization tools usually come in two forms – prohibitively expensive (on the order of tens to hundreds of thousands of dollars) and free. I like Raffy Marty’s open source project Afterglow because it is powerful and flexible enough to be used with many types of security data. The prefuse toolkit helps Java developers create powerful visualization applications. On the commercial side, I believe ArcSight, Splunk and Secure Decisions make very nice products. I’ve been very impressed with Zynamic’s BinNavi and BinDiff. Halvar Flake and his fellow researchers at Zynamics really know their stuff. There are a number of general purpose tools that can be used for security and other types of data, examples include IBM’s free Many Eyes service and TIBCO’s SpotFire. Finally, visualization is a very active research area. I’d recommend monitoring the output of VizSEC, the Workshop on Visualization for Computer Security and the National Visual Analytics Center as well as the VizSEC and SecViz portals for the latest developments. VizSEC 2008 will be held September 15, 2008 in conjunction with the Recent Advances in Intrusion Detection (RAID) Symposium and we invite people interested in visualization to attend. Here you’ll find bleeding edge ideas, before they turn into products.

You are the creator of the network and security visualization tool RUMINT. Introduce the main features of the tool as well as some of the possible usage scenarios.
RUMINT is a visualization tool designed to allow insightful analysis of network traffic, either in real-time as a packet sniffer or using historical pcap data. It uses a VCR-like interface to control playback of the packets in seven different visualization windows, similar to the way a Tivo allows a user to record, pause and play back video. Each window provides a different view on the same packets, at the same time, allowing analysts to correlate insights gained from each display. Analysts have found it useful for creating graphical fingerprints of network attacks, analyzing attack tool behavior, communicating Internet security threats to end users, understanding the operation of new protocols, and finding anomalies in network traffic.

Despite these strengths, I consider RUMINT a work in progress and I’m working on a complete rewrite of the source code now. The hacker and computer security communities have been very enthusiastic about the tool and are providing awesome feedback. While the latest version is stable, useful, and usable, I believe RUMINT hasn’t reached its full potential. I want to incorporate the feedback I’ve received to take the tool to the next level. My goal is to create a result that is compelling enough to make Fyodor’s Top 100 security tools list. This list is based on a survey of what tools the security analyst community finds most valuable. Fyodor’s list is an honor that must be earned, and I look forward to seeing RUMINT on it one day.

What kind of evolution can we expect in the upcoming versions of security visualization tools? What new features would you like to see?
I expect to see tools that help bridge the gap between visualization and machine processors. Human time and attention are rare resources, particularly that of experts. Once an analyst makes a discovery, such as the signature of a new form of malware, the visualization tool should make it very easy to offload the signature to machine processors, such as anti-virus programs and intrusion detection systems, to do the heavy lifting in the future. Currently, there is a distinct gap between security visualization systems and machine processors in many areas.

How long did it take you to write Security Data Visualization and what was it like? Any major difficulties?
It took about three years of research and about 14 months of writing to produce Security Data Visualization. The most frustrating aspect was banging into the edges of human knowledge. Visualization of security data is, in many ways a very young field, many aspects are relatively unexplored, such as visualization support for cryptanlysis and malware analysis, but bear great promise. I believe there are several decades before we start seeing the full range of possibilities.

There were some aspects of the writing process that helped greatly too. Without exception, the community of security visualization researchers freely contributed images and ideas to help with the project. Raffy Marty of Splunk contributed two excellent chapters. John Goodall of Secure Decisions served as technical reviewer and was the best person on the planet to fill that role. My editor, Tyler Ortman, exceeded all of my expectations by actively researching security visualization and pointing out valuable extensions and ideas. Near the end of the process I found out that he had studied physics before turning to editing. I couldn’t have asked for a better editor. Bill Pollock, founder of No Starch Press, was also a pleasure to work with. He puts his heart and soul into his books and never rushed the timeline; instead he was focused on creating the best possible book. I wholeheartedly recommend No Starch Press to people considering writing security books. I can’t overstate the importance of support like they provide when writing a book.

What are some of the interesting facts you discovered while researching for this book?
Perhaps the most interesting insight is that most software developers and attackers do not seem to anticipate visual analysis. It’s the classic security through obscurity problem. With the right visualization you are essentially lifting the lid on a previously invisible file structure or type of network activity. For example, I was visually examining password protected files produced by a popular word processor and found that the text was encrypted, but images were not. With the visualization tool I was building you could see the images as plain as day in a matter of seconds. The same analysis would have been much harder, if not impossible, with a hex editor. I found the same issue arises regarding network activity. While at a hacker conference in Atlanta, one of the capture the flag participants said after seeing an early version of RUMINT, “we’ll need to change the rules of capture the flag, once tools like that become available.” I agree with him, visualization helps change the rules of the security game. They help reduce security through obscurity.

I also learned that the hacker community is an excellent environment for incubating new ideas. Every time I presented research to Defcon, Interz0ne, and Black Hat, I received invaluable feedback and support. I’d like to see more interaction between academia, industry, government, and the hacker community, we will all benefit.

What are your future plans? Any exciting new projects?
Currently I’m working on several projects. I’m conducting research in the use of visualization to enhance reverse engineering and fuzzing of binary data, including data and executable files, storage devices, and RAM. In many cases the state of the art tool for low-level analysis is the classic hex editor. We have to be able to do better than that. Hexadecimal representations of text allow only a frustratingly tiny window onto very large chunks of data. The initial results are promising and I anticipate significant potential in gaining big picture context and understanding the behavior of closed file formats. Along with some friends, I hope to have a visual file analysis tool ready to distribute by this summer as an open source project.

I’m also working on a second book, tentatively entitled Googling Security. It explores how much information users pour into the databases of online companies through their use of online tools such as search, calendaring, email, and mapping. In many ways I see Googling Security as the inverse of Johnny Long’s excellent Google Hacking work. Google Hacking is all about how to retrieve interesting security information via online search. Googling Security focuses on how much information online companies can pull from users. I’m hoping to change people’s views regarding “free” online tools and the implications of such activity.

The views expressed here are those of the author and do not reflect the official policy or position of the United States Military Academy, the Department of the Army, the Department of Defense, or the U.S. Government.