Inside the largely unexplored world of mainframe security

The security of mainframe computers – the so-called “big iron”, which is mainly used by large organizations for critical applications, bulk data and transaction processing – is not a topic that has garnered much interest from the public. And, according to Phil “Soldier of Fortran” Young, the security community has not shown much interest so far, either.

“The two biggest misconceptions about mainframe computers is that they are unhackable, and that they are legacy and therefore don’t deserve our attention or focus,” says Young, who helps financial institutions protect their mainframes.

“The belief of them being ‘unhackable’ usually stems from a misunderstanding of how hacking works. They think someone has to find 0-days to exploit a mainframe, when in reality all they need to do is find a misconfigured webserver, a user account that has an easy-to-guess password, and so on.”

Even though US-CERT rates traditional mainframes as one of the most secure computer systems due to a small number of vulnerabilities (when compared to the thousands affecting Windows, Linux, and other similar systems), Young believes that another reason for the belief is that fewer people know about mainframes and even fewer target them. If they are less likely to be attacked then, yes, technically, mainframes are “more secure” – but not “unhackable”.

“As for the idea that these machines are ‘legacy’ and on the way out, that is totally false,” Young notes. “They are modern operating system with their own nomenclature. They offer the same, and sometimes better, controls that other operating systems offer. Just because the operating system originated in the 70s (and was re-written in the 90s) doesn’t make it legacy. Like I said in my BlackHat talk – parts of the NT likely still exist in Windows 10. But does that make it a ‘legacy’ operating system? No.”

The fact is, mainframes are extremely useful computers, and are at the basis of almost every big and important service and business – retailers, banks, insurers, governments. Mainframes are backward compatible, and have high hardware and computational utilization rates and extensive input-output facilities. And they are highly reliable, which makes them a much better alternative to a cloud infrastructure.

“These systems are nowhere near leaving the enterprise,” says Young. “Sure, we hear from time to time that a company is planning on switching to an alternative. But usually after looking at the costs they change their mind.”

Still, many organizations that use mainframes never test them – mostly because they are afraid that a penetration test could bring down one of their core systems.

“If a network security expert, with no knowledge about the mainframe, is able to bring it down with a simple Nmap scan, then that should be fixed, not ignored,” he opined. “However, the likelyhood of that happening today is almost zero and is fueled mostly by old wives’ tales from the late 90s when Nmap could bring the mainframe down under specific circumstances.

Young became interested in mainframe security in 2011. He scoured the Internet for tools, guides, anything to help him out do an audit of a mainframe and, when he found nothing, it became a problem that he set out to fix.

“When I say there was nothing online, I mean there was nothing – there was a link to a password cracker from 2000 and a post to the Nessus mailing list. That was it,” he explained.

“When I started my blog and talks I figured there would be no interest. But year after year I get more interest as people start to do research in this space. So much so, in fact, that we had a little ‘mainframe hacker meetup’ at DEF CON this year.”

Young and his colleague Chad “Big Endian Smalls” Rikansrud have been working on spreading the word about the issue of mainframe security.

They have been doing presentations about their work on security conferences (you can find some of the videos here), writing blogs, developing tools, and listing online resources in an effort to get the conversation and research started.

“I started out small with the tools. Mostly simple scripts as PoC. A perfect example of this is a shell script I wrote called Enumerate TSO, which would check user IDs of a mainframe (and works due to the way the TSO panel divulges information),” says Young. “It was slow but it worked. It has since been replaced with an Nmap script which does the same but is much faster:

“Generally, I’m making tools to make a penetration testing easier. Either I wrote it because I needed it to do a pentest or because I wondered if it could be done or not. Other tools for Nmap include a Network Job Entry (NJE) named node brute forcer.”

Another tool he wrote (and released at DEF CON) allows for submitting messages using NJE commands. “Once you have the node names (using the Nmap script) you can use iNJEctor to submit commands,” he explained.

“I think the most interesting thing I’ve learned in all the time I’ve been exploring mainframes is, by far, how unexplored this area really is. It almost feels like I’m exploring a vast unexplored island with an old castle – who knows if I’m going to find anything, but the journey is fun and interesting,” he shared.

“For example, I gave a talk at Skytalks about finding mainframes on the Internet. It started with a though ‘I bet there’s internet facing mainframes’, and it eventually led to me having the only database that I know of of internet facing mainframes.


“On top of that I’ve learned more about the inner workings of various tools and scipts than I ever would have if I just used those tools,” he noted. “At one time I wrote a Metasploit meterpreter in REXX (a mainframe scripting language). This required me to learn the inner workings of both Metasploit, meterpreter, REXX and z/OS. Another great example is writing scripts for the Nmap scripting engine – I am intimately more familiar with the inner workings of Nmap now than I ever was.”

Another plus of his research is the community. “I know I’ve ragged on them in the past in my talks (and if you ever dare go to mainframe forums, turn back, do not go there) but when I had the opportunity to meet people in real life at conferences it was truly far more welcoming and open than I had ever expected.”

In the end, I wanted to know what practical advice could he offer to organizations running mainframes to improve their security.

“Get a pentest done. Just do it. It won’t be great because the skillset isn’t there today but you have to start somewhere,” he advised. “After that, use standards like the DoD DISA STIG for z/OS to lock it down and make sure your auditors are actually auditing your mainframe properly. Finally, keep your mainframe up to date. If you have a mainframe you need to sign up for IBMs z/OS security portal where they will give you patches to security vulnerabilities for the system.”