Best practice methodology for industrial network security: SEC-OT

Secure Operations Technology (SEC-OT) is a methodology and collection of best practices inspired by a decade of experience working with secure industrial sites. The SEC-OT approach is counter-intuitive to many IT and even industrial control system (ICS) security practitioners. It turns out that secure industrial sites ask different questions and get different answers.

For example, industrial sites generally do not ask “how can we protect our information?” Instead, they ask, “how to we keep the site producing high quality outputs?” and “how do we control the site safely?” To this end, SEC-OT defines control system security as:

Control system security – Protecting safe and reliable physical operations by assuring correct and authorized control.

SEC-OT does not seek primarily to “protect the information” as information security does. SEC-OT observes that all cyber attacks are information and concludes that it is not information that needs protection, but physical operations that need protection from information – more specifically from cyber attacks that may be embedded in information.

SEC-OT principles

The first principle of SEC-OT therefore, observes that since all cyber attacks are information, every comprehensive inventory of information flows into an industrial site is also a comprehensive inventory of inbound attack vectors. Control these information flows and we control the attacks.

The second key principle observes that since all software has vulnerabilities, our primary defenses should be physical and hardware-enforced controls over information/attack flows entering ICS networks, rather than software-based protections. All software has defects after all – discovered and undiscovered – and some defects are security vulnerabilities. Using vulnerable firewalls, cryptosystems and security software to protect vulnerable ICS software is a lot like bailing a basement with a bottomless bucket.

First control incoming information/attack flows physically and then think about software security.

Offline / physical attacks

Offline information/attack flows include removable media, BYOD, laptops, cell phones, brand new computers from our vendors and even people with passwords and possibly malicious intent. To the greatest extent practical, SEC-OT advocates physical protection from offline information flows into ICS networks. For example: physically block or disable removable media ports and do the same for unused network connections, on computers, switches and wall jacks in ICS networks. Deploy secondary software protections as well, such as Active Directory policies to forbid mounting removable media and Network Access Control (NAC) to forbid connections from unauthorized laptops and other computers.

In addition, SEC-OT adapts the “near miss” concept from physical safety programs. When the NAC system reports an attempt to connect a non-ICS laptop to an ICS network, or an ICS laptop reports an attempt to connect it to an IT network, the Security Operations Center (SOC) contacts the offending individual and together they fill out a “security near miss” report. These reports are aggregated, analyzed and used to prioritize remedial actions in terms of training and awareness programs. With a strong “near miss” program in place, site personnel quickly learn never to attempt dangerous information transfers.

Certain information flows into ICS networks are unavoidable however – new software versions, anti-virus signature updates, new computers to replace aging ones, and so on. SEC-OT sites accommodate these needs most commonly through a combination of thoroughly patched anti-malware scanning kiosks and heavily-instrumented ICS test beds. The kiosks scan incoming media with typically 4-8 anti-malware engines, often augmented with cloud-based sandboxing systems. Approved files are copied to fresh ICS media and carried to a file server on the ICS test bed. Here the files are deployed and tested for threats to safe, reliable and secure operations, before being deployed on live control networks.

Insiders at the site are a different offline problem. Secure industrial sites generally do not ask “how can I convince my people to stop using USB drives?” That problem was addressed through a combination of SEC-OT physical, software, near-miss and test bed procedures above. Secure sites ask, “how can I prevent a trusted insider from mis-controlling the physical process?” There are no silver bullets here, but no mysteries either.

Industrial sites have been dealing with this problem since before computerization. SEC-OT draws on long-standing personnel security best practices including: regular personnel background checks and risk assessments, as well as deterrence in the form of detailed auditing, video surveillance and other surveillance, as much as is practical and permitted by local privacy laws.

Online / remote access attacks

Online information/attack flows include network connections, firewalls, serial connections, and wireless connections. This is where SEC-OT differs most sharply from less-fit-for-OT practices. SEC-OT requires hardware-based or physical controls over online connections to external networks such as IT networks and the Internet and forbids software-only mechanisms to control such flows. Encryption, whether hardware or software-based, firewalls and two-factor authentication are specifically called out as inadequate to the task of protecting control systems from external, online attacks.

To enable safe IT/OT and OT/cloud integration, secure ICS sites use hardware-enforced unidirectional gateways for connections from ICS networks to external networks. For readers not familiar with unidirectional gateway technology, the U.S. National Institute of Standards and Technology (NIST) defines such gateways as:

Unidirectional gateway – A combination of hardware and software. The hardware is physically able to send information in only one direction … [and] the software replicates databases and emulates protocol servers and devices. – U.S. NIST 800-82 Rev 2

Security practitioners not yet familiar with unidirectional gateway technology may find the technology and its applications counter-intuitive. To address this gap, the new book documents 18 network reference architectures using unidirectional gateways:

SEC-OT reference architectures

Database access

For example, many ICS networks consolidate data for IT/OT sharing in a historian or SQL database hosted in a DMZ network between the IT and ICS networks. The “database access” reference architecture explains how a replica database is established on the enterprise IT network and a unidirectional gateway replicates the industrial database to the enterprise network in real time. Enterprise users and applications access the IT replica database normally.

Remote support

Another example is remote support. Since unidirectional gateways replicate systems one way from ICS to IT networks, many practitioners assume that remote support is impossible. In fact, there are several reference architectures for secure remote support, the simplest of which is “Remote Diagnostics and Maintenance” which uses Remote Screen View (RSV). RSV replicates workstation screens from the ICS network to an external network across a unidirectional gateway.

This means remote experts can see the actual screen used by the on-site personnel and see the mouse moving and other actions in real time. These experts can then advise site personnel over the phone, providing real-time advice as to how to investigate and remediate complex problems. RSV delivers an experience resembling a physically-enforced “read only” remote desktop, allowing remote personnel to see, advise and assist, but prevents anyone from controlling the ICS network. Other remote support architectures meet other, specific remote access needs.

Universal central security monitoring

In addition to physical and hardware-based protections, SEC-OT advocates using a wide variety of secondary software protections, the most important of which is universal central security monitoring. Such monitoring usually consists of a central or cloud SIEM, ICS network intrusion detection system (NIDS) sensors and other ICS network-based and host-based sensors. Such security monitoring is vital to any corporate security program, essential to incident response and recovery efforts and is the basis of the near-miss program, which in turn is essential to offline/physical information flow/attack vector protections.

While security monitoring is a detective measure rather than a preventive one, when such monitoring is coupled with skilled, practiced and responsive incident response teams, the combination can often reduce or prevent the physical consequences of attacks. More fundamentally though, we can optimize only what we can measure. Measuring and improving security over time is important at industrial sites, and security monitoring is an essential tool for such improvement.

Security monitoring is enabled in the SEC-OT methodology by the “Central and Cloud SOCs” and “Network Intrusion Detection” reference architectures. The former sends information to central SOCs through unidirectional gateways, so that no compromise of SOC equipment or any part of the non-OT infrastructure can pose a threat to industrial operations. The latter sends network traffic captures from the ICS network to NIDS sensors through unidirectional gateways. This allows us to deploy the sensors on the IT side of the gateways where the sensors are easily managed, while the sensors receive ICS traffic captures with no risk of any attack being sent back into the monitored ICS network.

Future-proofing

Global trends work against the cause of ICS security:

• Computers continue to become smaller, more powerful and more ubiquitous, and where there are computers, there is vulnerable software.
• Networking continues to become faster, easier and more more ubiquitous, and as information flows increase, attack opportunities increase.
• Cyber attacks take advantage of these opportunities and continue to increase in number, consequence and sophistication.

On the surface, this looks bad for our heroes. These global trends though, are precisely what motivated the pioneers of SEC-OT principles and practices at secure ICS sites. Again:

1. All information flows are attack vectors, so inventory these flows / vectors and control them.

2. All software is vulnerable, so control attack vectors using physical and hardware mechanisms rather than software.

Neither of these principles needs to change as global trends continue, inexorably.

What does need to change is the current sole dependence of many industrial sites on software protections. As the sophistication of our attackers increases, so must the sophistication of our defenses. Given these global trends, it is inevitable that more and more industrial sites will choose a robust security posture and adopt SEC-OT principles.