What will it take to improve the ICS patch process?

While regular patching is indisputably good advice for IT networks, one of the main takeaways from the Petya and WannaCry attacks is that a lot of companies don’t do it. And with even more NSA exploits like EternalBlue scheduled to be released by The Shadow Brokers (TSB), it’s certainly not going to get any better.

Patching IT systems is hard enough, but it’s even more difficult to patch industrial control systems (ICS), commonly found in energy, manufacturing, pharmaceuticals and other verticals powered by Operational Technology (OT) networks that pre-date the Internet. These systems, which have been controlling complex machinery for years, run 24×7 and have limited maintenance windows, and often rely on custom software tethered to older, no-longer supported versions of Windows. A patch might very well crash the system, and in the ICS/SCADA world reliability and uptime are the number one priority.

Understandably this has led to an “if it ain’t broke don’t fix it” attitude towards security, where unpatched software – whether it’s vulnerable firmware in Programmable Logic Controllers (PLCs) or older versions of Windows in SCADA workstations – remains unpatched for extended periods of time.

But something’s got to give. The risk level from those unpatched systems won’t be acceptable for much longer. In FY 2016, the ICS-CERT Incident Response team completed work on 290 incidents and coordinated 305 vulnerabilities, but those numbers may be misleadingly low because most industrial organizations lack the technology and personnel to detect intrusions when they happen. As a result, many incidents are never reported. Also, unlike retail and financial services organizations that are mandated by law to report consumer data breaches, industrial organizations are not required to report ICS intrusions (and potentially expose themselves to lawsuits if they do).

Furthermore, the continued evolution of customizable, autonomous malware specifically developed to manipulate ICS devices, such as the newly discovered Industroyer/CrashOverride, will greatly reduce the level of expertise needed to launch attacks on OT networks. Just imagine an attack that blends the sophisticated propagation of Petya/WannaCry with the destructive payload of Industroyer/CrashOverride … now do you see why protecting vulnerable ICS systems is so important?

As bleak as that might sound, where there are challenges there are also opportunities. Here are some ideas on how to evolve ICS security processes to better match the real-world needs of OT organizations:

1. Evolve the culture

ICS networks often weren’t designed with security in mind, but neither were IT networks in their early days. It took years, an increasingly visible stream of mega-breaches and legislation (such a breach notification laws) to motivate business and IT leaders to make cyber defense a strategic business imperative. Once they did, and the culture shifted, CEOs and boards re-evaluated the importance of security and became willing to fund it accordingly. Today, there is much more transparency and information sharing around breaches, which has helped to drive new and better approaches for dealing with them.

The ICS patch process is ripe for such transformation. Sure, it’s no surprise that OT organizations find ICS patching painful or even impossible, given their absolute requirement for uptime and the risk of applying ICS patches without extensive testing. But Industrial Internet CEOs need to work with their industrial automation vendors and system architects to upgrade their ICS environments for a net-enabled future – one that weaves essential security and “patchability” into the fabric of their network operations.

2. Encourage stronger controls via more carrot, less stick

When companies were either behaving badly or refusing to take fundamental security precautions in the IT realm, legislation such as SOX and consumer data protection laws became a major lever for establishing required security baselines.

However, as cybersecurity policy experts including Richard Clarke have acknowledged, “regulation” is often a dirty word. So instead of punishing companies for not doing the right thing, let’s provide tax incentives and/or federal subsidies for implementing “Cyber Homeland Security” initiatives and upgrading our critical digital infrastructures – not just our roads and bridges.

3. Promote compensating controls

Legacy OT assets and protocols lack basic security capabilities such as strong authentication that we now take for granted in IT. The challenge is compounded as industrial organizations adopt digitization initiatives such as Industry 4.0 and Smart Grid to optimize their operations, thereby increasing their attack surface while removing the traditional “air-gap” separation between IT and OT networks.

When retrofitting outdated systems isn’t practical, implement mitigating controls that reduce the risk and impact of attacks on critical assets. Best practices include:

• Implementing continuous monitoring and behavioral anomaly detection to quickly detect and respond to suspicious activities in OT networks.
• Regularly assessing OT networks for vulnerabilities such as direct connections to the Internet, rogue devices, unmanaged connections between OT and IT networks, use of plaintext passwords, etc.
• Network segmentation and zoning to hinder lateral movement in case of an OT breach.

So, the OT security threat is real and the complexity of patching ICS/SCADA systems compounds it. When it comes to patching ICS networks, there’s plenty we have learned from the IT security side of the house, but it’s critical that adapt these processes to serve the unique needs of OT organizations — and not simply shame OT security teams for not following traditional IT processes such as patching.