Optimizing the patch management process

In this podcast recorded at Black Hat USA 2019, Jimmy Graham, Senior Director of Product Management at Qualys, discusses the importance of a tailored patch management process.

Security obviously will have some say in a patch management process because a lot of patching is security driven, but patching is beyond just security, there’s also stability performance updates that have to be taken into account.

Here’s a transcript of the podcast for your convenience.

Hi, my name’s Jimmy Graham and I’m the Senior Director of Product Management for vulnerability management and patch management at Qualys, and in today’s podcast we’re going to be talking about optimizing the patch management process.

You hear a lot about the vulnerability management lifecycle, we talk about it a lot at Qualys, how asset inventory feeds into vulnerability management. Then you prioritize those vulnerabilities, and then you patch.

Patch management process

And while patch management does get some input from vulnerability management, patch management really needs to be its own cyclical process. Vulnerability management should not be the only way that the patch management processes is engaged. It needs to really stand on its own. You have a new patch release. You prioritize the patch. You test and deploy that patch. You report on the patch deployment. You then audit that with VM and then you clean up.

And when I say clean up, that might mean going back and patching outliers. That might mean taking feedback from vulnerability management such as, we see this specific patch missing pervasively across the entire environment, to help you tune your patch management process so you can target those systems and get them into a cyclical patching process. And this cyclical process really needs to be IT driven, not security driven.

Security obviously will have some say in a patch management process because a lot of patching is security driven, but patching is beyond just security, there’s also stability performance updates that have to be taken into account. And again, it needs to be independent. It can’t just be driven by vulnerability management; it has to be its own cycle. And the way that you get to that is by implementing a few different components and I’ll go over a few of those.

Vulnerability management lifecycle

Patch management standard

So, you really need a patch management standard, a standard that stands on its own outside of the vulnerability management standard. There should be harmony between those two standards, but they need to be independent. One owned by IT, one owned by security. Obviously in smaller shops that may still be the same group, but if there are separate groups between IT and security then those standards need to be separately owned.

There should be documentation also for processes for each technology, the way that they’re patched, the cycle in which that they’re patched. You should have good metrics and reporting on the patch process. I’ll dive into that a little bit deeper in a minute. There should be a feedback loop into your gold images. So, when you do patch these systems, rather than just patching your production systems, you should also then feed that back into your gold images.

That may not be on the same cycle as your patching cycle, you’re maybe patching monthly, you don’t necessarily have to update your gold images monthly, but you know at least quarterly you should be going back and making sure that gold images for these products are out the door, in a patch state or at least closer to patch state.

Components of a security patch management program

What are the components of a security patch management program? I’ve gone over one of them already, the patch management standards. You really have to have IT executive buy-in because again it needs to be owned by IT, it needs to be IT driven. Within that standard, that’s where you set your SLAs and your OLAs for patches. That should be based on the severity of the vulnerabilities that are behind the patches and any threat intelligence that you have.

The type of threat intelligence that you would use for prioritization, does this vulnerability that this patch fixes, does it have an active exploit? Is it being actively attacked in the wild? That can help you set the OLA criteria so you know when these specific vulnerabilities or when these certain patches should be deployed.

For example, you may have a patch cycle that’s normally 30 days, where you take a high-risk vulnerability that is remediated within 30 days. But if you have a situation where there’s an exploit public, it’s being actively attacked and its public facing, you’re not going to wait the full 30 days.

Most organizations are going to prioritize that, but a lot of organizations don’t have a documented standard that says how that should happen. Those systems may need to be patched in 48 hours instead of 30 days.

That’s where you draw those lines and say, “when patches meet these specific criteria, we have this OLA versus this other OLA”. Again it should harmonize with your vulnerability management standard. It should be stricter than your vulnerability management standard. If vulnerability management says that vulnerabilities of a certain severity, you have to review or remediate in an X number of days, your patching cycle should be faster than that, and then you don’t have to worry about all of these vulnerabilities and all the prioritization coming out of the VM program.

Patching resources

You should also have dedicated resources for patching. That could mean an enablement team or an enablement individual for smaller organizations, where someone basically owns the standard and assists with the different various patching teams to help them get the proper tooling, get the proper processes for patching, and they’re there really just to drive the patching process.

Back to tools, you should have proper patching tools, you should be looking at on premise systems, remote systems, cloud systems – everything should be covered within the patching program. And then of course, you have to have reporting and metrics. There are various ways that you can divide that up. It could be by technology area, you may say “this is my standard for Windows versus Linux, versus Mac and so on”. It could be by business unit. If you have separate IT groups that are supporting those business units separately, you want to break it out that way, that will be very organization specific.

But you do want to track basically your adherence to the standard. So, what’s your meantime to remediation, what’s your percentage compliant with the standard? It’s OK to fail, especially when you start out with the metrics program and reporting. These are goals. Failing at these metrics, let’s say you do Windows patching for a round and you’re only 65, 66 percent compliant. That’s fine, that’s what drives additional resources. That doesn’t mean you should set it too strict, but you also don’t want to be too lenient, you want to set it to a real goal that’s within your organization’s risk appetite.

Does patch management tie into vulnerability management?

If patch management should stand on its own and should not be solely driven by vulnerability management how does patch management actually tie into vulnerability management?

Vulnerability management is really a check on patch management and to a degree configuration management. But about 80 percent, you’ll find out, deals with patch management. Vulnerability management’s there to give you insights into your patch management programs.

You should have some sort of patching tool that understands that vulnerability context as well. So, a lot of this can be automated, rather than trying to be extremely dependent on the vulnerability results to drive your patching program. If your patch management solution understands the vulnerability context meaning what severities does it patch, what threats are against the vulnerabilities that are fixed by this patch, then you can automate a lot of this within the patching program itself without having a high degree of dependency on vulnerability management.

Then the vulnerability management data can help to focus patch management. That means, if you see situations where a specific piece of software, could be something like Adobe Flash, could be Java, if you see that pervasively throughout your environment where your vulnerability management solution is reporting multiple vulnerabilities across multiple systems, you see five different vulnerabilities, 10 different vulnerabilities across multiple hosts, it means that those specific pieces of software are getting older and older, they’re aging out and they’ve been updated multiple times, and those updates have not been deployed.

Patch management best practices

Your patching program

That’ll help you determine which software should be in a cyclical patching program, because not all patching can be operationalized like that. You’re always going to have outliers, there will be some patching that’s driven by vulnerability management. But if you see a lot of systems with a lot of vulnerabilities stacking up for specific solutions, rather than trying to filter out superseded vulnerability results and come up with what patch to deploy right then just to get you caught up, you’re really never going to catch up.

You may have a situation where you are catching up and upgrading Java. That’s usually a big project, but after you go through that project and while you’re building that project, you should be focusing on getting that into a routine cycle. The Microsofts out there have a routine patching cycle, they have Patch Tuesday. Adobe is also on Patch Tuesday. Those types of patches should be rolled out within that patch ready cycle within those 30 days.

And if you do that then the WannaCrys aren’t an issue. The BlueKeeps aren’t an issue. You don’t see all this emergency patching. Everyone should have an emergency patching process, but for these vulnerabilities that are patched through Patch Tuesday, unless there are active attacks against those vulnerabilities, there’s typically no need to invoke an emergency patching process if you have a solid routine patching process.

Again, not all patching can be operationalized like that but a lot of it can and that’s really going to cut down on the vulnerability results, and then vulnerability management basically is there to help fill in those gaps and address the outliers.

Thank you for listening. If you want more information you can visit qualys.com, for more information on our patch management and vulnerability management solutions. Thank you.