Qualys platform study: Log4Shell, the menace continues

The anatomy of Log4Shell

By now, we are all familiar with the fact that Log4Shell is just about as critical as a critical vulnerability can get – scoring a 10 out of 10 on the National Institute of Standards and Technology’s CVSS severity scale.

As it targets a library – Apache Log4j2 – that nearly every Java application uses to log requests, this vulnerability is ubiquitous. Many applications use Log4j2 without even realizing it, meaning that even those with no apparent dependency on Log4j2 can still be at risk.

With its massive impact across nearly every industry, Log4Shell has taken its place in the cybersecurity hall of fame – among the likes of HeartBleed, WannaCry and ShellShock.

Difficult to locate but easy to exploit, remediating this vulnerability would prove incredibly complex, with several detection methods required. In fact, three months into Log4Shell, the Qualys Cloud Platform suggests that 30% of the Log4j instances still remain unpatched.

Qualys research team reveals the current state of Log4Shell

When it came to tracking the impact of Log4Shell, Qualys occupies a unique vantage point. The Qualys Cloud Platform indexes more than 10 trillion data points across its installed enterprise customer base and completed 6 billion IP scans per year with 75 million cloud agents deployed in hybrid IT environments globally. With that kind of scale, the Qualys Research Team was able to uncover unique insights into how global enterprises have and are managing Log4Shell:

Log4Shell exposure

• Qualys Cloud Platform scanned more than 150 million IT assets, across all geographies, flagging 22 million vulnerable app installations. Of these, more than 80% were open source applications.
• Log4Shell was detected in more than 3 million vulnerable instances.
• More than two months later, 30% of Log4j instances remain unpatched.

Log4Shell threat landscape

• Nearly 68,000 vulnerabilities were found in cloud workloads and containers across the U.S. and EMEA, reinforcing the recommendation that enterprises need to monitor running containers for flaws like Log4Shell.
• CISA and NCSC reported 1,495 products vulnerable to Log4Shell, and of those we observed 1,065 products across 52 publishers currently in use. This indexing proved very valuable to Qualys customers as this SBOM mapping is provided out of the box, providing immediate insights into their vulnerable software inventory.
• Surprisingly, more than 50% of application installations with Log4j were flagged as “end-of-support.” This means that these publishers will likely NOT be providing Log4Shell security patches for these apps.
• The vulnerability was detected in more than 2,800 web applications. Since web apps are publicly facing, this was the first line of defense for many enterprises looking to fend off early attacks. In the U.S., most detections occurred before/during the holiday period, while in the E.U. these spiked after the holidays.

Vulnerability trends

• The vast majority of the vulnerable assets (over 80%) were on Linux.
• A total of 98 distinct Log4j versions were observed in use, 55% of which were vulnerable versions.
• There was a 20% spike in detections as the new year arrived and employees returned to work.
• Within the first month after Log4Shell’s disclosure, we observed that 12% of total Log4j installations were vulnerable, while only 5% were not.

Remediation trends

• Average time to remediation after detection was 17 days. Systems which could be exploited remotely were patched faster (12 days) while internal systems were slower.
• After the first month, remediation efforts plateaued and began trending down, quite likely because security teams are finding it easier to mitigate Log4Shell rather than permanently fixing it.

Attack patterns

• Our Multi-Vector EDR solution detected 22,000 potential attacks per week at the height of the crisis. Many of these were scattershot “spray & pray” attacks trying to infect as many systems as possible quickly. Our data indicates that threat actors were trying to take advantage of the holiday season window of opportunity.
• Attacks also trended down into January, as mitigating controls and patches were rolled out by enterprise IT teams.

Unpacking Log4Shell’s continued peril

Log4j has been and will continue to be a headache for security professionals due to how difficult it is to fully understand where this vulnerability may be within an organization.

As with most vulnerabilities, understanding how and where the flaw will affect your business is crucial. Discovery processes are unique to each organization – meaning that depending on architecture and deployment, timetables vary.

This paired with obstacles such as the complexities of skeleton IT staff, potential lack of visibility into IT assets and an overall influx of other real-time sophisticated attacks and threats, could present a tumultuous road to immediate remediation.

Why are vulnerable Log4j versions continuing to be downloaded?

The main culprit for why vulnerable versions continue to be downloaded is likely because of automated build systems. These are configured to download a specific version build of their dependencies. Lesser maintained projects may automatically download a specific version to avoid conflicts with updated software, which has the potential to break their code. If the maintainer of that software hasn’t been paying attention to Log4j news their application is left open to the risk of exploitation.

Another scenario is the intentional download by researchers or adversaries to test exploitation of their latest wares. It is useful for both good and bad guys to continually validate that their exploitations or defenses are in working order outside of production areas.

Why are vulnerable Log4j versions still available for download?

Flawed forms of the code are still available because many other pieces of software still rely on them. Removing these downloads could potentially cause breakage in several systems if eliminated.

Further, The Qualys Research team found that more than 50% of application installations with Log4j were flagged as “end of support.” These publishers will likely not be providing Log4Shell security patches for these apps. End of life/support technology is one of the leading factors that put organizations at risk of being exploited by threat actors.

In fact, earlier this year, CISA developed a catalog of “Bad Practices” to showcase what is exceptionally risky. Landing at number one – especially for organizations supporting Critical Infrastructure or NCFs – was the use of unsupported software.

What’s next?

As we head into a new era of more sophisticated cyber-attacks and never-before-seen critical vulnerabilities, enterprises – large and small – must invest in best-in-class platforms. Utilizing such solutions that can aid security operations across IT asset management, vulnerability detection and response, cloud security, EDR/XDR and web app protection are crucial in building up and maintaining cyber posture.

Thankfully, critical vulnerabilities as severe as Log4Shell are a rare occurrence. However, the future discovery of another weakness just as bad (or worse) is inevitable. Real-time threat intelligence like that from the Qualys knowledgebase helps enterprises continuously assess, monitor and report on the latest and greatest security threats so that we’ll be ready when “next time” inevitably arrives.