The Long-Term Impact of Log4j

Build SBOMs, upgrade (or remove) everything

Another major exploit made headlines over the holidays when attackers went after the venerable and vulnerable Log4j utility. Developed over 20 years ago and continuously administered by Apache Software, the logging framework is deployed on hundreds of millions of systems around the world. The zero-day vulnerability was one of the most critical of the last decade and, unfortunately, it’s not likely to be the last. It’s time to take stock in the long-term implications for what this vulnerability and others in the future mean for security leaders.

First and foremost, it’s another wake-up call regarding supply chain security, component security, and open-source risk in general. These concerns have been around for a while, and there were rumblings of this coming since at least 2013 when “Using Components with Known Vulnerabilities” was added to the OWASP Top 10 in 2013. Discovery of the Heartbleed vulnerability in OpenSSL in 2014 provided a concrete example of the risk associated with open-source components, and the Equifax breach in 2017 based on an outdated version of the Apache Struts framework brought these risks into even greater focus. Supply chain security wake-up calls have come more recently with the SolarWinds and CodeCov breaches.

We’ve had enough wake-up calls. What should security managers for both producers and consumers of software do?

The least we can do: SBOMs

SBOM (Software Bill of Materials) is a list of proprietary and/or open-source components included in a software build that lets software consumers know what’s been packaged in the software they’re running. This has been a point of discussion for some time, and SBOMs recently gained greater prominence with their inclusion in Section 4 of the Biden Executive Order (EO) on improving the nation’s cybersecurity. It’s a simple concept whose time has come: if a vendor ships software, they ought to know what’s in it and be able to communicate that to the buyer. The real change will be forced by software consumers, who should make it their business to always request a rundown of what’s included in any software before they risk buying it. This should become best-practice common sense – it’s more work for developers, but software buyers deserve to know.

SBOMs are a great start, but are only a first step on a much longer journey. Consumers need systems in place to collect SBOMs across their entire portfolio – both internally developed and externally sourced software packages. This becomes even more complicated with SaaS and other cloud-native applications extending to containers, cloud servers and services.

In addition, SBOM inventory must have processes in place to flag vulnerable components and as new information becomes available. Software producers need to get comfortable providing SBOMs with software builds, and software consumers need to build out the processes and infrastructure to track and act on them.

No more component risk assessments: Upgrade or remove it

A challenge with Software Composition Analysis (SCA) tools has been that they generate a lot of findings that show the presence of vulnerable components, but not necessarily a reachable and exploitable vulnerability in the application. Because upgrading components costs development time, this puts development teams in a quandary – upgrade an allegedly vulnerable component that may not represent a true risk to the application, or build features that generate business value?

To support development teams, SCA vendors have added data and control flow analysis to help determine if vulnerable methods in vulnerable components are truly in use, indicating whether an application with the vulnerable component may ultimately be vulnerable itself. The unfortunate thing for software developers is that, after Log4j, software buyers will be less inclined to care and will simply demand that vulnerable components either be upgraded or be removed, regardless of reachability or exploitability. For them, this is far safer than trusting vendors and far easier than replicating the risk assessment the software vendor may have undertaken, just with less complete information.

The key point here is that buyer expectations of zero vulnerabilities will force development teams to rethink how they manage technical debt and how they schedule component upgrades in their backlog. Vulnerable component upgrades will no longer be discretionary or subject to risk assessments. SCA vendors will double down on providing automated remediation – via pull requests or diffs – for components vulnerabilities and DevSecOps teams will need to add automated verification to test out these proposed remediations

Better vendor security management through better questionnaires

In its aftermath, Log4j vulnerabilities put the spotlight on vendor management and supply chain security practices. Software suppliers should expect vendor security questionnaires to expand in scope and detail around application security practices. It’s relatively easy for software buyers to implement questionnaires. But it’s going to get harder on vendors, who will need to update their DevSecOps practices in anticipation of answering more thorough lines of questioning.

Every organization buys software from someone, and those developing software also buy software, so eventually this practice will pervade the industry. Standardization should evolve from the current chaos of bespoke questionnaires, allowing software producers to streamline their development practices and their responses to questions about their products.

After so many wake-up calls, the Log4j vulnerability may be the last one before we get to “I told you so” if we don’t up our vendor security management game. Log4j was big, and its impact will change software consumer behavior, which will evolve market incentives and software provider behavior. Forward-leaning software consumers will drive the policies, systems, and processes they use to manage software and cloud systems they consume, and all software vendors will be forced to improve their practices, tooling, and infrastructure to meet their customers’ new requirements.