The biggest update you’ll barely see

It’s been more than 10 years since ThreadFix had its first lines of code written by its creator, Dan Cornell, as a means of solving a very pervasive issue in the application security space. While it quickly became a popular talking point at conferences and app sec parties (they exist!), it was never expected that a decade later our product would be the centerpiece to Fortune 100 organizations’ SDLC processes across a variety of verticals from healthcare to financial services, and from telecommunication giants to the top global banking institutions. The “little-code-project-that-could” practically found itself by accident solving some of the most significant custom application security challenges faced by the largest of organizations.

ThreadFix became the tip of the spear in our space for uncovering exactly how big application programs can be, and it has always been on the front lines of the cloud-native development practices that have overtaken almost all custom development at this scale. This led ThreadFix to the biggest challenge of its success. As the only tool out there that could even approach these enterprise-scale needs, we were pushed harder than any other vendor within the application vulnerability consolidation and management space for solving the next order of magnitude scale needs. “Sure, you can handle a couple thousand applications getting new scans every week or two. But we have over 10,000 applications getting new scans multiple times a day!”

This brings us to the very fundamental shift we built into the product architecture when we released 3.0 in 2019 and are now leveraging for the first time in the application scan ingestion engine for the ThreadFix 3.1 release. Our entire deployment environment has shifted to enable a horizontally scalable architecture which represents a complete pull and rebuild of the core engine of our vulnerability ingestion and merge logic. ThreadFix is now a series of microservices running in a Kubernetes-managed container cluster, and this change is only our first major step in leveraging this new architecture.

So let’s talk a bit about what ThreadFix 3.1 means for you specifically. If you don’t need to handle thousands of applications and have near real-time vulnerability merging and de-duplication in a centralized platform, this release means little for you today. For those of you with smaller programs, please stay tuned as we’re going to be reimagining much of our product capabilities in the coming months and there is a lot we have in store for you soon! But for those of you who have been holding back on a larger roll-out of your secure software development lifecycle program, or have given up on finding a COTS solution to your vulnerability management needs because you know your scale is just too large to fit, this is the release for you.

Almost all the development into this release occurred under the hood. As I hinted at earlier, future releases will be working on rebuilding our user experience, expanding our reporting capabilities, and providing a SaaS environment for our users. ThreadFix 3.1, however, is all about the engine. We started by rewriting our entire ingestion and merge logic from scratch, looking for the most efficient ways to parse apart the incoming data and process it against your scan history to find the most merge opportunities with the least effort. This process alone yielded significant speed improvements over our old methods (more than 10x ingestion speed increase). But then we layered it on top of our new architecture allowing us to horizontally scale our ingestion efforts simply by spinning up additional services. With just a modest deployment of five processing services, we saw vulnerabilities being processed in 1/60^th the time it took previously. Scan ingestion queues that took an hour to process previously are being processed in one minute, all while maintaining our merge and deduplication process which has been refined and perfected over the years and is now protected by two patents. And this is just the beginning.

With 3.1 we’ve only scratched the surface of what the new architecture allows us to bring to you, our customers. I would like to personally thank all of you for your continued loyalty and passionate feedback as we’ve spent the last two years building out and refining this new foundation for our platform. This really is just the jumping off point for an entire new generation of features and expansion of capabilities. All major features have been waiting behind this architecture. Now that the floodgates are opened, we can’t wait to show you what’s next!