Review: ThreadFix 3.0

Maintaining a strong organizational security posture is a demanding task.

Most best practices – e.g. CIS Controls, the OWASP Vulnerability Management Guide – advocate a continuous program of asset discovery and vulnerability management. Due to fundamental changes in infrastructure provisioning and paradigm shifts like Infrastructure as Code (IaC), most organizations had to shift their regular vulnerability assessments to a more frequent pattern.

Adoption of Agile and DevOps practices in managing infrastructure, as well as frequent news about negligent secure practices that led to breaches in high-profile organizations, have pushed organizations to take security practices more seriously and to adopt a continuous vulnerability management process, which moves security tests and controls into earlier stages of the software development lifecycle and becomes a default prerequisite in the product requirements.

Security teams have a wide choice of vulnerability assessment tools, from static code analyzers (SAST) to dynamic ones (DAST). Most of these tools can be used to complement each other and give a wider view on the potential vulnerabilities found in the tested applications and infrastructure.

How to prioritize and follow up on findings of complementary vulnerability scanners? A lot of data is generated during the vulnerability assessment process and most of it should be double-checked to pinpoint only meaningful findings. Viewing a 100+ page PDF report or tracking the remediation status in a spreadsheet that takes an eternity to load can only result in headaches. A manual vulnerability management process also prevents us from tracking defined performance indicators and from achieving efficient collaboration.

This is a review of ThreadFix 3.0, a vulnerability management platform that helps organizations overcome these challenges and manage risky applications and infrastructure efficiently and in alignment with the agile development processes.

ThreadFix vulnerability resolution platform

ThreadFix is a software vulnerability aggregation and management system that can schedule vulnerability scans, organize and merge aggregated vulnerability reports, and integrate with popular security and software development tools. It addresses the problems of organizations that have an established vulnerability management program and have identified challenges in data management and collaboration between teams.

ThreadFix enables organizations to:

• Consolidate test results by de-duplicating and merging imported results from more than 40 commercial and open source dynamic (DAST), static (SAST), Software Composition Analysis (SCA) tools, and interactive (IAST) application scanning tools. ThreadFix can track manual findings and observations from code-reviews, threat modelling and penetration tests. Normalization and merging of test results between various types of tools is done by a patented technology called Hybrid Analysis Mapping. It also enables the correlation of found vulnerabilities at the network infrastructure and application level.
• Improve vulnerability management by integrating ThreadFix with various defect/bug trackers (Jira, Azure DevOps Server, IBM Rational ClearQuest, etc.) and developer tools. This removes the friction between software developers, system operations and security teams, and helps decrease the time spent on coordinating and fixing prioritized vulnerabilities. As software development and system operations teams resolve found deficiencies, ThreadFix detects these changes, enabling the security team to perform follow-up testing to confirm that these security holes have been closed.
• Schedule orchestrated scans with remote scanners. After the scan is finished the report is merged and becomes visible in ThreadFix.
• Prioritize risk decisions. ThreadFix reporting and analytics capabilities enable organizations to quickly identify vulnerability trends and make informed remediation decisions based on current vulnerability data. It gives visibility into how quickly the found vulnerabilities are resolved and supports reporting functions that provide security managers with up-to-date metrics needed to conduct data-driven conversations with upper management, which help estimate the efficiency of the security program or future actions. ThreadFix integrates with GRC tools, like ServiceNow GRC.
• Quickly isolate and pinpoint suspect vulnerability data using custom filters, to reprioritize their remediation plan.

Methodology

For this review, we used a test instance of ThreadFix that has been provisioned on Amazon Web Services by the vendor.

You can find straightforward instructions in the documentation to spin up your ThreadFix instance, should you wish to do so. All ThreadFix components are dockerized so it takes a Docker Compose one-liner to build the environment – this simplifies a lot the installation procedure and comes handy if you use container orchestration tools.

For testing purposes, we scanned intentionally vulnerable applications to get vulnerability reports that will populate the instance.

ThreadFix divides the vulnerability management in infrastructure- and application-related vulnerabilities.

We scanned the infrastructure with Qualys VM and Tenable Nessus. We uploaded manually the scan reports in ThreadFix, but this part can also be automated by configuring remote scanners.

We tested the application capabilities with various SAST and DAST tools: Burp Pro, Brakeman, the Acunetix web vulnerability scanner, Appscan, Fortify SCA, OWASP Zed Attack Proxy and Checkmarx, by scanning intentionally vulnerable applications: bodgeit, RailsGoat and Wavsep.

With the test instance running and the data present, we proceeded to evaluate ThreadFix by its main components:

• Infrastructure vulnerability management
• Applications vulnerability management
• Reporting & analytics
• Integrations (defect trackers and remote scanners)
• API

Tracking infrastructure assets and application vulnerabilities

After logging into ThreadFix, you are welcomed with a dashboard page containing statistics about the accumulated infrastructure scans (Figure 1).

The dashboard shows:

• Statistics about opened, closed and new vulnerabilities
• Vulnerability trends over several months
• A breakdown of operating system used
• Statistics about most vulnerable networks and hosts
• The most common CVEs found on your infrastructure.

The initial dashboard can be customized through the solution’s settings, like all the other ThreadFix dashboards.

Figure 1 – Infrastructure dashboard

Infrastructure

As previously mentioned, ThreadFix divides the vulnerability management in two sections: infrastructure and application.

The infrastructure is represented with internal networks and public networks related to data centers or cloud environments where the applications are deployed. The network IP ranges are manually configured and can be enriched with additional metadata (Location, Department, Description). These defined networks are later mapped to uploaded vulnerability reports.

ThreadFix has successfully recognized our efforts to upload duplicated or corrupted reports. After the reports were successfully uploaded, we started drilling down the findings either by focusing on the network or host level. Hosts are mapped based on scan results and populated with their FQDN (if available), IP and MAC addresses, and recognized operating systems.
You can also search through your assets based on the populated fields, which is efficient when you want to inspect the asset or look at the remediation status.

The infrastructure view enables us to efficiently choose hosts that we want to prioritize for remediation actions by using the available filters and sorting actions. For example, in the hosts table we can sort the vulnerabilities by their severity and choose the hosts with the most critical and high vulnerabilities (see Figure 2). Or, we can narrow down our search by filtering hosts with a specific operating system that share a specific vulnerability and can be fixed jointly.

Figure 2 – Using infrastructure filters can come handy when prioritizing specific remediation actions

By clicking on a host you can view vulnerabilities details and audit them (Figure 3), and you can filter vulnerabilities by their severity and status (Open, Closed, Mitigated, Accepted risk, False Positive). This workflow for auditing infrastructure is very user friendly, and the vulnerability mappings work pretty well. Filters are well covered in every infrastructure page, and we can stack vulnerabilities or inspect recurring ones.

Figure 3 – Host vulnerability report

Applications

ThreadFix automatizes the process of application vulnerability management by inspecting source code with a SAST tool and by scanning repeatedly the running application with a DAST tool. In ThreadFix every application is owned by a team and it can be tagged with custom tags that are helpful when used with filters. Tags are identifiers related to applications, vulnerabilities and vulnerability comments.

The applications portfolio shows the current teams and associated applications (Figure 4).

Applications can be connected with a Git repository, which can be used for static code analysis.

The uploading and parsing of vulnerability scans reports went without a hiccup. You can request a service engagement, which comes handy when you need a re-scan or a manual check after fixing an issue in the application. ThreadFix also allows you to define the penetration testing team that can collaborate on findings within the chosen application. After the pen test is done, you can see the results under Assessments (Figure 5).

Figure 5 – Summary of a manual penetration test

ThreadFix converts all uploaded vulnerability reports to the ThreadFix format and maps vulnerabilities to Common Weakness Enumeration (CWE) identifiers. It also detects and merges similar vulnerabilities found by different tools (Figure 6).

Figure 6 – ThreadFix merged a vulnerability found by different tools

ThreadFix offers unlimited search, filter and pivot options for application vulnerability reports. Pivot options allow you to narrow down interesting vulnerabilities but also hunt for potential false positives when comparing various tools (e.g., when you pivot by severity and scanner type). You can search reports by vulnerabilities, scanner, tags, CVE, paths affected, status, date range of scan, and more. ThreadFix also offers to save and export complex filters, which can be handy when you get back to a specific application report after some time.

Saved filters can be also used as a baseline for custom policies in ThreadFix. Policies are calls to action for your team, they usually compare the current remediation status against a desired baseline. Filter policy is a simple rule-based filter that shows a Pass or Fail indicator in the application dashboard if the application meets or fails to meet the defined requirements. For example, filter policy defines that, to pass, an application should have no critical or high vulnerabilities. Another interesting policy is the time-to-remediate policy. The concept is simple: you define a desired fix deadline for a severity level and ThreadFix sets a custom reminder that notifies your team about it.

A nice feature that improves collaboration between users is commenting and tagging vulnerabilities in the remediation process. You can filter vulnerabilities that have a comment and continue tracking their progress. This feature can be used during penetration testing engagements when multiple people work on the same application and record their progress but is also useful in the remediation phase (see Figure 7). While commenting, you can attach various files to the comment to clarify the vulnerability. Mature security teams that have established roles and a vulnerability management procedure will find tagging and commenting useful for pushing collaboration efforts while prioritizing next actions and tracking current actions.

Figure 7 – Application dashboard shows recent trends, uploads, and comments

Reporting and analytics

ThreadFix has exhaustive options to inspect application vulnerabilities with multiple views and level of detail. There are ten report types, each with its own filter set. This vast number of combinations enables users to be creative when producing data reports for the upper management.

The most popular report is the trending report that shows the remediation effects over time and is also visible under an application dashboard (see Figure 8). This simple but powerful visualization shows how your team is doing on solving found vulnerabilities through a selected time period.

Figure 8 – Trending report example

Other reports available in ThreadFix are:

• The Point in Time report shows a breakdown of the team/application vulnerability results by their severity
• The Progress by Vulnerability report is used for tracking the average age of vulnerability types as well as the time to close each vulnerability type
• The Most Vulnerable Applications report shows the most affected applications and gives insight into their vulnerability composition
• OWASP Top 10 maps found vulnerabilities to the OWASP Top 10 list
• The Portfolio report shows how fresh the current scans for each application in your portfolio are. This report can help target specific applications for follow-up scans so you can to stay up to date on your projects’ vulnerability status
• The DISA (Defense Information Systems Agency) STIG (Security Technical Information Guide) report displays information on your application’s compliance with DISA’s Application Security and Development STIG requirements. This report can help plan and execute remediation strategies for maintaining compliance with governmental application security standards
• The Scan Comparison Summary report gives a side-by-side look at how each scanner has been performing, showing the number and percentage of total vulnerabilities found and percent of total false positives discovered among them
• The Remediation report provides the trending report, as well as a more detailed table with starting and ending vulnerability counts to gauge progress
• Vulnerability Search allows to filter and explore vulnerabilities based on the set filters.

Defect tracker integrations

The integration with various defect/bug trackers enables security analysts to provide additional information to application developers about found vulnerabilities. This bi-directional communication removes the need for developers to use an external tool and helps them to quickly start fixing the vulnerabilities (Figure 9).

ThreadFix allows security teams to craft custom templates that will be used when creating a ticket and to define custom fields that will auto-populate the ticket in the defect tracker. If there are more teams managing the applications that use different trackers, it allows you to define and open tickets in multiple trackers. Security teams can define a defect tracker policy that automatically opens new tickets based on the severity level(s). For example, you can set a policy that opens tickets when the scanner has found vulnerabilities that have a severity of high or greater. Based on the policy setting, the vulnerabilities that match the criteria will be bundled together and attached to a ticket in the defect tracker.

Figure 9 – ThreadFix opened a vulnerability ticket in Jira

ThreadFix maintains a link to the issue created in the external defect tracking system, allowing security analysts to see developers’ progress in solving the reported vulnerabilities (see Figure 10). In addition, ThreadFix periodically checks the status of associated defects and updates that status in ThreadFix so that when development teams fix defects, security analysts can see their action and later check if the fix is working.

Figure 10 – Vulnerability contains the issue status that is monitored regularly

The ThreadFix API

ThreadFix allows users to perform vulnerability management actions via the ThreadFix API. The process follows the ThreadFix application and infrastructure workflow, but in a more step-by-step programmatic fashion and can be useful when automating tedious work. The API documentation is very helpful because it is up to date, well written, and comes with examples for every call.

We tried replicating the workflow of managing the application portfolio, so we created a new team, assigned to it a new application, uploaded a related scan report and tried changing the status of the open vulnerabilities. We found ThreadFix API pretty versatile and able to replicate all or almost all ThreadFix actions that are provided through the ThreadFix application.

To support the DevSecOps movement, the ThreadFix API can be used to integrate more complex security testing workflows and vulnerability reporting with the Continuous Integration/Continuous Development process. On specific events, you can schedule remote scanning and report found vulnerabilities back to the security team. Using the API you can also administer and provision ThreadFix more easily.

Conclusion

ThreadFix is a helpful and mature tool for vulnerability management. It bridges a lot of gaps between various teams, including those between the security and application developer team(s). Application management enables users to merge and manage vulnerabilities and track their remediation actions.

After adopting ThreadFix, you will say goodbye to tracking scan results and managing vulnerabilities via spreadsheets. ThreadFix supports industry standard vulnerability scanners with vulnerability merging technology that works well. In addition to that, most vulnerability scanners can be directly orchestrated through ThreadFix, and it also allows you to automate a lot of workflows with the API functionalities. ThreadFix enables you to drill down in every vulnerable aspect of your application without losing sight of the big picture that is delivered by the useful analytic charts and trendlines.

One thing that’s missing is the integration of the nmap scanner, which many security pros use for mapping infrastructure and scanning for low-hanging fruit vulnerabilities.

Another gap that Threadfix bridges is between operative security and compliance teams because ThreadFix also delivers a solid asset management tool that helps you to track and drive changes to meet the compliance or certification requirements.

CISOs will find ThreadFix analytics options useful because they enable identifying risk patterns in the scanned applications and infrastructure and a better estimate of risk levels and activities related to reducing risk. ThreadFix gives CISOs and other managers a valuable perspective into their software development lifecycle process and enables them to identify and measure segments that can be improved.

ThreadFix “humanizes” the security process with its collaboration features. Finding vulnerabilities is only one piece of the puzzle: remediation efforts are always the hardest and the most time-consuming piece. ThreadFix is not only used as a vulnerability unification and analytics platform but also as a collaboration platform where different teams and perspectives meet and engage in solving organizations deficiencies faster and better than before.