How do I select an automated red teaming solution for my business?

Red teaming is an essential part of an organization’s security assessment process. It is generally done manually to uncover possibile vulnerabilities and security gaps, but can automation simplify or even enhance the process?

To select a suitable automated red teaming solution for your business, you need to think about a variety of factors. We’ve talked to several industry professionals to get their insight on the topic.

Avihai Ben-Yossef, CTO, Cymulate

There are two considerations when selecting an automated red teaming solution.
First, from a business perspective, the solution must provide the capability of testing, visualizing, and explaining the risks and how to reduce them. It must keep track of how those risks rise and fall over time. These trends allow executives to clearly see how effective the company’s security investment is, how to optimize it and reduce risk.

The second is from a technical perspective. A red teaming solution must be comprehensive, covering the entire kill-chain in-depth. It should include reconnaissance intelligence gathering capabilities, phishing campaign testing, web and email gateway, web application firewall, end point, lateral movement, and data exfiltration. Testing must be atomic with the solution operating like real attackers, pivoting and continuing when they hit a block.

Since research and coding are resource intensive, the solution must provide a framework to make testing easy, automated and in depth by removing these arduous portions. It must be capable of running while in production, which is the only way to understand the risks and how to reduce them. It must also include actionable intelligence which is updated daily, 24/7. This will ensure that when new attacks occur in the wild, the practitioner can spend time testing against the new IoCs and TTPs rather than adding them manually.

Jay Christiansen, Manager, Red Team, Consulting, EMEA, Mandiant

When selecting an automated red team solution, one must acknowledge that no such product exists. Red teams are creative processes that pull many threads of information and weaknesses together to iteratively innovate an attack path, something that cannot be simulated. Yet.

There are, however, tools that can help train defenders or aid in discovering gaps in defensive investment. There are three initial considerations for these tools.

For the best defenders, identifying behavior, not static signatures or tools, is crucial. By correlating events and telemetry, they can spot new / unknown tools and react faster. To create this, the simulation tool must run complex chains of techniques based on the environment; checking the OS, downloading an implant, executing persistence, then searching local files before moving laterally, as an example.

Secondly, the solution’s techniques must be relevant, basing them on updated imitations of those observed from real actors. Use of threat intelligence will benchmark against genuine attackers instead of generic outdated threats, decreasing the likelihood of defensive gaps.

Finally, being able to get metrics on the performance of the current defensive set-ups it requires the solution to integrate with the SIEM. Without this, the ability to gain evidence on MITRE mapped control failing becomes cumbersome and error prone.

Other factors to consider are ease of deployment, compatibility with all types of platforms and trial options.

Frank Duff, GM of ATT&CK Evaluations, MITRE Engenuity

A major difference between the tools is their approach to ease of use vs. flexibility. Some solutions are built to be more point-and-click with predefined scenarios, whereas other solutions expect you to define your test scenarios and offer greater flexibility and customizability.

Additionally, others offer the flexibility, but with the expectation you leverage services to unlock their full capability. Each of these types of solutions address different customer needs and have tradeoffs.

While point and click solutions make it easier to get started, you can lose the ability to extend and customize. That said, customizability often correlates with complexity, meaning you need more specialized resources to execute and therefore greater cost. If your team is small, they aren’t familiar with red teaming, or they are already over-taxed, then you need to find a solution that will have a lower barrier of entry to get started. Conversely, if you have people who enjoy offensive testing, or even a dedicated red team, then being able to tailor tools and tests becomes very important as customizability will be key.

Regardless of which solution is right for you, the good news is that while red teaming has long been a nice to have, reserved for the upper echelon of companies, the emerging automated red team market has made it accessible to the masses.

Amitai Ratzon, CEO, Pentera

Automated red teaming solutions are an essential part of modern SecOps technology stacks, but not all are created equal.

When evaluating a solution for your business, there are several key factors to consider:

Vulnerability fatigue is real. There were more than 15,000 vulnerabilities found in 2020, while only 8% were exploited by attackers. Don’t implement a solution that floods already overworked and understaffed security teams with non-critical alerts.

Automate, don’t simulate. The most effective red teaming solutions for identifying and prioritizing critical weaknesses automate the actual tactics and techniques a malicious adversary would – reconnaissance, sniffing, spoofing, cracking, (harmless) malware injection, file-less exploitation, post-exploitation, lateral movement and privilege escalation — all the way to data exfiltration. This shows CISOs and SOC teams exactly how an attacker will exploit their network.

Continuous validation. The IT network is a living organ undergoing constant change. Point-in-time red teaming makes it impossible for organizations to reliably and continuously test their attack preparedness as new applications are introduced, privilege and access policies are updated, and further network changes are made. Real adversary tactics and techniques must be performed on a continual basis to effectively validate attack preparedness.

Rick Tomlin, Security Solutions Architect, Illusive

Attackers use automation to probe for access vectors into enterprise networks. As such, it is important for enterprises to utilize automation in defending their networks.

Consider the following when evaluating solutions:

Safe for environments: Do no harm. A solution that wreaks havoc across networks while exploiting security gaps and penetrating environments is not a net benefit to organizations. Determine what methods are being used and what impact the tool has on endpoints when it runs.

Comprehensive: What good is a tool that identifies weaknesses in half of attack surfaces? Attackers won’t limit themselves to just one part of the network, so ensure that tools are capable of simulating attacks across the entirety of the attack surface: different operating systems, on-premise vs. cloud, etc.

Variety of attacks: Look at the variety of attack scenarios that the tool is capable of simulating. Is it just limited to vulnerability identification, or can it actually exploit an identified vulnerability, thus testing mitigating controls? Can it deploy human attacks such as phishing? Can it harvest data from endpoints (Living off the Land) to move laterally and elevate privilege?

Level of automation: How much time and effort are required of an already overworked security staff? The more automation available, the less effort required. Tools that require a high level of effort are generally run less often.