NSS Labs test exposes weaknesses in NGFW products

Firewalls are the most widely deployed network security devices. Enterprises expect next generation firewalls (NGFWs) to prevent exploits and malware from infecting critical systems.

NSS Labs 2019 NGFW Group Test

NSS Labs announced the results of its 2019 NGFW Group Test. Twelve of the industry’s NGFW products were tested to compare NGFW product capabilities across multiple use cases. Products were assessed for security effectiveness, total cost of ownership (TCO), and performance.

This is the ninth year for testing NGFW products. NSS Labs raised the bar this year by performing a significantly harder test for security effectiveness, which exposed weaknesses not seen previously.

Test results showed that block rates for simple clear-text attacks remain strong (over 96%) for nine out of twelve products. However, while known/published exploits were frequently blocked, test engineers were able to bypass protection in all devices with minor modifications to known and blocked exploits.

In addition, only one of twelve products properly blocked exploits that were obfuscated using Complex Evasions (HTML / JavaScript / VBScript). Palo Alto Networks and WatchGuard stood out as the only products that didn’t miss major evasions this year.

“Often times, customers view security services as simple check boxes that are relatively identical across vendors throughout the industry. But there are many important technical differentiators that make certain security services more effective than others,” said Jack Waters, SVP of Engineering at WatchGuard Technologies.

Key takeaways

• Enterprises expect when they purchase products that they will remain viable over multiple years.
• While it is tempting to draw conclusions from one test, NSS Labs recommends enterprises favor vendors that consistently engage and improve over time.
• Scripting evasions are challenging for NGFWs since they require real-time code analysis in order to determine whether a function is legitimate or obfuscating an attack.
• Vendor claims to protect vulnerabilities (regardless of the exploit specifics) are largely dependent on the nature of the vulnerability and whether it lends itself to such protection. Test results found all products had room for improvement when confronted with unknown variants of known exploits.
• Research indicates that over 70% of Internet traffic is encrypted using TLS/SSL. NSS Labs recommends measuring the performance of devices both with and without TLS/SSL enabled. Failure to do so could result in unexpected performance bottlenecks.

“Given the ever increasing integration of the cyber and physical world, it is imperative that cybersecurity products work properly,” said Jason Brvenik, Chief Executive Officer at NSS Labs. “The good news is that while we found flaws, most vendors are committed to protecting their customers and are fixing their products. Stay tuned for follow-on reports,” added Brvenik.

Tested products

Of the twelve products tested, ten were rated as Recommended based on comparative scores for overall security effectiveness, TCO per protected Mbps, and performance:

• Barracuda Networks CloudGen Firewall F800.CCE v7.2.3
• Check Point Software Technologies 6500 Security Gateway R80.20
• Forcepoint 2105 NGFW v6.3.11
• Fortinet FortiGate 500E v6.0.4 build 0231
• Huawei USG6620E v600R006C00SPC310
• Palo Alto Networks PA-5220 PAN-OS 8.1.6-h2
• Sophos XG 750 Firewall SFOS v17.5
• SonicWall NSA 4650 SonicOS v6.5
• Versa Networks FlexVNF v16.1R2-S7
• WatchGuard Firebox M670 Firmware: 12.3 B589695 Ver-4.907