Which SD-WAN product should enterprises choose?

Adoption of Software Defined Wide Area Network (SD-WAN) has reached an inflection point and nearly every distributed business is deploying, evaluating, or planning to implement an SD-WAN as part of its IT vision.

To help enterprises understand the merits of products in the market and identify the capabilities best suited to meet their use case requirements, NSS Labs announced the results of the industry’s first Software Defined-Wide Area Network (SD-WAN) Group Test.

The capabilities that were assessed by NSS Labs include:

• Remote Initial Configuration
• WAN Impairment and Link Failover
• Dynamic Path Selection with SLA Measurements
• Path Conditioning and Application-Aware Steering
• Link Saturation and Congestion
• Application-Aware Traffic Steering
• Performance by Application Flows
• Raw Packet Processing Performance – UDP Throughput and UDP Latency
• Maximum Capacity
• HTTP Capacity and limits
• Application-Aware Response Time
• Security Effectiveness

While security effectiveness was not included in the final scores, it was evaluated in products offering security capabilities.

Why implement an SD-WAN?

The traditional WAN architecture is becoming increasingly expensive and ineffective as the adoption of cloud-based applications is changing the type of traffic remote branch offices are experiencing. This growth in the adoption of cloud-based applications, be they software-as a service (SaaS) or applications hosted in an infrastructure-as-a-service (IaaS), also disrupts the prominence of Multiprotocol Label Switching (MPLS)-based WAN connectivity to the branch.

The software-defined wide area network (SD-WAN) is the union of software-defined networking (SDN) with WAN technology. SD-WANs simplify link establishment and management, enable quality of service (QoS) features based on application and service requirements, and afford policy control capabilities.

The global market intelligence firm IDC estimates that the SD-WAN market will grow at a compound annual growth rate of 69%, reaching $8 billion by 2021.1 According to an NSS Labs research survey, 87.8% of US enterprises surveyed indicate that their organization have or will adopt SD-WAN in the next 12 months. Of the enterprises currently deploying SD-WAN, 52.9% plan on expanding their use of the technology.

The NSS Labs SD-WAN Group test provides the most comprehensive technical research available of SD-WAN products on the market today. Based upon hundreds of hours of rigorous testing, the SD-WAN Group Test includes detailed individual test reports for nine products, plus three comparative reports: a TCO Comparative Report, a Performance Comparative Report, and the NSS Labs Value Matrix Comparative Report.

Key findings:

• NSS Labs testing uncovered few outright flaws in the products tested, indicating that as a technology category, SD-WAN is primed for widespread adoption.
• NSS Labs research indicates that the SD-WAN market will continue to evolve, and both SD-WAN and security will become integral components of the modern WAN Edge architecture.
• Testing revealed the areas of primary differentiation for SD-WAN products to be Quality of Experience (QoE) for VoIP, QoE for video, and network performance. There was also a wide disparity in pricing and total cost of ownership (TCO) as would be expected in a high growth emerging market space.
  • On a QoE scale of 1 – 5 for VoIP, with 3.4 being the minimum viable NSS Labs considers to meet the use case for VoIP, and with a theoretical maximum score of 4.41, the tested range was 2.49 to 4.38 with eight of the nine tested products scoring above the minimum of 3.4.
  • On a QoE scale of 1 – 5 for video, with 3.4 being the minimum viable NSS Labs considers to meet the use case for video, and with a theoretical maximum score of 4.53, the tested range was 1.10 to 4.47 with seven of the nine tested products scoring above the minimum of 3.4

  Quality of Experience (QoE) refers to the measure of satisfaction an enterprise experiences with a voice or video service. NSS Labs tested nine SD-WAN products using a QoE scale ranging from 1 – 5 for VoIP and video. NSS Labs considers a QoE score below 3.5 for VoIP and video represents a significantly degraded voice call/video stream, out of a maximum achievable score of 4.41 and 4.53 respectively.

“The SD-WAN market holds strong potential and there are several impressive technologies available today for companies seeking to enhance or replace traditional WAN deployments,” said Jason Brvenik, Chief Technology Officer at NSS Labs.

“The NSS Labs 2018 SD-WAN Group Test is the industry’s first baseline assessment of these technologies. Our testing validates that this is a technology that is ready for prime time. We encourage enterprises to examine our findings for insights regarding the capabilities, performance, and cost of solutions as they continue to evolve their WAN architectures.”

Products tested

• Barracuda NextGen Firewall F-Series F80 v7.1.1
• Citrix Systems Inc. NetScaler SD-WAN 10.0.0.207
• Cradlepoint AER2200-600M v6.5.0
• FatPipe Networks MPVPN/SD-WAN v9.1.2
• Forcepoint NGFW 1101 vSMC 6.3.6, engine 6.3.6.19302
• Fortinet FortiGate 61E v6.0.1 GA build 5068
• Talari Networks Adaptive Private Networking (APN) Software vAPN 7.1
• Versa Networks FlexVNF v120
• VMware NSX SD-WAN by VeloCloud vEdge version 3.2

Unverified products:

• Cisco
• Silver Peak

NSS Labs is committed to providing empirical data and objective group test results that enable organizations to make educated decisions about purchasing and optimizing security infrastructure products and services.

As with all NSS Labs group tests, there is no fee for participation, and the test methodology is available in the public domain to provide transparency and to help enterprises understand the factors behind test results.

To download the NSS Labs Value Matrix, which provides a graphical comparison of the TCO per Mbps (Value) and the QoE scores for the VoIP and video of tested products, go here.