NetSecOpen, a collective of network-security companies and hardware testing organizations, is making significant progress in establishing testing and benchmark standards for network-security appliances. Despite the challenges and disagreements that have arisen between product makers and test labs in the past, NetSecOpen aims to create a consensus method for testing and benchmarking these devices, allowing for fair and accurate comparisons between different vendors.
The latest version of NetSecOpen’s network-security testing standard for next-generation firewall technology was published in May. The group sought feedback on this version as it moves closer to finalizing the standard. Once complete, this standard will enable multiple labs to execute the same test requirements using different tools, providing comparable results. This approach aims to address the issue of varying test methods employed by different third parties.
Brian Monkman, the executive director of NetSecOpen, emphasized the significance of this standard-setting effort, describing it as something unprecedented. He drew a comparison to the introduction of standard test requirements for measuring miles per gallon in different vehicles, which led to the creation of a universal standard. NetSecOpen aspires to achieve a similar outcome by establishing common testing guidelines for network-security appliances.
NetSecOpen was established in 2017 with the specific goal of bridging the gap between product makers and test labs. Its members include major network-security firms such as Cisco Systems, Fortinet, Palo Alto Networks, and WatchGuard, as well as testing equipment makers like Spirent and Ixia. Additionally, evaluators such as the European Advanced Networking Test Center (EANTC) and the University of New Hampshire InterOperability Laboratory (UNH-IOL) are also part of the collective.
While the current testing standards for firewalls are based on an internet standard published 20 years ago, NetSecOpen recognizes the need to update these guidelines to account for the dramatic changes in technology. The draft published by NetSecOpen states that security function implementations have evolved and diversified over the years, necessitating the establishment of well-defined and reproducible key performance indicators (KPIs) to enable fair comparisons between network security functions.
To increase the realism of their tests, NetSecOpen plans to use real-world data and simulate realistic network loads and security threats. Their attack traffic test set, for example, includes common vulnerabilities that attackers have exploited in the past decade. The draft recommends specific test architectures, traffic mixes, and enabled security features. It also highlights the importance of testing the capabilities of emulated browsers, attack traffic targeting known vulnerabilities, and throughput performance metrics.
Palo Alto Networks, a founding member of NetSecOpen, actively collaborates with the group to create and conduct tests for their firewalls. Samaresh Nair, director of product line management at Palo Alto Networks, emphasized that the testing process is standardized with accredited test houses. Customers can utilize these standardized results to evaluate various products effectively.
NetSecOpen recognizes that the threat landscape is constantly evolving and, as a result, aims to update its vulnerability test sets. The Cybersecurity and Infrastructure Security Agency (CISA) has demonstrated that smaller, seemingly insignificant vulnerabilities can be combined to form effective attacks. NetSecOpen acknowledges that these vulnerabilities, which were previously dismissed, now pose a significant threat and need to be addressed.
Looking ahead, NetSecOpen plans to expand its testing efforts to include cloud environments, such as distributed cloud firewalls and web application firewalls. Chris Brown, technical manager at UNH-IOL, which joined NetSecOpen in 2019, explained that while the mission for well-defined, open, and transparent standards remains the same, the inclusion of cloud security testing will broaden the range of products assessed. Brown emphasized that despite the benefits of cloud computing, network perimeter defense remains necessary.
In conclusion, NetSecOpen is making promising progress in establishing testing and benchmark standards for network-security appliances. By creating a consensus method for testing and benchmarking, the group aims to enable fair comparisons between different vendors’ devices. With the support of industry-leading companies and testing organizations, NetSecOpen is committed to improving the reproducibility and transparency of network-security tests.