Proponents Advocate for Performance Tests Over Certificates in Cybersecurity Hiring
In an innovative move to address the significant gap between the workforce needs and the available talent pool, the U.S. Department of Defense (DoD) is piloting new skills-based assessments aimed at transforming its cyber hiring strategy. This initiative was unveiled at the recent AFCEA Cyber Workforce Summit, where attendees had the opportunity to experience firsthand the new assessment methods.
According to Matt Isnor, the Division Chief for Cyber Workforce Development within the DoD’s Chief Information Officer’s office, reliance on traditional qualifications has proven inadequate in effectively evaluating candidates for cybersecurity roles. Many existing certifications, he noted, fail to accurately represent the practical skills necessary for success in real-world scenarios. Rather than adhering to outdated measures, Isnor emphasized the need for assessments that simulate actual operational conditions to better gauge the expertise required for the department.
The primary objective of this pilot program stems from alarming statistics indicating that around 20,000 military and civilian cyber positions remain unfilled within the DoD, translating to a vacancy rate of nearly 10 percent. This persistent shortage underscores the urgent need for a workforce recruitment strategy that expands the selection criteria beyond advanced degree holders and seasoned professionals. Isnor insisted that tackling these recruitment challenges is the department’s foremost priority, stating, “We have a huge, huge gap inside of the department.”
The skills-based assessments unveiled at the summit are developed to be comprehensive and reflective of real-life challenges that cybersecurity personnel encounter daily. These assessments featured two separate 30-minute tests, which are hosted in the cloud and designed to be interactive, requiring candidates to engage with the material actively rather than passively answering multiple-choice questions. Currently being piloted with both military and civilian staff, these tests are an important step in reshaping the hiring process.
In one notable assessment, candidates are tasked with analyzing malware. Participants are presented with a forensic copy of a thumb drive, which is paired with a variety of automated security reports. They must utilize this information to formulate responses to structured questions—information that simulates what would be necessary for preparing a comprehensive report post-cyber incident. Similarly, another assessment focuses on network data analysis, where candidates are required to scrutinize data captures to detect anomalies akin to logs that analysts review in real-world cybersecurity investigations.
These assessments have been crafted through collaboration with a pool of 40 to 50 subject matter experts, including hiring managers experienced in identifying the skills that are vital for cybersecurity roles. The intention is to align these evaluations with two of the more than 70 cybersecurity job roles within the DoD: Cyber Defense Analyst and Cyber Defense Incident Responder. Collectively, vacancies in these roles account for nearly half of the unfilled positions across the department’s cyber workforce.
Isnor illustrated that these short assessments serve as an initial screening tool, with a subsequent, more detailed 90-minute test expected to delve deeper into the specific skills and knowledge pertinent to each position. This two-tiered approach facilitates a more nuanced understanding of a candidate’s capabilities, increasing the likelihood of accurately identifying those qualified for roles critical to national security.
The validation process for these assessments is currently underway, involving qualified cyber analysts as well as control groups devoid of such skills. This meticulous evaluation is essential to ensuring the effectiveness of the tests. A contractor involved in the initiative, though opting to remain anonymous, highlighted the importance of achieving a passing rate for skilled candidates without inadvertently leading the control group to success, as that would indicate flaws in the testing process.
Looking ahead, Isnor confirmed that once the pilot phase concludes and evaluations are finalized, these assessments will be implemented for real-world hiring purposes. The goal is to incorporate these measures into 10 percent of new hires for vacant positions by the end of fiscal year 2027, which is approximately 18 months from now.
This forward-thinking approach represents a critical shift in how the DoD perceives and recruits cybersecurity talent, aiming not only to fill vacancies but to cultivate a workforce equipped with the necessary skills to tackle the increasingly complex landscape of cyber threats. By prioritizing performance assessments over traditional certifications, the department hopes to identify and attract the best and brightest candidates to meet its cybersecurity needs.