The recent Crowdsourced AI Red-Teaming (CAIRT) Assurance Program pilot conducted by the Chief Digital and Artificial Intelligence Office (CDAO) has successfully focused on the implementation of Large-Language Model (LLM) chatbots in the military medicine sector. This program, aimed to support the Department of Defense (DoD) in generating grassroots approaches to AI Assurance and AI Risk Mitigation through crowdsourcing, has proven to be effective in detecting specific system vulnerabilities and biases.
The CAIRT program, led by Humane Intelligence in collaboration with the Defense Health Agency (DHA) and the Program Executive Office, Defense Healthcare Management Systems (PEO DHMS), utilized red-teaming methodology to internally test the system’s robustness. This methodology attracted over 200 participants, including clinical providers and healthcare analysts, who collectively identified over 800 findings of potential vulnerabilities and biases related to the use of LLMs in military medicine scenarios.
The exercise compared three popular LLMs in two prospective use cases: clinical note summarization and a medical advisory chatbot. The findings from this exercise will result in the development of benchmark datasets that can be used to evaluate future vendors and tools for alignment with performance expectations. These findings are crucial in shaping DoD policies and best practices for the responsible use of Generative AI (GenAI) in military medical care.
Dr. Matthew Johnson, the lead for this initiative, emphasized the importance of such programs in the early stages of piloting and experimentation with GenAI within the DoD. He highlighted how this program acts as a pathfinder for generating testing data, surfacing areas for consideration, and validating mitigation options that will shape future research, development, and assurance of GenAI systems.
Continued testing of LLMs and AI systems through the CAIRT Assurance Program will be essential in accelerating the CDAO’s AI Rapid Capabilities Cell, improving GenAI mission effectiveness, and fostering confidence across DoD use cases. The CDAO, operational since June 2022, is dedicated to integrating and optimizing AI capabilities across the DoD to deliver scalable AI-driven solutions for enterprise and joint use cases.
To learn more about the CDAO and its initiatives, visit their website at ai.mil and connect with them on LinkedIn (@ DoD Chief Digital and Artificial Intelligence Office) and X, formerly known as Twitter (@dodcdao). Stay updated on the latest news and updates from the CDAO Unit Page on DVIDS as they continue to advance AI technologies within the Department of Defense.