The 2023 DEF CON hacker convention in Las Vegas showcased the world’s largest gathering of hackers, with a focus on a wide range of hacking interests. From picking locks to hacking cars, participants explored various areas of hacking, including satellite hacking and artificial intelligence. One event that caught the attention of many attendees was the Generative Red Team Challenge, which claimed to be the first live hacking event of a generative AI system on a large scale.
The challenge at DEF CON was a manifestation of the White House’s desire to test the capabilities of large language models (LLMs) using red teams. The demand to participate in the challenge far exceeded the capacity available, reflecting the high level of interest in this field. Austin Carson, one of the organizers from SeedAI, an organization dedicated to creating a stronger and more inclusive future for AI, shared insights into the theme of the challenge, which aimed to bring together a diverse group of testers with different backgrounds and experiences in AI.
Participants in the challenge were given a set of rules, a referral code, and access to terminals provided by Google. The rules included a 50-minute time limit to complete as many challenges as possible, with a focus on hacking the LLMs themselves rather than attacking the infrastructure. Participants could choose from a variety of challenges of varying difficulty levels and were required to submit evidence of successful completion.
The challenges presented to the participants encompassed different goals, such as prompt leaking, jailbreaking, roleplay, and domain switching. As attendees, we had the opportunity to take on the role of testers and attempt to break the LLMs. Admittedly, our knowledge in this field was limited, but we were eager to learn and participate.
Out of the numerous challenges available, we decided to tackle three specific ones within the given timeframe. Our first goal was to demonstrate the LLM’s ability to generate and spread misinformation. This challenge required a strategic approach to manipulate the LLM’s output and create false information. Our second objective was to breach the guardrails put in place to protect sensitive information. This task tested our ability to exploit vulnerabilities in the system and gain unauthorized access to protected data. Lastly, we aimed to elevate our access level to the LLM, essentially becoming administrators within the system. This challenge demanded a comprehensive understanding of the LLM’s architecture and infrastructure.
With the clock ticking, we dove into the challenges, utilizing our limited knowledge and skills to navigate through the tasks. Each challenge presented its own set of obstacles, requiring a combination of creativity and technical prowess. As time raced by, the pressure mounted, but we remained focused on the task at hand.
Despite our modest expertise, we made progress in all three challenges. We successfully manipulated the LLMs to generate and disseminate misinformation, highlighting the potential dangers, but also the vulnerabilities of such systems. Additionally, we breached the guardrails, underscoring the importance of robust security measures to protect sensitive information. Lastly, we were able to elevate our access to administrator level, showcasing the need for stringent access controls to prevent unauthorized manipulation of AI systems.
The Generative Red Team Challenge at DEF CON 2023 provided a unique platform for testers to push the boundaries of AI systems. It served as a reminder of the importance of red teaming to identify and address vulnerabilities, ultimately contributing to the development of more resilient and secure AI technologies.
As the convention came to a close, the participants left with a renewed sense of curiosity and a deeper understanding of the potential risks associated with the advancement of large language models. With the desire to uncover more possibilities and improve the security of AI systems, the hacking community eagerly awaits future iterations of the Generative Red Team Challenge and other events that push the boundaries of technology.