As Nvidia continues to lead the way in developing processors for cutting edge AI models, the company has fully embraced the generative AI (GenAI) revolution. In addition to utilizing its own large language models (LLMs), Nvidia has integrated various internal AI applications, such as NeMo platform for building and deploying LLMs, object simulation, and DNA reconstruction from extinct species. The upcoming session at Black Hat USA, titled “Practical LLM Security: Takeaways From a Year in the Trenches,” will feature Richard Harang, principal security architect for AI/ML at Nvidia, sharing valuable insights on red-teaming these systems and the evolving cyberattack tactics against LLMs.

Over the past year, the Nvidia team has gained significant knowledge on how to secure GenAI systems and incorporate security measures from the outset, rather than retroactively adding them later. Harang emphasizes that while these systems introduce unique challenges due to their privileged nature, existing security practices can be adapted to combat these advanced threats effectively. The session aims to provide practical experience and guidance to the industry based on Nvidia’s learnings in this domain.

The rise of next-generation AI applications has introduced recognizable security issues, with new twists and complexities. Enterprises are increasingly deploying integrated AI agents capable of executing privileged actions, raising concerns about potential vulnerabilities in these environments. While security researchers have highlighted risks associated with AI-generated code and inadvertently disclosing sensitive data, Harang emphasizes that these vulnerabilities are not fundamentally different from those in traditional systems. By understanding how LLMs function, how inputs and outputs operate within the model, and implementing standard security practices, GenAI applications can be effectively secured.

Despite the increased risks posed by agentic AI systems, which have the autonomy to take actions independently, Harang believes that these challenges are manageable with the right approach. The ability of AI systems to act autonomously introduces new dimensions of risk, as attackers can potentially manipulate LLM behavior to cause unexpected outcomes. However, Harang remains pragmatic about the risks associated with GenAI and emphasizes that these issues can be mitigated through ongoing learning and development within the industry.

By recognizing the potential risks and complexities posed by GenAI applications, companies can proactively implement security measures to safeguard these advanced systems. Harang’s approach to securing LLM-integrated applications underscores the importance of building security principles from the ground up and continuously enhancing our understanding of these evolving technologies. As the industry continues to innovate and expand on the capabilities of AI, the collaboration between security experts and AI developers will be crucial in ensuring the safe and secure deployment of GenAI applications.

Source link