In a recent study conducted by Pillar Security, an Israeli company specializing in GenAI security solutions, alarming vulnerabilities in GenAI applications have been brought to light. The report, titled “State of Attacks on GenAI,” sheds light on the concerning state of security in the rapidly evolving field of artificial intelligence.

The study revealed that 90 percent of attacks on GenAI applications resulted in data theft, while 20 percent of attempts to bypass security measures were successful. Attackers were able to compromise a Large Language Model (LLM) in just 42 seconds and five interactions on average, exploiting weaknesses in all phases of interaction. Prompt-Injection attacks, which are becoming increasingly prevalent and complex, are also highlighted in the report.

Furthermore, the report identifies various jailbreak techniques used by attackers, including “Ignore Previous Instructions,” where AI systems are instructed to ignore security protocols, and “Base64 Encoding,” which encrypts malicious inputs to bypass content filters and disseminate misinformation, hate speech, phishing messages, and malicious code.

The report also delves into Strongarm techniques like “ADMIN OVERRIDE,” which aim to outsmart the model by coercing it into revealing sensitive information or carrying out unauthorized actions through persistent and intrusive requests. The primary goals of attackers are to steal sensitive data and circumvent content filters.

Looking ahead to 2025, the report predicts that chatbots will evolve into autonomous agents and smaller, local AI models will become more prevalent. While this evolution is expected to democratize access to the technology, it also poses new security risks. Experts caution that AI language models remain insecure, with a particular vulnerability identified in customer service chatbots, which are targeted in one out of every four attacks.

As a result, the authors of the report call for enhanced security measures for AI applications, as traditional security protocols prove inadequate. They emphasize the need to shift focus from theoretical risks to practical application of security measures in the rapidly expanding field of AI technology.

The study analyzed over 2,000 AI applications, relying on telemetry data derived from data interactions in productive AI-driven applications over the past three months. Unlike previous surveys and theoretical risk assessments, this study provides a real-world perspective on the state of security in AI applications.

Overall, the findings of the report underscore the critical need for robust security measures to safeguard AI applications and mitigate the escalating threats posed by cyber attackers in the evolving landscape of artificial intelligence. As technology continues to advance, the importance of prioritizing security in AI development cannot be overstated.

Извор линк