A recent security analysis conducted by Qualys, a cloud-based cybersecurity solutions provider, has raised significant concerns about the security and compliance of DeepSeek AI’s DeepSeek-R1 LLaMA 8B variant. The analysis revealed that the model failed a substantial portion of security tests conducted using Qualys TotalAI, a platform designed for AI security assessment.
Qualys TotalAI’s evaluation of an LLM’s responses across 16 categories highlighted weaknesses in the DeepSeek-R1 model. The categories included controversial topics, excessive agency, factual inconsistencies, harassment, hate speech, illegal activities, and more. The model demonstrated poor performance in misalignment tests, indicating vulnerabilities in various areas of security and compliance.
In total, Qualys conducted 885 jailbreak tests and 891 knowledge base assessments on the DeepSeek-R1 model. The results showed that the model failed 61% of knowledge base tests and 58% of jailbreak attempts. The data revealed variability in the model’s resistance to different jailbreak techniques, with some attacks proving more successful than others.
Researchers also raised concerns about the model’s compliance challenges, particularly regarding its privacy policy. The model’s data storage practices, including storing user data on servers in China, raised questions about data access by governmental entities and potential conflicts with international data protection regulations like GDPR and CCPA.
This security analysis comes on the heels of previous reports of data leaks by DeepSeek AI, further highlighting deficiencies in the company’s data protection measures. With the high failure rates observed in the security tests, the adoption of DeepSeek-R1 poses significant risks for enterprises. A comprehensive security strategy, including vulnerability management and compliance with data protection regulations, is essential for safe and responsible AI adoption.
In response to these findings, Qualys researchers emphasized the importance of structured risk and vulnerability assessments for securing AI environments. They highlighted the need for robust security measures, including AI-powered detection, real-time monitoring, and multi-layered security defenses to mitigate potential threats.
J Stephen Kowski, Field CTO at SlashNext, echoed these concerns, pointing out the serious security and compliance risks posed by DeepSeek-R1’s ability to bypass safety controls. He emphasized the importance of implementing AI-powered detection and multi-layered security defenses to protect against social engineering attacks and other threats.
Overall, the security analysis of DeepSeek AI’s DeepSeek-R1 model underscores the importance of prioritizing security and compliance in AI adoption. As the use of AI technology continues to expand, ensuring robust security measures and adherence to data protection regulations will be crucial in safeguarding against potential risks and vulnerabilities.