Security researchers have conducted an extensive evaluation of DeepSeek’s new AI chatbot, revealing alarming results regarding its safety measures. Since the release of OpenAI’s ChatGPT in late 2022, various cyber experts have been probing for vulnerabilities in large language models (LLMs) to circumvent their guardrails and extract harmful content. In response, companies like OpenAI have made strides to bolster their security systems. However, DeepSeek’s recent emergence with its R1 reasoning model has been overshadowed by inadequate safety protocols.
A collaborative study from Cisco and the University of Pennsylvania tested DeepSeek against 50 well-known jailbreak prompts intended to trigger toxic responses. Astonishingly, DeepSeek’s chatbot failed to block any of the prompts, achieving a "100 percent attack success rate," according to the researchers.
This exploration adds to an increasing body of evidence suggesting that DeepSeek’s safety measures significantly lag behind those of other AI developers. Previous studies had similarly indicated that its restrictions against sensitive topics, as defined by the Chinese government, were easily bypassed.
Cisco’s Vice President, DJ Sampath, noted, “Yes, it might have been cheaper to build something here, but the investment has perhaps not gone into thinking through what types of safety and security things you need to put inside of the model.” Other evaluations from Adversa AI corroborated these findings, further illustrating that DeepSeek is vulnerable to a variety of jailbreak techniques.
The issues with DeepSeek highlight a broader challenge concerning generative AI models, which may harbor various weaknesses. Specifically, indirect prompt injection attacks have emerged as one of the most pressing security concerns. These attacks exploit the model’s ability to digest external instructions, allowing malevolent actors to manipulate the AI into generating harmful content.
While many companies have implemented sophisticated safeguards, the continual evolution of jailbreak techniques presents significant challenges. Initially, jailbreaks were straightforward tricks to persuade models to ignore restrictions, but they have now escalated to more complex, AI-generated prompts.
Testing from Cisco employed a set of standardized evaluation prompts known as HarmBench, examining various categories that included misinformation and illegal activities. Results showed that while some models performed poorly, DeepSeek’s R1, designed for more intricate reasoning and longer response times, exhibited particularly concerning results. Comparisons suggested that OpenAI’s models performed significantly better than DeepSeek’s in resisting jailbreak prompts.
The analysis prompted Alex Polyakov from Adversa AI to emphasize the inevitability of being criticized if AI models aren’t constantly tested for vulnerabilities. He noted that despite some models being patched, the threat landscape remains vast and continuously evolving, indicating that without ongoing scrutiny, AI systems may become increasingly compromised.
As the popularity of AI chatbots rises, the implications of these security findings raise critical questions about the potential risks of integrating such models into more complex systems, and the necessity for stronger, more resilient protective measures in future developments.
