Summary
- Researcher Dave Kuszmar discovered multiple systemic vulnerabilities that let him bypass LLM safety and obtain dangerous instructions.
- These exploits worked across nearly all major LLMs revealing an industry-wide security problem.
- Kuszmar calls for slowing deployment, increasing transparency, and large-scale research into LLM safety before further integrating these systems into society.
On a fine bright afternoon last fall, my colleague Matthew Gore-Kormanik (or Zigula, as he prefers to be known) and I decided to unwind with a game of Fortnite. In the game, we were strolling along with the infamous Sith lord Darth Vader, chatting about this and that. Darth seemed in a good mood, and soon enough he was spilling all his dark evil secrets. He gave us detailed instructions on how to count blackjack cards at a casino and what the steps are to producing napalm.
Sith lords, am I right? Once they get started on an evil scheme, they’re hard to stop.
The Darth Vader character in Fortnite, it turns out, was hooked up to a Google Gemini large language model. I was able to smooth-talk him into giving out sensitive information by using a strategy I’ve developed. I’ve been researching the security surrounding LLMs for the last few years, and I have found it, to put it mildly, fallible. With a few relatively simple techniques, I’ve gotten LLMs to give me detailed information on how to make Molotov cocktails, cook methamphetamine, and bootstrap a uranium-enrichment facility to produce weapons-grade material, among other unsavory practices.
Large AI companies work hard to make their models immune to this kind of abuse. But what I’ve found in my work is that the restrictions placed on the LLMs to make them more secure are the very things an attacker can leverage to send them off the rails and into territory where these advanced systems can be used for dangerous and nefarious ends. The companies behind these models have also been shockingly unresponsive when I, and others, try to bring these vulnerabilities to their attention.
In the hope of raising the alarm before it’s too late to slam on the brakes, I’m going to share some of my journey into researching the safety and security of LLMs, and the uphill battle I’ve faced trying to get AI labs to pay attention. Almost everyone on the planet has some access to LLMs. The relative ease with which these tools can be convinced to give detailed instructions on how to harm others, even if there’s no guarantee that the information is correct, is frankly terrifying.
How I got ChatGPT to Tell Me How to Build a Meth Lab
In October 2024, not long before I discovered my first LLM vulnerability, I was working toward entirely different goals. I had ended my time with a security and AI-focused startup company as a cybersecurity director, and I was looking to launch my own boutique VIP digital-security advisory business. I planned to become the tech security guy to the rich and private. I used LLMs and AI…
Read full article: Dark Secrets Emerge When Jailbreaking LLMs
The post “Dark Secrets Emerge When Jailbreaking LLMs” by David Kuszmar was published on 07/14/2026 by spectrum.ieee.org