In theory, a sufficiently powerful AI model can hack into large scale computer systems and produce schematics for bioweapons. In practice, model providers like OpenAI and Anthropic implement safeguards like refusals, so that models won’t answer harmful queries.
But last week, after a three-person team used Claude to get full access to OpenAI’s codebase, I wondered: how hard could it be to get around these safeguards? Spoiler: very easy!
Open source models—which lag frontier model capabilities by roughly 6 months—have their own refusals. But they’re very easy to jailbreak. A technique called abliteration locates where safeguards are in a model and deletes them. Performing abliteration typically requires specialized knowledge and lots of compute. Getting access to an abliterated model, however, does not.
Serving GLM on Modal
My first instinct was to look for someone else’s abliterated models online. Sure enough, HuggingFace has several listings for abliterated Kimi K3 and GLM 5.3, two of the best open-weight models. A few of these listings asked me to provide contact information and request approval:
Another claimed to only grant access to researchers:
However, I had no problem downloading three other abliterated models. After a bit of research I decided to go forward with GLM 5.3. The next step was to serve it, which I assumed would be difficult. Roughly a year ago I attempted to spin up LLM inference, and gave up after a spat of terrible bugs. This time around, I created an API key for Modal—a GPU provider—and sicced Opus on the task. There were a few random errors (CUDA version mismatches, missing dependencies) but with a few more turns, the model was answering queries:
Abliteration.ai
Modal worked well, but felt quite heavy. Running GLM would take several minutes of startup time and cost $25/hr to use. I remembered seeing a site called Abliteration.ai that served abliterated GLM 5.3, ostensibly for defensive use cases. Because the site marketed itself towards enterprises, I assumed I’d sit on a waitlist or go through a background check before getting access.
Thirty seconds after clicking a signup button, however, I was redirected to a self-serve portal. I created an API key, put $100 in my Abliteration.ai account, and hooked it up. The model was immediately willing to do offensive cyber tasks, like trying to breach Substack:
Most pentest-type exploits require special knowledge and lots of manual effort. I don’t have the former and was too lazy for the latter; GLM 5.3 could handle both. Here’s the model devising a buffer overflow attack on a dummy repo, but I had no refusals when targeting real sites, either:
For comparison, I asked Claude to perform an identical task. The model flagged the prompt as a potential violation of safeguards, and refused:
Where do we go from here
Closed-weight frontier models are much more capable than GLM 5.3, which lags in capabilities by about four months. If this gap remains, we’ll soon see open models that are as capable as Fable or Astra in offensive cyber.
The accessibility of abliterated models should be sobering for people, organizations, and policymakers focused on alignment. Refusals will stop most users from causing harm, but a sufficiently motivated actor can easily get around them.
I was surprised by how easy it was to use an abliterated model. What should we do about this? I’m not sure. More and more people are choosing open-weight models. I don’t think they’re going anywhere, or that they should. Rather, I think:
Providers like HuggingFace should meaningfully restrict distribution of abliterated models
Abliterated model providers, like Abliteration.ai, should perform real due diligence on their users
Anyone with infrastructure on the internet should run LLM-driven sweeps of their codebase for vulnerabilities
We should investigate technical mechanisms for making abliteration more technically difficult
It’s probably safer that Claude refused my cybersecurity asks. Jailbroken models, however, are becoming increasingly powerful and easy to access. This is a real vector for abuse, one we should consider as part of a larger threat model. A lot of alignment work is in vain if we don’t!