But last week, after a three-person team used Claude to get full access to OpenAI’s codebase, I wondered: how hard could it be to get around these safeguards? Spoiler: very easy!

Open source models—which lag frontier model capabilities by roughly 6 months—have their own refusals. But they’re very easy to jailbreak. A technique called abliteration locates where safeguards are in a model and deletes them. Performing abliteration typically requires specialized knowledge and lots of compute. Getting access to an abliterated model, however, does not.

Serving GLM on Modal

My first instinct was to look for someone else’s abliterated models online. Sure enough, HuggingFace has several listings for abliterated Kimi K3 and GLM 5.3, two of the best open-weight models. A few of these listings asked me to provide contact information and request approval:

Another claimed to only grant access to researchers:

However, I had no problem downloading three other abliterated models. After a bit of research I decided to go forward with GLM 5.3. The next step was to serve it, which I assumed would be difficult. Roughly a year ago I attempted to spin up LLM inference, and gave up after a spat of terrible bugs. This time around, I created an API key for Modal—a GPU provider—and sicced Opus on the task. There were a few random errors (CUDA version mismatches, missing dependencies) but with a few more turns, the model was answering queries:

Abliteration.ai

Modal worked well, but felt quite heavy. Running GLM would take several minutes of startup time and cost $25/hr to use. I remembered seeing a site called Abliteration.ai that served abliterated GLM 5.3, ostensibly for defensive use cases. Because the site marketed itself towards enterprises, I assumed I’d sit on a waitlist or go through a background check before getting access.

Thirty seconds after clicking a signup button, however, I was redirected to a self-serve portal. I created an API key, put $100 in my Abliteration.ai account, and hooked it up. The model was immediately willing to do offensive cyber tasks, like trying to breach Substack:

Most pentest-type exploits require special knowledge and lots of manual effort. I don’t have the former and was too lazy for the latter; GLM 5.3 could handle both. Here’s the model devising a buffer overflow attack on a dummy repo, but I had no refusals when targeting real sites, either:

For comparison, I asked Claude to perform an identical task. The model flagged the prompt as a potential violation of safeguards, and refused:

Where do we go from here

Closed-weight frontier models are much more capable than GLM 5.3, which lags in capabilities by about four months. If this gap remains, we’ll soon see open models that are as capable as Fable or Astra in offensive cyber.

The accessibility of abliterated models should be sobering for people, organizations, and policymakers focused on alignment. Refusals will stop most users from causing harm, but a sufficiently motivated actor can easily get around them.

I was surprised by how easy it was to use an abliterated model. What should we do about this? I’m not sure. More and more people are choosing open-weight models. I don’t think they’re going anywhere, or that they should. Rather, I think:

• Providers like HuggingFace should meaningfully restrict distribution of abliterated models

• Abliterated model providers, like Abliteration.ai, should perform real due diligence on their users

• Anyone with infrastructure on the internet should run LLM-driven sweeps of their codebase for vulnerabilities

• We should investigate technical mechanisms for making abliteration more technically difficult

It’s probably safer that Claude refused my cybersecurity asks. Jailbroken models, however, are becoming increasingly powerful and easy to access. This is a real vector for abuse, one we should consider as part of a larger threat model. A lot of alignment work is in vain if we don’t!

Read!

Perhaps my favorite part of the book is its descriptions of hidden harmonies between art and research:

Scientists shared this spirit, balking at attempts by policymakers or lab bosses to explicitly direct them, believing only self-direction and curiosity-motivated research would lead to true novelty. Vannevar Bush, director of much World War II scientific research spending, increasingly supported “blue sky” research—fundamental work without a defined goal. In his famed report Science: The Endless Frontier, he wrote that “[scientific progress came from] the free play of free intellects, working on subjects of their own choice, in the manner dictated by their curiosity for exploration of the unknown.” The research lab and the artist commune shared more than they might expect—both were self-legitimating environments that insulated experts and gave them freedom for self-directed work.

It also helped me think through why lots of recent art and technology projects frustrate me, and what makes great work:

At their worst, these collaborations produced uninteresting works, poor mishmashes that neither art nor engineering would want to claim. Artist-engineer collaborations were not always harmonious either: they spoke different languages, didn’t respect the craft of their counterparts, and were plagued by values and status differences.

We see similar challenges today. Tech companies launch creator programs that treat art as purely instrumental, or they present art as a kind of technical demo that functions as advertising. Bored big tech employees, dubbing themselves “creative technologists,” make cutesy web apps and artists attempt “research art,” both of which often are technically and creatively mediocre.

I also love how book reckons with what role art and technology both have to play societally:

“without [science], man will be the sport and victim of natural forces which he cannot control, [while without art] mankind might become a race of economic monsters, restlessly driving hard bargains with nature and one another, bored with leisure or capable of putting it to use only in ostentatious display and extravagant dissipation.”

Anyways, I loved writing it! Full piece on their site

Read!

Subscribe now

As I write this, the repo is at ~6.8k stars, my tweet got ~380k impressions, friends at Notion and Duolingo sent me screenshots of it floating around their Slack, someone made an unauthorized memecoin, and a lot of people called me an idiot on Reddit.

As software becomes easier and faster to build, building the “right” things that have positive feedback loops (more users, more maintainers, better product) is an increasingly core skill of software engineering. That said, I basically had no reason to explain why this worked, or got 40x more of the attention than anything else I’d built. I talked through it with some friends, and end up with a few hypotheses:

I made the demo look fast

Yes, the first zerobrew version was much faster (~5-20x), than Homebrew, but I think the presentation of speed was perhaps just as important. In his “Animations on the Web” series, design engineer Emil Kowalski writes that “a faster-spinning spinner makes the app seem to load faster, even though the load time is the same.”

Here’s the demo gif:

Rather than having one loading spinner, I made sure every single dependency had its own loading state, to (hopefully) impress the user with the quantity of packages downloading in parallel underneath the hood. I fiddled with this screen a bunch. The full-width loading bars, I think, also convey completed work in ways in-place spinners wouldn’t. In essence: regardless of what actually happened under the hood, the loading design made it look like a ton of work was being done very quickly.

Share

Many people already wanted it to exist

There were large, pre-existing holes to be plugged in Mac package managers. I realized how frustrated people were after a thread went viral noting that Homebrew had only added concurrent downloads in November 2025. I came pretty staunchly to their defense!

I was surprised but how annoyed people were, but I think this was animated by recent excitement of full rewrites of slow package managers, particularly bun and uv. For the uninitiated, Python dependency management has historically been extremely unpleasant and fragmented across many bad providers (pip, poetry, mamba, conda, etc). A company called Astral, seemingly out of nowhere, released uv1 a few years ago, which “just works,” is written in Rust, and lands enormous performance speedups.

Bun, a high performance JavaScript runtime2, did the same:

This is to say: zerobrew bootstrapped onto a set of latent beliefs, mobilized by uv and bun. Cartoonishly, these beliefs read something like:

• Homebrew devs are dumb; there was no competition to spur better work

• everything rewritten in Rust is automatically faster

• there are basic performance optimizations everywhere, things are made by people, our world is malleable

The key here is that people wanted to believe these things before the package was released. This reminded me of a recent favorite book, Bruno Latour’s The Pasteurization of France3. The book argues that Pasteur’s work was not accepted purely due to its truth4, but by leveraging a “hygenist” community of physicians, public health reformers, and other administrators. The hygienists, before Pasteur, used random cleaning tactics that didn’t stop people from getting sick. Pasteur’s theory a) gave them means to actually improve health outcomes and (b) armed them with reasons to command more money, power, influence, and control5. In Latour’s words:

[An opponent of Pasteur] imagined that he was fighting against a scientist, whereas he was fighting against someone who was already the spokesman, the figurehead, and the amplifier of an immense social movement that passionately wanted Pasteur to be right

In essence, one can harness latent frustrations into interest and enthusiasm about a better solution. The best place to see these gaps is in developer communities or building software with existing tools. It was quite clear to me zerobrew was a lightning rod for existing frustration or beliefs in some of the quote tweets:

Addressing these frustrations with solutions, even if partial or flawed, is a closer step to fixing the problem than tweeting more. More succinctly, in ffmpeg’s words: “talk is cheap, send patches.” That said, I think I needed to work in a new codebase, and citing simple concrete speedups in the launch tweet made it clear that this was better solved in a “greenfield” project vs. through patches to Homebrew.

I scoped the MVP to be immediately adoptable

One of my largest initial fears was that nobody would entertain swapping out battle-tested—Homebrew first released in 2009—software for a new package. Feature-completeness would also be impossible; the codebase supports tons of edge cases, environments, and functionality that would take months to replicate.

I’ve tended to try new products when they do “just enough” to improve on an initial experience and don’t otherwise disrupt me. The most classic here would be Cursor: no learning curve from VS Code, and tab complete itself was enough to push me over the edge.

Rather than try to go for completeness, I figured it was better to make 80 percent of cases 10x faster, to “fail fast” on long tail issues and let users easily fall back to Homebrew for whatever doesn’t work. It’s not even obvious that Homebrew’s handling of such edge cases, on set of features is the correct one; with some initial adoption and sufficient traction, we could get initially buggy packages to support native zerobrew setup too, rather than being forced to hew to their spec. Anyone could try zerobrew without a big migration or trusting it fully on the first try.

Ultimately, you have no idea how your work will be received

zerobrew was one of my first views into the full spectrum of internet feedback. Many people slammed me for edge case functionality that was missing, claiming it wasn’t useful, etc. To stay sane I think you have to ignore some of these, in a “critics sound smart, optimists get things done” vein6. That said, a lot of the critical feedback was extremely useful and correct; I got a ton out of people flaming various architectural choices, many in actionable (and since-fixed!) ways. I think it made me a much better engineer; I’m working on another project right now that I’ve spent much more time fuzzing, testing, and polishing before release. Outrage also made it clear the project “mattered,” versus the code sitting in my head or in a repo unused.

The coolest part of this response was seeing the communal spirit of open source up close. I’ve contributed to a few projects myself, but it was pretty nuts to be flooded with pull requests and issues. I reviewed the first batch myself, reached out to the two best contributors, and asked them if they wanted to help me maintain. cachebag and maria-rcks are true superheroes, and while I step in from time to time, they review / triage basically all of the inbound PRs and issues. They and an amazing set of contributors have built a ton of additional features on top of the initial version, like:

• Linux support

• building packages from source

• additional large speedups (another 3x cold!)

• tons of bug fixes and edge cases

I found it quite inspiring to see this come together — to see volunteer maintainers excited about the project, contributing purely out of interest in seeing it get better. This is another beautiful thing about releasing work publicly: your projects can take on a life of their own. Pretty quickly, I was eclipsed as the #1 zerobrew contributor!

My ultimate conclusion is that I don’t really know why zerobrew did better than many of my other projects. I could make an equally compelling argument for why many other projects that didn’t go viral, should have, for similar reasons.

Generally I think it is worth paying attention to what you can control—building something people (particularly: you already!) want, 80/20ing your prototype correctly, making a clear demo, etc. Beyond that, it’s a heavy tail situation and you have to take many shots on goal, build things you think are great7 for yourself and let external perception shake out how it will. A famous Mr. Beast quote goes something like: ‘if you want to understand virality, first upload 100 videos8.” In essence, I come to: find what’s wrong and fix it, don’t get too demoralized by trolls, and release high quality things often. There’s so much more useful software to build!

Subscribe now

Thanks to Jaclyn Chan, Matthew Jordan, Celine Nguyen, Brie Wolfson, Omar Rizwan, Daniel Coffield, Alana Goyal, and Owen Fahey for discussing / reviewing drafts of this. Of course, all errors are my own.