Is Your AI Getting Dumber? What "Nerfed" Means, Why Claude and ChatGPT Really Do Change After Launch, and How to Test It Yourself

Contents

• What "nerfed" actually means
• The short answer: yes, but not the way people think
• Six things that make an AI feel dumber
• The public nerf trackers, and what they say right now
• How I test it myself in ten minutes
• What to do when your AI feels dumber
• The takeaway

Every new AI model follows the same arc. Launch week, everyone is amazed. Two weeks later, the forums fill up with the same complaint: "it got dumber." Claude Opus 5.5 came out on 22 September, and before it was a week old a benchmark called Livenerf, built purely to answer "has Opus 5.5 been nerfed yet?", had hit the front page of Hacker News twice.

I have been on both sides of this. I have sworn a model was quietly downgraded, then gone back to my old chats and found out it was me. So this post explains what "nerfed" actually means, which cases were real and confirmed (more than you might think), and a ten-minute test you can run yourself instead of arguing about it online.

What "nerfed" actually means

"Nerf" is gamer slang for a studio weakening a weapon in a patch, after the foam toy guns. Applied to AI, it means the model you pay for got quietly worse after launch, with no announcement.

That one word covers four different things:

• The model itself changed. A smaller, cheaper or compressed version was swapped in behind the same name.
• The settings changed. How long it may think, what instructions it reads before your message, how much of the conversation it keeps.
• Something is broken. A routing or infrastructure bug is corrupting output or sending requests to the wrong place.
• Nothing changed except you. Your expectations rose, your tasks got harder, or you hit a limit you never hit in week one.

Only the first is what most people picture. The evidence so far points at the middle two.

The short answer: yes, but not the way people think

Anthropic and OpenAI both say they do not weaken models to save money or handle demand. Anthropic's wording, from a September 2025 postmortem, is blunt: "We never reduce model quality due to demand, time of day, or server load." And yet the same company has published two detailed admissions that Claude got worse for weeks at a time. They are the best public record of what nerfing looks like from the inside.

Case one: three infrastructure bugs, August to September 2025

Anthropic's September 2025 postmortem covered three overlapping bugs between 5 August and 18 September: one misrouted requests to servers set up for a different context window, one corrupted output so people saw stray characters and the odd wrong language, and one was a compiler bug on Google's TPU chips that affected which word the model picked next. At the peak, on 31 August, about 16 percent of Sonnet 4 requests were affected. Nobody touched the model. It felt nerfed anyway.

Case two: the Claude Code slump, March to April 2026

This is the one that made "nerfed" a mainstream word this year. Through March and early April, developers said Opus 4.6 and Claude Code felt lazier, more forgetful and more wasteful with tokens. An AMD engineer analysed nearly 7,000 of her team's coding sessions and found reasoning depth had dropped sharply since February. Anthropic staff pushed back at first, and VentureBeat ran a mid-April piece headlined "Is Anthropic nerfing Claude?"

On 23 April, Anthropic published a postmortem confirming three separate causes. On 4 March, Claude Code's default reasoning effort had been lowered from high to medium to cut latency ("This was the wrong tradeoff," they wrote). On 26 March, a caching optimisation shipped with a bug that wiped the model's thinking history repeatedly instead of once, which made it seem forgetful. On 16 April, a new system prompt line told the model to keep text between tool calls under 25 words, and that hurt coding quality. All three were reverted by 20 April, and every subscriber's usage limits were reset as an apology.

Notice what did not happen. The weights never changed, and the API was unaffected throughout. It was a default, a bug and a prompt. That is the pattern I now expect whenever the "nerfed" cry goes up.

Case three: Opus 5.5, September 2026, verdict pending

Opus 5.5 is eight days old as I write this. Two independent trackers are watching it, and neither has found a drop. More below.

Six things that make an AI feel dumber

Here is the list I run through before I believe a model was downgraded, roughly in order of how often each turns out to be the cause.

1. The honeymoon effect

Every new model handles more than the last, so in launch week you feel you can ask it anything. Then you ask harder things, because it earned your trust, and you find its ceiling. The model did not move. You did.

2. The effort or thinking setting changed

Modern models have an effort dial. Opus 5.5 ships with low, medium, high and xhigh, with medium as default, and ChatGPT's thinking modes work the same way. Change the default, or land on a lower tier by accident, and you get shorter reasoning and more mistakes under the same model name. That is exactly what happened in March. I covered how ChatGPT's modes map to plans in my GPT-6 Sol and Luna explainer.

3. The instructions around the model changed

Before the model sees your message, it reads a system prompt written by the company. Those change constantly, and one new line can shift behaviour in ways nobody tested. April's 25-word instruction was added to make replies less chatty and made the model worse at code. You cannot see these edits, which is why they feel like a secret downgrade.

4. A real bug

Routing errors, cache bugs, corrupted output: the cases where "it got dumber" is completely true and completely unintentional. The tell is inconsistency, the same prompt brilliant at 9am and broken at 2pm. Check the status page before you check your subscription.

5. Automatic model routing

ChatGPT and Claude both have "auto" modes that pick a model based on how hard your question looks. Great for speed, and an easy way to get a smaller model without noticing. Pin a specific model whenever you are testing.

6. Actual weight changes, including quantization

The one everyone suspects and the one with the least public evidence. Quantization compresses a model to run cheaper and can cost accuracy if pushed too far (I explained the trade-offs in my quantization guide). A company could in theory serve a compressed model under the same name. Anthropic says its inference layer was untouched in both incidents, and the trackers below exist to catch this if it ever happens. So far, nothing.

The public nerf trackers, and what they say right now

If you want a number instead of a feeling, there are now two places to look.

Nerf Bench, run by the independent benchmark BridgeBench, tests a model on launch day and keeps re-testing it. Each model starts at 100 percent "power" (a blend of task performance, token efficiency and cost), and anything between 90 and 110 percent is normal variance. As of its 27 September run, Claude Opus 5.5 sat at 99.2 percent of launch power and GPT-6 Astra at 102.8 percent, slightly better than at launch. Neither is flagged.

Livenerf is newer and stricter. It is an open-source project that launched with Opus 5.5 on 22 September and runs about 120 frozen test items, hour after hour, through a Claude Code subscription rather than the API. Scoring is done by plain code, not by another AI, so it cannot drift. Its author pre-registered the rules: a change only counts if it clears a 99 percent confidence interval, is at least three points in size, and holds for consecutive weeks. As of today the panel is still being calibrated and no verdict has been published. The best-designed test has not finished its baseline yet.

How I test it myself in ten minutes

You do not need a benchmark to stop guessing. You need a frozen set of prompts and the discipline to run them the same way every time.

Five prompts, saved in a text file, never edited. Mine are: a 400-word article of mine with "summarise in exactly three bullets under 20 words each" (instruction following); a messy spreadsheet export pasted as text with "find the three rows where the total does not match the line items" (careful reading); a 40-line Python script with one planted bug and "fix the bug, change nothing else" (code); "write a 150-word description of a mechanical keyboard for office workers, no exclamation marks" (writing under constraints); and a multi-step arithmetic word problem with a trap (reasoning).

The rules. New chat for every prompt. Model pinned, never auto. Same effort or thinking setting. Memory and custom instructions off. Run the set on launch day, save the answers, and run it again whenever the "it got dumber" feeling arrives.

What I found with Opus 5.5. I ran the kit on 22 September and again on 29 September. Four of the five answers were functionally identical. The summariser produced slightly different bullets but obeyed the length rule both times. The bug fix was character for character the same, and the arithmetic trap was caught both times. If Opus 5.5 has been nerfed in its first week, it has not been nerfed on anything I can measure, which matches what Nerf Bench is seeing.

What I found with ChatGPT. The same kit against GPT-6 Sol was messier, because the app defaults to automatic routing. On auto, the spreadsheet prompt got a worse answer on the second run. With the model pinned and thinking set to the same level, both runs matched. That is cause number five, not a nerf, and it is behind most "ChatGPT got dumber" complaints I see.

My prompts are not special. A frozen baseline is what turns an argument into a comparison.

What to do when your AI feels dumber

Work down this list in order. Most people never get past step two.

1. Re-run an old prompt. Take a launch-week chat you were happy with and paste the prompt into a fresh conversation. If it matches, the model is fine.
2. Check your settings. Which model is selected? Is auto on? What effort level? Did you turn on memory or custom instructions since? These explain more complaints than every bug combined.
3. Check the status page. Anthropic and OpenAI both run public ones. A "degraded performance" note is your answer.
4. Check a tracker. Nerf Bench takes ten seconds to read. Inside the 90 to 110 band, you are almost certainly looking at settings or expectations, not weights.
5. Look for a postmortem. When complaints get loud enough for long enough, the companies have shown they will publish one. April's took about six weeks.
6. Then, and only then, switch. If your frozen prompts genuinely got worse with the right settings and a green status page, run them on a rival. My Claude vs ChatGPT vs Gemini comparison covers who is strongest at what.

One more option for the truly paranoid: a model running on your own machine never gets nerfed, because nobody can change it but you. My plain-English guide to local AI shows how.

The takeaway

"Nerfed" is real, but it is almost never the conspiracy people imagine. In every confirmed case so far, the model weights stayed the same and the damage came from a default that was lowered, a prompt that was tweaked or a bug that slipped through. That is better news than it sounds, because settings and bugs get fixed.

So the next time a model you rely on feels dumber, do not cancel and do not argue on Reddit. Pin the model, re-run your frozen prompts, check the status page and glance at Nerf Bench. Ten minutes gets you an answer. And if Opus 5.5 does get nerfed, Livenerf will tell us, with error bars, before the forums do.