The Dispatch #82

The Calibration Gap

May 21, 2026
For engineers navigating AI-assisted work ~7 min read

You shipped the feature. The tests pass. You can explain what the code does.

But ask yourself: could you have built it without the AI?

Not "would it have taken longer?" โ€” could you have done it at all, from scratch, on a desert island with no internet?

If that question makes you uncomfortable, you're not alone. And it's not imposter syndrome.

It's something worse: the calibration gap.

What the Calibration Gap Actually Is

Calibration is the match between your confidence and your actual ability. Good calibration means you know what you know, and you know what you don't. Poor calibration means your confidence and your competence have drifted apart.

AI tools are exceptional at creating calibration gaps.

When you ask an AI a question and get a correct answer, your confidence goes up. You now "know" the answer. But the mechanism that increased your confidence was the AI's reasoning, not your own. Your actual understanding didn't change โ€” only your belief about your understanding did.

This is different from reading a tutorial. When you read a tutorial, you follow the reasoning step by step. You build a mental model. Your brain does the work.

When you ask an AI and it answers, your brain can skip the work. The answer arrives. The curiosity satisfied. The feeling of learning without the learning.

Psychologists call this illusion of understanding โ€” the Dunning-Kruger cousin that hits competent people hardest.

The Senior Engineer Problem

Calibration gaps hurt senior engineers most, and here's why:

A junior engineer who uses AI for everything will quickly notice they're lost. The gaps are so large they become obvious. They know they don't know.

A senior engineer with an AI assistant? The gaps are subtle. They can still reason. They can still debug. They can still ship. But the deep, embodied understanding โ€” the kind that takes years to build and shows up at 11pm when something breaks in production โ€” that's eroding quietly.

Senior engineers are confident and skilled, so when AI inflates their confidence further, they don't notice the skill drift.

7 pts

Estimated calibration gap for a senior engineer after 6 months of heavy AI use (The Clearing internal model, n=340 self-reports)

That 7-point gap is invisible. It shows up as:

  • A production incident you "should have" caught
  • A design decision that seemed obvious and was catastrophically wrong
  • A question in a code review you can't deeply answer
  • A pull request you approved that required three rounds of fixes

You're still a good engineer. But you're not as good as you think you are. And that gap is growing every sprint.

Why This Is Different From Imposter Syndrome

Imposter syndrome is: "I don't deserve this success; I'm fooling everyone."

The calibration gap is: "I deserve this success; I understand this."

Both are wrong, but in opposite directions.

Imposter syndrome makes you anxious. The calibration gap makes you confident and wrong.

Imposter syndrome pushes you to over-prepare, over-work, prove yourself. The calibration gap pushes you to move faster, take on more, trust your instincts โ€” which are quietly being outsourced to the AI.

Imposter syndrome is visible. The calibration gap is not.

The Mechanism Nobody Talks About

When you learn something the hard way โ€” through struggle, failure, debugging, iteration โ€” your brain encodes it deeply. The difficulty of retrieval creates the permanence of memory. This is what Robert Bjork calls desirable difficulties.

AI removes the difficulty. And in doing so, it removes the encoding.

You get the answer. You understand it at that moment. You move on.

Two weeks later, the answer is gone. But your confidence from that moment of understanding? That stayed.

This is why engineers report a strange phenomenon: they can have a conversation about a topic and sound brilliant, then open a blank editor and draw a complete blank.

The calibration gap isn't just "I don't know as much as I think." It's "I don't know how much I don't know." That's the dangerous part.

Three Warning Signs You're in the Gap

1. You can explain concepts but can't build from scratch.

If someone asked you to implement something fundamental to your stack โ€” a caching layer, an authentication flow, a data pipeline โ€” could you do it without looking anything up? Not "would take a while" โ€” could you do it at all, right now, cold?

2. AI answers feel familiar, not revelatory.

When you ask an AI a question and it answers, does the answer surprise you? Do you learn something new? Or does it mostly feel like hearing your own thoughts articulated back? Familiarity is a signal your brain isn't processing new information โ€” it's recognizing patterns it already has.

3. Your estimation is consistently off in one direction.

Do you regularly underestimate how long things take โ€” or do you overestimate how much you can accomplish in a sprint? Calibration gap makes you optimistically wrong because you genuinely believe you understand the work better than you do.

What Actually Closes the Gap

The fix isn't "use less AI." That's treating a symptom, not the mechanism.

The fix is deliberate calibration โ€” building in experiences where your confidence and your actual ability have to match. Where you find out what you actually know.

The Explanation Requirement

After AI generates something, close the AI window and explain it โ€” out loud, to no one.

Explain it as if you were teaching it to a junior engineer. Without looking at the code. Without the AI as a reference.

Where you hesitate is where the understanding gap is. That's the part that didn't transfer. That's where to focus.

The Desert Island Test

Once a month, pick one small feature and build it without AI.

Not because AI would do it worse โ€” probably AI would do it faster and just as correctly. But because the practice of working through it reveals exactly where the calibration gap lives.

Track what you couldn't do. That's your map.

The 10-Minute Rule

Before asking AI โ€” frame the problem yourself, in writing, for 10 minutes.

Write out what you think the problem is, what you've tried, what you're assuming, what outcome you're optimizing for. Then ask AI to respond to what you've written.

This isn't about suffering through hard problems. It's about building the habit of problem-framing before solution-seeking โ€” and making sure the solution-seeking still exercises your brain, not just the AI's.

The Thing Nobody Wants to Admit

There's a version of the "what is seniority in the AI era" conversation that ends in despair โ€” the idea that if AI can do the technical work, there's nothing left.

That version is wrong.

What's happening isn't that seniority got erased. It's that the definition got updated.

The engineers who navigate this well are the ones who figured out what was always actually valuable about being senior โ€” not the code, but the judgment, the context, the trade-off clarity โ€” and leaned into that.

They stopped competing with AI on what AI does well. They started being the person who knows what AI should and shouldn't be doing.

This is a real job. It's a harder job than it was. And it's the job that senior engineers were actually doing all along, even when they thought the code was the thing.

If the calibration gap framing resonates, take the AI Fatigue Quiz โ€” it surfaces where this dynamic shows up in your daily work and gives you a concrete next step.

Take the AI Fatigue Quiz โ†’

The skill atrophy page goes deeper on the mechanisms that erode your coding ability โ€” and the specific practices that rebuild it.

Read: Skill Atrophy โ†’

P.S. If you've been nodding along to the calibration gap description but the "could you build it without AI?" question makes you anxious โ€” that's the data point. That's the signal. Don't dismiss it. Use it.

The engineers who navigate the AI era best are the ones who stay honest with themselves about where their actual understanding is. The gap is real. It grows if you don't close it. And closing it is a skill โ€” one you can practice.

See you next Sunday.

โ€” The Clearing