The next coding models are not only trained on the public internet. They are trained on how agents already work inside editors.

That shift matters more than another arena screenshot. If a model has seen trillions of tokens of real tool traces, it may need fewer output tokens to finish a task you care about. Efficiency becomes a product claim: not just "smarter," but "cheaper per solved job." Solo builders should care — bills are personal.

They should also care about the dark twin of that story: contamination. When training data overlaps the test, scores inflate. Vendors that disclose contamination are doing you a favor. Vendors that do not are asking you to take a number on faith.

Why IDE-trace training changes the game

Classic pretraining teaches language. Agent-trace training teaches procedure: when to read a file, when to run a test, when to stop thrashing. That can show up as:

  • Fewer tokens per successful patch
  • Better tool choreography
  • Familiarity with editor-shaped workflows

It can also show up as overfitting to a particular IDE's habits. If your workflow looks nothing like the traces, the magic shrinks.

Contamination honesty as a feature

Public notes around recent coding-model launches included an uncomfortable admission: a codebase snapshot used in training contaminated a related benchmark and inflated a score. That sentence is more valuable than most marketing pages.

Why? Because it teaches the evaluation class:

  1. Private benches can leak into training.
  2. Inflated scores can ship anyway.
  3. Disclosure is how adults operate.

Your response should not be cynicism. It should be a protocol that does not trust vendor benches as purchase orders.

A contamination-aware eval protocol

Use tasks the vendor cannot have memorized as your product:

  1. Private repo slice — a module only you have, with tests.
  2. Fresh bug — opened after the model's train cutoff if you know it.
  3. Domain chore — something ugly and specific to your stack.
  4. Same prompt pack across candidate models.
  5. Score accepts, not eloquence.

Record tokens or dollars. Record whether the first patch was accepted. Ignore any leaderboard that cannot be reproduced on that pack.

If a model wins public benches and loses your private suite, believe your suite.

Efficiency claims without self-deception

"Quarter the output tokens of model X per solved task" is only meaningful if:

  • The task set is fixed
  • "Solved" means tests green or human accept
  • Failures are counted
  • Tool time is acknowledged

Otherwise you are comparing poetry speeds.

The catch

Self-disclosure can become a PR move: admit a small contamination to look honest while larger evaluation problems stay quiet. Still prefer disclosure to silence — then verify.

The second catch is IDE lock-in by training distribution. A model that is brilliant in one agent product may be average in another. Test in the harness you actually use.

Bottom line

Trace-trained coding models can improve real economics. Contaminated benches can improve fake ones. Solo builders need a private eval pack that survives both hype and humility. Trust disclosure. Trust your suite more.