Slow models force patience. Fast models force process.

Coding agents that generate on the order of a thousand tokens per second change the feel of a session. You stop waiting for the paragraph. You start drowning in diffs. If your workflow was "let it cook, then skim," high throughput turns skim into a liability.

Speed is not free quality. Speed is a multiplier on whatever loop you already have — including a bad one.

What actually gets faster

Vendor tok/s numbers usually measure generation, not the full agent loop. Tool calls, test runs, installs, and human reading still take wall-clock time. A model that streams instantly can still spend most of the minute waiting on your test suite.

Still, when generation stops being the long pole, three things shift:

  1. You can try more patches per hour.
  2. You can over-generate garbage per hour.
  3. Your review queue becomes the scarce resource.

Solo builders are the review queue.

Redesign the loop for high TPS

Before (slow model habits):
large prompt → long wait → big patch → hope tests pass → manual cleanup.

After (fast model habits):
tiny goal → small patch → automatic tests → accept/reject → next tiny goal.

Concrete rules:

  • Prefer patches that touch few files
  • Require tests on every accept path
  • Cap steps before the agent can thrash
  • Keep irreversible actions behind a human gate
  • Measure accepted patches per hour, not tokens per second

If you cannot instrument accepts, you will optimize for motion.

Metrics worth keeping

Metric Why
Time to first plausible patch Feels like speed
Time to green tests Real speed
Accept rate Quality
Rework rate after accept Honesty
Dollars per accepted patch Business

A model that wins tok/s and loses accept rate is a toy.

The catch

Vendor speed claims may exclude tool wait, may apply only inside a closed product, and may not be available as a public API SKU. Design against the path you can actually run.

The second catch is false confidence. Fast output looks authoritative. It is still guesswork with better UX. Your tests remain the adult in the room.

The third catch is context thrash. High-speed agents love to rewrite the same file six ways. Without a goal checklist, they burn money polishing noise.

Practical defaults on my desk

  • Fast model for small, tested edits
  • Workhorse for multi-file features
  • Flagship for architecture and gnarly bugs
  • Always a stop condition
  • Never "Ultra harder" as a response to a failing test — fix the task framing first

What not to do

Do not remove tests to "keep up" with the model. Do not accept a 40-file rewrite because it arrived in twelve seconds. Do not confuse streaming UI with correctness. And do not buy a faster SKU as a substitute for a goal checklist — that is how you purchase thrash at a higher sampling rate.

If the agent is fast and lost, you will only notice when the diff is already large. Prefer small goals that fail closed.

Bottom line

A thousand tokens per second does not remove engineering. It removes the excuse that generation was slow. Build a loop that verifies as quickly as the model types. Instrument accepts. Cap thrash. Keep humans on irreversible steps. Otherwise you have only invented a faster way to create review debt — and a more expensive one.