The flywheel, and how it fails silently
Our data loop was wired, tested, and moving nothing. Every dashboard was green.
Every robotics deck has a flywheel diagram. More machines, more data, better models, more machines. Ours is real in the sense that it is implemented in code, in a specific call, on a specific hot path — and it was still, for a while, completely inert.
The mechanism is one HTTP request. Before the router picks a model for a stage, it asks WorldGraph how often this kind of work has succeeded before, and weights its choice accordingly. Data improves decisions; decisions generate data. That call was written, wired and covered by a passing test.
GET /v1/priors?skill=open-door&fingerprint=fp1_9a3f
→ { attempts: 41022, success: 0.947, confident: true }The bug
Priors were keyed by skill, fingerprint and morphology. Not by provider.
Which means every candidate model received the same number. In a weighted comparison, a term identical across all candidates contributes nothing — it shifts every score by the same amount and changes no ordering. The flywheel was spinning a wheel that was not connected to the axle.
“How often does open-door work?” ranks nothing. “How often does it work when GR00T handles it?” is a routing signal.
Nothing errored. The prior appeared in the routing trace, so an audit would show it was consulted. The test asserted a prior was fetched and used, and it was. A dashboard showing “priors: wired” would have been telling the truth.
What caught it
A test name. The test was called “lets a confident prior overturn a cheaper, faster provider”, and it passed — but when we read the assertion it only checked that the prior appeared in the output, not that the ranking had changed. The name described a behaviour the test did not verify.
Rewritten to assert the flip — a provider 50× cheaper and 60× faster must lose once the evidence exists — it failed immediately. The fix was a provider dimension on the prior key.
Why this class of bug matters more than most
A crash gets fixed on Tuesday. A silent no-op gets fixed when somebody eventually asks why eighteen months of accumulated data has not improved anything — by which point the company has been telling investors it has a compounding advantage, and believing it.
Three others in the same family turned up in the same fortnight, all found by tests rather than by use:
- The offline plan cache was written under one fingerprint and read with another. Offline autonomy never worked, and could not have, and nothing said so.
- An injected empty cache was silently discarded, because an empty cache is falsy in Python and the code said `cache or default()`. A fleet would have quietly shared one stale file.
- A seventeen-task job cost exactly $0.00. Rounding on each accrual erased every sub-cent line — a systematic downward bias in a money path, biased toward flattering our own margin.
The rule we took from it
If a mechanism is load-bearing for the business, there must be a test that fails when the mechanism stops working — not one that fails when it throws. Those are different tests, and only the first one is worth writing.