The bill didn't come from the volume
What we saw
Our cost report for an AI pipeline was 2.4 times higher than reality, and the real bill still went up about tenfold in a week.
How we found it
Comparing our per-call cost log with the provider's usage page, line by line.
Cause
Two things. We didn't count cached input tokens, which were 91% of input and are billed much cheaper. And every time we changed the prompt, the pipeline reprocessed every row from scratch.
Fix
Logged cached tokens separately. Stamped each row with the prompt version, so only rows made with an older prompt get reprocessed.
The rule now
Measure cost per task from day one, including cache. Reprocessing every row after a prompt change is what costs money.