Measure AI cost per useful result.

Trace a complete run

Record the input, model calls, tool calls, retries, output and review outcome for a representative task. Keep sensitive data out of logs where possible. The useful unit is the completed task, not the cheapest individual request.

Distinguish a successful response from an accepted result. A generated document that needs substantial rewriting has a different operating cost from one that meets the brief on the first review.

Remove unnecessary context

Separate stable instructions from task-specific material. Retrieve the relevant passage instead of sending the entire library when the task permits it. Check whether repeated history is helping the model or merely increasing the request size.

Provider features such as prompt caching may reduce the cost of repeated input under their documented conditions. They do not make irrelevant context useful, and their behaviour should be checked against current provider documentation.

Match effort to the task

Use an evaluation set before moving work to a cheaper model or a shorter prompt. Include difficult and ambiguous examples, not only the cases the existing system handles easily. Savings are meaningful only if the result remains acceptable.

Put a limit around retries. An agent that keeps trying without a clear stopping rule can spend more while making the failure harder to understand.

Include the human work

Measure review and correction time alongside provider charges. A slightly more expensive generation step may be economical if it materially reduces rework; a cheaper one may be appropriate for a low-risk draft.

Keep a cost-quality record

For each change, compare cost per accepted result, completion rate, latency and review effort. The transcript that prompted this article used a dramatic savings claim. Treat that as someone else's reported result, not a forecast. Your workflow needs its own baseline and evidence.