Understanding inference costs

Feature set: Inference Contact our support for access.

Spend is the cost of model traffic in a period, together with the tokens that traffic consumed. Akka computes it from the exchanges that the gateway recorded and prices each exchange from a versioned pricing catalog. No figure is estimated or entered by hand.

A provider’s invoice states what an account cost. It does not say which agent, use case or system id the cost came from, and it arrives after the period is over. Deciding where a smaller model reduces cost, holding a team to a budget, or showing what a change saved all need figures that are divided the way the system is built and that can be read again later with the same result. This page explains how those figures are produced.

How it works

From exchange to figure

  1. An agent’s model request passes through the gateway, and the exchange is captured. See concepts:model-training.adoc#capture-and-grading.

  2. The capture refinery derives a derived capture from it, with the served name, the use case, the system id, the tags, the conversation and the tokens of each class.

  3. The portfolio service prices the derived capture from the pricing catalog when it stores it. If its served name has no price at that time, the service prices it later, once a published version prices that served name at the instant it was served.

  4. Spend for a period is read from the priced derived captures.

Traffic that the gateway did not record does not appear in spend.

The pricing catalog

The pricing catalog holds the price of each served name as rates per million tokens. Input and output have rates, and so can cache reads, cache writes and reasoning tokens. A price applies to a served name: the model field in the body of the model’s response. A response without that field is unpriced. See Served names.

The catalog is versioned, and a published version is never edited. Each version states the instant from which its prices apply, which can be in the past or the future but must be later than the latest price of each served name it carries. A version can price some served names and leave out others, and the names it leaves out keep their earlier price. A price is marked as assigned when your organization set the rate, for example a negotiated rate, and as a list price otherwise. The catalog holds one currency.

When a price applies

The rate that applies to an exchange is the one that the catalog holds for its served name at the instant the exchange was served. The portfolio service prices the exchange when it stores it, from the catalog as it stands then:

  • When the served name has a price at that instant, the service prices the exchange, or leaves it unpriced if a token class it used has no rate. Either way the exchange keeps that catalog version. A later version does not change it, also when the later version takes effect from an instant in the past.

  • When the served name has no price at that instant, the service stores the exchange unpriced, with no catalog version. Once a published version prices that served name at that instant, the service gives the exchange that version. The exchange is then priced, or remains unpriced if a token class it used has no rate in that version. The service starts this shortly after the publish, and a long range takes longer.

The service keeps the record of each exchange for a record period of at least 90 days, and prices recorded traffic after the fact for that period. Hours older than the record period keep their figures and their unpriced counts. It follows that:

  • The spend of a period can rise after a publish, and its unpriced count fall, also when the period is complete.

  • Traffic recorded soon after a publish can still be priced at the previous rate.

  • A new rate for a served name that already had a price applies to traffic served after the rate takes effect and recorded after it is published.

Unpriced traffic and lower bounds

An exchange is priced only when every token class it used has a rate. A rate left out means the class has no price, not that it costs nothing, so a free class is given a rate of zero. An unpriced exchange counts in the exchanges and tokens of a period and not in its spend. The cause is one of two: the served name has no price, or a token class it used has no rate. An exchange whose response names no served name, such as a refused request, has no price. Dividing the period by model shows which served names have unpriced traffic.

A token count marked with + is a lower bound. Some exchanges did not report every token class, so the true count can be higher. This is common, because providers do not report every class: OpenAI responses report cache-write tokens only for the models that charge for them, GPT-5.6 and later, and Anthropic responses do not report reasoning tokens. The command output names the classes that were not reported.

Completeness

A period is a window of whole UTC hours. Each answer states whether its figures are complete:

  • complete: every hour of the window has been reported on, and no exchange was lost,

  • incomplete: some hours have not been reported on yet, for example because the window includes the current hour, or some exchanges were lost,

  • unknown: no hour of the window has been reported on yet.

An hour is reported on about a day after it ends, with or without traffic, when the gateway router recorded for the whole hour. Until then, a window that holds it is not complete. An hour with a minute in which no gateway router instance was recording, for example during a restart, can stay unknown, and a window that holds it then never reads complete.

A figure with no value says why, and never reads as zero.

Attribution

A period can be divided into rows, and the rows always add up to the total:

  • by model, one row for each served name,

  • by use case, the use case that the router assigned, which is the same use case that targeting ranks and the trainer specializes for,

  • by system id, with one row, unidentified, for traffic that named none,

  • by route, how the gateway router reached the destination, with each revision of a routing ruleset as a row of its own,

  • by tag, the value that a tagging ruleset gave the exchange, with one row for traffic that the ruleset did not decide for.

The divisions combine, for example model and use case together. A system id names the AI system that a request was sent for, in the x-akka-system-id header. See Attribute spend to systems.

Conversations are ranked by spend rather than divided, so the most expensive conversations of a period can be found. That ranking does not add up to the period total. Two periods can be compared, and a difference has a value only when both periods have one in the same currency. A period can also be read as a series of hours or days, each answered as a period of its own. The most recent exchanges can be listed one by one, each with its system id, outcome, tokens and spend.

Guarantees

Failure or situation What Akka does Precondition

A figure is read again later

It is the same, unless a publish has since priced traffic of the period that was unpriced. A published catalog version is never edited, and an exchange that a version priced keeps that price.

The window is complete.

A served name or token class has no price

The exchange counts in exchanges and tokens, not in spend, and the answer states the cause.

None.

A provider does not report a token class

The count is marked as a lower bound, and the output names the classes not reported.

None.

A window includes hours not yet reported on

The answer is marked incomplete or unknown.

None.

Limits and trade-offs

  • Spend covers traffic that the gateway recorded. A call made directly to a provider does not appear.

  • A period is whole UTC hours.

  • The catalog holds one currency.

  • A new price cannot take effect earlier than the latest price for the same served name. A later rate does not change an exchange that an earlier version priced or left unpriced, and an exchange older than the record period stays as it is. Publish a price for every served name before its traffic is served.

  • The service does not check a system id. A client sends the value that it chooses, unless your deployment overwrites the header at its own edge. The service keeps every system id that it sees, also past the record period, and nothing deletes one. Use the name of a system as the system id, never personal data.

Next steps