Inference cost is what you pay each time an AI model processes a request: input tokens (instructions, context and question) and output tokens (the answer), each with its own price per million. Multiplied by volume and retries, and net of the cache discount, it becomes a monthly cost and a cost per user, the figure that decides whether the feature pays for itself.
How to use
- Enter how many requests the feature makes per day and the average input and output tokens of each.
- Type the prices per million tokens from your provider’s current price list; the example prices are illustrative.
- If part of the input repeats, enter the share read from cache and the cache price; add retries caused by errors.
- To see the cost per user, enter monthly active users and, optionally, monthly revenue per user.
- Read the sensitivity table and export as PNG, PDF, CSV or Markdown to take to the decision.
What each field asks for
| Field | What to enter |
|---|---|
| Currency | The same currency as the prices and revenue you enter. |
| Requests per day | How many times a day the feature calls the model, across all users. |
| Input tokens per request | Instructions, context and question sent to the model, on average. |
| Output tokens per request | Average length of the generated answer. |
| Input price | Price per million input tokens, from your provider’s current price list. |
| Output price | Price per million output tokens. It is usually higher than the input price. |
| Input read from cache | Share of input tokens that repeat and are billed as cache (fixed instructions, shared context). |
| Cached input price | Price per million tokens read from cache. Empty: same as the input price. |
| Retries | Share of requests called again after an error, a timeout or a rejected answer. |
| Monthly active users | Optional: to work out the cost per user. |
| Monthly revenue per active user | Optional: to compare with the cost per user. |
Example
A fictional customer service app summarises conversations with AI: 20,000 requests per day, 2,000 input tokens and 400 output tokens per request, illustrative prices of $0.50 and $2.00 per million, 40% of input read from cache at $0.05 and 3% retries. Each request costs $0.00148, a day comes to $29.66 and a month to $889.92. With 8,000 active users, that is $0.111 per user, 5.6% of the $2.00 revenue. Output accounts for 56% of the cost: capping answer length is the strongest lever.
When to use
- Before approving an AI feature, to check whether the cost per user fits the product’s margin.
- When comparing models or providers: change the prices and see the effect on the month.
- When planning growth: the sensitivity table shows the cost at double or ten times the volume.
- To decide where to optimise: shorter answers, smaller context, more cache or fewer retries.
Frequently asked questions
What is a token?
It is the unit in which a model reads and writes text: part of a word, a short word or a symbol. The count varies by language and by model. To estimate, measure a few real requests with your provider’s counting tool and use the average.
Why does output often weigh more than input?
Many providers charge more per generated token than per token read. That is why a long answer can cost more than a large context. The calculator shows which part dominates and which lever to pull first.
How does the cache work in the calculation?
When the same instructions or context repeat across requests, some providers charge less for tokens read from cache. Enter the share of input that repeats and the cache price. Without a cache price, the calculator applies no discount.
Why count retries?
Requests that fail, time out or have their answer rejected and are called again pay for input and output once more. A 3% rate looks small, but it shows up in the month and grows with volume.
Are the calculator’s prices real?
No. The example values are illustrative and only show how the calculation works. Always use your provider’s current price list for the chosen model, because prices change often.
How much of revenue can inference take?
It depends on the product’s margin and other costs. The calculator compares cost per user with revenue per user and warns when the cost passes half of it or exceeds it. That is the starting point for deciding on price, usage limits or optimisation.
Does the calculation include infrastructure and people?
No. It covers only the cost of calling the model. Hosting, storage, quality evaluation, monitoring and the team running the feature go separately into the total cost.