Tokens Are the Unit

Disclaimer: What follows was true as of 6/3/2026. Considering the speed at which the AI space is moving, the dollars referenced below will change.

“What’s this going to cost per month?”

Now that I sit in the CTO seat, I understand why that answer matters. It is not a curiosity question. It is a staffing, margin, and roadmap question.

Before we start: this is an intentionally simplified academic exercise for a blog post. The numbers are pedagogical, not a production forecast.

So here is the answer I wish I had given on day one.

For the support-drafting feature in this post, at 200 tickets per day:

• Naive first pass: about $82 per month
• Same feature with prompt caching: about $45 per month
• Caching plus batch plus routing: about $13 per month

That is the budget story in exercise form: same product behavior, roughly a 6x spread depending on call structure.

The rest of this post is the math behind those numbers so you can adapt the method to your own environment.

What an LLM Actually Charges For

Almost every commercial LLM today charges by the token — a sub-word unit the model actually sees. Tokens are not words. The string hello is one token. The word observability is three. The emoji 🐛 is two or three. JSON is dense in tokens because every brace and quote is its own token. Tokenization efficiency also varies by language and script, sometimes by a lot, so per-character comparisons are only rough heuristics. If cost matters, run representative text through your target model’s tokenizer and measure.

The provider charges you twice on every call:

• Input tokens — everything you send: system prompt, user message, conversation history, tool definitions, any documents you’ve pasted in.
• Output tokens — everything the model writes back.

Output is usually about 5× more expensive than input. Part of that is technical (decoding is sequential), and part is pricing policy. Either way, output tokens are typically where budgets drift first.

A rough mental model for mid-2026 (illustrative only, check live pricing before you budget):

• A Sonnet-class mid-tier model runs about $3 per million input tokens and $15 per million output.
• A Haiku-class cheap model is often around $1 in, $5 out (older generations were closer to $0.80 and $4).
• An Opus-class frontier model is often around $5 in, $25 out (older generations reached $15 and $75).

Those numbers will be wrong in six months. The shape is what matters: there are still clear pricing tiers, and model choice can dominate your cost profile.

The Baseline Math (Academic Exercise)

Here is the deliberately simplified scenario.

The feature was a customer-support drafter. A model reads an incoming support ticket and drafts a reply for a human to review before it sends. Volume: about 200 tickets a day.

Per ticket, the model sees:

• A system prompt with the support voice guide and escalation rules — about 500 tokens
• The FAQ, returns policy, and shipping rules — about 2,000 tokens
• The customer’s actual email — about 500 tokens

And it writes back a draft reply of around 300 tokens.

That’s 3,000 input tokens and 300 output tokens per ticket. At Sonnet-class pricing:

Daily input:
200 tickets × 3,000 tokens = 600,000 tokens
× $3 per million = $1.80 / day

Daily output:
200 tickets × 300 tokens = 60,000 tokens
× $15 per million = $0.90 / day

Daily total: $2.70
Annual: ~$985

Less than a thousand dollars a year. For a feature that drafts 200 customer responses every day, all year. The economics of LLM-powered features are usually not where engineering instinct lands.

Monthly equivalent: about $82.

For the purpose of this exercise, that is enough for a first-pass answer.

The Same Math, With Caching

Look at the breakdown again. Of the 3,000 input tokens per ticket, 2,500 are the same every time. The system prompt and the knowledge base don’t change between tickets. Only the customer’s email (about 500 tokens) is new.

Most modern LLM providers have a feature called prompt caching. When you send the same prefix repeatedly, the provider keeps the computed state in a hot cache and reuses it. Cached input is typically about 10% of the price of fresh input. One order of magnitude. In practice, first writes are usually priced above base input, and minimum cacheable length plus TTL windows can affect hit rates.

If I mark the stable 2,500 tokens as cacheable, and assume a 90% cache hit rate within the cache window, the input math becomes:

Stable portion (cached 90% of the time):
450K cached × $0.30 / M = $0.135
50K fresh × $3.00 / M = $0.150

Variable portion (always fresh):
100K tokens × $3.00 / M = $0.300

Daily input: $0.585
Daily output (unchanged): $0.900

Daily total: $1.485
Annual: ~$542

The bill dropped 45% for one configuration change. The trick was structuring the prompt so the stable content goes at the front (cacheable prefix) and the variable content goes at the end (the user’s email). The model didn’t notice. The cache did.

Prompt caching is usually the biggest immediate lever in LLM cost optimization.

Monthly equivalent: about $45.

The Stacked Version (Exercise Outcome)

A few more optimizations stack on top.

Batch API. On providers that support it, batch mode is often around 50% of normal cost, with results returned within ~24 hours. For workloads that don’t need to be synchronous — and a support-draft job almost never does, because a human reviews the draft anyway — batch is free money.

With batch: $1.485 × 0.5 = $0.74 / day
Annual: ~$272

Model routing. About 70% of incoming support tickets are simple — “where’s my order,” “how do I reset my password,” “do you ship to Canada.” A cheap Haiku-class classifier can decide “is this simple or complex?” in a fraction of a cent, then route the easy ones to a Haiku-class drafter and the hard ones to Sonnet. The classifier costs almost nothing; the routing pays for itself by lunchtime on day one.

With routing: Annual ~$160

So the same workload that started at $985/year is now $160. About six times cheaper.

Monthly equivalent: about $13.

None of these optimizations required switching providers, retraining models, or a major rewrite. They were structural choices about how to call the model, not which model to worship.

After doing this exercise repeatedly, the direction has held even when exact rates changed: first caching, then batch (if your workflow tolerates async), then right-size model routing.

What I Now Check Before Shipping an LLM Feature

The bill rarely surprises the team that measured. It almost always surprises the team that did not.

The short list:

Wire token counting into the request log on day one. Three fields: input_tokens, output_tokens, cached_input_tokens. Every request. Without them, every cost-engineering conversation in six months is a guess.

Estimate the bill before building. Tokens per call × calls per period × price. Sanity-check against the real numbers after the first week. If they don’t match within 2×, find out why before they get worse.

Cache stable prefixes. Put the system prompt and any reference content at the front of the prompt. Put the user’s actual content at the end. Mark the prefix cacheable. This costs 30 seconds of configuration and routinely saves 40–70% of input cost.

Cap the loop. Long-running agents — including any tool-use loop — must have a turn cap and a token cap. The most expensive incident I’ve heard of in this space was a bug that put an agent in a retry loop, sending 40,000 tokens per attempt. The Slack message from finance came the following Monday.

Demand structured outputs. “No commentary. JSON only.” is not just clearer — it’s cheaper. Every word of preamble the model writes before answering is a billed output token at the most expensive per-token rate in the whole pipeline.

Pick the right model size. The cheapest token is the one that didn’t run through the frontier model. A cheap-tier classifier in front of an expensive-tier worker can drop a workload’s bill by 60–80% on its own.

That’s the list. Six things, all roughly free, all measurable.

If you want one operator rule: do not approve production until you can show a monthly estimate and a first-week actual that reconcile within 2x.

What I Wish I’d Known Earlier

The honest version of what I’d tell first-time-me, staring at that pricing page:

A token is not a word. “Observability” is three. JSON has more tokens than the same data in CSV. Code has more tokens than the same logic in English. The estimate is a multiplier, not a count.

Output costs about 5× more than input per token. The single most impactful thing you can do for cost is shorten the output. “No prose. JSON only.” can cut output spend materially but measure the delta on your own traffic.

You pay for what you put in the context window, not for the window itself. A 1-million-token context window used for a 2,000-token prompt costs the same as a 200K-token window used for a 2,000-token prompt. Window size is capacity, not a fee.

Prompt caching can change your input bill by an order of magnitude if the prompt has a stable prefix. Most prompts do, once you arrange them right.

Many long-running agent implementations re-send substantial conversation history each turn unless they use managed state. Without caching or state controls, this gets expensive quickly.

And the bill is the math. The math is mostly choices.

The Mantra

Tokens are the unit. Caching is the multiplier. Model size is the bigger multiplier. Batch is the cleanest discount. Output shape is the one most teams overlook.

If you measure, the bill rarely surprises. If you don’t, it always does.