What Is a Token in AI? How AI Coding Tools Are Priced

June 23, 2026 · updated June 25, 2026 · The Hidden Cost of AI Coding (part 1)

▶ Watch on YouTube & subscribe to The Stack Underflow

Every limit, every bill, every caching discount in AI tooling is measured in the same unit: the token. Most cost calculators throw numbers at you before you have any intuition for what you’re counting. This tutorial builds that intuition from the ground up — because without it, none of the pricing math, none of the optimization advice, and none of the alarming bill screenshots on developer Twitter will make sense.

This is episode 1 of “The Hidden Cost of AI Coding” series. We start with the unit before we touch the economics.

The one-sentence version: A token is a small chunk of text produced by a trained vocabulary splitter, direction of travel (in vs. out) determines most of what you pay, and in agentic coding workflows the same context is re-sent dozens of times — which is what actually makes bills explode.

What a token actually is

A token is not a word. It is not a character. It is whatever chunk of text the model’s tokenizer decided to group together when the model was trained on a large corpus of text and code. That vocabulary was fixed during training. You do not control it at inference time, and you cannot change it by rephrasing your prompt cleverly.

The dominant tokenizer technique is Byte Pair Encoding (BPE), which starts with individual characters and iteratively merges the most frequent adjacent pairs into single tokens. A tokenizer trained on typical web and book text will merge “the”, “ing”, ” is”, and “function” into single tokens because they appear together so often. It will not merge “camelCasedVariableName” because that string is rare. Every token corresponds to an integer ID in a fixed vocabulary, and the model sees only those integers — never the original characters.

Some useful rules of thumb for English:

These are approximations for standard English. The actual count depends on vocabulary frequency: “cat” is likely one token; “mycorrhizal” is probably three or four.

The tokenizer changed — and it matters

Starting with Claude Opus 4.7, Anthropic introduced a new tokenizer shared by Opus 4.7, Opus 4.8, and the newer Fable 5 and Mythos 5 models. Anthropic’s own pricing docs note this new tokenizer “may use up to 35% more tokens for the same fixed text” compared to Claude Opus 4.6 and earlier models (docs.anthropic.com, 2026). Real-world measurements on technical documentation have shown a 1.47x multiplier, exceeding the official estimate on code-heavy content.

Claude Haiku 4.5 and Sonnet 4.6 retain the older tokenizer (as of June 2026), which is why a given codebase may cost measurably different amounts depending on which model you route it to — independent of the per-token price.

Code tokenizes differently than prose

This is the part that consistently surprises developers. Code is not prose. The same information expressed as Python costs significantly more tokens than the same idea expressed in English, for three compounding reasons:

Indentation is tokenized character by character or in small merged groups. Every leading space or tab in Python is counted. A heavily nested file spends a meaningful fraction of its token budget on whitespace alone.

Punctuation density is much higher in code. Curly braces, square brackets, angle brackets, semicolons, and colons all cost tokens — often one each — and code repeats them constantly.

Rare identifiers do not compress well. The BPE tokenizer learns to merge tokens that appear together frequently. camelCase variable names, company-specific module names, and long function signatures are rare strings that fragment into many sub-tokens.

Research into this problem even has a name: TokDrift — the fundamental mismatch between how LLM tokenizers (trained primarily on natural language) fragment code versus how programming language tokenizers (from compilers and interpreters) parse it deterministically. The TokDrift paper (arXiv 2510.14972, 2025) demonstrated that identical programs can produce inconsistent or fragmented token representations across model tokenizers, reducing effective context length for code-heavy workloads.

Rough token density comparison (characters per token):

English prose:    ~4.3 chars/token  (efficient)
Python source:    ~3.5 chars/token  (15-25% more tokens)
TypeScript:       ~2.7 chars/token  (with new tokenizer, Opus 4.7+)
JSON / minified:  ~2.5-3.0 chars/token  (worst case)

When you paste 200 lines of a real codebase into a model context, the token count climbs fast — and it climbs faster with certain models than others.

# This function looks short. It is not cheap to tokenize.
def calculate_total(
    items: list[dict],
    discount: float = 0.0,
) -> float:
    return sum(item["price"] for item in items) * (1 - discount)

# Every indent space, every bracket, every type annotation,
# every camelCase dict key is getting split into tokens.
# A 100-line Python file can cost twice the tokens
# of a 100-word paragraph.

The model lives in tokens, you live in words

Your message (words)
        |
        v
  [ Tokenizer ]
  words --> integer IDs
        |
        v
  [ Model forward pass ]
  sees only integer sequences
        |
        v
  [ Detokenizer ]
  integer IDs --> words
        |
        v
Model response (words)

Every input you send gets converted to a sequence of integers before the model sees anything. Every word the model writes back is generated as tokens, one at a time, using a forward pass through the network. The model has no concept of “word” or “line” — it attends over a flat sequence of token IDs.

This is why a model’s context window is measured in tokens. When Anthropic says Claude Opus 4.8 has a 1 million token context window, that is roughly 555,000 words — but the exact equivalent in your codebase depends on how that specific model’s tokenizer handles your code (docs.anthropic.com, 2026). Claude Haiku 4.5 has a 200,000 token context window; Sonnet 4.6 reaches 1 million tokens at the same price tier.

Not all tokens cost the same

This is where most developers’ mental model breaks. Tokens are priced by direction:

• Input tokens — everything you send to the model: your system prompt, your message, your pasted code, your conversation history, tool definitions, tool results
• Output tokens — everything the model generates back: the response, the generated code, the reasoning trace

Output tokens typically cost 5 times more than input tokens. Sometimes more.

Current pricing (June 2026, Claude API):

Model            Input / MTok    Output / MTok
-----------------------------------------------
Claude Haiku 4.5    $1.00           $5.00
Claude Sonnet 4.6   $3.00          $15.00
Claude Opus 4.8     $5.00          $25.00
Claude Fable 5     $10.00          $50.00

(MTok = million tokens)

This 5x asymmetry exists because generating tokens is computationally more expensive than reading them. For every output token, the model runs a complete forward pass through the full context. Reading input tokens — processing the context — happens in a single forward pass shared across all input tokens.

The shape of the task decides where your money goes

Code generation is expensive on the output side. Agentic sessions are expensive on the input side for a reason covered in the next section.

The agentic cost explosion: context re-sent on every turn

A single-turn chat is a simple transaction: you pay for the input you sent and the output you received. An agentic session — where a model runs autonomously across dozens of tool calls — has a completely different cost structure.

Every API call in an agentic session re-sends the full accumulated context as input tokens: the system prompt, every file the agent has read, every edit it has made, every tool result returned, the full conversation history. Nothing is stored server-side between calls by default. Each turn, the context grows. Each turn, the full context is re-sent.

Turn 1  context:    ~5,000 input tokens  (system prompt + initial task)
Turn 10 context:   ~20,000 input tokens  (+ 9 turns of tool results)
Turn 30 context:   ~35,000 input tokens  (+ file reads, edits, outputs)
Turn 50 context:   ~60,000 input tokens  (+ failures, retries, more files)

Cumulative input across 50 turns: ~1,000,000 tokens
Cumulative output across 50 turns: ~40,000 tokens

Input-to-output ratio: 25:1

A 50-turn agentic coding session consumes roughly 1 million input tokens and 40,000 output tokens — a 25:1 ratio (Vantage Engineering, 2026). At Sonnet 4.6 pricing, that session costs approximately $3.60. But a feature-length task with 100 API calls can reach 2 million input tokens: $6+ on Sonnet, $10+ on Opus 4.8.

Analysis of real developer usage shows re-sent context accounts for 62% of total agent token costs — more than output, more than system prompts, more than anything else (LeanOps, 2026). This is why the same developer who spends $15/month on casual AI chat can spend $500-$2,000/month the moment they switch to an autonomous coding agent.

Prompt caching: the most important lever you’re probably not using

Prompt caching lets you mark portions of your input as cacheable, so subsequent calls that send the same prefix read from cache at a fraction of the cost instead of re-processing it from scratch. Anthropic’s implementation (platform.claude.com, 2026):

A cache hit costs 10% of the normal input price — a 90% savings on anything that gets re-sent. For a system prompt or large static file that is included on every turn of an agentic session, caching converts a recurring per-turn cost into a nearly-free read after the first write.

Without caching (50-turn session, 10,000-token system prompt):
  50 turns x 10,000 tokens x $3/MTok = $1.50 in system prompt alone

With caching (1-hour TTL):
  Turn 1 write: 10,000 tokens x $6/MTok = $0.06
  Turns 2-50: 49 turns x 10,000 tokens x $0.30/MTok = $0.147
  Total: $0.207  (86% savings on that portion)

The minimum cacheable prompt length varies by model: 1,024 tokens for Haiku 4.5 and Sonnet 4.6, 2,048 for Opus 4.7 (docs.anthropic.com, 2026). Anything shorter is processed without caching — no error is returned, which is why many teams think caching is working when it is not.

How to apply this right now

1. Count your tokens before you optimize costs. Use the anthropic Python SDK’s token counting method or the Anthropic Console tokenizer before you assume you know how expensive a prompt is. Code-heavy inputs cost more than they look, especially on Opus 4.7+ with the new tokenizer.

2. Route tasks by model tier deliberately. Use Haiku 4.5 for classification, extraction, and tool routing. Use Sonnet 4.6 for most production coding tasks. Reserve Opus 4.8 for genuinely complex multi-step reasoning. The 5x price difference between Haiku and Opus is real; using the right tier for the task reduces costs to roughly 12% of an all-Opus workflow (LeanOps, 2026).

3. Cache your system prompt and static context. If your system prompt exceeds 1,024 tokens and is the same on every call, add a cache_control block. For agentic sessions longer than 5 minutes, use the 1-hour TTL. Set up a pre-warm call at startup so the cache is hot when users arrive.

4. Prune accumulated context aggressively. In long agentic sessions, tool results from many turns ago are often stale. Summarizing or dropping old tool results before they bloat the context can shrink a 2M-token session to under 1M without degrading output quality. The “context rot” problem — where quality actually drops as context grows — means pruning can improve results while cutting cost (see Context Rot Explained).

5. Tell the model to be concise. Output tokens are the expensive direction. Instructing the model to return only changed lines instead of full files, to skip boilerplate explanations, and to be concise reduces the output token count. This is not just a stylistic preference — it is a cost lever.

6. Set per-developer budget caps. Median developer spend on agentic tools is ~$480/month but variance is extreme: 20x between light and heavy users, with extreme cases reaching $4,200 in a single weekend (LeanOps, 2026). Set daily hard cutoffs in your API gateway before you get an invoice surprise.

Common misconceptions

“Tokens are just words.” They are not. A single word can be multiple tokens, especially compound words, names, and technical identifiers. A common short word may share a token with a space character. The 0.75-words-per-token rule is a rough starting heuristic, not a law — and it degrades for code, JSON, and non-English text.

“Output tokens are the cheap part — the model is just typing.” This is backwards. Output tokens are the most expensive direction of travel, typically 5x more per token than input. Tasks that generate a lot of text — code generation, writing documentation, producing structured output — are dominated by the expensive side of the price schedule.

“More context always means better results at the same cost.” More context means more input tokens on every single turn, compounded across every turn in an agentic session. In practice, larger contexts correlate with degraded quality (see Bigger Context Windows, Worse Memory). More expensive and worse is the failure mode, not more expensive and better.

“The per-token price dropped, so AI coding is getting cheaper.” Per-token prices have dropped dramatically — roughly 98% since late 2022 for GPT-4-class performance. But enterprise AI budgets grew an estimated 320% from 2024 to 2026. What changed is usage patterns: teams moved from single-turn chat to multi-turn agentic sessions that burn 200x more tokens per task. Cheaper per-token but much more tokens consumed per task means bigger bills, not smaller ones.

Frequently asked questions

How do I find out exactly how many tokens my prompt will use?

The Anthropic Python SDK exposes a count_tokens method that returns a token count without making a full inference call. The Anthropic Console also includes a tokenizer tool where you can paste text or code and see an exact count. These counts reflect the actual tokenizer for the model family you are using — remember that Opus 4.7 and later use a different tokenizer that produces up to 35% more tokens for the same text.

Does the token count include the system prompt?

Yes. Every token sent to the model — system prompt, conversation history, your current message, tool definitions, tool results, any retrieved context — counts as input tokens and is billed accordingly. Tool use adds additional overhead: Claude Opus 4.8 adds 290 tokens per request for the tool-use system prompt when tool_choice is auto (docs.anthropic.com, 2026). This is why large system prompts and large tool definition sets have a measurable cost at scale.

If output tokens are 5x more expensive, should I tell the model to give shorter answers?

Yes, deliberately. Instructing the model to return only the changed lines rather than full files, to skip explanatory preamble, and to be concise can substantially reduce output token usage. For agentic tasks this is especially valuable because longer tool-call responses get re-sent as context on every subsequent turn, compounding the input cost as well.

What is prompt caching and when does it pay off?

Prompt caching marks a prefix of your input as cacheable. On the first request, the prefix is written to cache at 1.25x the normal input price (for the 5-minute TTL). Every subsequent request that sends the same prefix reads from cache at 0.1x the normal input price — a 90% savings. The break-even point is after just one cache read for the 5-minute TTL. For system prompts and large static documents that are re-sent on every turn of an agentic session, caching is one of the highest-leverage cost reductions available.

How does agentic token usage compare to chat token usage?

The same task handled by an autonomous agent burns roughly 10-100x more tokens than the same task handled by a single chatbot call. At 50 agent steps, the multiplier exceeds 30x (LeanOps, 2026). The driver is context re-sent on every turn: the input-to-output ratio in a typical agentic session is 25:1, compared to roughly 2:1 in single-turn chat. This means input pricing — not output pricing — is the dominant cost lever in agentic workflows.

Does switching from Sonnet to Opus cost exactly 5/3 more?

Not quite. The per-token ratio is indeed $5/$3 on input. But if you are also switching from a model using the older tokenizer to one using the newer tokenizer (Opus 4.7 or later), the same text produces up to 35% more tokens. A switch from Sonnet 4.6 to Opus 4.7 can effectively cost 2.2x per request on code-heavy workloads, not 1.67x. Always measure with a real prompt before assuming the ratio from the pricing page.

Where this fits in the series

This tutorial is episode 1 of The Hidden Cost of AI Coding — a series that traces where AI coding bills actually come from. Understanding the token as a unit is the prerequisite for everything that follows.

Episode 2 covers what happens when you chain tokens into a context window and why longer is not always better: Bigger Context Windows, Worse Memory.

Episode 3 traces the specific failure mode where context fills up with stale, low-value tokens until the model starts making mistakes: Context Rot Explained.

If you want to understand the full agentic cost picture — why a 50-turn session is structurally different from 50 individual chats — Why AI Coding Bills Explode is the direct follow-on.

For the caching mechanics covered briefly above, the full treatment is in Prompt Caching: Anthropic vs OpenAI.

Browse all tutorials in the series.