What Is a Context Window? Token Limits by Model (2026)

A context window is the maximum number of tokens a model can handle in one request, counting the input and the output together.

Unit:             tokens, not words or characters (~4 English chars per token, but it varies)
Scope:            one request; not memory, does not persist between calls
Counts toward it: system prompt, full history, tool definitions, tool results, reasoning, the reply
Typical size:     200K on older or cheaper models, 1M on most current flagships
Largest in 2026:  1,131,072 (Qwen 3.8 Max); the even 1,000,000 is the low end of the 1M tier
If you exceed it: HTTP 400, no output; nothing is silently truncated
Does not tell you: how accurately the model uses the far end of that window

What Is a Context Window?

It is the token budget for a single API request. Everything the model reads and everything it writes has to fit inside one number.

The API is stateless. It does not remember the last call. On every turn your application resends the entire conversation, and the context window is the ceiling on how big that resend plus the coming reply is allowed to be.

That is why the window is not memory in any useful sense. A chat product that appears to remember your name across sessions is storing that text somewhere and pasting it back into the window on each request.

What Counts Toward the Context Window?

Everything in the request, plus everything in the response. The parts people forget are usually the expensive ones:

• System prompt. Counted on every single turn, not once per session.
• Full message history. Every prior user and assistant turn you resend.
• Tool definitions. Names, descriptions, and JSON schemas for every tool you declare. On Claude these also add a per-model tool-use system prompt, from 286 tokens on Opus 5 to 675 on Opus 4.7.
• Tool results. Often the biggest single item in an agent loop; a directory listing or an API response can be thousands of tokens.
• Reasoning tokens. On models with thinking enabled, the reasoning counts and is billed even when the text is not returned to you.
• The reply itself. Which is why max_tokens and the context window interact: a request that leaves no room for output truncates.

That last point catches people. On Claude Opus 5 thinking is on by default, so a request sized tightly around its answer on an older model can now run out of room mid-response.

How Big Is Each Model’s Context Window?

1M tokens is the top of the mainstream tier, and 256K is the most common size overall. Across the 119 text models in the ofox catalog on 2026-08-12, 25 sit at 256K and 22 at 1M.

The flagship tier, with numbers as published by each vendor:

Two things worth noticing. Qwen 3.8 Max holds the largest published window of the group, and Kimi K3 is the only model here whose max output equals its full window, so it can in principle write a million tokens in one response. DeepSeek V4 Flash is second on output at 384,000, three times what most of this tier allows, which matters if you generate long documents rather than read them. And “1M” is nine different numbers, from an even 1,000,000 to 1,131,072, a 13% spread before you have measured anything.

One caution about where these come from. Gateway catalogs and vendor documentation do not always agree: on 2026-08-12 the ofox catalog listed Grok 4.20 at 2,000,000 while xAI’s own model docs state 1,000,000. The table above uses the vendor number. Check the vendor page when the figure matters.

Why Doesn’t “1M Tokens” Mean the Same Thing on Every Model?

Because a token is not a fixed amount of text, and the difference between tokenizers is larger than most people assume. We sent one identical 2,638-character English document (420 words, a service postmortem) to nine models through one endpoint and read prompt_tokens off each response.

Measured 2026-08-12. A 1.56x spread on identical input. Claude Opus 5 is the outlier because Anthropic documents that Claude 4.7 and later use a newer tokenizer producing “approximately 30% more tokens for the same text.”

Combine the two tables and the advertised window stops being the useful number. Here is how many copies of that same 420-word document actually fit:

Every one of these advertises roughly 1M, and the real capacity for this document varies by 1.60x.

The ratio is not constant across content types, so do not carry these numbers to a different workload. On a TypeScript file the spread was 1.53x and GLM-5.2 was leanest rather than Grok. On Chinese prose the spread widened to 1.88x, and both Claude models landed close to one token per Chinese character, at 1.00 chars/token on that sample against 1.87 for Grok. Treat that as a sample, not a rule: on a second, punctuation-heavier Chinese passage the same two models came in at 0.98 and 0.96 chars/token, meaning some characters cost more than one token. If your input is code or non-English, measure it rather than assuming.

The practical rule: measure your own content on the models you are choosing between. One request with max_tokens: 1 returns prompt_tokens and costs almost nothing.

What Happens When You Exceed the Context Window?

You get an HTTP 400 and no output. Nothing is silently truncated. We sent an over-sized request to a 32,000-token model to see the exact behavior:

{"error":{"code":null,
  "message":"<400> InternalError.Algo.InvalidParameter: Range of input length should be [1, 30720]",
  "type":"invalid_request_error"}}

Read that limit carefully. The model advertises 32,000, and the enforced input ceiling is 30,720, with the remainder reserved for output. The advertised window is the total, not your input allowance, and the enforced number can be lower than the marketing one.

Error shapes differ by vendor, so do not pattern-match on the message string:

• OpenAI-compatible endpoints generally return a 400 with a context_length_exceeded-style code.
• Claude can instead finish the turn with stop_reason: "model_context_window_exceeded", which is distinct from max_tokens and needs its own branch in your code.

Handle both. A response that stops early because the window filled is not the same failure as one that stopped because your max_tokens was small, and the fixes differ.

Does a Bigger Context Window Cost More?

Sometimes, and it depends entirely on the vendor. Two pricing models are in play in 2026:

• Flat. Anthropic bills the full 1M window at standard rates with no long-context premium, so a 900K-token request costs the same per token as a 9K one.
• Tiered past 200K. Gemini 3.1 Pro goes from $2 to $4 per million input tokens and $12 to $18 output once a prompt crosses 200K. Grok 4.20 goes from $1.25 to $2.50 input and $2.50 to $5.00 output at the same threshold.

So the model with the cheaper headline rate can be the more expensive one for long-document work. If your workload routinely crosses 200K, check for a tier break before comparing sticker prices, and note that our prompt caching cost math applies on top of whichever tier you land in.

Is the Advertised Window the Same as Usable Context?

No, and this is the most important caveat on the page. A model accepting 1M tokens is not the same as a model reliably finding the fact you buried at token 800,000.

Retrieval accuracy degrades with distance for every public model, and the size of that gap is a benchmark question rather than a spec-sheet one. We covered it separately in LLM context windows: real accuracy past 200K tokens, which walks through what RULER, MRCR v2, and NoLiMa actually measure.

Treat the advertised window as an upper bound on what the API will accept, and the benchmark numbers as the guide to what the model will use well.

How Do You Fit More Into the Window You Have?

None of these makes the window bigger; they all reduce what you spend inside it. In rough order of payoff:

1. Prompt caching. A stable prefix (system prompt, tool definitions, a document you keep asking about) is billed at roughly 10% of the input rate on a cache hit on Anthropic and several other vendors, though the exact discount varies and runs deeper at some. This is the single largest lever for repeated calls, and it changes cost, not capacity.
2. Compaction. Server-side summarization of earlier turns once the conversation approaches the limit, so a long agent session keeps running instead of erroring.
3. Context editing. Clearing stale tool results and old reasoning blocks out of the transcript. Agent loops fill up on tool output more than on conversation, and our Claude Code token optimization notes cover the coding-agent case specifically.
4. Pick the right tokenizer for your content. As the tables above show, that decision alone is worth up to 1.6x of effective capacity before you optimize anything else.

How Do You Compare Token Counts Across Vendors Without Nine Accounts?

The honest comparison needs the same request sent to models from different vendors, and each vendor normally means its own key, its own SDK, and its own billing line. That is the friction: nobody sets up nine accounts to answer a sizing question, so the question usually gets answered with a rule of thumb instead of a measurement.

Because every model above speaks the same OpenAI-compatible HTTP shape, one client and one loop over model IDs is enough. Send max_tokens: 1 and read prompt_tokens; the model never writes an answer, so the measurement costs a fraction of a cent.

from openai import OpenAI

client = OpenAI(base_url="https://api.ofox.ai/v1", api_key="YOUR_OFOX_KEY")
text = open("your_document.txt").read()

for model in [
    "anthropic/claude-opus-5",
    "openai/gpt-5.6-sol",
    "google/gemini-3.1-pro-preview",
    "moonshotai/kimi-k3",
    "z-ai/glm-5.2",
]:
    r = client.chat.completions.create(
        model=model, max_tokens=1,
        messages=[{"role": "user", "content": text}],
    )
    n = r.usage.prompt_tokens
    print(f"{model:32} {n:>7,} tokens   {len(text)/n:.2f} chars/token")

That loop produced every measurement on this page. Run it on your own content before choosing a model on window size, because on the evidence above the advertised number is off by up to 1.6x in either direction.

References

• Context windows
• Pricing, including the tokenizer note
• Prompt caching
• Compaction
• Gemini API pricing
• xAI: Grok 4.20 model card
• OpenAI: GPT-5.6 Sol model card
• ofox model catalog