Model Context Window Calculator

The context window is the hard ceiling on how many tokens a model can hold in a single forward pass, and it is shared by everything in the request: system prompt, retrieved documents, conversation history, image tokens, and the completion the model is allowed to produce. It exists because self-attention builds a position-by-position score matrix, so both compute and key-value cache memory grow with sequence length, and because positional encodings are trained over a bounded range. A second, separate limit governs generation: GPT-4o accepts 128,000 tokens of context but caps a single response at 16,384, and Claude 3 Opus accepts 200,000 while capping output at 4,096. A request can fit the window comfortably and still fail on the output ceiling. Estimating usage means converting characters to tokens, and the ratio is model-specific: near 4 characters per token for GPT tokenizers on English prose and closer to 3.5 for Claude, with code and non-Latin scripts consuming more. Images are billed as blocks of tokens rather than characters, on the order of several hundred to about 1,600 depending on resolution and detail setting. Two caveats matter in production. Overrunning the window returns an error rather than silently truncating on most modern APIs, so applications should keep a 10 to 15 percent safety margin. And a bigger window is not uniformly usable: retrieval accuracy measurably degrades for material buried in the middle of a very long context, so placement of critical text still matters.

Context length has expanded by roughly three orders of magnitude in six years. The original transformer of Vaswani and colleagues in 2017 and the BERT models of 2018 worked over 512 tokens. GPT-2 in 2019 used 1,024, and GPT-3 in 2020 doubled that to 2,048. The constraint was quadratic attention cost, and a research programme grew around it: Transformer-XL by Zihang Dai and co-authors in 2019 added segment recurrence, Longformer from the Allen Institute for AI in 2020 introduced sparse sliding-window attention, and Rotary Position Embedding, published by Jianlin Su and colleagues in 2021, gave a positional scheme that extrapolates and can be interpolated to longer ranges. The practical jump came from systems work rather than architecture alone. FlashAttention, published by Tri Dao and co-authors in 2022, made exact attention IO-aware and removed the need to materialise the full score matrix, and grouped-query attention, described by Joshua Ainslie and colleagues at Google in 2023, cut key-value cache growth. Commercial windows followed rapidly: Anthropic announced a 100,000-token Claude context in May 2023 and 200,000 with Claude 2.1 in November 2023, OpenAI shipped GPT-4 Turbo with 128,000 tokens in November 2023, and Google announced a 1 million token window for Gemini 1.5 Pro in February 2024. Nelson Liu and co-authors published Lost in the Middle in 2023, documenting that models use the beginnings and ends of long contexts better than the middle.