AI Model Selection Calculator

Hosted language models are metered in tokens, not requests, and input and output tokens are priced separately. A token is a sub-word unit averaging roughly four characters of English, so 500 input tokens is about 375 words. Monthly spend is therefore requests times input tokens divided by one million times the input price, plus the same product for output. Output almost always costs three to five times more than input, and the reason is architectural: prefill processes the whole prompt in parallel and saturates compute, while decoding emits one token at a time and is bound by memory bandwidth, so each generated token consumes far more of the serving budget. Prompts that are long but answers that are short are consequently much cheaper than the token counts alone suggest. Three constraints then filter the candidate set. Latency here is a per-call figure, and interactive chat generally needs a first response inside roughly 500 milliseconds to feel immediate, while batch classification can tolerate seconds. Accuracy is a per-task estimate, not a single benchmark score, because relative model rankings shift by task - a model strong at summarization may trail at code. Budget caps monthly spend outright. The value score divides task accuracy by cost per thousand requests, which deliberately favours small models unless the accuracy floor is set high. Two caveats: published prices move often and downward, so any stored table dates quickly, and prompt caching plus asynchronous batch endpoints commonly cut effective cost by half, which this straightforward per-token model does not capture.

The token as a billing unit descends from a compression technique. Byte pair encoding was described by Philip Gage in 1994 as a data compression algorithm, and Rico Sennrich, Barry Haddow and Alexandra Birch adapted it to neural machine translation in 2016 to handle rare and unseen words by splitting them into sub-word pieces. When Vaswani and colleagues at Google published the transformer architecture in 2017, sub-word tokenization was already the standard input representation, and the token became the natural unit of both computation and cost. Commercial metering followed GPT-3, which OpenAI released in 2020 and exposed through an API priced per thousand tokens. Anthropic, Google, Mistral and others adopted the same unit, making cross-vendor price comparison possible for the first time. Prices fell steeply and repeatedly as serving efficiency improved and smaller distilled models arrived, and Meta's release of LLaMA in early 2023 followed by Llama 2 under a commercial licence in July 2023 added a self-hosting option that trades per-token fees for GPU-hour costs. Evaluation matured alongside pricing: the MMLU benchmark introduced by Dan Hendrycks and colleagues in 2020, Stanford CRFM's HELM framework in 2022, and the crowd-sourced LMSYS Chatbot Arena from 2023 gave buyers accuracy evidence to weigh against price. Selecting a model by cost-per-quality rather than by raw capability became routine practice only once both numbers were publicly comparable.