Llm Cost Calculator

Projecting the spend of a language-model feature is a unit-economics exercise rather than a subscription one. Cost per request is the prompt tokens times the input rate plus the completion tokens times the output rate, both quoted per million tokens; multiplying by request volume gives the daily, monthly, and annual figures. The spread between rate cards is what makes this worth computing. Among the models listed here, input pricing runs from about 0.06 US dollars per million for Llama 3.1 8B through hosting partners up to 15.00 for Claude 3 Opus and the o1 reasoning model, a range of roughly 250 to 1. Output is consistently the steeper side, typically 2 to 5 times the input rate, because generation is sequential while prompt processing is a single parallel prefill pass. The practical consequence is that the shape of the workload matters as much as the model choice. A retrieval or summarisation pipeline with a long prompt and a short answer is dominated by input tokens, so cheap-input models and prompt caching pay off. A drafting or agent workload with short instructions and long generations is dominated by output tokens, where a model with a low output rate matters more. Watch for three caveats: reasoning models bill hidden thinking tokens as output, so observed cost can exceed the visible response; averages hide a long tail, since a small fraction of requests with maximal context can carry a disproportionate share of spend; and published rates move, with batch endpoints and cached prefixes commonly discounting 50 percent.

Charging for computation by the token is a recent convention. Cloud services through the 2010s billed by instance-hour, request count, or gigabyte transferred; Amazon Web Services launched EC2 hourly billing in 2006 and AWS Lambda per-invocation billing in 2014. When OpenAI opened the GPT-3 API in June 2020, it priced access in units of 1,000 tokens with separate tiers for the Ada, Babbage, Curie, and Davinci models, tying cost directly to the quantity of text processed rather than to time or calls. Competition reshaped the rate cards quickly. Anthropic began commercial Claude access in 2023, Google opened the PaLM and later Gemini APIs the same year, and hosting providers such as Together AI, Fireworks, and Groq began serving Meta Llama weights after the Llama 2 release in July 2023, pushing open-weight inference prices far below proprietary equivalents. Two structural changes followed. Providers split pricing into distinct prompt and completion rates once the asymmetry between prefill and decode became a serving reality, and from 2023 onward they introduced discount mechanisms tied to that same asymmetry: asynchronous batch endpoints at roughly half price, and prompt caching that charges a reduced rate for a repeated prefix already resident in the serving cache. Quoting per million tokens instead of per thousand became standard as production volumes grew.