Fine Tuning ROI Calculator

Fine-tuning trades a known upfront cost for a lower marginal cost per request, so the decision reduces to a payback-period calculation. The upfront figure has two parts that are easy to underestimate separately: the training job itself, often a few hundred dollars, and the human labour of assembling, cleaning, and reviewing the training set, which at 40 hours and 75 dollars an hour is 3,000 dollars and usually the larger number. The recurring saving is the per-token price gap between a large prompted model and a smaller tuned one - for example 10 and 30 dollars per million input and output tokens against 3 and 6 - applied to monthly volume. Break-even in months is upfront cost divided by monthly saving, and the answer is dominated by volume: 200,000 requests a month can repay the investment in under three, while 5,000 a month may take over two years. A second saving is often omitted from this arithmetic. A tuned model has absorbed the formatting rules and few-shot examples that a prompted model needs restated on every call, so prompts frequently shrink by hundreds of tokens, compounding the rate advantage. Against that, several factors argue for staying with the larger model. A small tuned model rarely matches a frontier model on open-ended reasoning, however well it performs on a narrow task. Models drift as products and policies change, and quarterly retraining reintroduces both cost lines. Weights are provider-specific, deepening lock-in, and some providers charge a premium on inference against custom models that narrows the rate gap this calculation assumes.

The pretrain-then-adapt pattern is the organizing idea of modern natural language processing. Its decisive demonstrations came in 2018: Jeremy Howard and Sebastian Ruder published ULMFiT, showing that a language model pretrained on a general corpus could be fine-tuned to beat task-specific architectures with far less labelled data; Alec Radford and colleagues at OpenAI described generative pretraining followed by discriminative fine-tuning in the first GPT paper; and Jacob Devlin and colleagues at Google released BERT, whose fine-tuning recipe became the default across the field. As models grew, updating every weight became impractical, and parameter-efficient methods emerged to keep adaptation cheap. Neil Houlsby and co-authors introduced adapter modules in 2019, inserting small trainable layers into a frozen network. Edward Hu and colleagues at Microsoft published LoRA in 2021, representing the weight update as a pair of low-rank matrices and cutting trainable parameters by orders of magnitude, and QLoRA from Tim Dettmers and co-authors in 2023 combined it with quantization so that large models could be tuned on a single GPU. Commercially, OpenAI opened fine-tuning on its base GPT-3 models in 2021 and extended it to gpt-3.5-turbo in August 2023, which is the point at which the build-versus-prompt question became a routine budgeting exercise for engineering teams rather than a research undertaking.