Cloud GPU Cost Calculator

Accelerator spend has three components and only one of them is the GPU. Compute is billed per instance-hour, and on hyperscalers the headline rates correspond to full multi-accelerator nodes - AWS p4d and p5 instances, for example, carry eight A100 or eight H100 devices respectively - which is why on-demand figures of 30 to 65 dollars an hour look large next to single-card pricing. Storage is charged per terabyte-month, roughly 20 to 23 dollars on the major clouds for standard object storage, and egress is charged per gigabyte, around 0.09 dollars on AWS, which quietly dominates the bill for teams that repeatedly pull checkpoints or datasets out of the provider. Utilization is the single most consequential input. A month has about 730 hours, so 8 hours a day across 22 days is 176 hours, a 24 percent duty cycle, and at that level on-demand pricing usually beats any commitment. Beyond roughly 60 percent sustained utilization, reserved instances and committed-use discounts of 30 to 60 percent, or spot and preemptible capacity at 60 to 90 percent off, change the ranking entirely. Specialist providers such as Lambda and RunPod price well below the hyperscalers because they omit the surrounding enterprise services, though they also offer weaker availability guarantees and thinner regional coverage. Two caveats: published rates change frequently and any stored table should be re-verified before a purchase decision, and the cheapest hourly rate is not always the cheapest job, since a faster accelerator that finishes in a third of the time can win on total cost despite a higher rate.

General-purpose computation on graphics hardware became practical when NVIDIA released CUDA in November 2006, giving developers a C-like interface to the parallel units previously reachable only through graphics shader tricks. The decisive demonstration for machine learning came in 2012, when Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton trained the AlexNet convolutional network on two consumer GTX 580 cards and won the ImageNet competition by a wide margin, establishing GPU training as the default for deep learning. Rented capacity followed almost immediately. Amazon Web Services introduced Cluster GPU instances in November 2010, and the accelerator line advanced with each NVIDIA data-center generation: Kepler-based K80 instances arrived in 2016, Volta V100 instances in 2017 with the first tensor cores, Ampere A100 instances in 2020, and Hopper H100 instances from 2023. Google, Microsoft Azure and others tracked the same cadence, and Google additionally offered its own Tensor Processing Units from 2018. Demand for training and inference capacity after ChatGPT's late-2022 release created acute H100 scarcity through 2023 and 2024, and that shortage opened space for a cohort of specialist GPU clouds - CoreWeave, Lambda Labs, RunPod and others - competing chiefly on price per accelerator-hour and on availability. Comparing providers on a normalized hourly basis became standard practice for machine learning teams during that period.