GPU Memory Calculator
Free Gpu memory Calculator for ai & ml. Enter parameters to get optimized results with detailed breakdowns. Free to use with no signup required.
Reviewed by Daniel Agrici, Founder & Lead Developer
Formula
VRAM = Model Weights + KV Cache + Activations + Overhead
Model weights = parameters × bytes per parameter. KV cache = 2 × layers × batch × seq × kv_heads × head_dim × precision. Add ~10% for CUDA/framework overhead. Training additionally requires gradients (same as weights) and optimizer states (2× weights for AdamW in FP32).
Worked Examples
Example 1: Llama 3.1 7B in FP16
Problem:Estimate VRAM needed to run Llama 3.1 7B in FP16 with batch size 1 and 2048 context.
Solution:Model weights: 7B × 2 bytes = 14 GB\nKV cache: ~0.5 GB (32 layers × 2048 seq × 32 heads × 128 dim × 2 bytes × 2)\nActivations: ~0.1 GB\nOverhead: ~10%\nTotal: ~16 GB
Result:~16 GB — fits on RTX 4080 (16GB) or RTX 4090 (24GB)
Example 2: 70B Model in INT4
Problem:Can a 70B model run on consumer hardware with 4-bit quantization?
Solution:Model weights: 70B × 0.5 bytes = 35 GB\nKV cache: ~2-4 GB at 2048 context\nTotal: ~40 GB\nNo single consumer GPU has 40+ GB (except RTX 5090 at 32 GB — tight)
Result:Requires 40+ GB — best on A100 40GB, or use 2× RTX 3090/4090 with model parallelism
Frequently Asked Questions
How is GPU memory (VRAM) calculated for LLMs?
LLM VRAM consists of: (1) Model weights — parameters × bytes per parameter (4B for FP32, 2B for FP16, 1B for INT8, 0.5B for INT4). A 7B parameter model in FP16 needs ~14 GB just for weights. (2) KV cache — stores key/value pairs for attention, scaling with batch size and sequence length. (3) Activations — intermediate computation results. (4) Framework overhead — CUDA context, memory fragmentation (~10%). Total VRAM = weights + KV cache + activations + overhead.
References
Reviewed by Daniel Agrici, Founder & Lead Developer · Editorial policy