LLM VRAM Requirement Calculator
Estimate how much GPU memory (VRAM) is needed to run a language model, from its parameter count.
Inputs
Accounts for the KV cache, activations and framework overhead beyond raw model weights — 1.2 (20%) is a common rule-of-thumb minimum.
Saved Scenarios
— select 2+ to compare| Metric | |
|---|---|
Estimated VRAM (GB)
16.8
Spark says
How it's calculated
Formula
- Overhead
- — Extra memory for KV cache/activations beyond raw weights
What is the LLM VRAM Requirement Calculator?
A model's raw weight size is roughly parameters × bytes-per-parameter — but running inference also needs memory for the KV cache and intermediate activations, so a common rule of thumb adds 20%+ on top of raw weight size.
How to use it
- 1 Enter the model's parameter count in billions.
- 2 Choose the precision it will run at (lower precision = less memory, some quality trade-off).
- 3 Adjust the overhead factor if you have a more precise figure for your setup.
Worked examples
Limitations
- •Actual VRAM usage also depends on batch size, sequence length, and the specific inference framework — this gives a reasonable ballpark, not an exact figure.
Frequently asked questions
Why does quantization reduce VRAM needs?
Quantization stores each parameter in fewer bits (e.g. 8 or 4 instead of 16) — this shrinks memory usage roughly proportionally, at some cost to output quality.