ByteCalculators/ AI Infrastructure/ LLM VRAM Calculator

LLM VRAM Calculator 2026 Hardware Guide

Estimate GPU memory requirements for running local LLMs. Compare quantization levels, context windows, batch sizes, standard GQA attention and DeepSeek-style MLA KV cache.

⚡ Model presets Loads model size and attention configuration
Use total parameters for MoE models, not active parameters per token.
Bits per weight is an estimate; actual file formats vary.
Assumes each sequence uses the selected context length.
This tool estimates inference memory. It does not calculate training or fine-tuning memory, CPU offload, speculative decoding, prefix-cache sharing, or framework-specific fragmentation.
Estimated VRAM budget
6.9
GiB (binary) — before extra safety margin
Model weights + allowance4.5 GiB
KV cache0.50 GiB
Runtime / activation reserve1.5 GiB
Suggested capacity (+10%)7.6 GiB
Hardware compatibility estimate
8GB GPU class
Estimated fit — check actual usable VRAM.
A multi-GPU recommendation requires software support for model sharding or tensor parallelism. VRAM capacities cannot simply be added for a single-GPU workload.

LLM VRAM estimate comparison

Calculated using the same estimation method as the calculator: 4.5 bits/weight, 8% weight allowance, 8,192-token context, batch 1, FP16 KV cache and a 1.5 GiB runtime reserve. Values are estimates, not measured benchmark results.

ModelArchitecture usedEstimated totalCapacity target (+10%)Indicative setup
Llama 3.2 3B28 layers · 8 KV heads——Single GPU class
Llama 3.3 8B32 layers · 8 KV heads——Single / multi GPU depends on quantization
Qwen 2.5 14B48 layers · 8 KV heads——Single / multi GPU depends on quantization
Qwen 2.5 32B64 layers · 8 KV heads——Multi-GPU or large unified memory
Gemma 2 27B46 layers · 16 KV heads——Multi-GPU or large unified memory
Llama 3.3 70B80 layers · 8 KV heads——Multi-GPU
Mistral Large 2 123B88 layers · 8 KV heads——Multi-GPU / large unified memory
DeepSeek V3 / R1 671B61 layers · MLA——Distributed multi-GPU

Local GPU vs. Cloud API: TCO considerations

There is no universal break-even point. Local inference economics depend on the hardware purchase price, utilization, electricity, maintenance, model performance and the API workload being replaced.

1. Hardware utilizationA GPU workstation that is idle much of the day still ties up capital. Compare cost per generated token at realistic utilization, not theoretical 24/7 capacity.
2. Electricity and coolingEstimate whole-system wall power, not only GPU TDP. Multiply measured kW by operating hours and your current electricity tariff; add cooling where relevant.
3. Multi-GPU overheadTwo 24GB cards provide 48GB aggregate physical VRAM only when the inference stack can distribute model layers or tensors. Interconnect bandwidth affects performance.
4. API price changesHosted inference prices change frequently and differ by input, output, cached tokens, service tier and provider. Use current provider pricing before making a purchasing decision.

Model weights

Raw weight memory is approximated as parameters × bits per weight ÷ 8. The adjustable allowance accounts for the fact that real quantized files and runtime loaders do not always match the ideal bit count.

Standard GQA KV cache

2 × layers × KV heads × head dimension × context × batch × bytes. The factor of two represents keys and values. Grouped-query attention reduces KV memory by using fewer KV heads than query heads.

MLA and MoE models

For the DeepSeek-style MLA preset, the cache estimate uses a 576-element latent representation rather than the standard K/V head formula. MoE weight memory uses total model parameters: active parameters per token describe compute, not the full weight footprint when all experts are resident.

Inference is not fine-tuning

This calculator estimates inference only. Training and fine-tuning can require substantially more memory for gradients, optimizer states, activations and checkpointing. Use a dedicated fine-tuning estimator for those workloads.

Scroll to Top