Local Model Memory Estimator

Model & context Reference model

Changing models restores its parameters and architecture, and returns to parameter-based sizing. Custom starts from the current values; edit its architecture below.

Total parameters (billions)

Total loaded parameters, including embeddings. 1 billion = 1,000,000,000.

Weight format estimate

Bits per weight include block scales. Mixed GGUF formats such as Q4_K_M vary; refine with an allowance or measured size below.

Context tokens per session

Total reserved context: instructions, history, prompt, and generated output.

Parallel sessions

Each session has its own full context cache; model weights are shared.

Your memory budget

Memory pool

Assumes weights and cache occupy this pool. Do not add separate RAM and VRAM capacities.

Available memory (GiB)

Memory you can allocate after reserving room for the OS, display, and other apps.

Refine architecture & overhead

Weight sizing Weight sizing method

KV cache architecture

Transformer layers

Full-attention layers that store K and V.

KV heads per layer

Use key/value heads, not query heads, for GQA and MQA models.

Head dimension

Assumes equal K and V dimensions. Check the model config; hidden size / query heads is common.

KV cache format

Independent of weight quantization. Quantized KV requires runtime support and may affect quality.

Planning allowances

Editable planning assumptions, not measured requirements. Adjust for your backend and workload.

Weight allowance (%)

Extra for mixed-precision tensors, embeddings, and metadata beyond the block estimate. Not applied to a known size.

Runtime buffers (GiB)

Compute, activations, allocator, and backend buffers. Prefill and batching may require much more.

Safety margin (%)

Added to weights + KV cache + runtime buffers. Does not reserve OS memory again.

Assumptions, formula & sources

Weights = billions of parameters × 1,000,000,000 × effective bits per weight ÷ 8, plus the weight allowance. Quantization block scales are already included in effective bits. Mixed formats and non-quantized tensors vary, so this is not an exact GGUF file-size calculator. A known weight size replaces both the parameter estimate and its allowance.

KV cache bytes = 2 (K and V) × layers × KV heads × head dimension × context tokens × parallel sessions × bytes per cache element. FP16/BF16 uses 2 bytes; Q8_0 uses 34/32 and Q4_0 uses 18/32 including block scales. Quantized cache estimates require a head dimension divisible by 32. Real allocation may include padding and alignment.

Planning target = (weights + KV cache + runtime buffers) × (1 + safety margin / 100). The defaults of 10% extra weights, 1 GiB runtime buffers, and 20% safety are editable planning choices, not measured results or universal recommendations. 1 GiB = 1,073,741,824 bytes; 1 GB = 1,000,000,000 bytes.

Scope: one dense, full-attention decoder model with equal K/V head dimensions, one shared set of weights, and an independent full cache for each parallel session. This does not model training, beam search, MoE residency, MLA, hybrid/recurrent/sliding-window attention, vision encoders, speculative draft models, multi-GPU splits, or CPU/GPU offloading. Unified memory is one pool, not additional RAM plus VRAM. Memory fit does not predict speed, runtime compatibility, or guarantee that a model will load; check real peak usage with your backend.

Calculated in your browser. No model downloads, hardware scans, API keys, or saved inputs.