MEMORY PLANNING · 23 SEPTEMBER 2026
Local LLMs for 8, 16 and 32 GB RAM
A GGUF file that fits on disk may still exceed your working memory. Budget for model weights, context cache, runtime buffers and other applications.
See current comparisons for every memory tier from 4 to 128 GB or set your own reserve in the memory tool.
The budgets below are editorial starting points, not tested fit guarantees. All values use decimal GB. They assume one model and one user; measure actual free memory before loading.
| Installed RAM | Initial weight budget | Space left for everything else |
|---|---|---|
| 8 GB | Up to 4 GB | 4 GB |
| 16 GB | Up to 10 GB | 6 GB |
| 32 GB | Up to 22 GB | 10 GB |
8 GB RAM: where to start
Start small. Close other applications and use a short context. A lightweight CPU runner may be more practical than a full desktop AI app.
These three records have the highest published MMLU-Pro scores within the 4 GB weight budget in our 14 October 2026 catalog. This selection is computed from existing records; it is not a memory or quality test of the quantized files. Protocols differ.
| Model / file | Weights | Published MMLU-Pro |
|---|---|---|
| Qwen3.5-4B | 2.741 GB | 79.1 / 100 |
| MiniCPM5 2B | 1.561 GB | 70.8 / 100 |
| Granite 4.2 3B | 2.244 GB | 67.84 / 100 |
16 GB RAM: where to start
Leave room for the operating system and the context cache. Check whether the backend uses CPU RAM, integrated graphics or a separate GPU.
These three records have the highest published MMLU-Pro scores within the 10 GB weight budget in our 14 October 2026 catalog. This selection is computed from existing records; it is not a memory or quality test of the quantized files. Protocols differ.
| Model / file | Weights | Published MMLU-Pro |
|---|---|---|
| Qwen3.5-9B | 5.681 GB | 82.5 / 100 |
| Qwen3.5-4B | 2.741 GB | 79.1 / 100 |
| gemma-4-12B-it | 7.122 GB | 77.2 / 100 |
32 GB RAM: where to start
You can consider larger weight files, but long contexts and multiple sessions still use extra memory. More RAM alone does not guarantee fast generation.
These three records have the highest published MMLU-Pro scores within the 22 GB weight budget in our 14 October 2026 catalog. This selection is computed from existing records; it is not a memory or quality test of the quantized files. Protocols differ.
| Model / file | Weights | Published MMLU-Pro |
|---|---|---|
| Qwen3.6-27B | 16.817 GB | 86.2 / 100 |
| Qwen3.5-27B | 16.741 GB | 86.1 / 100 |
| gemma-4-31B-it | 18.324 GB | 85.2 / 100 |
Choose quantization, not just the smallest file
A smaller GGUF can make a model fit, but fitting is not evidence that the compressed artifact preserves the capability you need. An October 2026 preprint tested 27 quantized artifacts across 13 model families from 0.35B to 14B parameters. Its paired test asked models both to use 429 rare English words and to define the same words.
Within this study, Q4_K_M remained clean on the tested rare-word measure for every artifact at or above 1B parameters. More aggressive Q2 compression was much less predictable: every tested sub-2B artifact lost 20–67% of its baseline definition performance, even when output could still look fluent. Results also differed between builds of the same model.
That does not make Q4_K_M universally best or Q2 universally unusable. The paper is a preprint, evaluates a narrow lexical task, and does not measure your application, token speed, context memory or hardware compatibility. Treat its result as a reason to validate the exact downloaded file on representative prompts before buying hardware or accepting a smaller quantization. Read the paper and test design.
RAM, VRAM and unified memory
System RAM and dedicated GPU VRAM are separate pools. A 16 GB RAM computer with a 6 GB GPU does not provide a single 22 GB GPU pool. CPU offload may let a larger model run, with different performance. Unified memory is shared by CPU, GPU and operating system; installed capacity is not the amount available to one process.
LM Studio recommends at least 16 GB RAM and, on Windows, at least 4 GB dedicated VRAM. Its app requirements do not certify any particular model. See LM Studio system requirements and llama.cpp backend and quantization support.
A measured 128 GB example: weights are not peak memory
Microsoft reports that its 53 GB on-device MAI Code 1.1 Flash build reached 75.5 GB peak memory at 256K context on a Surface Laptop Ultra with NVIDIA RTX Spark. That is a 22.5 GB gap between model size and measured peak use. It is a concrete reason not to spend an entire memory pool on weights.
| Measurement | Reported result | Keep in view |
|---|---|---|
| Quantized model | 53 GB · mixed precision, approximately 3.3 bits per weight | 137B total / 6.8B active parameters |
| Peak memory | 75.5 GB at 256K context | Peak working memory, not weight size |
| Prompt processing | 923.5 tokens/s at 64K context | Prompt throughput, not output-token generation speed |
| Prompt processing | 769.8 tokens/s at 128K context | Prompt throughput, not output-token generation speed |
The tested build used DFlash2 sliding-window speculative decoding and a Windows ARM64 llama.cpp CUDA runtime. Microsoft reports 70.8% on SWE-Bench Verified (500 tasks) and 66.29% on Terminal-Bench 2.1 (89 tasks). These publisher results preserve task-quality context, but they do not prove the same fit or speed on another 128 GB computer.
Check fit before buying
- Choose the exact GGUF quantization and all required shards. Record their combined bytes.
- Set your intended context length and number of parallel sessions. The context cache grows with workload and model architecture.
- Load the model, run a representative prompt and record peak RAM/VRAM and any swapping.
- Compare generation speed on your task, then increase context gradually. A successful load is not a useful speed test.
Use the reproducible test worksheet, hardware comparison. Advertised maximum context and locally practical context are different; the catalog does not yet contain verified context limits for every model.