Storage holds the model, RAM or VRAM runs it, and extra headroom is required for the operating system, context, cache, and applications.
Don't Download Any AI Model on Ollama Before Watching This
This Deviceterra walkthrough focuses on the mistake behind many slow or inaccurate Ollama setups: choosing a model before checking the job and the machine. The article supplies the memory rules behind that choice.
Visit the Deviceterra YouTube channel ↗Three different resources
Storage is the SSD or hard drive where the downloaded model file lives. RAM is the computer's general working memory. VRAM is high-speed memory attached to a dedicated GPU. Apple Silicon uses unified memory shared by the CPU and GPU.
A 5 GB model file needs roughly 5 GB of storage, but it also needs working memory when loaded. The runtime, prompt context, key-value cache, operating system, and other applications add overhead. That is why “the file fits” is not a safe compatibility rule.
Why quantization changes the equation
Model weights can be stored at lower precision through quantization. A quantized model is smaller and usually easier to run, with some possible quality loss. Tags such as Q4, Q5, and Q8 commonly indicate different precision and size choices, but naming can vary across model formats.
For first tests, a reputable 4-bit quantization is often a practical balance. Move upward only when your hardware has headroom and evaluation shows a meaningful quality benefit.
RAM: the compatibility floor
If a model runs primarily on the CPU, system RAM carries most of the load. Leave room for Windows, macOS, or Linux and for the applications you need alongside the model.
As a conservative rule, do not allocate every reported gigabyte to model weights. An 8 GB machine should target very small models. A 16 GB machine has useful options in the compact 3B–8B range. A 32 GB machine opens larger quantized models, but context size and speed still matter.
Recommendations require the model file plus operating-system and runtime headroom. This is deliberately more cautious than a bare download-size comparison.
VRAM: acceleration, not magic
A supported GPU can process model operations much faster than a CPU. If all layers fit in VRAM, performance is often best. Partial GPU offload can still help when the complete model does not fit.
VRAM does not repair a poor task match. A fast model that produces unreliable answers is still the wrong model. GPU support also depends on the runtime, operating system, drivers, and hardware generation.
Context consumes memory too
Context is the text the model can consider in one session. A published maximum context length is a capability limit, not proof that the maximum will fit your computer. Longer conversations and documents increase cache memory and can slow the system.
Begin with a moderate context window and expand only after monitoring memory use. For document assistants, retrieval is usually better than pouring an entire archive into one prompt.
A practical buying and testing checklist
- Confirm total system RAM and, if present, dedicated VRAM.
- Check the exact quantized file size - not only the model's parameter count.
- Reserve memory for the operating system and normal applications.
- Test the context length you actually need.
- Measure speed on the real machine; do not trust generic tokens-per-second claims.
- Evaluate answer quality with representative work before buying new hardware.
Find a model your computer can run.
The LocalLens advisor applies conservative memory rules and tells you when the evidence is insufficient.
Run the free compatibility check →This guide is educational and reviewed for practical accuracy. Model software, licenses and hardware support change; verify official sources before a production deployment.