Best Local AI for Private Documents on a 32GB Computer
Model and engine guidance for private document search on a 32GB computer.
Yes. A computer with 32GB RAM can run small local AI models. A safe first choice from the current MamiLens list is Qwen 3 Embedding 8b. This is a memory-fit estimate, not a speed promise.
The device we checked
MamiLens keeps 6GB for the operating system and background work. That leaves a working model budget of about 26GB.
Best models to start with
Qwen 3 Embedding 8b
Turns text into vectors for multilingual search and private-document retrieval. It does not write chat answers; use the embedding API.
- Calculated weights
- 4.7 GB
- Estimated total RAM
- 12 GB
- RAM left
- 20 GB
- Expected path
- CPU / RAM
- Speed
- Benchmark required
- Evidence
- Calculated estimate
ollama pull qwen3-embedding:8bOpen evidence-labelled model profile →Granite 3.2 8B
Granite 3.2 8B is a curated dense profile for general, reasoning, tools workloads.
- Calculated weights
- 5 GB
- Estimated total RAM
- 28 GB
- RAM left
- 4 GB
- Expected path
- CPU / RAM
- Speed
- Benchmark required
- Evidence
- Calculated estimate
ollama run granite3.2:8bOpen evidence-labelled model profile →Granite 3.3 8B
An IBM text assistant for instruction following, reasoning, tools and retrieval workflows. Compare its answers on your actual business documents.
- Calculated weights
- 5 GB
- Estimated total RAM
- 28 GB
- RAM left
- 4 GB
- Expected path
- CPU / RAM
- Speed
- Benchmark required
- Evidence
- Calculated estimate
ollama run granite3.3:8bOpen evidence-labelled model profile →Granite 4 7B-A1B Hybrid
An IBM instruction model for business text, private-document workflows and tools. Hybrid variants use a different architecture and need a compatible current runtime.
- Calculated weights
- 4.2 GB
- Estimated total RAM
- 27 GB
- RAM left
- 5 GB
- Expected path
- CPU / RAM
- Speed
- Benchmark required
- Evidence
- Calculated estimate
ollama run granite4:7b-a1b-hOpen evidence-labelled model profile →Qwen 3 Embedding 4b
Turns text into vectors for multilingual search and private-document retrieval. It does not write chat answers; use the embedding API.
- Calculated weights
- 2.5 GB
- Estimated total RAM
- 10 GB
- RAM left
- 22 GB
- Expected path
- CPU / RAM
- Speed
- Benchmark required
- Evidence
- Calculated estimate
ollama pull qwen3-embedding:4bOpen evidence-labelled model profile →Recommended inference engine
LM Studio or Ollama
A CPU-first setup is the safest assumption when dedicated GPU support is unknown.
More CPU cores can help, but memory bandwidth and engine settings still matter.How accurate is this result?
This page uses the same calculator rules as the live MamiLens tool. It includes model weights, runtime overhead, operating-system reserve, context reserve and simultaneous-user cache when selected. A displayed speed range is calculated from published memory bandwidth. It is not a benchmark from this exact computer.
Start with the first model. Test a normal task, record response speed and watch memory use before using it for important work.