On this pageThe dependable starting pointA real CPU-only testKeep context and parallel requests modestMeasure usefulness, not just tokens per secondSources
DT

Written and reviewed by DeviceterraPractical guidance · Updated November 11, 2026

THE SHORT ANSWER

Start with Qwen3 1.7B Q4 on CPU-only systems with limited memory. On 16 GB or more, compare Qwen3 4B or 8B only if the smaller model fails your task.

THE FULL GUIDE

Here is what matters, why it matters and how to check it on your own setup.

01

The dependable starting point

A CPU can run local text models without a graphics card. The main trade-off is speed: model weights and intermediate work move through system memory, which is typically slower than dedicated GPU memory.

For a low-memory CPU-only machine, use Qwen3 1.7B Q4 (about 1.4 GB). If you have 8 GB RAM, this smaller artifact gives Windows or Linux more room than a 4B model. With 16 GB or more, Qwen3 4B Q4 (about 2.6 GB) is a reasonable comparison; try 8B (about 5.2 GB) only when you have spare memory and accept slower generation.

Small CPU-first test

ollama run qwen3:1.7b-q4_K_M
02

A real CPU-only test

One published community test used Ollama on four virtual CPU cores with 6 GB RAM. It reported about 26–28 tokens per second for Qwen3 1.7B and about 12.7–13.4 for Qwen3 4B on its tasks; Qwen3 8B was killed by the out-of-memory system in that 6 GB VM.

This result is useful because the hardware, workload and failure are disclosed. It is not a benchmark for your laptop: a virtual machine, CPU model, operating system and background software can change the outcome.

03

Keep context and parallel requests modest

Start at 4K context, one request at a time. Longer context and simultaneous sessions use more memory. A CPU-only machine with 8 GB total RAM should generally use a smaller model for daily use and test the larger one only with applications closed.

04

Measure usefulness, not just tokens per second

  • Try a short summary, a simple question and a task from your normal work.
  • Record first-answer delay and total completion time.
  • Check correctness; a quick but wrong answer is not a good result.
  • Run the test once with few apps open and again with your normal apps.
  • If the computer freezes, swaps heavily or takes too long, use the smaller model or a cloud model for the occasional difficult task.
PLAIN ENGLISH

Words you may see in this guide

RAM
The working memory used by your computer and apps.
VRAM
The memory on a dedicated graphics card. Shared graphics may use system RAM.
Context
The text and other input the model can use for its current answer.
Quantization
A way to make model weights smaller. It can change memory use and answer quality.
Engine
The software that loads the model and runs it.
RAG
A workflow that finds useful document passages before asking a model to answer.
Benchmark
A repeatable test. A result is useful only when you know the setup and task.
RESEARCH SOURCES

Official facts and real user evidence

Official documentation supports product and model facts. Community discussions show real setups, failures, and questions. A community result is supporting evidence, not a promise that another computer will perform the same way.

YOUR NEXT STEP

Watch a real test. Check what fits your computer.

See practical local AI tests from DeviceTerra, then use MamiLens to build a hardware-aware shortlist for your own setup.

This guide is educational. Model software, licenses, and hardware support can change. Check official sources before an important deployment.