On this pageChoose a first model by available memoryMeasure usefulness instead of accepting a slow answerWhat a CPU can doChoose a small modelKeep the conversation shortSimple setup stepsTasks to leave for laterMake a CPU test worth your timeShould I choose a tiny model?Sources
DT

Written and reviewed by DeviceterraPractical guidance · Updated November 11, 2026

THE SHORT ANSWER

Yes, a CPU can run local text AI. Start small, keep the chat short and choose by answer quality and waiting time on your computer.

THE DIRECT ANSWER

What should you choose?

Yes, you can run a local text model without a dedicated GPU. On an 8 GB computer, start with Qwen3 1.7B. With 16 GB, try Qwen3 4B first. Use a short chat and check whether the answer is useful at a waiting time you accept.

This recommendation assumes:

One user, CPU processing and a short context. Enough free RAM and disk space are still required. These are starting points, not measured speed results.

8 GB or older laptop

Qwen3 1.7B

A small first download for short rewrites and simple questions.

Limit: Complex questions may need a more capable model.
ollama run qwen3:1.7b
16 GB computer

Qwen3 4B Q4

A larger candidate for general text tasks when memory allows.

Limit: A slow CPU can still make replies take too long.
ollama run qwen3:4b
What to avoid

Do not choose from download size alone. The engine, conversation and other apps also need working memory.

Check this choice on your computer

Starts with an 8 GB Windows example. Change RAM, operating system and task to match your computer.

Check my CPU-only model options →

Evidence: Model tags and commands were checked against official sources on October 8, 2026. These suggestions are not measured benchmarks on your hardware. See how MamiLens checks recommendations.

What to record in your own test

CPU, RAM, GPU, driver, engine version, exact model tag, context, first-answer time, repeat-answer time, peak memory and factual errors. Use the same prompts for each model.

Follow the repeatable test guide →
THE FULL GUIDE

Here is what matters, why it matters and how to check it on your own setup.

01

Choose a first model by available memory

On an 8 GB computer, begin with a 1B to 2B quantized text model and one short task. A 16 GB computer has more room to test a 3B to 8B model, but a larger model may respond too slowly on a laptop CPU. Installed RAM includes the memory Windows and your other apps already use.

For a reproducible small test, Qwen3 1.7B is available as an Ollama download of about 1.4 GB. That is the file size, not the total memory needed while it runs. Use the hardware calculator with your real free disk space before downloading.

A useful first task

Give it a short message and ask for a clearer rewrite without changing names, dates or amounts. Compare the result with your original. A fluent answer that changes the facts fails the test.

02

Measure usefulness instead of accepting a slow answer

Save three short examples from the work you actually do, with private details removed. Run each twice: the first attempt includes model loading, while the second better represents a model already in memory. Record how long you wait, whether the facts survived, and whether the computer remained usable.

Set your own limit before testing. For example, a draft that takes 30 seconds may be fine for occasional writing and unacceptable for live customer support. This is an example acceptance threshold, not a predicted speed.

  • If disk activity stays high and the computer freezes, reduce model size and close other apps. Do not assume adding virtual memory will make it fast.
  • If later replies slow down, start a new chat and reduce the context.
  • If answers are wrong, a faster processor will not repair them. Try a better model or narrow the task.
03

What a CPU can do

A CPU is the main processor in your computer. It can run small AI models for writing, summaries, sorting, simple coding help, and private document questions.

A CPU is usually slower than a good GPU. That may be fine if one person uses the model and the task does not need an instant answer.

04

Choose a small model

On an 8 GB computer, start with a model between 1B and 4B. On a 16 GB computer, you can often try models between 3B and 8B.

Choose a Q4 file first. Q4 is a smaller version of the model that uses less memory. These ranges are starting points, not guarantees.

Do not chase size

The best first model is the smallest one that can complete your task.

05

Keep the conversation short

Context is the text the model reads during one conversation. Long chats and large documents need more memory.

Start with short prompts. For many documents, use a search system that sends only the most useful passages to the model.

06

Simple setup steps

  • Close apps that use a lot of memory.
  • Install Ollama or LM Studio from its official website.
  • Use MamiLens to choose a small model.
  • Download one model.
  • Test five real prompts and record the waiting time.
  • Try a larger model only if the small one cannot do the job.
07

Tasks to leave for later

Large coding agents, very long documents, live web research, and many users need more memory and speed.

Start with one user and one short task. Buy better hardware only after your test shows clear value.

08

Make a CPU test worth your time

A laptop without a dedicated GPU can still be useful for a short email rewrite. It may be a poor choice for a long coding task that needs many answers quickly. The question is whether the waiting time is acceptable for your job.

Close heavy apps and test one small model with a short prompt. Time the first answer and a second answer after the model is loaded. Try a normal work sample next. If it is too slow, reduce the model size before buying hardware.

A mistake to avoid

More CPU cores do not guarantee a fast answer. Memory speed, cooling, model size and the app settings also matter.

09

Should I choose a tiny model?

Choose a small starting point, then test quality. A tiny model that repeatedly gets the job wrong costs time too. Move up only when you have enough memory and a clear quality reason.

TEST THE RECOMMENDATION

Run a small model on a normal computer

You can try local AI without a dedicated graphics card. It will usually be slower, so start with a small model and a narrow task.

  1. 1

    Close heavy apps

    Save your work. Close games, video editors, and browser tabs you do not need.

  2. 2

    Install Ollama

    Visit ollama.com/download and run the installer for your operating system.

  3. 3

    Open PowerShell or Terminal

    Windows: click Start, type PowerShell, and open it. Mac: press Command and Space, type Terminal, and press Enter.

  4. 4

    Run a small model

    Type this command and press Enter. Wait for the first download to finish.

    ollama run qwen3:1.7b
  5. 5

    Test one short job

    Ask it to rewrite a short message. Do not paste a long document into the first test.

How to know the choice is right
  • The model answers without the computer freezing.
  • Normal memory use stays below the total available RAM.
  • The waiting time is acceptable for your task.
PLAIN ENGLISH

Words you may see in this guide

RAM
The working memory used by your computer and apps.
VRAM
The memory on a dedicated graphics card. Shared graphics may use system RAM.
Context
The text and other input the model can use for its current answer.
Quantization
A way to make model weights smaller. It can change memory use and answer quality.
Engine
The software that loads the model and runs it.
RAG
A workflow that finds useful document passages before asking a model to answer.
Benchmark
A repeatable test. A result is useful only when you know the setup and task.
RESEARCH SOURCES

Official facts and real user evidence

Official documentation supports product and model facts. Community discussions show real setups, failures, and questions. A community result is supporting evidence, not a promise that another computer will perform the same way.

YOUR NEXT STEP

Watch a real test. Check what fits your computer.

See practical local AI tests from DeviceTerra, then use MamiLens to build a hardware-aware shortlist for your own setup.

This guide is educational. Model software, licenses, and hardware support can change. Check official sources before an important deployment.