How to run local AI without a dedicated GPU

Choose a first model by available memory

On an 8 GB computer, begin with a 1B–2B quantized text model and one short task. A 16 GB computer has more room to test a 3B–8B model, but a larger model may respond too slowly on a laptop CPU. Installed RAM includes the memory Windows and your other apps already use.

For a reproducible small test, Qwen3 1.7B is available as an Ollama download of about 1.4 GB. That is the file size, not the total memory needed while it runs. Use the hardware calculator with your real free disk space before downloading.

Measure usefulness instead of accepting a slow answer

Save three short examples from the work you actually do, with private details removed. Run each twice: the first attempt includes model loading, while the second better represents a model already in memory. Record how long you wait, whether the facts survived, and whether the computer remained usable.

Set your own limit before testing. For example, a draft that takes 30 seconds may be fine for occasional writing and unacceptable for live customer support. This is an example acceptance threshold, not a predicted speed.

• If disk activity stays high and the computer freezes, reduce model size and close other apps. Do not assume adding virtual memory will make it fast.
• If later replies slow down, start a new chat and reduce the context.
• If answers are wrong, a faster processor will not repair them. Try a better model or narrow the task.

What a CPU can do

A CPU is the main processor in your computer. It can run small AI models for writing, summaries, sorting, simple coding help, and private document questions.

A CPU is usually slower than a good GPU. That may be fine if one person uses the model and the task does not need an instant answer.

Choose a small model

On an 8 GB computer, start with a model between 1B and 4B. On a 16 GB computer, you can often try models between 3B and 8B.

Choose a Q4 file first. Q4 is a smaller version of the model that uses less memory. These ranges are starting points, not guarantees.

Keep the conversation short

Context is the text the model reads during one conversation. Long chats and large documents need more memory.

Start with short prompts. For many documents, use a search system that sends only the most useful passages to the model.

Simple setup steps

• Close apps that use a lot of memory.
• Install Ollama or LM Studio from its official website.
• Use MamiLens to choose a small model.
• Download one model.
• Test five real prompts and record the waiting time.
• Try a larger model only if the small one cannot do the job.

Tasks to leave for later

Large coding agents, very long documents, live web research, and many users need more memory and speed.

Start with one user and one short task. Buy better hardware only after your test shows clear value.