On this page
Start with a smaller modelCheck which processor is doing the workUse a small context and testSourcesTry Qwen3 1.7B Q4 first on 4 GB VRAM. Qwen3 4B Q4 has a 2.6 GB weight file and may fit on some systems, but desktop display memory and runtime overhead can leave too little room.
Here is what matters, why it matters and how to check it on your own setup.
Start with a smaller model
Use Qwen3 1.7B Q4 as the safer first GPU test. A 4 GB card's full capacity is not necessarily available to the model: a monitor, desktop, browser and runtime may use some of it.
Qwen3 4B Q4_K_M is a second candidate (about 2.6 GB download). The artifact size alone does not establish that the model plus context and GPU buffers will fit. Keep system RAM available in case the engine moves some work to the CPU.
Safer first model
ollama run qwen3:1.7b-q4_K_MOptional next test
ollama run qwen3:4b-q4_K_MCheck which processor is doing the work
Ollama supports different GPU backends depending on the card, driver and operating system. Confirm GPU support in the current official documentation, then check the running process or app's hardware monitor to see whether the GPU is actually being used.
If the model loads but most layers run on the CPU, it can still answer but may be slower. Partial GPU offload is a fallback, not proof that the card can run the full model in VRAM.
Use a small context and test
Start with 4K context and one request. If the desktop becomes unstable, reduce context or return to the smaller model. Do not try a long image or document prompt until short text tests are stable.
- Confirm the exact GPU model and available VRAM.
- Close GPU-heavy apps and note idle VRAM use.
- Run the short model command and watch VRAM and system RAM.
- Test a short real task and check the answer.
- Record driver, runtime, model tag and settings before comparing results.
Words you may see in this guide
- RAM
- The working memory used by your computer and apps.
- VRAM
- The memory on a dedicated graphics card. Shared graphics may use system RAM.
- Context
- The text and other input the model can use for its current answer.
- Quantization
- A way to make model weights smaller. It can change memory use and answer quality.
- Engine
- The software that loads the model and runs it.
- RAG
- A workflow that finds useful document passages before asking a model to answer.
- Benchmark
- A repeatable test. A result is useful only when you know the setup and task.
Official facts and real user evidence
Official documentation supports product and model facts. Community discussions show real setups, failures, and questions. A community result is supporting evidence, not a promise that another computer will perform the same way.
Watch a real test. Check what fits your computer.
See practical local AI tests from DeviceTerra, then use MamiLens to build a hardware-aware shortlist for your own setup.
This guide is educational. Model software, licenses, and hardware support can change. Check official sources before an important deployment.
