On this pageStart with the smaller card-friendly modelWhat community benchmarks sayAvoid assuming 8 GB means an 8 GB modelVerify on your cardSources
DT

Written and reviewed by DeviceterraPractical guidance · Updated November 11, 2026

THE SHORT ANSWER

For 8 GB VRAM, Qwen3 4B Q4 is a cautious first model; Qwen3 8B Q4 (about 5.2 GB) is a stronger candidate if your app and context leave enough free VRAM.

THE FULL GUIDE

Here is what matters, why it matters and how to check it on your own setup.

01

Start with the smaller card-friendly model

Try Qwen3 4B Q4_K_M (about 2.6 GB) first. If it runs stably and you need more capability, test Qwen3 8B Q4_K_M (about 5.2 GB). The 8B model's file size leaves about 2.8 GB of the card's advertised capacity for context, temporary buffers and display use, before accounting for runtime details.

Lower-memory test

ollama run qwen3:4b-q4_K_M

Larger comparison

ollama run qwen3:8b-q4_K_M
02

What community benchmarks say

An RTX 3060 12 GB community benchmark repository reports measurements across several quantizations and real GPUs. A third-party dataset lists an RTX 3060 12 GB running Qwen3 8B with llama.cpp at a 16K context. These are useful adjacent data points, but neither proves an 8 GB card will fit the same settings; the RTX 3060 has four more gigabytes of VRAM.

03

Avoid assuming 8 GB means an 8 GB model

The model file is only part of runtime memory. Context, image processing, KV cache, GPU backend and monitor use can push the total above the card limit. Start with 4K context and check the app's current VRAM use.

04

Verify on your card

  • Identify the exact GPU and its available VRAM, not only the product family.
  • Run Qwen3 4B first, then check whether the runtime reports GPU layers loaded.
  • Try Qwen3 8B only after the smaller model works and memory remains available.
  • Test a short coding or writing prompt and note answer quality, speed and peak VRAM.
  • If the model offloads heavily or fails, use a smaller quantization or model size.
PLAIN ENGLISH

Words you may see in this guide

RAM
The working memory used by your computer and apps.
VRAM
The memory on a dedicated graphics card. Shared graphics may use system RAM.
Context
The text and other input the model can use for its current answer.
Quantization
A way to make model weights smaller. It can change memory use and answer quality.
Engine
The software that loads the model and runs it.
RAG
A workflow that finds useful document passages before asking a model to answer.
Benchmark
A repeatable test. A result is useful only when you know the setup and task.
RESEARCH SOURCES

Official facts and real user evidence

Official documentation supports product and model facts. Community discussions show real setups, failures, and questions. A community result is supporting evidence, not a promise that another computer will perform the same way.

YOUR NEXT STEP

Watch a real test. Check what fits your computer.

See practical local AI tests from DeviceTerra, then use MamiLens to build a hardware-aware shortlist for your own setup.

This guide is educational. Model software, licenses, and hardware support can change. Check official sources before an important deployment.