An RTX 3060 12 GB is strongest with 7B to 14B Q4 models that fit fully or almost fully in VRAM. Full GPU fit usually matters more than chasing a larger model.
Use the explanations below when you want to know why each step matters.
The short answer
Start with an 8B model when you want the fastest normal chat. Test a 12B or 14B Q4 model when answer quality matters more. Do not choose a 30B dense model as your first daily model because much of it must use system RAM.
NVIDIA lists the desktop RTX 3060 with 12 GB of GDDR6 memory. Hardware references report a 192-bit bus and about 360 GB per second of memory bandwidth. The large VRAM capacity keeps this older card useful for local models.
Usually fits fully with room for context
A strong test if the exact file fits
Often needs system RAM and becomes slower
What full GPU fit means
When the model and its working memory fit in VRAM, the GPU can do most of the work without repeatedly waiting for normal RAM. In simple words, the model stays on the fast desk beside the worker.
If the file is too large, llama.cpp and related engines can keep some layers in system RAM. The model may still run, but it is not the same experience as a full GPU load.
This model can sit fully inside your selected GPU. That usually gives the fastest and most stable starting point.
Why speed numbers vary
Community benchmark collections show useful RTX 3060 results, but the exact rate changes with model architecture, quantization, context, driver, cooling, and engine. MamiLens should never promise one fixed token rate from the GPU name alone.
An 8B Q4 model should feel responsive when fully loaded. A 14B Q4 may still be comfortable, but it needs an exact fit check and real speed test.
More weights require more work
Avoids slow transfers to system RAM
Long prompts need more memory and time
The backend path must be verified
Choose by job
- General writing and chat: current Qwen or Llama instruction models around 8B.
- Stronger reasoning and multilingual work: a 12B to 14B Q4 model that fits safely.
- Coding: a code-tuned 7B to 14B model tested on your repository tasks.
- Images and screenshots: a vision model whose text file and image projector are supported.
- Private PDFs: an 8B to 14B writer plus an embedding and retrieval system.
Use a short context first
A model may advertise 128K context, but 128K is not a sensible default on every 12 GB card. The KV cache grows as the conversation grows. Long prompts also take longer to process.
Begin at 8K or 16K. Increase only when a real task needs more text. Use retrieval for large document collections instead of placing every page in one prompt.
Verify the GPU path
- Open Windows Task Manager, select Performance, and then GPU.
- Run one prompt after the model loads.
- Check dedicated GPU memory, not shared GPU memory.
- Read the engine log for layers sent to the GPU.
- Compare against a CPU-only run if possible.
- Do not call the setup GPU accelerated until the evidence shows real GPU work.
Should you upgrade?
If you already own an RTX 3060 12 GB, keep it until a real task proves that 12 GB is too small or the speed is too low. It remains a useful one-person local AI card.
When buying used hardware, compare price, power, warranty, cooling, and VRAM. Newer cards can be faster, but an 8 GB card may block a model that fits on the 3060.
Official facts and real user evidence
Official documentation supports product and model facts. Community discussions show real setups, failures, and questions. A community result is supporting evidence, not a promise that another computer will perform the same way.
Find a model your computer can run.
MamiLens checks your hardware and shows a careful starting point.
Run the free compatibility check →This guide is educational. Model software, licenses, and hardware support can change. Check official sources before an important deployment.