On this page
Start with the job and budgetUnderstand the memory that mattersRead benchmark evidence properlyTreat price as a dated, local observationUse this before payingSourcesChoose hardware by the largest model and task you actually need, then compare total system cost, memory headroom, software support, power, and measured results. Do not shop by AI TOPS alone.
Here is what matters, why it matters and how to check it on your own setup.
Start with the job and budget
Write down the task first: short writing, coding, PDF search, image understanding, or several people using one machine. Then decide how much waiting is acceptable and what you can spend on the complete system, not only the GPU.
For a first personal assistant, a computer you already own with 16 GB RAM can be a sensible trial. If you want a GPU, the used RTX 3060 12 GB is a common budget candidate, but local prices and card condition vary. For large models, compare systems with 64–128 GB unified memory and high memory bandwidth.
Understand the memory that matters
Dedicated VRAM is memory on a graphics card. Unified memory is one pool shared by the CPU and GPU, as on Apple Silicon and AMD Ryzen AI Max systems. Neither pool is fully available to the model: the operating system, app, context, and other programs use part of it.
A model download size is not a safe memory budget. Runtime buffers and conversation context need additional room. When a model exceeds VRAM, it may still run by moving work to system RAM, often with a large speed penalty.
| Hardware memory | Practical first experiments | Purchase meaning |
|---|---|---|
| 8 GB system RAM, CPU only | Very small 1B–3B quantized text models | Try an existing computer first; upgrade RAM or whole system only if a measured need appears |
| 16 GB system RAM, CPU or Apple base model | Small models; selected 4B–8B quantized models with careful context | Good entry point for learning and one-user tasks, not a guarantee of speed |
| 8 GB dedicated GPU VRAM | Small to mid-size models, typically around 7B–9B at low quantization | Check GPU support and leave room for context and desktop display |
| 12 GB dedicated GPU VRAM | Many 7B–14B quantized experiments, exact fit depends on model/context | Strong value can come from used cards; inspect condition and power requirements |
| 24–32 GB dedicated GPU VRAM | Larger dense models or faster smaller ones, depending on quantization | High performance, but total PC, PSU, cooling, and card cost are substantial |
| 64–128 GB unified memory | Larger models can load locally, generally with speed trade-offs | For users who need model capacity and prefer compact systems over discrete GPU builds |
Read benchmark evidence properly
Tokens per second measures how quickly generated text appears; it is not answer quality. A useful comparison names the exact model, quantization, engine, context, prompt/output length, hardware configuration, and whether the result is measured, community-submitted, or estimated.
For example, a public 2026 RTX 3060 test reports Qwen2.5 14B Q4_K_M around 26–29 tokens per second under its stated setup. That result is evidence that one configuration worked; it does not predict every RTX 3060, model version, or long-context workload. A 2026 16 GB Mac mini community benchmark project reports task accuracy as well as resource use, which is more useful for classification than a speed-only score.
Treat price as a dated, local observation
Manufacturer launch prices describe a region and a date. Retail prices change, stock can disappear, and import costs, shipping, taxes, warranty, and local voltage or repair access affect what you really pay.
In a November 2026 US sold-listing snapshot, the RTX 3060 12 GB showed a $340 median from 296 sales. Another tracker reported about $300 from only three qualified sales in September. The disagreement is a useful reminder: show sample size and collection date, and check your own market before buying. NVIDIA's RTX 5090 official starting price is $1,999; the DGX Spark founder MSRP was changed to $4,699 in February 2026. These figures are not local checkout quotes.
Use this before paying
- Check the exact memory capacity and whether it is dedicated or unified.
- Check model/runtime support for the operating system and GPU backend.
- Price the entire system: GPU, case, motherboard, power supply, RAM, storage, shipping, and power backup where needed.
- Use local sold listings when possible. For used GPUs, test under load and check temperature, fan noise, ports, warranty, and return terms.
- Choose two models and one real task. Run a repeatable test before committing to a costly upgrade.
Words you may see in this guide
- RAM
- The working memory used by your computer and apps.
- VRAM
- The memory on a dedicated graphics card. Shared graphics may use system RAM.
- Context
- The text and other input the model can use for its current answer.
- Quantization
- A way to make model weights smaller. It can change memory use and answer quality.
- Engine
- The software that loads the model and runs it.
- RAG
- A workflow that finds useful document passages before asking a model to answer.
- Benchmark
- A repeatable test. A result is useful only when you know the setup and task.
Official facts and real user evidence
Official documentation supports product and model facts. Community discussions show real setups, failures, and questions. A community result is supporting evidence, not a promise that another computer will perform the same way.
- Arynwood: RTX 3060 12 GB model tests and benchmark setup ↗
- Mac mini M4 16 GB task benchmark dataset ↗
- llama.cpp official benchmark and runtime project ↗
- GPU Sold: live US eBay sold-price tape and sample counts ↗
- NVIDIA Developer Forum: DGX Spark MSRP update ↗
- NVIDIA RTX 5090 specifications and starting price ↗
Watch a real test. Check what fits your computer.
See practical local AI tests from DeviceTerra, then use MamiLens to build a hardware-aware shortlist for your own setup.
This guide is educational. Model software, licenses, and hardware support can change. Check official sources before an important deployment.
