Storage holds the download
An SSD keeps model files after shutdown. RAM holds working data while the model runs. Leave free disk space for updates and other models. More SSD space does not replace enough RAM.
AMD Radeon RX 6700 XT 12 GB
Raspberry Pi 5 plus external RX 6700 XT; patched amdgpu kernel and llama.cpp Vulkan; not an ordinary plug-and-play desktop configuration
Chat & learning
Meet the hardware
Photographs and manufacturer images, with source credits. Select a picture to inspect the full image. The pictured variant may differ from the reviewed unit.
Processor: what does the work?
This product is a graphics card, not a complete computer. Its GPU does parallel model calculations. Your separate host PC supplies the CPU, system RAM, storage and power supply. The reviewed host details remain important when judging a result.
Read the right numbers
CPU cores and GPU compute units describe different designs. Compare the same model, engine and settings before treating one chip as faster. An NPU or Neural Engine specification does not prove that your chosen language-model application uses it.
Memory: where the model works
The memory on this graphics card is dedicated VRAM. It is separate from your host PC’s RAM. The model, working buffers and context cache need space, so the model download size alone does not prove that everything fits in VRAM.
Capacity is not the same as speed
Capacity tells you how much memory is available. Bandwidth tells you how quickly the chip can read it. A model can fit in memory and still respond slowly. Longer context also uses memory for the model’s working cache.
Learn RAM, VRAM and model size →Storage, connections and upgrades
You cannot add ordinary desktop RAM to increase this card’s VRAM. Check card dimensions, motherboard slot, power connectors, PSU capacity and cooling. Board-partner cards can differ from NVIDIA or AMD reference designs.
Connections shape the setup
Check the monitor, network and drive ports you need. USB-C is a connector shape; check its actual speed and display support. A fast network does not automatically merge two computers’ memory.
Detailed system specifications
These are manufacturer product-family specifications and options. The creator’s exact reviewed configuration is listed separately. Confirm the model number and options before buying.
| Specification | Detail | Source |
|---|---|---|
| Cpu | Host-dependent; no CPU included | Source ↗ |
| Gpu | RDNA 2 Radeon RX 6700 XT | Source ↗ |
| Memory | 12 GB dedicated VRAM | Source ↗ |
| Memory type | GDDR6 | Source ↗ |
| Storage | Host-dependent | Source ↗ |
| Power | 230 W typical board power; manufacturer 650 W PSU recommendation | Source ↗ |
| Additional Power Connector | 1x8-Pin + 1x6-Pin | Source ↗ |
| Base Frequency | 2321 MHz | Source ↗ |
| Boost Frequency | Up to 2581 MHz | Source ↗ |
| Game Frequency | 2424 MHz | Source ↗ |
| Typical Board Power (Desktop) | 230 W | Source ↗ |
| AMD Infinity Cache Technology | 96 MB | Source ↗ |
| Memory Speed | Up to 16 Gbps | Source ↗ |
| Max Memory Size | 12 GB | Source ↗ |
| Memory Interface | 192-bit | Source ↗ |
| Memory Bandwidth | Up to 384 GB/s | Source ↗ |
| Effective Memory Bandwidth | Up to 1278 GB/s | Source ↗ |
| DisplayPort™ | 1.4a | Source ↗ |
| Length | 267 mm | Source ↗ |
Software checks before you start
Confirm your operating system, driver, engine version, model format and quantization. The creator’s named test below is a starting point, not a guarantee that every model or engine works. Check support for this exact AMD or Intel GPU and OS. ROCm, Vulkan and other backends have different compatibility limits.
Plan your model and engineWatch the real test
A GPU-powered Pi for more efficient AI?
By Jeff Geerling · 2024-11-19
Primary companion provides commands and a GPU-memory screenshot.
What this test does not prove
Experimental Pi driver patch; larger models exposed inefficiencies. This does not verify current ROCm or Windows compatibility.
Terms used in this review
A plain-English reading companion for this review and its hardware specifications. Select a term to read its meaning and why it matters. These explanations are not a transcript or quotations from the creator.
24 terms
CPU and CPU cores
The CPU runs the operating system and general instructions. A core is a unit inside it that can work on instructions.
Why it matters here: More cores can help some jobs, but core counts from different chip families are not a shared speed score.
GPU
A graphics processor can do many similar calculations at once. Local AI engines can use that ability to run a model.
Why it matters here: The engine must support the exact GPU and driver. A powerful GPU is not useful to an unsupported engine.
RAM
The computer’s working memory. The operating system, applications and a running model use it.
Why it matters here: A downloaded model can fit on your SSD and still be too large to load into working memory.
VRAM
Memory dedicated to a separate graphics card.
Why it matters here: When a model fits in VRAM with space for its working context, the engine may avoid moving data through slower system memory.
GB/s and Gbps
GB/s means gigabytes per second; Gbps means gigabits per second. A byte contains eight bits.
Why it matters here: A10Gbps network number is not the same as10GB/s of memory bandwidth. Check the units before comparing specifications.
Memory bandwidth
How much data the memory system can move per second. Capacity and bandwidth describe different things.
Why it matters here: A machine may hold a very large model but generate its answers slowly. Check actual model tests.
SSD, NVMe and M.2
An SSD stores files. NVMe is a fast storage protocol. M.2 describes a module and connector format.
Why it matters here: Model downloads live on storage, while a running model needs working memory. Check supported drive sizes and slots before upgrading.
Tokens and tokens per second
A token is a small piece of text. Tokens per second describes how quickly a model produces or processes those pieces.
Why it matters here: Reading the prompt and writing the answer are different measurements. Ask which stage the reviewer measured.
Context window
The amount of text a model can work with at once, including the prompt, conversation and other supplied text.
Why it matters here: A larger setting uses more working memory and can increase loading or response time. Start with enough for your real task.
KV cache
Working memory that keeps information from earlier tokens so the model can use it while continuing an answer.
Why it matters here: Longer context can make this cache larger. Model-file size alone does not describe total memory use.
Quantization and Q4
A way to store model numbers using fewer bits. Q4 usually describes a roughly four-bit approach, with different formats and tradeoffs.
Why it matters here: Lower-bit files often need less memory, but quality and speed can change. Compare the exact file, not just the model name.
Q4_K_M and other suffixes
A particular quantization recipe used by some GGUF files. The whole suffix identifies more than just the bit count.
Why it matters here: Two files for the same model can have different size, quality and memory needs.
Parameters and “B”
Parameters are the model’s learned numbers. B means billion, so 8B means about eight billion parameters.
Why it matters here: More parameters do not guarantee a better result for every task. The model must also fit and respond at a useful speed.
GPU offload
Running some or all of a model’s work on the GPU instead of the CPU.
Why it matters here: Partial offload can allow a model to run, but it is not proof that the complete model fits in the GPU’s memory.
Inference engine
The software that loads a model and performs its calculations to produce an answer.
Why it matters here: The same hardware can behave differently with different engines, drivers and model files.
llama.cpp
A local model inference runtime, commonly used with GGUF files and supported CPU or GPU backends.
Why it matters here: Its backend and build options matter. A result from one setup does not prove all other builds perform the same.
GGUF
A model-file format used by llama.cpp and several local AI applications.
Why it matters here: Choose the model format your engine supports. A downloadable model is not automatically compatible with every application.
CUDA
NVIDIA’s software platform for GPU computing.
Why it matters here: CUDA-based tools need compatible NVIDIA hardware and software. A CUDA tutorial is not an AMD installation guide.
ROCm and Vulkan
Software technologies some engines use to run calculations on supported graphics hardware.
Why it matters here: Support depends on the GPU, engine, driver and operating system. Check the exact setup in the review.
FP8 and FP16
Number formats that use eight or sixteen bits for a stored value.
Why it matters here: Memory use and hardware support differ. A model with the same name but a different format is a different configuration to test.
Time to first token
The wait before the model starts producing its answer.
Why it matters here: A system can write quickly once started but still spend a long time reading a large document first.
RAG
Retrieval-augmented generation: finding relevant passages from documents and supplying them to a model with your question.
Why it matters here: This can help answer from your files, but you still need to check the cited passages and answer accuracy.
PCIe and GPU host
PCIe connects components such as a graphics card to a computer. The host is the rest of that computer.
Why it matters here: A GPU card alone needs a compatible motherboard, CPU, RAM, storage, power supply and cooling.
TDP and power supply rating
TDP is a chip cooling or power-design figure. The power supply rating is the amount the supply is designed to provide.
Why it matters here: Neither number alone is a measurement of the full system’s electricity use during your AI task.
Further reading: LM Studio load settings · Quantization · KV cache
What the creator tested
These observations belong to this source and configuration. Changing the model file, context or software can change the result.
Llama 3.2 3B Instruct
Engine: llama.cpp Vulkan · Quantization: Q4_K_M
Creator confirms model in RX 6700 XT VRAM and GPU inference.
Explore this model family and size in our catalog →
The catalog artifact, quantization or variant may differ from the review. This link does not extend the tested result.
Is it useful for your work?
Useful evidence for Linux experimentation; do not choose this unusual eGPU setup for a beginner appliance.
Check before buying
- Exact host RAM and storage not confirmed; no universal engine compatibility claim.
Check current price, regional availability, returns, warranty and the exact memory configuration with the seller. This guide does not guarantee a specific speed.
Community and reproducibility
No separately verified owner report is attached to this profile yet. The creator test above is the evidence currently recorded.
Have this hardware? Share the exact model file, engine version, memory, context length, speed and any setup problems.
Contribute your testMatch the exact configuration
Check memory, storage, model number, seller warranty, returns and shipping to your country. A product-family name does not identify every component inside.
View manufacturer options ↗