Pick Hardware That Can Actually Run Models

A 65GB file does not fit in 16GB of memory. That sentence settles most hardware questions before you buy anything, and the rest of this part is about getting to the number on the left before you spend money on the number on the right.

Every figure below comes from a published page: the tag tables in Ollama's model library, the measurements in llama.cpp's quantize README, and Ollama's own hardware-support page. Where a page declines to publish a number, that gap is called out rather than filled in.

Memory is the wall, and one multiplication tells you where it sits

A model file is a pile of weights. The size on disk is the parameter count times the average bits spent per weight, divided by eight. That is the entire calculation.

llama.cpp publishes the measured bits per weight for every quantization it ships, benchmarked on Llama-3.1-8B. Four rows from that table:

So an 8B model at half precision is roughly 15GB, at 8-bit roughly 8GB, at the common 4-bit K-quant roughly 4.6GB. Double the parameters and every row doubles with it. That is why a 70B model at Q4_K_M lands near 40GB and a 235B model lands well past any single consumer card.

The weights are the floor, not the total. Your operating system, your browser, and the key-value cache for whatever context you configure all sit in the same memory. Ollama does not publish a formula for that overhead, so treat the file size as a minimum you must clear with room left over.

Read the tag table instead of trusting a hardware guide

Ollama's library publishes an exact download size for every tag, and that is the most reliable hardware signal available anywhere. The qwen3 tags page shows the same model at three precisions:

Those three 8B rows are the arithmetic from the previous section, published by a vendor with no reason to flatter your laptop. 16GB, 8.9GB, 5.2GB. The ratios track the 16.0005, 8.5008 and 4.8944 bits-per-weight figures llama.cpp measured.

Gemma 4 gives you the same ladder at larger sizes. The gemma4 tags page lists gemma4:12b-it-q4_K_M at 7.6GB, gemma4:26b-a4b-it-q4_K_M at 18GB, and gemma4:31b-it-q4_K_M at 20GB, with the bf16 versions of the same three at 24GB, 52GB and 63GB.

One number worth staring at: gemma4:26b-a4b-it-qat is 16GB, smaller than the 18GB q4_K_M build of the same weights. QAT means quantization-aware training, and it is a different production route from post-hoc GGUF quantization. Part 4 of this series decodes those labels one piece at a time.

The one model card that publishes a memory floor

Ollama's library pages give you download size and nothing else. No minimum RAM, no minimum VRAM, no per-tag guidance. The FAQ does not fill the gap either.

The gpt-oss card is the exception, and it is worth quoting because it is the only vendor statement of its kind in the library. OpenAI's weights are described as quantized to "4.25 bits per parameter" using MXFP4, which allows "the smaller model to run on systems with as little as 16GB memory" and "the larger model to fit on a single 80GB GPU".

Now put that against the tags page for the same model. gpt-oss:20b downloads at 14GB. gpt-oss:120b downloads at 65GB. So the small model is claimed to work with about 2GB of headroom over its own weights, while the large one is given about 15GB. Those two ratios are not the same, nothing on either page reconciles them, and neither figure is broken out into weights plus cache plus overhead.

Do not resolve that by picking whichever number flatters your machine. Read it as the vendor telling you 16GB is tight rather than comfortable for a 14GB download, and budget accordingly.

What Ollama says it supports, by chip

Capacity is one question. Whether the runtime will use your GPU at all is a separate one, and Ollama's hardware-support page answers it precisely.

• NVIDIA. "Ollama supports Nvidia GPUs with compute capability 5.0+ and driver version 550 and newer." There is a second clause that catches older cards: "Nvidia GPUs with compute capability 5.0 through 6.2 require driver version 570 or newer." The published compatibility table runs from 5.0 (GTX 750 Ti) up to 12.1 (GB10 DGX Spark).
• AMD. "Ollama requires the AMD ROCm v7 driver on Linux." On Windows it "requires an AMD ROCm v7 / HIP7-capable driver stack". Supported families listed include Radeon RX, Radeon AI PRO, Radeon PRO, Ryzen AI and Instinct MI100 through MI350X.
• Apple. The macOS page lists "Apple M series (CPU and GPU support)" against x86 processors at CPU only. An Intel Mac is a CPU-only machine for this purpose no matter what is in it.

Multi-GPU selection uses the vendors' own environment variables: CUDA_VISIBLE_DEVICES for NVIDIA, ROCR_VISIBLE_DEVICES for AMD, and GGML_VK_VISIBLE_DEVICES to pick specific Vulkan devices.

What the hardware page does not say is what happens when a model is larger than your VRAM. It publishes no VRAM minimums and no layer-offloading rules. The troubleshooting page only notes that models fall back to CPU when GPU initialization fails or times out during discovery, which is a different failure from simply not fitting.

The practical check is a command rather than a spec sheet. After a model loads, run:

ollama ps

The processor column reports 100% GPU, 100% CPU, or a split such as 48%/52% CPU/GPU. Anything other than full GPU means part of the work is happening in slower system memory, and that is your signal to drop to a smaller tag.

Context length is charged on top of the weights

Ollama's context-length page ties the default directly to how much VRAM you have. It assigns 4k tokens under 24GB of VRAM, 32k between 24GB and 48GB, and 256k at 48GB or more. For web search, agents and coding assistance it recommends "at least 64000 tokens".

Read that as a hardware requirement in disguise. A 24GB card gets 4k tokens by default, and getting to the recommended 64k means paying for the cache out of the same pool that is already holding the weights. The page states the tradeoff plainly: expanding context length increases memory demands proportionally, and you should verify sufficient VRAM before making the change.

So the honest capacity question is not "does the 20GB file fit in 24GB". It is "does the 20GB file fit in 24GB alongside the context I actually intend to use". Frequently it does not.

Three budgets and the biggest tag each one holds

Sizes below are the published download figures from the tag pages named above. They are file sizes, not certified memory floors, because no such floors exist for these models.

Two things that table cannot tell you. First, a Mac's unified memory is shared with everything else the machine is doing, so the number in the About panel is not the number available to a model. Second, an 8GB discrete GPU inside a 32GB machine is an 8GB budget for full-GPU placement, not 32GB, and ollama ps is what tells you which of those two you actually got.

If your machine lands in the 8GB row, the useful move is to stop shopping for a bigger model and start with a 1.7b or 4b tag to confirm the whole pipeline works. DeepSeek, Qwen and Gemma all ship small variants for exactly this. Then read the coding-model case in part 3, where Laguna XS.2 ships its smallest build at 19GB and the model card's only published memory figure is a Mac with 36GB of RAM.

Buying advice, since people ask: memory capacity beats memory bandwidth beats raw compute, in that order, for the specific job of getting a model to load at all. A card that cannot hold the weights runs them at CPU speed no matter how fast it is.