Local AI buying guide
What fits in 128GB for local AI? A practical memory budget
Estimate model weights, context and runtime headroom before buying a 128GB laptop or mini PC. Capacity calculations are not speed benchmarks.
By Laptop Deals UK · Updated
Start with usable memory, not the label on the box
A 128GB computer does not give a language model an empty 128GB GPU. The operating system, other applications, graphics allocation, runtime buffers and context cache all matter. On a discrete-GPU PC, 128GB system RAM is not 128GB dedicated VRAM; moving work to the CPU can change performance substantially.
The useful question is: can your exact model file, chosen context and runtime fit within the usable allocation, with headroom? A model loading successfully does not prove acceptable response speed.
First estimate: weights only
The idealised calculation is parameters × bits per parameter ÷ 8. The table converts the resulting bytes to GiB (2³⁰ bytes). These are arithmetic lower-bound illustrations, not measured model-file sizes or fit guarantees. Quantisation scales, metadata, mixed-precision tensors and format overhead add memory.
| Total parameters | Ideal 4-bit weights | Ideal 8-bit weights | Ideal 16-bit weights |
|---|---|---|---|
| 8 billion | 3.7 GiB | 7.5 GiB | 14.9 GiB |
| 32 billion | 14.9 GiB | 29.8 GiB | 59.6 GiB |
| 70 billion | 32.6 GiB | 65.2 GiB | 130.4 GiB |
| 120 billion | 55.9 GiB | 111.8 GiB | 223.5 GiB |
| 200 billion | 93.1 GiB | 186.3 GiB | 372.5 GiB |
For example, ideal 70-billion-parameter weights at four bits occupy 35 billion bytes, or about 32.6GiB, before overhead. At 16 bits the same parameter count needs about 130.4GiB for weights alone, already beyond a nominal 128GB machine. Use the downloaded file's actual size and the runtime's allocation report for the next step.
A mixture-of-experts model's active parameter count is not its total stored weight count. Unless your specific runtime supports another loading/offload strategy, budget for the full resident model rather than only the experts used by one token. Lower precision may also affect output quality; the smallest file is not automatically the right model.
Then add context, concurrency and working buffers
During generation the KV-cache stores attention keys and values. Its size depends on architecture, retained context, KV-head count, cache precision and the number of simultaneous sequences. A long-context coding session or several users can need much more memory than a short single-user demonstration.
As a deliberately hypothetical dense-attention example, 80 layers × 8 KV heads × 128 values per head × 32,768 cached tokens × 2 bytes × 2 for keys and values is 10GiB for one sequence. This is not a prediction for every 70B model: grouped/sliding attention, cache quantisation and runtime implementation can change it. Other buffers and multimodal inputs are additional.
Before buying, test the actual runtime at your intended context, batch and concurrency. Leave space for the OS and ordinary applications rather than planning to use every last byte. Check both prompt-processing memory peaks and generation, not just the memory shown immediately after loading.
How the three 128GB desktop routes differ
- Ryzen AI Max+ / Strix Halo: AMD documents up to 96GB graphics allocation on 128GB systems. Check the exact machine's firmware, driver and runtime. That is not 128GB of dedicated VRAM, and a platform limit is not proof that every device exposes it.
- Mac Studio / Apple Silicon: CPU and GPU share memory. Check the runtime's usable allocation and leave macOS headroom; do not transplant AMD's 96GB figure to a Mac.
- DGX Spark / GB10: NVIDIA specifies 128GB unified system memory. Model, cache and system use still share it. Verify Linux-on-Arm package support and required CUDA operators as well as capacity.
Compare the exact UK desktop configurations. A 128GB laptop also has different portability and cooling compromises from a permanent desktop; see the 128GB laptop shortlist.
A four-step buying test
- Choose the model and format. Write down the total parameter count, actual file size, quantisation and any vision/audio components. Verify the model's licence and runtime compatibility.
- Set a real workload. Specify context, concurrent users, batch size and whether retrieval or other applications run alongside it.
- Measure the working allocation. Include weights, cache, buffers and the OS. Use a supported machine or a reproducible report for the exact setup; do not substitute an NPU TOPS figure.
- Check usability and the UK total. Prompt processing, generation, cooling, noise, warranty and delivered cost can matter even when a model fits. There is no universal tokens-per-second figure for a memory capacity.
These calculations concern inference. Full training and fine-tuning add different requirements, including activations, gradients and optimiser state. A model fitting for inference is not a promise that it can be fully trained in the same memory.
Primary references
- AMD: variable graphics memory and model-size guidance
- NVIDIA: DGX Spark hardware and unified memory
- MLX: Apple Silicon shared-memory framework
- llama.cpp: supported backends, quantisation and runtime options
Arithmetic examples above are illustrative, not hands-on benchmarks. Compare desktop configurations · All Local AI buying guides