5 Things to Know About RAM for Local LLMs

32GB is the minimum for practical use If you want to run anything larger than tiny 7B parameter models, 32GB is where things actually start working. I would not build a machine with less.

16GB only works for the smallest models 7B models can technically run on 16GB, but only with quantized versions and after closing everything else. You will likely hit swap memory.

13B models are unusable on 16GB Forget running 13B parameter models on 16GB. You will end up in swap city, making your LLM slower than reading a PDF.

32GB handles the sweet spot With 32GB, you can run 7B models comfortably at 4-bit quantization. Even 13B models fit with room to spare for your OS and browser.

Larger models still need more RAM Higher parameter models like 30B or 70B require 64GB or more. But 32GB unlocks practical local inference without constant frustration.