When sizing a VPS for self-hosted LLM inference, RAM is what decides whether a model runs at all, and vCPU count only decides how fast it runs once it does. Get the RAM wrong and no amount of extra cores fixes it. This piece is the sizing reference the earlier cost breakdown and Ollama setup guide both point back to.
Why RAM is the real constraint
A language model’s weights have to sit in memory for inference to happen. If they don’t fit, the process either fails to load, or the OS starts swapping to disk, which is dramatically slower than RAM and effectively makes the model unusable rather than just slow. vCPU count affects tokens-per-second once the model is loaded; it doesn’t change whether it loads in the first place.
The rule of thumb
For a quantized model, a workable estimate is:
RAM needed ≈ (parameters in billions) × (bytes per parameter at your quant level) + 1-2 GB overhead
Q4 quantization (the common default for running models on a VPS) works out to roughly 0.5-0.6 GB per billion parameters. So a 7B model at Q4 needs roughly 4-5 GB, and an 8B model needs roughly 5-6 GB. Add headroom for the OS and whatever else is running on the box, and the practical minimums look like this:
Matching models to Hetzner CAX tiers
| Tier | RAM | Realistic model size (Q4) |
|---|---|---|
| CAX11 (2 vCPU, 4 GB) | 4 GB | Too tight for reliable inference alongside the OS |
| CAX21 (4 vCPU, 8 GB) | 8 GB | 3-4B models comfortably, e.g. Qwen3.5 4B |
| CAX31 (8 vCPU, 16 GB) | 16 GB | 7-8B models, e.g. Llama 3.3 8B or Qwen 3 7B |
| CAX41 (16 vCPU, 32 GB) | 32 GB | 13B-class models, or a 7-8B model with room for a longer context window |
Quantization changes the math
Q4 is the common starting point because it roughly halves memory use compared to Q8 with a modest quality trade-off, and it’s a quarter the size of the unquantized F16 weights. If a model feels noticeably worse than you expected on a given VPS, the quant level is often why — dropping from Q8 to Q4 to fit a box saves RAM but does cost some output quality, so it’s worth testing both if you have the headroom.
What vCPU count actually buys you
Once the model fits in RAM, more vCPUs mean faster generation, not a bigger model. Ollama and llama.cpp both parallelize inference across available cores, so a CAX31’s 8 vCPUs will generate noticeably faster than a CAX21’s 4, even running the identical model. If speed matters more than model size for your use case, it can be worth over-provisioning vCPU relative to what the RAM tier would suggest.
Don’t rely on swap
It’s tempting to add swap space to squeeze a slightly-too-big model onto a smaller VPS. In practice this turns “doesn’t fit” into “technically loads but is too slow to use,” since disk I/O is orders of magnitude slower than RAM for the random access pattern inference needs. If a model needs swap to load, the honest fix is a bigger RAM tier, not more swap.