If you’re paying per token for GPT-4o mini or Claude Haiku and wondering whether a self-hosted LLM on a cheap ARM VPS could do the job for a flat monthly fee, the short answer is: it depends on your volume, not your ideology. Below a few thousand requests a month, API pricing usually wins outright. Above a certain threshold, and if you can tolerate slower responses and a real quality step-down, running a self-hosted LLM on a modest VPS starts to make financial sense.
Self-Hosted LLM vs API: The Two Cost Models
An API charges per token, metered to the request. A VPS charges a flat monthly fee no matter how many requests you send. That means the API is cheaper at low volume and the VPS is cheaper at high volume, so the real question is where the crossover point sits.
What ARM VPS capacity actually costs right now (Hetzner CAX line, August 2026)
| Tier | vCPU | RAM | Price/mo | What fits |
|---|---|---|---|---|
| CAX11 | 2 | 4 GB | €4.49 | Too tight for reliable LLM inference alongside the OS |
| CAX21 | 4 | 8 GB | €7.99 | A quantized 3-4B model, e.g. Qwen3.5 4B at Q4 |
| CAX31 | 8 | 16 GB | €15.99 | A quantized 7-8B model: Llama 3.3 8B, Mistral Small 3, Qwen 3 7B |
netcup’s Ampere Altra Max ARM VPS line is a comparable option when it’s back in stock, if you already have an account there. For a deeper look at how ARM stacks up against x86 for VPS workloads generally, see our ARM vs x86 VPS comparison.
What the API side costs
- GPT-4o mini: $0.15 per million input tokens, $0.60 per million output tokens
- Claude Haiku 4.5: $1.00 per million input tokens, $5.00 per million output tokens
Working the break-even
Take a CAX31 at €15.99/month, roughly $17.50, running a self-hosted LLM at the 7-8B size, compared against GPT-4o mini, the cheapest frontier-adjacent API on the market.
Assume a fairly typical request shape: 500 input tokens, 500 output tokens. On GPT-4o mini that works out to roughly $0.000375 per request.
To spend $17.50/month at that rate, you’d need roughly 46,000 requests a month, about 1,500 a day, before the VPS’s flat fee becomes the cheaper option.
Against Claude Haiku 4.5, the same request costs about $0.003, so the break-even drops to roughly 5,800 requests a month, under 200 a day.
That’s the real shape of the decision: self-hosting rarely beats GPT-4o mini on cost alone unless you’re at real volume, but it can beat Haiku-class pricing surprisingly fast.
What the math doesn’t show you
- Speed. CPU-only inference on an ARM VPS, with no GPU, is meaningfully slower than a hosted API. Expect noticeably longer generation times, especially on longer outputs. If you need sub-second responses at scale, this isn’t that.
- Quality. A self-hosted LLM at 7-8B parameters is not a frontier model. It’s good enough for structured, narrow tasks like summarization, classification, and templated generation, and noticeably weaker at open-ended reasoning than GPT-4o mini or Haiku.
- No autoscaling. The VPS handles one request at a time reasonably; if usage spikes, latency degrades instead of a provider absorbing it for you.
- Your time. Someone has to set up Ollama or llama.cpp, manage the model files, monitor the box, and handle updates. That’s not free, even if the invoice says so.
Where self-hosting genuinely wins
- Privacy-sensitive workloads where sending data to a third-party API isn’t an option, regardless of cost.
- Steady, predictable, high-volume workloads, such as background jobs, batch classification, and internal tooling, where a smaller model handles the task fine and the flat fee beats metered pricing at your actual volume.
- Learning and experimentation, where marginal cost per request isn’t the point at all.
Where it doesn’t
- Low-to-moderate volume, general-purpose chat: the API is both cheaper and better.
- Anything customer-facing where response latency matters.
- Tasks that need frontier-level reasoning quality.
If you want to try it
Start on a CAX21 (€7.99/mo) with a 4B model like Qwen3.5 to get comfortable running a self-hosted LLM; Ollama makes this close to a one-command install. Only move up to a CAX31 and an 8B model once you’ve confirmed the smaller model doesn’t already do what you need. The RAM headroom, not the vCPU count, is what actually gates which model you can run.