Ollama VPS
for private LLMs on CPU

What to expect from an LLM on a CPU VPS

None of our plans include a GPU, so Ollama runs models on the CPU. That works well for small quantized models in the 1B to 8B range, and it does not work for large models with tens of billions of parameters. Speed is modest: a few tokens per second on a 7B or 8B model, faster on 3B and 4B models, depending on the number of vCPU.

That speed suits background work better than live chat. Classifying support tickets, extracting fields from text, summarising documents overnight, tagging content or generating embeddings are jobs where nobody watches the text appear. For an interactive assistant, a smaller model gives a more pleasant response time, and streaming the answer token by token makes the wait feel shorter.

How much RAM each model size needs

The whole model has to fit in memory with room for the context window and the operating system. For 3B and 4B models in 4 bit quantization, 8 GB of RAM is a workable start, which is VPS-4 with 4 vCPU. More vCPU means more tokens per second, so the core count matters almost as much as memory.

For 7B and 8B models, plan on 16 to 24 GB. The weights alone take several gigabytes, and a longer context, a second model kept loaded or an embedding model next to it quickly uses the rest. VPS-6 with 16 GB is a good fit for one such model, and VPS-7 with 24 GB and 8 vCPU gives the best speed we offer.

• 8 GB RAM: 3B to 4B quantized models and embedding models.
• 16 GB RAM: one 7B to 8B quantized model.
• 24 GB RAM: 7B to 8B models with long context or a second model.

Setting up Ollama and keeping it private

Installation is manual and quick. Our step by step guide covers connecting over SSH, installing Ollama on Ubuntu, pulling a model and testing it from the command line. Ollama runs as a system service and starts again after a reboot, so the API is ready whenever your applications call it.

By default Ollama listens only on localhost, and it has no built in authentication. Keep it that way if your apps run on the same server. If you need to reach it from elsewhere, use an SSH tunnel or put it behind a reverse proxy with HTTPS and a password, and never open the Ollama port to the whole internet.

Pairing Ollama with agents and automation

Ollama becomes more useful when other tools call it. n8n can send each new form entry to a local model for classification. OpenClaw and other agents can use it for cheap subtasks while a hosted model handles harder reasoning. A Telegram bot can answer simple questions from your own documents without per message fees.

Running everything on one VPS keeps latency low and data in one place. Size the server for the model first, then add a few gigabytes for the other services. Because billing is hourly, you can try a model on a larger plan for a day, measure the speed and then settle on the plan that fits.