How to run Ollama on a CPU only VPS

Ollama runs open language models such as Gemma, Llama and Qwen on your own machine and gives them a simple HTTP API. You do not need a GPU: Ollama falls back to the processor, and small models answer at a usable speed on a few vCPU cores. This guide installs Ollama on an Ubuntu VPS, pulls a small model and shows two safe ways to use the API from outside the server.

What you need

• A VPS with Ubuntu 22.04 or 24.04. DataPasa servers have no GPU, so everything runs on the CPU. RAM decides which models fit, and more vCPU cores make answers faster. We recommend VPS-4 (4 vCPU, 8 GB RAM, 128 GB NVMe): it runs models up to about 8 billion parameters, and four cores keep small models responsive. More options are on the Ollama VPS page.
• SSH access to the server, see how to connect via SSH.
• Realistic expectations. On a CPU a 1B to 4B model is fine for chat, classification, summaries and bots. Large models run, but slowly.

How much RAM you need

The model must fit in RAM together with the system and Ollama itself. As a rule of thumb, take the download size of the model and add 1 to 2 GB. Download sizes below are from the Ollama library for the default 4 bit quantized tags.

Model example	Download size	Plan that fits
gemma3:270m, qwen3:0.6b	0.3 to 0.5 GB	VPS-2 (1 vCPU, 2 GB)
gemma3:1b, llama3.2:1b, qwen3:1.7b	0.8 to 1.4 GB	VPS-3 (2 vCPU, 4 GB)
llama3.2:3b, qwen3:4b, gemma3:4b	2.0 to 3.3 GB	VPS-4 (4 vCPU, 8 GB)
qwen3:8b	5.2 GB	VPS-4 (4 vCPU, 8 GB)
gemma3:12b, qwen3:14b	8.1 to 9.3 GB	VPS-6 (6 vCPU, 16 GB)
gemma3:27b	17 GB	VPS-7 (8 vCPU, 24 GB), slow on CPU

Context uses memory too

A longer context window needs more RAM on top of the model. If a model barely fits, keep the default context length instead of raising it.

Step 1. Connect and prepare the server

ssh root@YOUR_SERVER_IP
apt update && apt upgrade -y

The Ollama installer downloads a .tar.zst archive, so install zstd first:

apt install -y zstd

Step 2. Install Ollama

Run the official install script:

curl -fsSL https://ollama.com/install.sh | sh

The script creates an ollama system user, installs a systemd service called ollama and starts it. On a server without a graphics card it prints a warning that no NVIDIA or AMD GPU was detected and Ollama will run in CPU only mode. That is expected.

Check the version and the service:

ollama -v
systemctl status ollama

Step 3. Pull and run a model

Download a small model. gemma3:1b is a good first test on any plan from VPS-3:

ollama pull gemma3:1b

Chat with it in the terminal:

ollama run gemma3:1b

Type a question and press Enter. Type /bye to exit. Useful commands:

ollama list
ollama ps
ollama rm gemma3:1b

ollama list shows downloaded models, ollama ps shows models loaded in memory, and ollama rm deletes a model. Models are stored in /usr/share/ollama/.ollama/models.

Step 4. Use the API on localhost

The Ollama API listens on 127.0.0.1:11434, only inside the server. Test it with curl:

curl http://localhost:11434/api/chat -d '{
  "model": "gemma3:1b",
  "messages": [{ "role": "user", "content": "Why is the sky blue?" }],
  "stream": false
}'

Apps on the same server, such as a bot, n8n or OpenClaw, can call http://localhost:11434 directly. Nothing else is needed.

Do not open port 11434 to the internet

The Ollama API has no authentication. If you set OLLAMA_HOST=0.0.0.0 and open the port, anyone can use your CPU and your models. Use one of the two options below instead.

Step 5. Option A: an SSH tunnel for personal use

If only you need the API, forward the port over SSH. Run this on your own computer:

ssh -L 11434:127.0.0.1:11434 root@YOUR_SERVER_IP

While the session is open, http://localhost:11434 on your computer reaches Ollama on the server. Desktop chat apps and code editors that support Ollama can use this address as is.

Step 6. Option B: Caddy with HTTPS and a password

For access from other servers or services, put Ollama behind Caddy with basic authentication. You need a domain, for example ollama.example.com, with an A record pointing to the server IP.

Install Caddy from its official repository:

apt install -y debian-keyring debian-archive-keyring apt-transport-https curl
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | tee /etc/apt/sources.list.d/caddy-stable.list
chmod o+r /usr/share/keyrings/caddy-stable-archive-keyring.gpg
chmod o+r /etc/apt/sources.list.d/caddy-stable.list
apt update
apt install -y caddy

Create a password hash. The command asks for the password and prints the hash:

caddy hash-password

Replace the contents of /etc/caddy/Caddyfile with:

ollama.example.com {
    basic_auth {
        apiuser PASTE_THE_HASH_HERE
    }
    reverse_proxy 127.0.0.1:11434 {
        header_up Host localhost:11434
    }
}

The header_up line mirrors the proxy example in the Ollama documentation, which passes localhost:11434 as the host. Apply the config and open the web ports:

systemctl reload caddy
ufw allow OpenSSH
ufw allow 80/tcp
ufw allow 443
ufw enable

Caddy gets a Let's Encrypt certificate on the first request. Test from any machine:

curl -u apiuser:YOUR_PASSWORD https://ollama.example.com/api/tags

Ollama itself still listens only on localhost, so the password protected domain is the only way in.

Keep models in memory longer

By default Ollama unloads a model 5 minutes after the last request, and the next request waits while it loads again. On a server with enough RAM you can keep it loaded longer:

systemctl edit ollama.service

Add under [Service]:

[Service]
Environment="OLLAMA_KEEP_ALIVE=24h"

Then apply the change:

systemctl daemon-reload
systemctl restart ollama

Update Ollama

Run the install script again. It replaces the binary and keeps your models:

curl -fsSL https://ollama.com/install.sh | sh

Logs are available with journalctl -e -u ollama.

Summary

Ollama installs with one command and runs well on CPU for models up to a few billion parameters. Keep the API on localhost, use an SSH tunnel for yourself or Caddy with a password for other services, and choose RAM by the model size. Plans for this setup are on the Ollama VPS page.