Ollama Hosting: Run Open LLMs on a Dedicated GPU

Ollama has become the default way to run open-weight LLMs — Llama, Mistral, Qwen, Gemma — with one command instead of a CUDA yak-shave. But the laptop it runs on closes at night, shares its VRAM with your browser, and can't serve your teammates. Ollama hosting on a cloud GPU fixes all three. SnapDeploy gives you a one-click Ollama template on your own dedicated NVIDIA GPU — a T4 (16 GB VRAM) at a flat $499/month or an A10G (24 GB VRAM) at $999/month — running 24/7 with a public HTTPS URL. This guide covers which models fit where, how the template works, and the honest cost math against per-second serverless GPUs.

What Is Ollama?

Ollama is an open-source LLM server that wraps model downloading, quantization handling, GPU memory management, and inference behind a simple HTTP API. You ask it to pull a model by name — llama3.1:8b, mistral:7b, qwen2.5:14b — and it fetches the quantized weights from its registry and serves them on port 11434. It speaks its own API (/api/chat, /api/generate, /api/pull) and also exposes OpenAI-compatible endpoints under /v1, so most existing OpenAI SDK code works against it by changing the base URL.

That combination — self-host any open LLM, keep your data on your own infrastructure, keep OpenAI-compatible client code — is why "self-host Llama" increasingly just means "run Ollama somewhere". The question is where.

Why Hosted Ollama Beats Your Laptop

• VRAM. A dedicated T4 gives your models 16 GB of VRAM with nothing else competing for it — no Chrome, no display compositor, no Docker Desktop. The A10G gives 24 GB, which changes which models you can run at all.
• Uptime. A laptop sleeps, reboots for updates, and leaves the Wi-Fi. A hosted Ollama server answers at 3 a.m., keeps weights warm in VRAM, and survives your commute.
• Teams and apps. An LLM on localhost serves exactly one person. A hosted endpoint with a public HTTPS URL serves your whole team, your staging environment, and your product — the same way an OpenAI key would, except the model and the data stay yours.

Which Models Fit: 16 GB T4 vs 24 GB A10G

Ollama serves 4-bit quantized models by default (Q4_K_M), which is what makes mid-size LLMs practical on single GPUs. Rule of thumb: a Q4 model needs roughly 0.6 GB of VRAM per billion parameters for weights, plus headroom for the KV cache that grows with context length. Approximate figures for popular models:

Translation: 8B-class models are the T4's sweet spot — fast, cheap, plenty of KV-cache headroom. 14B and quantized 32B models want the A10G. And 70B doesn't fit either tier — we'd rather say that plainly than sell you a subscription that disappoints. If your quality bar demands 70B, you need 48 GB-class or multi-GPU hardware.

Step-by-Step: Deploy Ollama on SnapDeploy

1. Create a free SnapDeploy account and subscribe to Dedicated GPU from the billing page — T4 for 8B models, A10G for 14B+. See full pricing.
2. Click "New Container" and select GPU as the compute type.
3. Pick the Ollama template from the template gallery.
4. Set the OLLAMA_MODEL environment variable — e.g. llama3.1:8b. This is the model the container pulls automatically on startup.
5. Click Deploy. The build takes 2-3 minutes; the server serves on port 11434 at a public HTTPS URL.

The template is designed around a background pull: the Ollama server comes up and passes health checks immediately, while the multi-gigabyte model download runs behind it. You get a live URL right away; the first chat completions work as soon as the weights land (typically a few minutes for an 8B model). The template is open source at github.com/AAR-Labs/ollama-gpu-snapdeploy — fork it to bake in different defaults.

Using the API

Pull an additional model (Ollama downloads it in the background and streams progress):

curl https://your-app.snapdeploy.app/api/pull \
-d '{"model": "qwen2.5:7b"}'

Chat with a model:

curl https://your-app.snapdeploy.app/api/chat \
-d '{
  "model": "llama3.1:8b",
  "messages": [
    {"role": "user", "content": "Explain KV cache in one paragraph."}
  ],
  "stream": false
}'

For single-turn completions there's also /api/generate, and both endpoints stream tokens by default — set "stream": false when you want one JSON response instead.

Or point any OpenAI SDK at https://your-app.snapdeploy.app/v1 and use /v1/chat/completions as usual. One caveat: Ollama ships with no authentication, so treat your URL as a secret or fork the template and add an API-key check in front.

The Honest Cost Math

An always-on LLM server is the workload where flat-rate pricing earns its keep. Over ~730 hours of 24/7 runtime, the T4 tier works out to roughly $0.68/hr effective ($499/mo flat; ₹48,403 in India) and the A10G to ~$1.37/hr ($999/mo; ₹96,903). Per-second serverless GPUs advertise lower hourly rates — but an Ollama server that answers instantly has to keep the model resident in VRAM, which means the meter runs 24/7 anyway, plus you inherit cold starts (re-loading gigabytes of weights) whenever the platform scales you to zero. A forgotten always-on instance on a metered cloud bills whatever it bills; here the number cannot move.

The honest flip side: if you only need an LLM for a few hours of batch work a week, per-second billing is genuinely cheaper — spin up, run, tear down. Flat-rate dedicated wins when the server is a standing service: team inference, product backends, always-warm chat. Full comparison tables in our GPU cloud pricing guide.

Frequently Asked Questions

Which Ollama models fit on a 16 GB T4?

Any 4-bit quantized model up to roughly 8B parameters runs comfortably — Llama 3.1 8B, Mistral 7B, Qwen2.5 7B, Gemma 2 9B — with headroom left for the KV cache. 14B-class models load on a T4 but leave little room for context; for 14B and quantized 32B models, choose the 24 GB A10G tier.

Can I run Llama 70B on SnapDeploy?

No — and we'd rather tell you that up front. A 4-bit quantized 70B model needs roughly 40+ GB of VRAM, which exceeds both tiers. The practical ceiling on the A10G is a quantized 32B-class model. For 70B you need multi-GPU or 48 GB-class hardware, which SnapDeploy does not currently offer.

Is my Ollama API exposed to the public internet?

Your container gets a public HTTPS URL, and Ollama itself does not ship with authentication. Treat the URL as a secret, or fork the template and add an auth layer — an API-key check in a reverse proxy or middleware — before sharing it beyond your team.

Can I use Ollama's OpenAI-compatible API?

Yes. Ollama exposes OpenAI-compatible endpoints under /v1 (including /v1/chat/completions), so most OpenAI SDKs work by pointing base_url at your container's public URL. If you want higher-throughput OpenAI-compatible serving, SnapDeploy also has a one-click vLLM template.

Do models survive a redeploy?

The template is designed so you don't have to care: on every start it pulls the model named in OLLAMA_MODEL in the background. After a redeploy, the model re-downloads automatically and the server answers /api/pull and /api/chat as soon as the weights are in place.

Getting Started

Everything about the Dedicated GPU tiers — hardware, what's included, the full FAQ — lives on the GPU hosting page. When you're ready:

1. Create a free SnapDeploy account
2. Subscribe to Dedicated GPU — T4 for 8B models, A10G for 14B and up
3. Deploy the Ollama template with OLLAMA_MODEL set to your model
4. Point your OpenAI SDK or curl at the public URL — your LLM, your data, your GPU

Generating images instead of text? See our companion guide to Stable Diffusion hosting with ComfyUI.