Turn Any PC Into a Home AI Server: The Complete 2026 Setup Guide

Most people run Ollama on their main workstation and treat local AI as a single-machine, single-user experience. That works fine until your spouse wants to use it too, or you want AI available on your phone without carrying a gaming PC. The better setup — one that more people are moving to in 2026 — is a dedicated home AI server: an always-on machine that runs Ollama and Open WebUI, accessible to every device on your network. This guide walks you through the full setup, from hardware to remote access.

• ▸Always-on — models stay loaded in VRAM 24/7; first token arrives in under a second instead of waiting for a cold load
• ▸Every device gets AI — phone, tablet, second laptop, smart home systems, scripts — all hit the same local endpoint
• ▸Zero interference — the AI server runs independently; no dropped frames in games, no competition for GPU memory
• ▸Multi-user — Open WebUI gives each family member or team member their own login, conversations, and settings
• ▸Privacy by default — traffic never leaves your network; Tailscale encrypts remote access without opening a public port

• ▸$200 — Intel N100 / AMD mini PC with 16-32GB RAM. CPU inference only. Runs 7B models at 8-15 tok/s. Silent, uses ~15W idle. Great for light household use.
• ▸$550 — Used RTX 3060 (12GB) desktop. Runs all 7B-13B models at GPU speed (40-60 tok/s). Best entry point for real-time multi-device chat.
• ▸$850 — Used RTX 3090 (24GB). The sweet spot. Runs Llama 70B at Q4, all 13B models at Q8. Fast enough for multiple concurrent users.
• ▸$1,400 — RTX 4070 Ti (16GB). Faster than the 3090 on 7B-13B. Better for latency-sensitive coding inference.
• ▸$2,400 — RTX 4090 (24GB). No compromise. Runs any model that fits in 24GB at maximum speed.

1. 1.Ollama — the inference engine. Manages model downloads, VRAM loading, and serves an OpenAI-compatible API on port 11434. Everything else talks to Ollama.
2. 2.Open WebUI — the browser interface. A full-featured chat UI with conversation history, model switching, system prompts, multi-user accounts, and RAG document upload.
3. 3.Tailscale (optional) — encrypted remote access. Lets your phone and laptop reach the home server from anywhere without opening any router ports.

By default, Ollama only listens on localhost. One environment variable opens it to your local network:

# Install Ollama
curl -fsSL https://ollama.ai/install.sh | sh

# Enable as a systemd service (auto-starts on boot)
sudo systemctl enable ollama
sudo systemctl start ollama

# Open a drop-in override to listen on all network interfaces:
sudo systemctl edit ollama
# Add these lines in the editor:
# [Service]
# Environment="OLLAMA_HOST=0.0.0.0"
# Environment="OLLAMA_ORIGINS=*"
# Environment="OLLAMA_KEEP_ALIVE=24h"

# Reload and restart
sudo systemctl daemon-reload && sudo systemctl restart ollama

# Test from another machine on your LAN
curl http://YOUR-SERVER-IP:11434/api/tags

# Install Docker
curl -fsSL https://get.docker.com | sh && sudo usermod -aG docker $USER

# Run Open WebUI — auto-connects to Ollama on the host
docker run -d \
  --name open-webui \
  --restart always \
  -p 3000:8080 \
  --add-host=host.docker.internal:host-gateway \
  -v open-webui:/app/backend/data \
  ghcr.io/open-webui/open-webui:main

# Open WebUI is now at http://YOUR-SERVER-IP:3000
# First account you create becomes the admin

Tailscale creates a WireGuard-encrypted mesh network between your devices so your phone and laptop can reach the home server securely from anywhere — without opening any router ports. Free for personal use (up to 3 users, 100 devices).

# On the server: install and enable Tailscale
curl -fsSL https://tailscale.com/install.sh | sh
sudo tailscale up

# Find the server's Tailscale IP (looks like 100.64.X.X)
tailscale ip -4

# Install Tailscale on any other device (laptop, phone) via tailscale.com
# Sign in with the same account — devices can now reach each other
# Access Open WebUI remotely at: http://100.64.X.X:3000

• ▸Do NOT expose port 11434 or 3000 directly to the internet — Ollama has no authentication by default
• ▸Do NOT use router port forwarding for these services — use Tailscale or a properly authenticated reverse proxy
• ▸DO use a strong password on your Open WebUI admin account
• ▸DO keep Ollama and Open WebUI updated — both push regular security patches

version: "3.8"
services:
  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    restart: always
    ports:
      - "3000:8080"
    extra_hosts:
      - "host.docker.internal:host-gateway"
    volumes:
      - open-webui:/app/backend/data
    environment:
      - OLLAMA_BASE_URL=http://host.docker.internal:11434
      - WEBUI_SECRET_KEY=change-this-to-a-random-string

volumes:
  open-webui:

• ▸12GB VRAM — Preload one 7B-8B model (Llama 3.1 8B or Qwen3.5-9B at Q4). Load 13B on demand.
• ▸24GB VRAM — Keep two models warm: a fast 8B for quick tasks and a 32B or 70B at Q3 for deeper work.
• ▸CPU-only mini PC — Preload a single 7B at Q4. Slower but always-on beats nothing.

• ▸Continue.dev (VS Code) — Set base_url to http://YOUR-SERVER-IP:11434/v1. All autocomplete and chat go local, zero API cost.
• ▸Aider — Use --openai-api-base http://YOUR-SERVER-IP:11434/v1 for a full local code editing agent.
• ▸AnythingLLM — Connect to your Ollama server for a private RAG system over your own documents.
• ▸Home Assistant — Community integrations connect HA to Ollama for local voice AI and automation scripting.

The RTX 3090 server pays for itself in under eight months compared to three ChatGPT Plus subscriptions. After that your marginal cost is electricity — roughly $120 a year. The savings scale dramatically for anyone previously paying developer API rates at $50-200 per person per month.