Ollama - Self-Hosted Your AI Chat

Ollama

Get up and running with large language models, locally.\

Prerequisite

1. Docker
2. Git

Installation

git clone ollama webui

git clone https://github.com/ollama-webui/ollama-webui.git
cd ollama-webui

edit docker-compose.yml, I am running with my nvidia GPU, so please uncomment gpu part

version: '3.6'

services:
  ollama:
    # Uncomment below for GPU support
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities:
                - gpu
    volumes:
      - ollama:/root/.ollama
    # Uncomment below to expose Ollama API outside the container stack
    # ports:
    #   - 11434:11434
    container_name: ollama
    pull_policy: always
    tty: true
    restart: unless-stopped
    image: ollama/ollama:latest

  ollama-webui:
    build:
      context: .
      args:
        OLLAMA_API_BASE_URL: '/ollama/api'
      dockerfile: Dockerfile
    image: ollama-webui:latest
    container_name: ollama-webui
    depends_on:
      - ollama
    ports:
      - 3000:8080
    environment:
      - "OLLAMA_API_BASE_URL=http://ollama:11434/api"
    extra_hosts:
      - host.docker.internal:host-gateway
    restart: unless-stopped

volumes:
  ollama: {}

run docker-compose

docker-compose up -d --build

WebUI

go to http://localhost:3000

go setting download a model
ps. please take a look hardware requirement with the LLM.

select with you just downloaded model.

ask some questions.

References

https://github.com/ollama-webui/ollama-webui

Additional

You can expose ollama api for Litellm API integration. I’m learning this, and will share it later.