Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -33,3 +33,6 @@ servers/*/server.yaml

# Python package metadata
src/*.egg-info/

# Docker volumes
docker/volumes/
22 changes: 22 additions & 0 deletions docker/.env.cpu.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# CPU-only compose vars
UI_PORT=5050
MILVUS_PORT=19530
MILVUS_METRICS_PORT=9091
MINIO_API_PORT=9000
MINIO_CONSOLE_PORT=9001
MINIO_ROOT_USER=minioadmin
MINIO_ROOT_PASSWORD=minioadmin

# ----------------------------
# UI defaults for external LLM/Embedding providers
# These values auto-fill in the UI settings panel.
# ----------------------------
# ULTRARAG_UI_DEFAULT_AI_PROVIDER options: openai | custom | azure | anthropic
ULTRARAG_UI_DEFAULT_AI_PROVIDER=openai
ULTRARAG_UI_DEFAULT_AI_BASE_URL=https://api.openai.com/v1
ULTRARAG_UI_DEFAULT_AI_API_KEY=
ULTRARAG_UI_DEFAULT_AI_MODEL=gpt-5-mini

ULTRARAG_UI_DEFAULT_EMB_BASE_URL=https://api.openai.com/v1
ULTRARAG_UI_DEFAULT_EMB_API_KEY=
ULTRARAG_UI_DEFAULT_EMB_MODEL_NAME=text-embedding-3-small
52 changes: 52 additions & 0 deletions docker/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# ----------------------------
# UltraRAG UI
# ----------------------------
UI_PORT=5050

# ----------------------------
# Milvus / MinIO ports (host)
# ----------------------------
MILVUS_PORT=19530
MILVUS_METRICS_PORT=9091
MINIO_API_PORT=9000
MINIO_CONSOLE_PORT=9001
MINIO_ROOT_USER=minioadmin
MINIO_ROOT_PASSWORD=minioadmin

# ----------------------------
# LLM (vLLM OpenAI-compatible API)
# ----------------------------
# Example:
# LLM_MODEL_PATH=Qwen/Qwen3-8B
# LLM_MODEL_NAME=qwen3-8b
LLM_MODEL_PATH=Qwen/Qwen3-8B
LLM_MODEL_NAME=qwen3-8b
LLM_DTYPE=auto
LLM_MAX_MODEL_LEN=32768
LLM_GPU_MEMORY_UTILIZATION=0.9
LLM_API_PORT=8000

# ----------------------------
# Embedding (vLLM OpenAI-compatible API)
# ----------------------------
# Example:
# EMB_MODEL_PATH=Qwen/Qwen3-Embedding-0.6B
# EMB_MODEL_NAME=qwen-embedding
EMB_MODEL_PATH=Qwen/Qwen3-Embedding-0.6B
EMB_MODEL_NAME=qwen-embedding
EMB_DTYPE=auto
EMB_GPU_MEMORY_UTILIZATION=0.8
EMB_API_PORT=8001

# ----------------------------
# UI defaults (auto-filled in settings panel)
# ----------------------------
# ULTRARAG_UI_DEFAULT_AI_PROVIDER options: openai | custom | azure | anthropic
ULTRARAG_UI_DEFAULT_AI_PROVIDER=custom
ULTRARAG_UI_DEFAULT_AI_BASE_URL=http://vllm-llm:8000/v1
ULTRARAG_UI_DEFAULT_AI_API_KEY=dummy
ULTRARAG_UI_DEFAULT_AI_MODEL=qwen3-8b

ULTRARAG_UI_DEFAULT_EMB_BASE_URL=http://vllm-emb:8000/v1
ULTRARAG_UI_DEFAULT_EMB_API_KEY=dummy
ULTRARAG_UI_DEFAULT_EMB_MODEL_NAME=qwen-embedding
117 changes: 117 additions & 0 deletions docker/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
# UltraRAG One-Click Compose

This folder provides two compose setups:

- `docker-compose.yml` (GPU): UI + Milvus + vLLM(LLM) + vLLM(Embedding)
- `docker-compose.cpu.yml` (CPU-only): UI + Milvus (no vLLM)

GPU setup includes:

- UltraRAG UI
- Milvus (standalone with etcd + minio)
- vLLM OpenAI-compatible LLM endpoint
- vLLM OpenAI-compatible Embedding endpoint

## 1) Prepare env

```bash
cd /Users/rqq/UltraRAG/docker
cp .env.example .env
```

Edit `.env` at least for these fields:

- `LLM_MODEL_PATH`
- `LLM_MODEL_NAME`
- `EMB_MODEL_PATH`
- `EMB_MODEL_NAME`
- Optional UI auto-fill defaults:
- `ULTRARAG_UI_DEFAULT_AI_BASE_URL`
- `ULTRARAG_UI_DEFAULT_AI_MODEL`
- `ULTRARAG_UI_DEFAULT_EMB_BASE_URL`
- `ULTRARAG_UI_DEFAULT_EMB_MODEL_NAME`

## 2) Start all services

```bash
docker compose up -d --build
```

Open UI:

- [http://localhost:5050](http://localhost:5050)

## 3) Recommended UI settings

Inside UltraRAG UI:

- Knowledge Base -> Milvus URI should be `tcp://milvus-standalone:19530`
- Already pre-mounted from `docker/config/kb_config.json`.
- Embedding config:
- `Base URL`: `http://vllm-emb:8000/v1`
- `Model Name`: same as `EMB_MODEL_NAME` in `.env`
- `API Key`: optional (for local vLLM you can use any non-empty value if required by UI)
- AI assistant / generation config:
- `Base URL`: `http://vllm-llm:8000/v1`
- `Model`: same as `LLM_MODEL_NAME` in `.env`
- `API Key`: optional (for local vLLM you can use any non-empty value)

## 4) Stop

```bash
docker compose down
```

To remove volumes too:

```bash
docker compose down -v
```

## Notes

- `vllm-llm` and `vllm-emb` require NVIDIA GPU + working Docker GPU runtime.
- First startup may take a long time due to model/image downloads.
- Persistent data is under `docker/volumes/`.

---

## CPU-only setup (no vLLM)

If your machine has no NVIDIA GPU, use this flow.

### 1) Prepare env

```bash
cd /Users/rqq/UltraRAG/docker
cp .env.cpu.example .env
```

### 2) Start CPU stack

```bash
docker compose -f docker-compose.cpu.yml up -d --build
```

Open UI:

- [http://localhost:5050](http://localhost:5050)

### 3) UI settings for CPU mode

- Knowledge Base -> Milvus URI:
- `tcp://milvus-standalone:19530`
- Already pre-mounted from `docker/config/kb_config.json`.
- Embedding and Generation models:
- Use external OpenAI-compatible API (OpenAI/Azure/other hosted endpoint).
- You can prefill UI via `.env`:
- `ULTRARAG_UI_DEFAULT_AI_*`
- `ULTRARAG_UI_DEFAULT_EMB_*`
- In UI you can still manually change provider `Base URL`, `Model`, `API Key`.
- CPU compose does not start local vLLM endpoints.

### 4) Stop CPU stack

```bash
docker compose -f docker-compose.cpu.yml down
```
21 changes: 21 additions & 0 deletions docker/config/kb_config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{
"milvus": {
"uri": "tcp://milvus-standalone:19530",
"token": "",
"id_field_name": "id",
"vector_field_name": "vector",
"text_field_name": "contents",
"id_max_length": 64,
"text_max_length": 60000,
"metric_type": "IP",
"index_params": {
"index_type": "AUTOINDEX",
"metric_type": "IP"
},
"search_params": {
"metric_type": "IP",
"params": {}
},
"index_chunk_size": 1000
}
}
84 changes: 84 additions & 0 deletions docker/docker-compose.cpu.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
services:
ultrarag-ui:
build:
context: ..
dockerfile: Dockerfile.base-cpu
container_name: ultrarag-ui-cpu
restart: unless-stopped
env_file:
- .env
environment:
PYTHONUNBUFFERED: "1"
command:
- ultrarag
- show
- ui
- --admin
- --port
- "5050"
- --host
- 0.0.0.0
ports:
- "${UI_PORT:-5050}:5050"
volumes:
- ../data:/ultrarag/data
- ../output:/ultrarag/output
- ../logs:/ultrarag/logs
- ./config/kb_config.json:/ultrarag/data/knowledge_base/kb_config.json
depends_on:
- milvus-standalone
networks:
- ultrarag-net

etcd:
container_name: ultrarag-etcd-cpu
image: quay.io/coreos/etcd:v3.5.12
restart: unless-stopped
command: >
etcd
--advertise-client-urls=http://etcd:2379
--listen-client-urls=http://0.0.0.0:2379
--data-dir=/etcd
volumes:
- ./volumes/etcd:/etcd
networks:
- ultrarag-net

minio:
container_name: ultrarag-minio-cpu
image: minio/minio:RELEASE.2024-12-18T13-15-44Z
restart: unless-stopped
environment:
MINIO_ROOT_USER: "${MINIO_ROOT_USER:-minioadmin}"
MINIO_ROOT_PASSWORD: "${MINIO_ROOT_PASSWORD:-minioadmin}"
command: minio server /minio_data --console-address ":9001"
ports:
- "${MINIO_API_PORT:-9000}:9000"
- "${MINIO_CONSOLE_PORT:-9001}:9001"
volumes:
- ./volumes/minio:/minio_data
networks:
- ultrarag-net

milvus-standalone:
container_name: ultrarag-milvus-cpu
image: milvusdb/milvus:v2.5.4
restart: unless-stopped
command: ["milvus", "run", "standalone"]
environment:
ETCD_ENDPOINTS: etcd:2379
MINIO_ADDRESS: minio:9000
ports:
- "${MILVUS_PORT:-19530}:19530"
- "${MILVUS_METRICS_PORT:-9091}:9091"
volumes:
- ./volumes/milvus:/var/lib/milvus
depends_on:
- etcd
- minio
networks:
- ultrarag-net

networks:
ultrarag-net:
name: ultrarag-net
Loading