Skip to content

Home

Jump to bottom
loglux edited this page Mar 4, 2026 · 17 revisions

Documentation

Welcome to the Knowledge Base Platform documentation!

Current Notes

  • API auth is enabled for protected routes after setup completion.
  • Reranking is implemented (Voyage/Cohere) and configurable in Chat/Settings.
  • Verify reranking in UI source cards (reranked, rerank, pre) or API logs (Rerank applied).
  • PDF support added: text-based PDFs are now supported (TXT, MD, FB2, DOCX, PDF). Heading structure is extracted at upload time for all formats and stored as section_heading/section_path/section_level in Qdrant payload.

Metadata Processing Model

The platform uses two complementary metadata layers during ingestion:

  • Structural Metadata Indexing: deterministic structure extraction (section_heading, section_path, section_level, page mapping for PDF).
    Use this for precise section/page grounding and stable behavior without extra model cost.
  • Metadata Enrichment: optional semantic enrichment (currently contextual_description) generated by an LLM during ingestion.
    Use this when retrieval quality matters more than ingest latency/cost.

These layers are not alternatives: structural metadata tells the system where content lives; enrichment helps describe what a chunk means in document context.

Contextual Description Controls

contextual_description_enabled is resolved with strict precedence:

  1. Request override on operation (upload/reprocess)
  2. KB override (knowledge_bases.contextual_description_enabled)
  3. Global default (app_settings.contextual_description_enabled)
  4. Fallback default (false)

Practical guidance:

  • Enable globally only if most KBs require highest retrieval quality.
  • Keep global off and enable per-KB for targeted quality-sensitive corpora.
  • Use request-level force enable/disable for one-time bulk operations (quality reindex vs fast ingest).

Quick Links

Interactive Documentation

When the API is running, you can access:

Update After git pull (Local Docker)

Recommended quick update:

git pull
docker compose down
docker compose up -d --build

Notes:

  • Avoid deleting volumes unless you want to reset data.
  • If builds are slow, skip cache pruning; use docker builder prune -f only when needed.
  • docker-compose (v1) is legacy; prefer docker compose.
  • Ensure ./secrets/db_password exists and matches the current DB password.

Chat UI Notes

MCP / Automation Notes

  • For search-only integrations (MCP tools), use POST /api/v1/retrieve/ to avoid creating chat conversations.
  • KB-level retrieval defaults can be stored via PUT /api/v1/knowledge-bases/{kb_id}/retrieval-settings.
  • MCP endpoint is mounted at /mcp (use the backend or frontend proxy).
  • Auth options:
    • Static MCP tokens (Admin UI → MCP tab)
    • OAuth (Gateway) via POST /oauth/token (password + refresh flow)
  • OAuth TTLs are configurable in Admin UI → MCP tab.

KB Transfer (Export/Import)

  • UI: Settings → KB Transfer

  • Export: POST /api/v1/kb/export (downloads .tar.gz)

  • Import: POST /api/v1/kb/import (multipart)

  • Chats (Markdown): POST /api/v1/kb/export-chats-md (separate .zip for reading)

  • Merge rules: for targeted merge, select a single-KB archive and a target KB with the same embedding model/provider/dimension.

  • Each knowledge base can have multiple conversations.

  • Conversations can be renamed or deleted from the chat list.

  • Conversation settings are saved per chat and can be revisited later.

Windowed Retrieval (Context Expansion)

When enabled, the system expands each retrieved chunk by including neighboring chunks from the same document. This helps recover context that was split during chunking.

  • Setting: Context Expansion → enable Windowed Retrieval
  • Window size: context_window = number of chunks on each side (0–5)
  • API fields: context_expansion: ["window"], context_window: N

Prompts & Self-Check (UI)

Prompt editing is available in Settings → Prompts:

  • Create and save system prompt drafts (KB chat prompt).
  • Activate a prompt version to make it the default for chat.
  • View previous versions and load them into the editor.

Self-check prompts are managed in the same tab:

  • Create/activate self-check prompt versions.
  • Toggle self-check per chat in the chat settings.

Optional setting:

  • Show prompt versions in chat (adds a small badge to assistant messages for traceability).

Documentation Files

API_DOCUMENTATION.md

Use this for: Detailed API reference, complete examples, data models

Contains:

  • Complete endpoint descriptions
  • Request/response schemas
  • Authentication details
  • Error handling
  • Data models (TypeScript interfaces)
  • Comprehensive examples

Best for: Integration, understanding business logic, troubleshooting

ENDPOINTS_QUICK_REFERENCE.md

Use this for: Quick lookup during day-to-day use

Contains:

  • Endpoint list by category
  • Key parameters
  • Common patterns
  • HTTP status codes
  • Code snippets

Best for: Daily usage, quick reference, bash scripts

API_MAP.md

Use this for: Understanding system architecture

Contains:

  • Visual API tree structure
  • Data flow diagrams
  • Integration points
  • Use case examples
  • Performance tips

Best for: Onboarding, architecture review, optimization

Getting Started

1. First-Time Setup

# Check API health
curl http://localhost:8004/api/v1/health

# Get API info
curl http://localhost:8004/api/v1/info

# View available embedding models
curl http://localhost:8004/api/v1/embeddings/models

2. Create Your First Knowledge Base

# Create knowledge base
curl -X POST http://localhost:8004/api/v1/knowledge-bases/ \
  -H "Content-Type: application/json" \
  -d '{
    "name": "My First KB",
    "description": "Test knowledge base",
    "chunking_strategy": "semantic",
    "embedding_model": "text-embedding-3-small"
  }'

3. Upload a Document

# Upload document (replace KB_ID with your KB ID)
curl -X POST http://localhost:8004/api/v1/documents/ \
  -F "file=@/path/to/document.pdf" \
  -F "knowledge_base_id=KB_ID"

4. Monitor Processing

# Check document status (replace DOC_ID)
curl http://localhost:8004/api/v1/documents/DOC_ID/status

5. Query the Knowledge Base

# Ask a question (replace KB_ID)
curl -X POST http://localhost:8004/api/v1/chat/ \
  -H "Content-Type: application/json" \
  -d '{
    "question": "What is this document about?",
    "knowledge_base_id": "KB_ID",
    "top_k": 5
  }'

Common Tasks

Check System Status

# Health check
curl http://localhost:8004/api/v1/health

# Dependency check
curl http://localhost:8004/api/v1/ready

# Ollama status (if using local models)
curl http://localhost:8004/api/v1/ollama/status

List Resources

# List knowledge bases
curl http://localhost:8004/api/v1/knowledge-bases/

# List documents for a KB
curl "http://localhost:8004/api/v1/documents/?knowledge_base_id=KB_ID"

# List conversations
curl http://localhost:8004/api/v1/chat/conversations

Manage Settings

# Get current settings
curl http://localhost:8004/api/v1/settings/

# Update settings
curl -X PUT http://localhost:8004/api/v1/settings/ \
  -H "Content-Type: application/json" \
  -d '{
    "llm_model": "gpt-4o-mini",
    "temperature": 0.5,
    "top_k": 10
  }'

# Reset to defaults
curl -X POST http://localhost:8004/api/v1/settings/reset

Manage Prompts

# List chat prompt versions
curl http://localhost:8004/api/v1/prompts/

# Create a new chat prompt (draft or activated)
curl -X POST http://localhost:8004/api/v1/prompts/ \
  -H "Content-Type: application/json" \
  -d '{
    "name": "My Chat Prompt",
    "system_content": "You are a helpful assistant...",
    "activate": true
  }'

# List self-check prompt versions
curl http://localhost:8004/api/v1/prompts/self-check

Response Formats

Success Response

{
  "id": "uuid",
  "status": "success",
  "data": { ... }
}

Paginated Response

{
  "items": [...],
  "total": 100,
  "page": 1,
  "page_size": 10,
  "pages": 5
}

Pagination uses page and page_size (default 10, max 100).

Error Response

{
  "detail": "Error message",
  "path": "/api/v1/endpoint",
  "suggestion": "Helpful suggestion"
}

Development Workflow

Typical Development Flow

  1. Read API Map to understand system architecture
  2. Reference Quick Reference for endpoint details
  3. Deep Dive API Documentation for complete specifications
  4. Test using Swagger UI at http://localhost:8004/docs

Before Creating New Endpoints

  1. ✅ Check if endpoint already exists in documentation
  2. ✅ Review naming conventions in Quick Reference
  3. ✅ Follow existing patterns from API Map
  4. ✅ Update all three documentation files when adding new endpoints

Support

Getting Help

Reporting Issues

When reporting API issues, include:

  • Endpoint URL
  • Request method and body
  • Response status code and body
  • Expected vs actual behavior

Version Information

  • API Version: v1
  • Documentation Version: 1.0
  • Last Updated: 2026-02-05

Happy Building! 🚀

📝 Questions? Open an issue | 🌟 Like it? Star the repo | 📖 API Docs: Swagger UI

📚 Documentation

Getting Started

API Reference

Operations

Links

Version: v1.0
Updated: 2026-02-08

Clone this wiki locally