Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions .github/workflows/unified-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
name: Unified Service Tests

on:
pull_request:
push:
branches: [ main ]

jobs:
test:
runs-on: ubuntu-latest

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: "3.11"

- name: Install deps
run: |
python -m pip install --upgrade pip
pip install -r services/unified/requirements.txt
pip install pytest

- name: Run tests
env:
UNIFIED_MODE: forward
PYTHONPATH: services/unified
run: |
python -m pytest -q services/unified/tests
129 changes: 50 additions & 79 deletions docs/PR_DESCRIPTION.md
Original file line number Diff line number Diff line change
@@ -1,103 +1,74 @@
## Summary

This PR prepares the stack for shipping with env-driven LLM routing, clean service boundaries, and Railway deployment guidance.

### Included changes

- AI backend:
- Added env-based LLM routing (`LLM_PROVIDER`) with Ollama default and optional OpenAI fallback.
- Removed hardcoded model path; model/provider selected from env in one place.
- Bot:
- Kept bot as thin transport layer to AI backend (`/api/chat` + `X-API-Key`).
- Removed ticker-specific local prompt branching.
- Deploy/docs:
- Added copy/paste env examples and Railway 3-service deployment docs.
- Added smoke test scripts and env templates.

## Railway Deploy Order
## PR Title

1. **RAG** (FastAPI)
2. **AI backend** (FastAPI)
3. **Bot** (worker)
`chore(unified): scaffold modular unified service with forward-only defaults and test wiring`

## Required Env Vars Per Service

### Service A: RAG

```env
INNER_CALLS_KEY=change-me-shared-secret
# optional: COFFEE_KEY
```
## Summary

Start command:
This PR creates the first real `services/unified` skeleton as a safe, mergeable foundation for incremental architecture work.

```bash
uvicorn main:app --host 0.0.0.0 --port $PORT
```
- No runtime cutover.
- No behavior migration.
- Existing services remain source of truth via forwarding.
- Default mode is forward; unified is not used in prod yet.

### Service B: AI backend

```env
INNER_CALLS_KEY=change-me-shared-secret
RAG_URL=https://<rag-domain>
LLM_PROVIDER=ollama
OLLAMA_URL=http://127.0.0.1:11434
OLLAMA_MODEL=qwen2.5:1.5b
# optional:
# OPENAI_API_KEY=
# OPENAI_MODEL=gpt-4o-mini
```
## Why

Start command:
We need a single-root modular structure ready for growth (auth, ai, rag, wallet, tasks, feed) without introducing deployment risk.

```bash
uvicorn main:app --host 0.0.0.0 --port $PORT
```
## What Changed

### Service C: Bot
### New unified service scaffold

```env
BOT_TOKEN=123456:telegram-token
DATABASE_URL=postgresql://user:pass@host:5432/db
AI_BACKEND_URL=https://<ai-domain>
API_KEY=change-me-shared-secret
# optional: APP_URL
```
- Added `services/unified` service package with:
- `app/main.py` (FastAPI entrypoint)
- `app/config.py` (env-driven global + per-route modes)
- `app/health.py` (`/health` payload + route mode visibility)
- `app/api/routers/` for `auth`, `ai`, `rag`
- `app/forwarding/` for legacy upstream calls
- `app/modules/` placeholders for `auth`, `ai`, `rag`, `wallet`, `tasks`, `feed`
- `app/observability/` and `app/shared/` placeholders

Start command:
### Forward-only behavior (default)

```bash
python bot.py
```
- `UNIFIED_MODE=forward` by default.
- Added route-level mode vars (`UNIFIED_AUTH_MODE`, `UNIFIED_AI_MODE`, `UNIFIED_RAG_MODE`, etc.) inheriting from `UNIFIED_MODE`.
- Live routes currently forward to legacy services:
- `POST /auth/telegram` -> bot
- `POST /ai/chat` and `POST /api/chat` -> ai backend
- `POST /rag/query` and `POST /query` -> rag backend

## How To Smoke Test
### Operational files

### Telegram checks
- Added `Dockerfile`, `railway.json`, `requirements.txt`, `scripts/run_local.sh`, and updated `README.md`.

Send:
### Tests

1. `$DOGS`
2. `что такое DOGS?`
3. `$TON`
- Added `services/unified/tests/test_health.py`
- Added `services/unified/tests/test_forwarding.py`
- Added CI workflow `.github/workflows/unified-tests.yml` to run `pytest` for `services/unified`

Confirm:
## Risk / Safety

- RAG receives `/tokens/{symbol}` and returns `200` for valid symbols.
- AI backend does not log `RAG verification failed` for those requests.
- Bot replies cleanly.
- Low risk: scaffold-only PR with forward defaults.
- No legacy code moved or deleted.
- No API contract changes on live forwarded endpoints.

### Scripted checks
## Verification

Bash:
Local (inside `services/unified`):

```bash
export API_KEY=...
./smoke.sh
pip install -r requirements.txt
pytest -q
```

PowerShell:
CI:

```powershell
$env:API_KEY="..."
.\shell\smoke.ps1
```
- `Unified Service Tests` workflow runs on push/PR and executes `pytest` for `services/unified`.
- CI validates the scaffold test harness even when local environments do not have `pytest` installed.

## Follow-ups (Not in this PR)

1. Add wallet/tasks/feed routers with forward stubs.
2. Introduce `shadow` mode diff logging.
3. Migrate one bounded context at a time (starting with wallet/auth policy).
128 changes: 128 additions & 0 deletions docs/UNIFIED_SERVICE_ARCHITECTURE_PROPOSAL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
# Unified Service Architecture Proposal (M2)

## Goal

Build a single-root `services/unified` service that starts as a safe gateway/orchestrator and incrementally becomes the business-logic owner per domain, without risky cutover.

## Proposed Folder Structure

```text
services/unified/
app/
main.py
config.py
health.py
observability/
logging.py
metrics.py
tracing.py
api/
dependencies.py
routers/
auth.py
ai.py
rag.py
wallet.py
tasks.py
feed.py
modules/
auth/
service.py
contracts.py
ai/
service.py
contracts.py
rag/
service.py
contracts.py
wallet/
service.py
crypto.py
contracts.py
repository.py
tasks/
service.py
contracts.py
feed/
service.py
contracts.py
forwarding/
client.py
policy.py
registry.py
shared/
errors.py
types.py
idempotency.py
tests/
smoke/
contract/
integration/
scripts/
run_local.sh
README.md
requirements.txt
Dockerfile
railway.json
```

## Unified Service Role (Clear Boundary)

1. `M2` role: API gateway + orchestrator + policy enforcement.
2. `M3` role: Own business logic for one domain at a time (start with wallet).
3. `M4` role: Primary owner for auth/wallet/tasks/feed while AI/RAG can remain forwarded until stabilized.

Rule: route shape and security stay in unified from day one; domain logic migrates behind unchanged APIs.

## Migration Strategy (No Big-Bang Cutover)

1. Keep all current endpoints and compatibility aliases.
2. Introduce per-route mode flags:
- `forward` -> call legacy service
- `local` -> execute unified module logic
- `shadow` -> execute both, return forward response, log diffs
3. Migrate one bounded context at a time:
- Auth verification policy in unified
- Wallet create/read/update paths
- Tasks and feed aggregation
- AI/RAG orchestration and rate policy
4. Remove legacy calls only after contract and shadow parity pass.

## Wallet Security Ownership

1. Mnemonic generation:
- Generated client-side only (WebCrypto/secure RNG).
- Never sent to unified or legacy backends.
2. PIN handling:
- PIN never stored directly.
- Client derives key material (KDF) and encrypts mnemonic locally.
3. Backend responsibility:
- Store only non-secret wallet metadata (address, state, timestamps, optional device-scoped envelope metadata).
- Enforce auth, rate limits, replay protection, audit events.
4. Session/logout behavior:
- Clear in-memory secrets on logout/app lock.
- Require PIN re-entry to decrypt local encrypted mnemonic.
5. Recovery UX:
- Explicit "I saved 24 words" checkpoint before activation.
- Soft-block sensitive actions until backup confirmation.

## API and Domain Contracts

1. API routers only parse/validate HTTP requests and map to module services.
2. Module services hold domain rules and return typed results.
3. Forwarding layer is infrastructure-only; no domain logic in forwarding code.
4. Shared errors define stable error codes across all modules.

## Immediate Next PR (Low Risk)

1. Create folders/files in `services/unified` with stubbed modules and routers.
2. Keep behavior unchanged by default (`forward` mode for all routes).
3. Add:
- `/health` with module mode visibility
- `/ready` dependency checks
- smoke tests for auth/ai/rag forwarding contracts
4. Add route-level mode config (env-driven) to enable incremental migration.

## Message You Can Send in the Morning

I prepared a unified architecture proposal with a single-root modular layout and route-level migration flags. I will keep no-cutover behavior (forward-first), then migrate domains one by one starting with wallet/auth policy, with contract parity checks before switching ownership.
25 changes: 20 additions & 5 deletions services/unified/README.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,29 @@
# Unified Service (Milestone 1)
# Unified Service

This folder contains the first unified runtime skeleton for bot + ai + rag.
This folder contains the unified runtime skeleton for bot + ai + rag.

Current status:
- `UNIFIED_MODE=forward` (default): forwarding stubs to existing services.
- `UNIFIED_MODE=local`: reserved for future in-process implementation.
Current milestone keeps runtime behavior safe:
- Default mode is `forward` for all live routes.
- Legacy services remain source of truth.
- Unified service owns routing, health, and migration controls.

## Endpoints

- `GET /health`
- `GET /ready`
- `POST /auth/telegram` -> forwards to bot `/auth/telegram`
- `POST /ai/chat` -> forwards to ai `/api/chat`
- `POST /api/chat` -> compatibility alias for `/ai/chat`
- `POST /rag/query` -> forwards to rag `/query`
- `POST /query` -> compatibility alias for `/rag/query`

## Route Modes

Each route supports mode flags:
- `forward` (default)
- `local` (reserved)
- `shadow` (reserved)

## Run local

```bash
Expand All @@ -26,6 +35,12 @@ uvicorn app.main:app --host 0.0.0.0 --port 8090
## Environment

- `UNIFIED_MODE` (`forward` by default)
- `UNIFIED_AUTH_MODE` (default inherits `UNIFIED_MODE`)
- `UNIFIED_AI_MODE` (default inherits `UNIFIED_MODE`)
- `UNIFIED_RAG_MODE` (default inherits `UNIFIED_MODE`)
- `UNIFIED_WALLET_MODE` (default inherits `UNIFIED_MODE`)
- `UNIFIED_TASKS_MODE` (default inherits `UNIFIED_MODE`)
- `UNIFIED_FEED_MODE` (default inherits `UNIFIED_MODE`)
- `UNIFIED_FORWARD_TIMEOUT_SECONDS` (`30` by default)
- `UNIFIED_FORWARD_CONNECT_TIMEOUT_SECONDS` (`5` by default)
- `BOT_BASE_URL` (`http://127.0.0.1:8080` by default)
Expand Down
2 changes: 1 addition & 1 deletion services/unified/app/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
# Unified service package marker.
"""Unified service package."""
1 change: 1 addition & 0 deletions services/unified/app/api/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
"""HTTP API layer for unified service."""
1 change: 1 addition & 0 deletions services/unified/app/api/dependencies.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
"""Shared API dependencies placeholder."""
1 change: 1 addition & 0 deletions services/unified/app/api/routers/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
"""Unified API routers package."""
21 changes: 21 additions & 0 deletions services/unified/app/api/routers/ai.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
from __future__ import annotations

from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import Response

from app.config import settings
from app.forwarding.client import forward_post

router = APIRouter()


@router.post("/ai/chat")
async def ai_chat(request: Request) -> Response:
if settings.ai_mode == "local":
raise HTTPException(status_code=501, detail="UNIFIED_AI_MODE=local is not implemented yet")
return await forward_post(request, f"{settings.ai_base_url}/api/chat")


@router.post("/api/chat")
async def ai_chat_compat(request: Request) -> Response:
return await ai_chat(request)
Loading