Skip to content

Releases: Agentic-Analyst/api-runner

v1.0.0 — Async Orchestration Layer with Docker-in-Docker Execution

01 Apr 04:58
@zanwenfu zanwenfu
v1.0.0
c7bf731
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v1.0.0 — Async Orchestration Layer with Docker-in-Docker Execution Latest
Latest

v1.0.0 — Async Orchestration Layer with Docker-in-Docker Execution

The central control plane for VYNN AI. This service does not run financial analysis in-process — it orchestrates the full lifecycle of AI-driven analysis workflows: receiving requests, spawning isolated Docker containers for each pipeline run, streaming real-time progress back to clients via SSE, managing dual WebSocket channels for live market data, and delivering structured outputs (PDF reports, Excel financial models, markdown summaries).

10,598 lines of Python across 30 source files. Deployed in production on Hetzner Cloud.

Docker-in-Docker Execution

The defining architectural decision:

  1. User submits analysis request via POST /run
  2. API Runner spawns an ephemeral Docker container (fuzanwenn/stock-analyst:latest, ~975 MB) via the Docker SDK, mounting the host Docker socket (/var/run/docker.sock)
  3. Container runs the full agent pipeline in isolation, sharing only the persistent stockdata volume for I/O
  4. Logs stream back to the client via Server-Sent Events with typed event taxonomy
  5. Results persist to MongoDB; artifacts written to the shared volume
  6. Container is automatically removed on completion, timeout, or graceful stop (SIGTERM with 10s timeout, then force-kill)

Why DinD: Complete process isolation. A crashed or hung analysis container does not affect the API server or other users' requests. Each job is sandboxed with its own environment. Horizontal scaling is achieved by running multiple analysis containers concurrently.

6 pipeline types: comprehensive · financial-only · model-only · news-only · model-to-price · news-to-price

Real-Time Communication

Server-Sent Events (SSE) — /jobs/{job_id}/logs/stream

Per-job event streams with typed events:

Event Description
connection Initial connection established
log Log line with type field: NL (LLM natural language output) or LOG (system log)
status Progress status update
completed Analysis finished (includes session_id for chat pipelines)
error Error occurred

Automatic heartbeats keep connections alive through reverse proxies. Final drain logic ensures no events are lost on completion. Session ID extraction for chat jobs enables multi-turn conversation continuity.

WebSocket Hub — 2 distinct channels

  • Market Prices (/api/realtime/ws) — yfinance polling every 10s with subscriber-based architecture, intelligent caching, and per-symbol subscription management
  • News Feed (/api/news/ws) — per-ticker subscriptions with article deduplication (by ID, URL hash, or URL), 30-second ping/pong keep-alive, and background auto-updater with smart rate limiting for SerpAPI quotas

Both channels support subscribe, unsubscribe, ping, and health check messages.

REST API

  • Analysis job CRUD: create (/run), status polling (/jobs/{id}/status/detailed), stop (/jobs/{id}/stop with graceful SIGTERM), log snapshots (/jobs/{id}/logs)
  • Chat pipeline: POST /chat with natural language ticker identification and session continuity
  • File downloads: financial models (.xlsx), professional reports (.pdf), financial/news summaries, complete tar archives
  • Historical OHLCV data with configurable timeframes (1D · 1W · 1M · 3M · 1Y · ALL)

Authentication

Three methods, all under /auth:

  • Google OAuth 2.0 — Authlib OIDC flow
  • GitHub OAuth — Authlib OAuth2 flow
  • Email verification codes — passwordless login with HMAC-SHA256 token generation

Session management via Starlette middleware with HTTP-only cookies. Development mode supports hardcoded login codes for local testing.

Daily Intelligence Reports

  • Pre-market scheduler at 8:30 AM ET (Mon–Fri) with automatic weekend skip
  • Company-level and sector-level reports generated as batch Docker container jobs
  • Markdown-to-PDF conversion via ReportLab with custom styling, table of contents, and internal links
  • Batch status tracking: individual job status + aggregated batch status endpoints

Job Lifecycle Management

Status Description
pending Job queued, waiting to start
running Analysis actively executing in container
completed Finished with all results available
failed Analysis failed — check error field
llm_timeout LLM provider overloaded — retryable
stopping Stop request received, shutting down container
stopped Successfully stopped by user — retryable

Health Monitoring

Multi-level health checks across Docker connectivity, real-time price service, WebSocket server, news feed, backend image availability, data volume, and API key configuration. Kubernetes-style liveness probe at /healthz.

Data Architecture

Per-user, per-ticker, per-timestamp directory structure on a persistent Docker volume:

/data/{email}/{TICKER}/{timestamp}/
├── info.log                          # Pipeline execution log
├── models/
│   ├── {TICKER}_financial_model.xlsx  # DCF Excel workbook
│   └── {TICKER}_price_adjustment_explanation.md
├── reports/
│   └── {TICKER}_Professional_Analysis_Report.md
├── summaries/
│   ├── {TICKER}_financial_summary.md
│   └── {TICKER}_news_summary.md
├── searched/                          # Raw scraped articles
└── filtered/                          # NLP-filtered articles + index.csv

News articles stored in MongoDB via vynn_core in per-ticker collections.

Known Limitations

  • Session store is cookie-based (Starlette middleware), not backed by a distributed store — adequate for current scale but would need Redis-backed sessions for multi-instance deployments
  • WebSocket channels use in-memory connection state; not horizontally scalable without a pub/sub broker
  • News auto-updater rate limiting is keyed to SerpAPI quotas, not per-user

Tech Stack

Python 3.11 · FastAPI 0.104 · Uvicorn · Docker SDK 7.1 · MongoDB (Motor 3.5 + PyMongo 4.5) · Authlib · Starlette · ReportLab · yfinance · SerpAPI · SSE · WebSocket

Assets 2
Loading