Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .github/workflows/benchmark-community-world.yml
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,21 @@ jobs:
sleep 2
done

- name: Start SurrealDB
if: ${{ inputs.service-type == 'surrealdb' }}
run: |
docker run -d --name surrealdb -p 8000:8000 \
surrealdb/surrealdb:v3 \
start --user root --pass root --bind 0.0.0.0:8000 memory
echo "Waiting for SurrealDB to be ready..."
for i in {1..30}; do
if curl -sf http://localhost:8000/health &>/dev/null; then
echo "SurrealDB is ready"
break
fi
sleep 2
done

- name: Setup environment
uses: ./.github/actions/setup-workflow-dev
with:
Expand Down Expand Up @@ -143,3 +158,4 @@ jobs:
run: |
docker stop mongodb 2>/dev/null || true
docker stop redis 2>/dev/null || true
docker stop surrealdb 2>/dev/null || true
16 changes: 16 additions & 0 deletions .github/workflows/e2e-community-world.yml
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,21 @@ jobs:
sleep 2
done

- name: Start SurrealDB
if: ${{ inputs.service-type == 'surrealdb' }}
run: |
docker run -d --name surrealdb -p 8000:8000 \
surrealdb/surrealdb:v3 \
start --user root --pass root --bind 0.0.0.0:8000 memory
echo "Waiting for SurrealDB to be ready..."
for i in {1..30}; do
if curl -sf http://localhost:8000/health &>/dev/null; then
echo "SurrealDB is ready"
break
fi
sleep 2
done

- name: Setup environment
uses: ./.github/actions/setup-workflow-dev
with:
Expand Down Expand Up @@ -135,3 +150,4 @@ jobs:
run: |
docker stop mongodb 2>/dev/null || true
docker stop redis 2>/dev/null || true
docker stop surrealdb 2>/dev/null || true
2 changes: 1 addition & 1 deletion docs/content/docs/deploying/world/meta.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{
"title": "World",
"pages": ["local-world", "vercel-world", "postgres-world"]
"pages": ["local-world", "vercel-world", "postgres-world", "surrealdb-world"]
}
251 changes: 251 additions & 0 deletions docs/content/docs/deploying/world/surrealdb-world.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,251 @@
---
title: SurrealDB World
description: Community world using SurrealDB with LIVE SELECT for queueing and real-time streaming.
type: integration
summary: Deploy workflows to your own infrastructure using SurrealDB.
prerequisites:
- /docs/deploying
related:
- /docs/deploying/world/local-world
- /docs/deploying/world/postgres-world
---

The SurrealDB World is a community-maintained backend for self-hosted deployments. It uses [SurrealDB](https://surrealdb.com) for durable storage and leverages `LIVE SELECT` for reactive queue processing and real-time streaming.

Use the SurrealDB World when you want a single-binary database that handles storage, queuing, and streaming without additional infrastructure like a separate job queue.

<Callout type="info">
This is a community world maintained at [sepcnt/workflow-world-surrealdb](https://github.com/sepcnt/workflow-world-surrealdb). For bug reports and feature requests, please use that repository.
</Callout>

## Installation

Install the SurrealDB World package in your workflow project:

```package-install
workflow-world-surrealdb
```

Configure the required environment variables:

```bash title=".env"
WORKFLOW_TARGET_WORLD="workflow-world-surrealdb"
WORKFLOW_SURREAL_URL="ws://127.0.0.1:8000/rpc"
WORKFLOW_SURREAL_NAMESPACE="workflow"
WORKFLOW_SURREAL_DATABASE="workflow"
WORKFLOW_SURREAL_USERNAME="root"
WORKFLOW_SURREAL_PASSWORD="root"
```

Run the setup script to create the necessary tables and functions in your database:

```bash
npx workflow-surreal-setup
```

<Callout type="info">
The setup is idempotent and can safely be run as a post-deployment lifecycle script.
</Callout>

## Running SurrealDB

Start a SurrealDB instance using Docker:

```bash
docker run -d -p 8000:8000 \
surrealdb/surrealdb:v3 \
start --user root --pass root --bind 0.0.0.0:8000 memory
```

Or use the included `docker-compose.yml`:

```bash
docker compose up -d
```

For persistent storage, replace `memory` with a file path:

```bash
docker run -d -p 8000:8000 -v surrealdb-data:/data \
surrealdb/surrealdb:v3 \
start --user root --pass root --bind 0.0.0.0:8000 \
"surrealkv:///data/workflow.db?versioned=true"
```

## Starting the World

To subscribe to the queue via LIVE SELECT, your workflow app needs to start the world on server start. Here are examples for a few frameworks:

<Tabs items={["Next.js", "SvelteKit", "Nitro"]}>

<Tab value="Next.js">

Create an `instrumentation.ts` file in your project root:

```ts title="instrumentation.ts" lineNumbers
export async function register() {
if (process.env.NEXT_RUNTIME !== "edge") {
const { getWorld } = await import("workflow/runtime");
await getWorld().start?.();
}
}
```

<Callout type="info">
Learn more about [Next.js Instrumentation](https://nextjs.org/docs/app/guides/instrumentation).
</Callout>

</Tab>

<Tab value="SvelteKit">

Create a `src/hooks.server.ts` file:

```ts title="src/hooks.server.ts" lineNumbers
import type { ServerInit } from "@sveltejs/kit";

export const init: ServerInit = async () => {
const { getWorld } = await import("workflow/runtime");
await getWorld().start?.();
};
```

<Callout type="info">
Learn more about [SvelteKit Hooks](https://svelte.dev/docs/kit/hooks).
</Callout>

</Tab>

<Tab value="Nitro">

Create a plugin to start the world on server initialization:

```ts title="plugins/start-surreal-world.ts" lineNumbers
import { defineNitroPlugin } from "nitro/~internal/runtime/plugin";

export default defineNitroPlugin(async () => {
const { getWorld } = await import("workflow/runtime");
await getWorld().start?.();
});
```

Register the plugin in your config:

```ts title="nitro.config.ts"
import { defineNitroConfig } from "nitropack";

export default defineNitroConfig({
modules: ["workflow/nitro"],
plugins: ["plugins/start-surreal-world.ts"],
});
```

<Callout type="info">
Learn more about [Nitro Plugins](https://v3.nitro.build/docs/plugins).
</Callout>

</Tab>

</Tabs>

<Callout type="info">
The SurrealDB World requires a long-lived worker process that maintains LIVE SELECT subscriptions. This does not work on serverless environments. For Vercel deployments, use the [Vercel World](/worlds/vercel) instead.
</Callout>

## Configuration

All configuration options can be set via environment variables or programmatically via `createWorld()`.

### `WORKFLOW_SURREAL_URL`

SurrealDB WebSocket URL. Supports `ws://`, `wss://`, `http://`, and `https://` protocols (HTTP is auto-converted to WS).

Default: `ws://127.0.0.1:8000/rpc`

### `WORKFLOW_SURREAL_NAMESPACE`

SurrealDB namespace. Default: `workflow`

### `WORKFLOW_SURREAL_DATABASE`

SurrealDB database. Default: `workflow`

### `WORKFLOW_SURREAL_USERNAME` / `WORKFLOW_SURREAL_PASSWORD`

Root or namespace-level credentials for authentication.

### `WORKFLOW_SURREAL_TOKEN`

Alternative to username/password. Use a JWT token for authentication.

### `WORKFLOW_SURREAL_QUEUE_CONCURRENCY`

Number of concurrent jobs to process. Default: `10`

### `WORKFLOW_SURREAL_QUEUE_LANE_SHARDS`

Number of queue lane partitions for horizontal scalability. Default: `16`

### `WORKFLOW_SURREAL_QUEUE_LEASE_MS`

Job lease duration in milliseconds before a job is considered stale. Default: `300000` (5 minutes)

### `WORKFLOW_SURREAL_QUEUE_HEARTBEAT_MS`

Heartbeat interval for extending job leases. Default: `10000` (10 seconds)

### Programmatic configuration

{/* @skip-typecheck: incomplete code sample */}
```typescript title="workflow.config.ts" lineNumbers
import { createWorld } from "workflow-world-surrealdb";

const world = createWorld({
url: "ws://127.0.0.1:8000/rpc",
namespace: "workflow",
database: "workflow",
auth: {
username: "root",
password: "root",
},
queueConcurrency: 20,
queueLaneShards: 32,
});
```

## How It Works

The SurrealDB World uses SurrealDB as a unified backend:

- **Storage** — Workflow runs, events, steps, and hooks are stored in SurrealDB tables with SCHEMAFULL validation
- **Job Queue** — Lane-sharded queue with `LIVE SELECT` subscriptions for instant wake-up and automatic lease recovery
- **Streaming** — `LIVE SELECT` on stream chunk tables enables real-time event distribution with periodic reconciliation

This architecture keeps the infrastructure minimal — a single SurrealDB instance handles storage, queuing, and streaming without additional services. For implementation details, see the [source code](https://github.com/sepcnt/workflow-world-surrealdb).

## Deployment

Deploy your application to any cloud that supports long-running servers:

- Docker containers
- Kubernetes clusters
- Virtual machines
- Platform-as-a-Service providers (Railway, Render, Fly.io, etc.)

Ensure your deployment has:
1. Network access to your SurrealDB instance
2. Environment variables configured correctly
3. The `start()` function called on server initialization

<Callout type="info">
The SurrealDB World is not compatible with Vercel deployments. On Vercel, workflows automatically use the [Vercel World](/worlds/vercel) with zero configuration.
</Callout>

## Limitations

- **Requires long-running process** — Must call `start()` on server initialization; not compatible with serverless platforms
- **SurrealDB infrastructure** — Requires a SurrealDB instance (self-hosted or [Surreal Cloud](https://surrealdb.com/cloud))
- **Not compatible with Vercel** — Use the [Vercel World](/worlds/vercel) for Vercel deployments
- **Community-maintained** — Not an official Vercel package; support via [GitHub Issues](https://github.com/sepcnt/workflow-world-surrealdb/issues)

For local development, use the [Local World](/worlds/local) which requires no external services.
4 changes: 2 additions & 2 deletions scripts/create-community-worlds-matrix.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -50,9 +50,9 @@ const matrix = {
let serviceType = 'none';
if (world.services && world.services.length > 0) {
// Use the first service's name as the service type
// Currently supports: mongodb, redis
// Currently supports: mongodb, redis, surrealdb
const serviceName = world.services[0].name;
if (['mongodb', 'redis'].includes(serviceName)) {
if (['mongodb', 'redis', 'surrealdb'].includes(serviceName)) {
serviceType = serviceName;
}
}
Expand Down
32 changes: 32 additions & 0 deletions worlds-manifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,38 @@
}
]
},
{
"id": "surrealdb",
"type": "community",
"package": "workflow-world-surrealdb",
"name": "SurrealDB",
"description": "SurrealDB world using LIVE SELECT for queueing and real-time streaming",
"repository": "https://github.com/sepcnt/workflow-world-surrealdb",
"docs": "/docs/deploying/world/surrealdb-world",
"features": [],
"env": {
"WORKFLOW_TARGET_WORLD": "workflow-world-surrealdb",
"WORKFLOW_SURREAL_URL": "ws://127.0.0.1:8000/rpc",
"WORKFLOW_SURREAL_NAMESPACE": "workflow",
"WORKFLOW_SURREAL_DATABASE": "workflow",
"WORKFLOW_SURREAL_USERNAME": "root",
"WORKFLOW_SURREAL_PASSWORD": "root"
},
"services": [
{
"name": "surrealdb",
"image": "surrealdb/surrealdb:v3",
"ports": ["8000:8000"],
"healthCheck": {
"cmd": "curl -sf http://localhost:8000/health || exit 1",
"interval": "5s",
"timeout": "3s",
"retries": 10
}
}
],
"setup": "pnpm exec workflow-surreal-setup"
},
{
"id": "jazz",
"type": "community",
Expand Down