Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
162 changes: 162 additions & 0 deletions .agents/skills/adapter-dev/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
---
name: adapter-dev
description: "OpenCLI adapter development guide — creating YAML pipeline or TypeScript adapters for new websites/apps. Use this skill whenever you need to create a new opencli adapter, add a command for a new platform, write a YAML pipeline, write a TypeScript CLI adapter, choose between YAML vs TS, debug pipeline steps, fix adapter issues, or understand opencli's adapter architecture. Triggers on: editing files in src/clis/, creating .yaml or .ts adapter files, 'add command for X', 'create adapter', 'new site', pipeline debugging."
---

# Adapter Development Guide

> 为 opencli 创建新 adapter 的完整指南。涵盖 YAML pipeline 和 TypeScript adapter 两种方式。

> [!IMPORTANT]
> **开始前必须阅读 [CLI-EXPLORER.md](../../../CLI-EXPLORER.md)**,它包含 API 发现工作流和认证策略决策树。
> 快速模式(一个命令)看 [CLI-ONESHOT.md](../../../CLI-ONESHOT.md)。

## 选择实现方式

| 场景 | 方式 | 原因 |
|------|------|------|
| Read + 简单 API(纯 fetch/select/map) | **YAML pipeline** | 声明式,通常 10-30 行 |
| Read + GraphQL/分页/签名/复杂逻辑 | **TypeScript adapter** | 需要 JS 逻辑 |
| Write 操作(DOM 点击/输入) | **TS + `Strategy.UI`** | UI 自动化 |
| Write + API(直接 POST) | **TS + `Strategy.COOKIE/HEADER`** | API 调用 |

## 收口规则(必须遵守)

1. 主参数优先用 positional arg — 不要默认做成 `--query` / `--id` / `--url`
2. 预期中的 adapter 失败优先抛 `CliError` 子类,不要直接 throw 原始 `Error`
3. 新增 adapter 时同步更新 `docs/adapters/index.md`、sidebar、README

## YAML Pipeline Adapter

Create `src/clis/<site>/<name>.yaml`,自动发现无需手动注册:

### Cookie 策略(最常见)

```yaml
site: mysite
name: hot
description: Hot topics
domain: www.mysite.com
strategy: cookie
browser: true

args:
limit:
type: int
default: 20

pipeline:
- navigate: https://www.mysite.com
- evaluate: |
(async () => {
const res = await fetch('/api/hot', { credentials: 'include' });
const d = await res.json();
return d.data.items.map(item => ({
title: item.title, score: item.score,
}));
})()
- map:
rank: ${{ index + 1 }}
title: ${{ item.title }}
score: ${{ item.score }}
- limit: ${{ args.limit }}

columns: [rank, title, score]
```

### Public API 策略(无需 browser)

```yaml
strategy: public
browser: false

pipeline:
- fetch:
url: https://api.example.com/hot.json
- select: data.items
- map:
title: ${{ item.title }}
- limit: ${{ args.limit }}
```

## TypeScript Adapter

Create `src/clis/<site>/<name>.ts`,同样自动发现(不要手动 import):

```typescript
import { cli, Strategy } from '../../registry.js';

cli({
site: 'mysite',
name: 'search',
description: '搜索',
strategy: Strategy.INTERCEPT,
args: [{ name: 'query', required: true, positional: true }],
columns: ['rank', 'title', 'url'],
func: async (page, kwargs) => {
await page.goto('https://www.mysite.com/search');
await page.installInterceptor('/api/search');
await page.autoScroll({ times: 3, delayMs: 2000 });

const requests = await page.getInterceptedRequests();
let results = [];
for (const req of requests) {
results.push(...req.data.items);
}
return results.map((item, i) => ({
rank: i + 1, title: item.title, url: item.url,
}));
},
});
```

**何时用 TS**:XHR interception (`page.installInterceptor`)、infinite scrolling (`page.autoScroll`)、cookie extraction、GraphQL unwrapping 等复杂逻辑。

## Pipeline Steps Reference

| Step | Description | Example |
|------|-------------|---------|
| `navigate` | Go to URL | `navigate: https://example.com` |
| `fetch` | HTTP request (browser cookies) | `fetch: { url: "...", params: { q: "..." } }` |
| `evaluate` | Run JS in page | `evaluate: \| (async () => { ... })()` |
| `select` | Extract JSON path | `select: data.items` |
| `map` | Map fields | `map: { title: "${{ item.title }}" }` |
| `filter` | Filter items | `filter: item.score > 100` |
| `sort` | Sort items | `sort: { by: score, order: desc }` |
| `limit` | Cap result count | `limit: ${{ args.limit }}` |
| `intercept` | Declarative XHR capture | `intercept: { trigger: "navigate:...", capture: "api/hot" }` |
| `tap` | Store action + XHR capture | `tap: { store: "feed", action: "fetchFeeds", capture: "homefeed" }` |
| `snapshot` | Page accessibility tree | `snapshot: { interactive: true }` |
| `click` | Click element | `click: ${{ ref }}` |
| `type` | Type text | `type: { ref: "@1", text: "hello" }` |
| `wait` | Wait for time/text | `wait: 2` or `wait: { text: "loaded" }` |
| `press` | Press key | `press: Enter` |

## Template Syntax

```yaml
${{ args.query }} # 参数引用
${{ args.limit | default(20) }} # 带默认值
${{ item.title }} # 当前 item(map/filter 中)
${{ item.data.nested.field }} # 嵌套字段
${{ index }} # 0-based 索引
${{ index + 1 }} # 1-based
```

## Verification Checklist

```bash
npx tsc --noEmit # TypeScript 编译检查
opencli list | grep <site> # 确认命令已注册
opencli <site> <command> --limit 3 -f json # 实际运行
opencli <site> <command> --limit 3 -v # verbose 看 pipeline
```

## Common Pitfalls

| 问题 | 原因 | 解决 |
|------|------|------|
| `Target page context` error | evaluate 之前没有 navigate | 加 `navigate:` 步骤 |
| Empty table | evaluate 返回的数据路径错误 | 用 `-v` 查看 pipeline 输出 |
| Cookie 失效 | Chrome 未登录目标网站 | 先在 Chrome 登录 |
| TS adapter 未注册 | 手动 import 了文件 | 删除手动 import,自动发现 |
2 changes: 1 addition & 1 deletion .agents/skills/cross-project-adapter-migration/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: cross-project-adapter-migration
description: "Cross-project CLI command migration workflow for opencli. Use when importing commands from external CLI projects (python/node) like rdt-cli, twitter-cli, etc. Covers: source analysis → gap matrix → batch migration → README/SKILL.md update."
description: "Cross-project CLI command migration workflow for opencli. Use whenever importing, porting, or migrating commands from external CLI projects (Python/Node/Go) into opencli — e.g. rdt-cli, twitter-cli, or any third-party CLI tool for a platform opencli supports. Covers: source project analysis → feature gap matrix → batch adapter implementation → doc updates. Triggers on: '把 xxx-cli 迁移过来', 'migrate commands from X', '对齐 xxx 功能', 'import from external CLI', 'port X commands', or when expanding opencli support for a platform that already has a third-party CLI."
---

# Cross-Project Adapter Migration
Expand Down
109 changes: 109 additions & 0 deletions .agents/skills/record-workflow/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
---
name: record-workflow
description: "OpenCLI record workflow — capture browser API calls by manually operating a page, then generate YAML adapter candidates. Use this skill when the user mentions 'record', 'opencli record', wants to capture API calls from a webpage, needs to generate adapters from manual browsing, converts captured JSON to YAML/TS adapters, or troubleshoots record output. Also triggers on: editing .opencli/record/ files, 'captured.json', 'candidates/', record debugging."
---

# Record Workflow

> `record` 是为「无法用 `explore` 自动发现」的页面准备的手动录制方案。
> 适用场景:需要登录操作、复杂交互、SPA 内路由的页面。

## 工作原理

```
opencli record <url>
→ 打开 automation window 并导航到目标 URL
→ 向所有 tab 注入 fetch/XHR 拦截器(幂等,可重复注入)
→ 每 2s 轮询:发现新 tab 自动注入,drain 所有 tab 的捕获缓冲区
→ 超时(默认 60s)或按 Enter 停止
→ 分析捕获 JSON:去重 → 评分 → 生成候选 YAML
```

**拦截器特性**:
- 同时 patch `window.fetch` 和 `XMLHttpRequest`
- 只捕获 `Content-Type: application/json` 的响应
- 过滤纯对象少于 2 个 key 的响应(避免 tracking/ping)
- 跨 tab 隔离,幂等注入

## 使用步骤

```bash
# 1. 启动录制
opencli record "https://example.com/page" --timeout 120000

# 2. 在 automation window 里正常操作页面
# 打开列表、搜索、点击条目、切换 Tab — 触发网络请求的操作都会被捕获

# 3. 按 Enter 停止(或等超时)

# 4. 查看结果
cat .opencli/record/<site>/captured.json # 原始捕获
ls .opencli/record/<site>/candidates/ # 候选 YAML
```

### 命令参数

```bash
opencli record <url> # 录制,site name 从域名推断
opencli record <url> --site mysite # 指定 site name
opencli record <url> --timeout 120000 # 自定义超时(毫秒,默认 60000)
opencli record <url> --poll 1000 # 缩短轮询间隔(毫秒,默认 2000)
opencli record <url> --out .opencli/record/x # 自定义输出目录
```

### 输出结构

```
.opencli/record/<site>/
├── captured.json ← 原始捕获数据(url/method/body)
└── candidates/*.yaml ← 高置信度候选适配器(score ≥ 8,有 array 结果)
```

## 页面类型与捕获预期

| 页面类型 | 预期捕获量 | 说明 |
|---------|-----------|------|
| 列表/搜索页 | 多(5~20+) | 每次搜索/翻页触发新请求 |
| 详情页(只读) | 少(1~5) | 首屏数据一次性返回 |
| SPA 内路由 | 中等 | 路由切换触发新接口,但首屏请求在注入前已发出 |
| 需要登录 | 视操作而定 | 确保 Chrome 已登录目标网站 |

> **注意**:SSR 页面在导航完成前就发出的请求会被错过。
> 解决方案:手动触发新请求(搜索、翻页、展开折叠项等)。

## 候选 YAML → TS CLI 转换

候选 YAML 是起点,复杂场景需要转为 TypeScript:

**候选 YAML(自动生成)**:
```yaml
site: tae
name: getList
strategy: cookie
browser: true
pipeline:
- navigate: https://...
- evaluate: |
(async () => {
const res = await fetch('/approval/getList.json?procInsId=...', { credentials: 'include' });
const data = await res.json();
return (data?.content?.operatorRecords || []).map(item => ({ ... }));
})()
```

**转换为 TS**(参考 adapter-dev skill 的模板):

转换要点:
1. URL 中的动态 ID 提取为 `args`
2. `captured.json` 里的真实 body 确定正确数据路径
3. 认证方式:cookie(`credentials: 'include'`),通常不需要额外 header
4. 文件放入 `src/clis/<site>/`,`npm run build` 后自动发现

## 故障排查

| 现象 | 原因 | 解法 |
|------|------|------|
| 捕获 0 条请求 | 拦截器注入失败或页面无 JSON API | `curl localhost:19825/status` 检查 daemon |
| 捕获量少(1~3 条) | 详情页首屏数据已在注入前发出 | 手动操作触发更多请求(搜索/翻页) |
| 候选 YAML 为 0 | 捕获的 JSON 都没有 array 结构 | 直接看 `captured.json` 手写 TS |
| 新 tab 没有被拦截 | 轮询间隔内 tab 已关闭 | 缩短 `--poll 500` |
Loading