Skip to content

CODEX_TEST.md

Latest commit

 

History

History
106 lines (89 loc) · 5.26 KB

CODEX_TEST.md

File metadata and controls

106 lines (89 loc) · 5.26 KB

AgenticFlow CLI — Full Feature Test Plan

You are testing the AgenticFlow CLI (af). The binary is at:

node /Users/sean/WIP/Antigravity-Workspace/agenticflow-js-cli/packages/cli/dist/bin/agenticflow.js

Create a shell alias first:

alias af='node /Users/sean/WIP/Antigravity-Workspace/agenticflow-js-cli/packages/cli/dist/bin/agenticflow.js'

Services Running

Test Plan — Execute All Steps In Order

Phase 1: CLI Basics

  1. af --version — should print 1.0.6
  2. af --help — should list all commands including paperclip and gateway
  3. af doctor --json — should show all checks passing
  4. af whoami --json — should show authenticated user
  5. af discover --json — should list all CLI capabilities
  6. af playbook --list — should list 10 playbooks including quickstart, gateway-setup, deploy-to-paperclip

Phase 2: Playbooks

  1. af playbook quickstart — read and verify it has 5 steps
  2. af playbook gateway-setup — verify it covers webhook, paperclip, linear channels
  3. af playbook deploy-to-paperclip — verify it has full step-by-step guide
  4. af playbook agent-channels — verify it covers all 3 channel types

Phase 3: Agent Operations

  1. af agent list --limit 3 --json — should return array of agents
  2. Pick the first agent ID, then: af agent get --agent-id <ID> --json — should return full agent details
  3. Note the agent name and model from the response

Phase 4: Gateway

  1. af gateway channels — should list paperclip, linear, webhook
  2. curl -s http://localhost:4100/health — should show gateway healthy with connectors
  3. Test generic webhook: Send a simple task to the first agent from Phase 3:
curl -s -X POST http://localhost:4100/webhook/webhook \
  -H "Content-Type: application/json" \
  -d '{"agent_id": "<AGENT_ID>", "message": "Write a one-paragraph summary of what you can do", "task_id": "test-001"}'

Verify: response has status: "completed" and response_length > 0

Phase 5: Paperclip Company Setup

  1. af paperclip company create --name "Codex Test Corp" --budget 50000 — note the company ID
  2. af paperclip company list — verify the company appears
  3. af paperclip company get --company-id <ID> — verify details

Phase 6: Deploy Agents to Paperclip

  1. Deploy the first agent: af paperclip deploy --agent-id <AF_AGENT_ID> --company-id <COMPANY_ID> --role engineer
  2. Note the Paperclip agent ID from the response
  3. af paperclip agent list --company-id <COMPANY_ID> — verify agent is listed
  4. af paperclip connect --company-id <COMPANY_ID> — connect to gateway

Phase 7: Goals and Tasks

  1. af paperclip goal create --company-id <COMPANY_ID> --title "Test all CLI features" --level company --status active — note goal ID
  2. af paperclip goal list --company-id <COMPANY_ID> — verify goal appears
  3. af paperclip issue create --company-id <COMPANY_ID> --title "Write a test report" --description "Summarize all test results" --priority high --assignee <PC_AGENT_ID> --goal-id <GOAL_ID> — note issue ID
  4. af paperclip issue list --company-id <COMPANY_ID> — verify issue appears
  5. af paperclip issue comment --id <ISSUE_ID> --body "Please be thorough in your report" — add a comment

Phase 8: Agent Execution via Gateway

  1. Trigger the agent via the Paperclip channel:
curl -s -X POST http://localhost:4100/webhook/paperclip \
  -H "Content-Type: application/json" \
  -d '{"agentId": "<PC_AGENT_ID>", "runId": "<generate-uuid>", "context": {"issueId": "<ISSUE_ID>", "taskKey": "<IDENTIFIER>", "wakeReason": "Work on your assigned task", "wakeSource": "on_demand"}}'

Use uuidgen or any UUID for runId. Wait for response (may take 30-60s). Verify: response has status: "completed"

  1. af paperclip issue comments --id <ISSUE_ID> — verify agent posted a response comment
  2. af paperclip dashboard --company-id <COMPANY_ID> — check dashboard stats

Phase 9: Approval & Lifecycle

  1. af paperclip approval list --company-id <COMPANY_ID> — list approvals (may be empty)
  2. af paperclip agent pause --id <PC_AGENT_ID> — pause the agent
  3. af paperclip agent get --id <PC_AGENT_ID> — verify status is "paused"
  4. af paperclip agent resume --id <PC_AGENT_ID> — resume the agent
  5. af paperclip agent get --id <PC_AGENT_ID> — verify status is back

Phase 10: Cleanup

  1. af paperclip issue delete --id <ISSUE_ID> — delete the issue
  2. af paperclip goal delete --id <GOAL_ID> — delete the goal
  3. af paperclip company delete --company-id <COMPANY_ID> — delete the company
  4. af paperclip company list — verify company is gone

Reporting

After completing all steps, write a test report to /tmp/af-test-report.md with:

  • Total tests: 40
  • Passed / Failed count
  • Any errors encountered
  • Summary of what works and what doesn't

Important Notes

  • Always add 2>/dev/null to suppress TLS warnings in output
  • Use --json flag where available for parseable output
  • UUIDs from previous steps are needed in later steps — save them
  • The gateway is already running on port 4100
  • Paperclip is running on port 3100