Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions .claude/skills/sdk-docs/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
---
name: databricks-sdk-docs
description: Use this skill when the user asks about Databricks SDK methods, API signatures, parameter types, return types, or how to use specific Databricks APIs programmatically. Triggers on questions like "how do I create a job", "what parameters does X take", "SDK method for Y", or "JobSettings fields".
allowed-tools: mcp__databricks-mcp__databricks_query_sdk_docs
---

# Databricks SDK Documentation Skill

When users ask about Databricks SDK usage, API methods, or type definitions, use the `databricks_query_sdk_docs` MCP tool to find accurate documentation.

## When to Use This Skill

- User asks "how do I create a job/cluster/pipeline using the SDK?"
- User needs method signatures: "what's the signature for Jobs.Create?"
- User asks about type fields: "what fields does CreateJob have?"
- User needs enum values: "what are the possible run lifecycle states?"
- User is confused about SDK API parameters or return types

## How to Query

Use the `databricks_query_sdk_docs` tool with these parameters:

```json
{
"query": "search terms",
"category": "methods|types|enums|services", // optional filter
"service": "jobs|clusters|pipelines|...", // optional filter
"limit": 10 // default 10, max 50
}
```

## Example Queries

| User Question | Tool Query |
|---------------|------------|
| "How do I create a job?" | `{"query": "create job", "category": "methods"}` |
| "What fields does JobSettings have?" | `{"query": "JobSettings", "category": "types"}` |
| "What are the run states?" | `{"query": "run lifecycle state", "category": "enums"}` |
| "List all jobs API methods" | `{"query": "jobs", "service": "jobs", "category": "methods"}` |

## Response Guidelines

After querying, provide:
1. The method signature with parameter types
2. A brief description of what the method does
3. Key parameters the user likely needs
4. A simple code example if applicable

Keep responses focused on what the user asked - don't dump all documentation.

## CLI Fallback

If MCP is unavailable, use the helper script:

```bash
# From the CLI repo root
.claude/skills/sdk-docs/query-sdk-docs.sh "create job"
.claude/skills/sdk-docs/query-sdk-docs.sh "JobSettings" types
.claude/skills/sdk-docs/query-sdk-docs.sh "list" methods jobs 20
```

The script searches the embedded SDK docs index directly using `jq`.
129 changes: 129 additions & 0 deletions .claude/skills/sdk-docs/query-sdk-docs.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
#!/usr/bin/env bash
#
# Query Databricks SDK documentation from the command line.
# Usage: ./query-sdk-docs.sh <query> [category] [service] [limit]
#
# Examples:
# ./query-sdk-docs.sh "create job"
# ./query-sdk-docs.sh "JobSettings" types
# ./query-sdk-docs.sh "list" methods jobs
# ./query-sdk-docs.sh "cluster" methods compute 20
#
# Categories: methods, types, enums, services
# Services: jobs, clusters, pipelines, workspace, etc.

set -euo pipefail

QUERY="${1:-}"
CATEGORY="${2:-}"
SERVICE="${3:-}"
LIMIT="${4:-10}"

if [[ -z "$QUERY" ]]; then
echo "Usage: $0 <query> [category] [service] [limit]"
echo ""
echo "Examples:"
echo " $0 'create job' # Search for 'create job'"
echo " $0 'JobSettings' types # Search types for 'JobSettings'"
echo " $0 'list' methods jobs # Search jobs service methods for 'list'"
echo ""
echo "Categories: methods, types, enums, services"
exit 1
fi

# Build the JSON input for the MCP tool
build_json_input() {
local json="{\"query\": \"$QUERY\""

if [[ -n "$CATEGORY" ]]; then
json+=", \"category\": \"$CATEGORY\""
fi

if [[ -n "$SERVICE" ]]; then
json+=", \"service\": \"$SERVICE\""
fi

json+=", \"limit\": $LIMIT}"
echo "$json"
}

# Try to find the SDK docs index file for direct search
SDK_DOCS_INDEX="${SDK_DOCS_INDEX:-}"
if [[ -z "$SDK_DOCS_INDEX" ]]; then
# Look for the index in common locations
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
CLI_ROOT="$(cd "$SCRIPT_DIR/../../.." && pwd)"

POSSIBLE_PATHS=(
"$CLI_ROOT/experimental/aitools/lib/providers/sdkdocs/sdk_docs_index.json"
"./sdk_docs_index.json"
)

for path in "${POSSIBLE_PATHS[@]}"; do
if [[ -f "$path" ]]; then
SDK_DOCS_INDEX="$path"
break
fi
done
fi

# If we have jq and the index file, do a direct search
if command -v jq &>/dev/null && [[ -n "$SDK_DOCS_INDEX" && -f "$SDK_DOCS_INDEX" ]]; then
echo "Searching SDK docs for: $QUERY"
echo "---"

QUERY_LOWER=$(echo "$QUERY" | tr '[:upper:]' '[:lower:]')

# Search methods
if [[ -z "$CATEGORY" || "$CATEGORY" == "methods" ]]; then
echo ""
echo "## Methods"
jq -r --arg q "$QUERY_LOWER" --arg svc "$SERVICE" '
.services | to_entries[] |
select($svc == "" or .key == $svc) |
.key as $service |
.value.methods // {} | to_entries[] |
select(
(.key | ascii_downcase | contains($q)) or
(.value.description // "" | ascii_downcase | contains($q))
) |
"- \($service).\(.key): \(.value.description // "No description")[signature: \(.value.signature // "N/A")]"
' "$SDK_DOCS_INDEX" 2>/dev/null | head -n "$LIMIT" || echo " (no matches)"
fi

# Search types
if [[ -z "$CATEGORY" || "$CATEGORY" == "types" ]]; then
echo ""
echo "## Types"
jq -r --arg q "$QUERY_LOWER" '
.types // {} | to_entries[] |
select(
(.key | ascii_downcase | contains($q)) or
(.value.description // "" | ascii_downcase | contains($q))
) |
"- \(.key): \(.value.description // "No description")"
' "$SDK_DOCS_INDEX" 2>/dev/null | head -n "$LIMIT" || echo " (no matches)"
fi

# Search enums
if [[ -z "$CATEGORY" || "$CATEGORY" == "enums" ]]; then
echo ""
echo "## Enums"
jq -r --arg q "$QUERY_LOWER" '
.enums // {} | to_entries[] |
select(
(.key | ascii_downcase | contains($q)) or
(.value.description // "" | ascii_downcase | contains($q))
) |
"- \(.key): \(.value.values // [] | join(", "))"
' "$SDK_DOCS_INDEX" 2>/dev/null | head -n "$LIMIT" || echo " (no matches)"
fi
else
# Fallback: show how to use the MCP tool
echo "SDK docs index not found locally. Use the MCP tool instead:"
echo ""
echo "databricks_query_sdk_docs with input:"
build_json_input
echo ""
echo "Or set SDK_DOCS_INDEX environment variable to point to sdk_docs_index.json"
fi
3 changes: 3 additions & 0 deletions .github/workflows/check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -55,3 +55,6 @@ jobs:
run: |-
make checks
git diff --exit-code

- name: Verify SDK docs index is up to date
run: make verify-sdk-docs-index
114 changes: 114 additions & 0 deletions .github/workflows/update-sdk-docs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
name: update-sdk-docs

# This workflow automatically updates the SDK docs index when:
# 1. Manually triggered
# 2. On a schedule (daily) to catch SDK updates
# 3. When go.mod changes (SDK version bump)

on:
workflow_dispatch:
inputs:
create_pr:
description: 'Create a PR with the changes'
required: false
default: 'true'
type: boolean

schedule:
# Run daily at 6 AM UTC to check for SDK updates
- cron: '0 6 * * *'

push:
branches:
- main
paths:
- 'go.mod'
- 'go.sum'

jobs:
update-sdk-docs:
runs-on: ubuntu-latest

# Don't run on forks
if: github.repository == 'databricks/cli'

permissions:
contents: write
pull-requests: write

steps:
- name: Checkout repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
# Use a token that can trigger workflows
token: ${{ secrets.GITHUB_TOKEN }}

- name: Setup Go
uses: actions/setup-go@4dc6199c7b1a012772edbd06daecab0f50c9053c # v6.1.0
with:
go-version-file: go.mod

- name: Get current SDK version
id: sdk-version
run: |
SDK_VERSION=$(go list -m -json github.com/databricks/databricks-sdk-go | jq -r '.Version')
echo "version=$SDK_VERSION" >> $GITHUB_OUTPUT
echo "Current SDK version: $SDK_VERSION"

- name: Regenerate SDK docs index
run: make sdk-docs-index

- name: Check for changes
id: changes
run: |
if git diff --quiet experimental/aitools/lib/providers/sdkdocs/sdk_docs_index.json; then
echo "changed=false" >> $GITHUB_OUTPUT
echo "No changes detected in SDK docs index"
else
echo "changed=true" >> $GITHUB_OUTPUT
echo "Changes detected in SDK docs index"
git diff --stat experimental/aitools/lib/providers/sdkdocs/sdk_docs_index.json
fi

- name: Commit and push changes (direct to main)
if: steps.changes.outputs.changed == 'true' && github.event_name == 'push'
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git add experimental/aitools/lib/providers/sdkdocs/sdk_docs_index.json
git commit -m "Auto-update SDK docs index for ${{ steps.sdk-version.outputs.version }}"
git push

- name: Create Pull Request
if: steps.changes.outputs.changed == 'true' && (github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')
uses: peter-evans/create-pull-request@v7
with:
token: ${{ secrets.GITHUB_TOKEN }}
commit-message: "Auto-update SDK docs index for ${{ steps.sdk-version.outputs.version }}"
title: "Auto-update SDK docs index for SDK ${{ steps.sdk-version.outputs.version }}"
body: |
This PR was automatically generated to update the SDK documentation index.

**SDK Version:** ${{ steps.sdk-version.outputs.version }}

The index was regenerated because:
- ${{ github.event_name == 'schedule' && 'Scheduled daily check detected changes' || 'Manual workflow trigger' }}

## What changed
The SDK docs index (`experimental/aitools/lib/providers/sdkdocs/sdk_docs_index.json`)
was regenerated from the current Go SDK to include any new or updated:
- Service methods and signatures
- Type definitions
- Enum values

## Verification
- [ ] CI checks pass
- [ ] Index contains expected services

---
*This PR was auto-generated by the update-sdk-docs workflow.*
branch: auto-update-sdk-docs
delete-branch: true
labels: |
automated
sdk-docs
8 changes: 7 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,12 @@ snapshot-release:
schema:
go run ./bundle/internal/schema ./bundle/internal/schema ./bundle/schema/jsonschema.json

sdk-docs-index:
go run ./tools/gen_sdk_docs_index.go

verify-sdk-docs-index:
python3 ./tools/verify_sdk_docs_index.py

docs:
go run ./bundle/docsgen ./bundle/internal/schema ./bundle/docsgen

Expand Down Expand Up @@ -171,7 +177,7 @@ generate:
$(GENKIT_BINARY) update-sdk


.PHONY: lint lintfull tidy lintcheck fmt fmtfull test test-unit test-acc test-slow test-slow-unit test-slow-acc cover showcover build snapshot snapshot-release schema integration integration-short acc-cover acc-showcover docs ws wsfix links checks test-update test-update-templates generate-out-test-toml test-update-aws test-update-all generate-validation
.PHONY: lint lintfull tidy lintcheck fmt fmtfull test test-unit test-acc test-slow test-slow-unit test-slow-acc cover showcover build snapshot snapshot-release schema sdk-docs-index verify-sdk-docs-index integration integration-short acc-cover acc-showcover docs ws wsfix links checks test-update test-update-templates generate-out-test-toml test-update-aws test-update-all generate-validation

test-exp-aitools:
make test TEST_PACKAGES="./experimental/aitools/..." ACCEPTANCE_TEST_FILTER="TestAccept/apps"
Expand Down
1 change: 1 addition & 0 deletions experimental/aitools/lib/prompts/flow.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
- **databricks_discover**: MUST call first - returns scaffolding commands
- **invoke_databricks_cli**: Execute CLI commands including init-template for scaffolding
- **databricks_configure_auth**: Switch workspace profile/host
- **databricks_query_sdk_docs**: Search SDK documentation for methods, types, and examples

## Critical Workflow Rules
1. ALWAYS call databricks_discover FIRST to get scaffolding guidance
Expand Down
Loading