Skip to content

Add PPL bugfix skill for Claude Code#5307

Open
qianheng-aws wants to merge 10 commits intoopensearch-project:mainfrom
qianheng-aws:bugfix-skill
Open

Add PPL bugfix skill for Claude Code#5307
qianheng-aws wants to merge 10 commits intoopensearch-project:mainfrom
qianheng-aws:bugfix-skill

Conversation

@qianheng-aws
Copy link
Copy Markdown
Collaborator

@qianheng-aws qianheng-aws commented Apr 3, 2026

Summary

Adds a /ppl-bugfix slash command for Claude Code that automates PPL bug fixing end-to-end: triage, classification, TDD-style fix, testing, and PR creation — all driven by a structured harness.

Who is this for?

Developers working on the OpenSearch SQL plugin who use Claude Code as their coding assistant. The command turns a GitHub issue number into a draft PR with fix, tests, and decision log.

How it works

/ppl-bugfix #1234                    # Fix a single issue
/ppl-bugfix PR#5293                  # Follow up on an existing PR (CI failures, review feedback, merge conflicts)
/ppl-bugfix #1234 #5678 PR#9012     # Process multiple in parallel

Each issue/PR dispatches an autonomous subagent in an isolated git worktree (no interference with your working directory). The subagent follows a structured harness:

  1. Phase 0 — Reproduce bug, classify layer (Grammar / AST / Type System / Optimizer / Execution / DI), route to fix path
  2. Phase 1 — Implement fix following layer-specific guidance with historical commit references
  3. Phase 2 — TDD: write failing test first, then fix, then complete coverage (unit + integration + YAML REST)
  4. Phase 3 — Verify (spotless + full test suite), commit with DCO, create draft PR, post Decision Log

Permission modes control how much autonomy the subagent has:

  • --yolo (default) — bypassPermissions, zero prompts, fastest
  • --safeacceptEdits only, Bash commands require manual approval

Files

File Purpose
.claude/commands/ppl-bugfix.md Slash command definition — input parsing, issue/PR resolution, subagent dispatch
.claude/harness/ppl-bugfix-harness.md Full bugfix workflow (Phase 0-4) with fix paths, test templates, case index
.claude/harness/ppl-bugfix-followup.md Post-PR follow-up: review feedback, CI failures, merge conflicts, commit cleanup
.claude/settings.json Pre-approved tool permissions (gradle, gh, git commands)
CLAUDE_GUIDE.md User-facing documentation for all slash commands

Example PRs produced by this command

Test plan

  • Run /ppl-bugfix #<issue> on a real open issue — verify subagent dispatches in worktree, follows harness phases, creates draft PR
  • Run /ppl-bugfix PR#<number> on an existing PR with review comments — verify follow-up agent addresses feedback
  • Run /ppl-bugfix #1234 #5678 — verify parallel dispatch of multiple subagents
  • Run /ppl-bugfix #1234 --safe — verify acceptEdits mode (Bash commands prompt for approval)

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 3, 2026

PR Code Analyzer ❗

AI-powered 'Code-Diff-Analyzer' found issues on commit e2ff864.

PathLineSeverityDescription
.claude/harness/ppl-bugfix-harness.md109mediumHardcoded fallback fork owner 'qianheng-aws': if git user config is missing or wrong, the AI agent will attempt to push branches and create PRs to this specific user's fork. Any contributor without a properly configured git identity would silently redirect their code contributions to this third-party fork.
.claude/settings.json1mediumThis settings file is now tracked in version control (see .gitignore change), meaning all repo contributors who use Claude Code will automatically inherit these broad permissions including git push, force-with-lease push, and all gh pr/issue/run operations — without opting in individually. This widens the blast radius of any compromised or misconfigured AI agent session.
.claude/commands/ppl-bugfix.md22lowbypassPermissions mode is set as the default for subagent dispatch. This means the AI agent will execute git push, gh pr create, and other write operations against GitHub without any user confirmation prompt. While the subagent runs in an isolated worktree, the GitHub-side actions (PR creation, issue comments, issue close) are not isolated and are immediately visible externally.
.claude/harness/ppl-bugfix-followup.md55lowThe follow-up harness instructs the agent to perform force pushes via 'git push fork clean-branch: --force-with-lease' during commit history cleanup. While --force-with-lease is safer than --force, combining this with bypassPermissions default means history-rewriting on shared PR branches can occur without user review.

The table above displays the top 10 most important findings.

Total: 4 | Critical: 0 | High: 0 | Medium: 2 | Low: 2


Pull Requests Author(s): Please update your Pull Request according to the report above.

Repository Maintainer(s): You can bypass diff analyzer by adding label skip-diff-analyzer after reviewing the changes carefully, then re-run failed actions. To re-enable the analyzer, remove the label, then re-run all actions.


⚠️ Note: The Code-Diff-Analyzer helps protect against potentially harmful code patterns. Please ensure you have thoroughly reviewed the changes beforehand.

Thanks.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 3, 2026

PR Reviewer Guide 🔍

(Review updated until commit e2ff864)

Here are some key observations to aid the review process:

🧪 No relevant tests
🔒 Security concerns

Sensitive information exposure:
The .claude/settings.json file is committed to the repository and grants broad bypassPermissions-compatible Bash access including gh pr:*, gh api:*, and git push commands. Any contributor or CI system with access to this repo inherits these permission grants. Additionally, the harness hardcodes a specific GitHub username (qianheng-aws) as a fallback fork owner, which could expose internal contributor information or cause unintended pushes to that user's fork.

✅ No TODO sections
🔀 Multiple PR themes

Sub-PR theme: Add PPL bugfix Claude Code slash command and harness files

Relevant files:

  • .claude/commands/ppl-bugfix.md
  • .claude/harness/ppl-bugfix-harness.md
  • .claude/harness/ppl-bugfix-followup.md
  • .claude/harness/ppl-bugfix-reference.md
  • .claude/settings.json

Sub-PR theme: Update developer documentation for Claude commands and build reference

Relevant files:

  • CLAUDE.md
  • CLAUDE_GUIDE.md

⚡ Recommended focus areas for review

Overly Broad Permissions

The settings.json grants broad Bash permissions including git reset --soft:*, git push --force-with-lease:*, and git merge:* with wildcards. Combined with bypassPermissions mode being the default, a subagent could potentially overwrite branches or reset history in unintended ways. Consider scoping these permissions more tightly or documenting the risk explicitly.

{
  "permissions": {
    "allow": [
      "Bash(./gradlew *)",
      "Bash(gh issue:*)",
      "Bash(gh pr:*)",
      "Bash(gh api:*)",
      "Bash(gh search:*)",
      "Bash(gh run:*)",
      "Bash(git add:*)",
      "Bash(git commit:*)",
      "Bash(git stash:*)",
      "Bash(git show:*)",
      "Bash(git diff:*)",
      "Bash(git status:*)",
      "Bash(git log:*)",
      "Bash(git branch:*)",
      "Bash(git remote:*)",
      "Bash(git fetch:*)",
      "Bash(git checkout:*)",
      "Bash(git push -u:*)",
      "Bash(git push --force-with-lease:*)",
      "Bash(git merge:*)",
      "Bash(git cherry-pick:*)",
      "Bash(git reset --soft:*)"
    ]
  }
}
Default Yolo Mode

The default permission mode is bypassPermissions (--yolo), meaning all subagents run with no prompts by default. While the worktree isolation mitigates some risk, a misconfigured or misbehaving subagent could still push branches, create PRs, or close issues without any human confirmation. Consider making --safe the default and requiring explicit opt-in for --yolo.

- `--yolo``bypassPermissions` mode. Fully trusted, no prompts. Subagent runs in an isolated worktree so this is safe. (Default)
Hardcoded Fork Owner

The harness falls back to a hardcoded fork owner qianheng-aws when the git user name cannot be inferred. This will cause push failures for any other contributor using this command, and could silently push to the wrong fork remote.

Do NOT add Co-Authored-By lines. Use the git user name to infer the fork owner, or fall back to "qianheng-aws".
Missing worktree setup

The follow-up harness instructs the agent to gh pr checkout <pr_number> but does not include steps to initialize the worktree (e.g., git worktree add) before checkout. If the worktree is truly fresh/isolated as described, the branch may not exist locally and the checkout could fail without proper setup instructions.

The follow-up agent runs in a fresh worktree. First checkout the PR branch, then load state:

```bash
# Checkout the PR branch in this worktree
gh pr checkout <pr_number>

# Resolve fork remote — the worktree may only have origin (upstream)
git remote -v
# If no fork remote exists, add it:
git remote add fork https://github.com/<fork_owner>/sql.git

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 3, 2026

PR Code Suggestions ✨

Latest suggestions up to e2ff864

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
Possible issue
Ensure git push permissions cover all required variants

The git push -u permission pattern may not match git push -u fork as intended,
since the -u flag is part of the argument string. Additionally, a plain git push
(without -u or --force-with-lease) is not covered, which could block legitimate push
operations. Consider adding a broader git push pattern or verifying the exact
pattern syntax used by Claude Code's permission system.

.claude/settings.json [21-22]

-"Bash(git push -u:*)",
-"Bash(git push --force-with-lease:*)",
+"Bash(git push:*)",
Suggestion importance[1-10]: 5

__

Why: The concern about git push -u pattern matching is valid — if the permission system matches on prefix, git push -u may not match git push -u fork <branch>. However, replacing with a broad git push:* could be overly permissive. The suggestion is reasonable but the improved code may be too broad.

Low
Resolve fork owner placeholder in follow-up harness

The <fork_owner> placeholder is never resolved in the follow-up harness — unlike the main
harness which instructs the agent to infer it from the git user name or fall back to
"qianheng-aws". Without this instruction, the follow-up agent may fail to push to
the correct fork remote. Add the same fork-owner resolution guidance here.

.claude/harness/ppl-bugfix-followup.md [20]

-git remote add fork https://github.com/<fork_owner>/sql.git
+# Infer fork_owner from git config user or fall back to "qianheng-aws"
+git remote add fork https://github.com/$(git config user.name || echo "qianheng-aws")/sql.git
Suggestion importance[1-10]: 4

__

Why: The <fork_owner> placeholder is indeed unresolved in the follow-up harness, unlike the main harness. However, the improved_code uses a shell command that may not work as intended (git config user.name returns a full name, not a GitHub username), making the suggested fix potentially incorrect.

Low
General
Handle merge conflicts explicitly before pushing

The merge step happens after the commit but before pushing, which means merge
conflicts could arise silently and the resulting merge commit would be pushed
without re-running the full test suite. Consider making the merge and
re-verification a conditional step with explicit conflict-resolution guidance, or
move the merge to before the commit.

.claude/harness/ppl-bugfix-harness.md [79-80]

 git fetch origin && git merge origin/main
+# If conflicts arise, resolve them before continuing
+./gradlew spotlessApply
 ./gradlew test && ./gradlew :integ-test:integTest -Dtests.class="*<YourIT>"
+# Commit merge resolution if needed
+git commit -s -m "Merge origin/main into fix branch"
Suggestion importance[1-10]: 3

__

Why: The suggestion to add explicit conflict-resolution guidance is reasonable, but the improved_code only adds comments and a spotlessApply call without fundamentally changing the flow. The concern is valid but the improvement is minor and the harness already has a dedicated merge conflict section in the follow-up file.

Low

Previous suggestions

Suggestions up to commit 42caa29
CategorySuggestion                                                                                                                                    Impact
Possible issue
Remove hardcoded username fallback for fork owner

Hardcoding a specific GitHub username (qianheng-aws) as a fallback fork owner is a
critical bug — this will cause pushes to fail or push to the wrong fork for any
other contributor. The fallback should instead prompt the user or fail with a clear
error message rather than silently using a hardcoded username.

.claude/harness/ppl-bugfix-harness.md [241]

-# Use the git user name to infer the fork owner, or fall back to "qianheng-aws"
+# Use the git user name to infer the fork owner; if unknown, prompt the user for the fork owner before proceeding
Suggestion importance[1-10]: 6

__

Why: Hardcoding qianheng-aws as a fallback fork owner is a real issue that would cause failures or incorrect pushes for any other contributor. The suggestion to prompt the user instead is valid, though the improved code only changes a comment rather than adding actual resolution logic.

Low
Resolve unresolved fork owner placeholder

The <fork_owner> placeholder is never resolved in this file — there is no instruction on how to
determine the fork owner in the follow-up harness, unlike the main harness which at
least mentions inferring from git user name. This will cause the command to fail
literally. Add explicit instructions to resolve the fork owner (e.g., from gh api
user --jq .login) before running this command.

.claude/harness/ppl-bugfix-followup.md [20]

-git remote add fork https://github.com/<fork_owner>/sql.git
+# Resolve fork owner from GitHub CLI
+FORK_OWNER=$(gh api user --jq '.login')
+git remote add fork https://github.com/${FORK_OWNER}/sql.git
Suggestion importance[1-10]: 5

__

Why: The <fork_owner> placeholder in the follow-up harness is indeed unresolved with no instructions on how to determine it. Using gh api user --jq '.login' is a concrete and practical solution, though this is a documentation/instruction file so the agent is expected to interpret placeholders contextually.

Low
General
Use dynamic fork owner resolution via CLI

The fork owner resolution is vague — "use the git user name" doesn't specify which
git config key to read, and the fallback is a hardcoded username. Replace this with
a concrete command to resolve the fork owner dynamically using the GitHub CLI, which
is already an allowed tool.

.claude/harness/ppl-bugfix-harness.md [240-241]

-git remote add fork https://github.com/<fork_owner>/sql.git
-# Use the git user name to infer the fork owner, or fall back to "qianheng-aws"
+FORK_OWNER=$(gh api user --jq '.login')
+git remote add fork https://github.com/${FORK_OWNER}/sql.git
Suggestion importance[1-10]: 5

__

Why: Replacing the vague "use the git user name" instruction and hardcoded fallback with gh api user --jq '.login' is more concrete and portable. This overlaps significantly with suggestion 2 but provides a more actionable implementation.

Low
Security
Tighten overly broad git push permission

The permission Bash(git push -u:) is overly broad and would also match git push -u
fork main which could push to unintended branches. More importantly, git push
without --force-with-lease is missing as a standalone permission, but the current
pattern git push -u:
won't match plain git push fork calls used in the harness.
Verify that all git push variants used in the harness are covered and that the
patterns are as restrictive as possible.

.claude/settings.json [21-22]

-"Bash(git push -u:*)",
+"Bash(git push -u fork:*)",
+"Bash(git push fork:*)",
 "Bash(git push --force-with-lease:*)",
Suggestion importance[1-10]: 4

__

Why: The concern about git push -u:* being overly broad has some merit, but the pattern matching in Claude's permission system may work differently than standard glob matching. The suggestion to add Bash(git push fork:*) is reasonable but the security impact is limited since the harness already runs in isolated worktrees.

Low
Suggestions up to commit 45bbb51
CategorySuggestion                                                                                                                                    Impact
Security
Remove hardcoded username fallback for fork remote

Hardcoding a specific GitHub username (qianheng-aws) as a fallback for the fork
owner is a security and correctness concern — it could cause code to be pushed to
the wrong repository if the git user name cannot be resolved. This fallback should
be removed or replaced with a prompt to the user to provide the correct fork owner.

.claude/harness/ppl-bugfix-harness.md [241]

-# Use the git user name to infer the fork owner, or fall back to "qianheng-aws"
+# Use the git user name to infer the fork owner; if unknown, ask the user before proceeding
Suggestion importance[1-10]: 6

__

Why: Hardcoding qianheng-aws as a fallback fork owner is a real concern — it could cause pushes to the wrong repository for other contributors. Replacing it with a prompt to ask the user is a meaningful correctness and security improvement.

Low
General
Add conflict check before committing merge resolution

The merge conflict resolution runs all integration tests unconditionally, which can
be very slow and may not be necessary for simple conflicts. More critically, if git
merge results in conflicts that need manual resolution, the subsequent commands will
fail or commit an incomplete state. Consider adding a conflict-check step before
committing.

.claude/harness/ppl-bugfix-followup.md [80-83]

 git fetch origin && git merge origin/main  # Resolve conflicts
+# Check for unresolved conflicts before proceeding
+git diff --check
 ./gradlew spotlessApply && ./gradlew test && ./gradlew :integ-test:integTest  # Re-verify
 git commit -s -m "Resolve merge conflicts with main"
-git push -u fork <branch_name>
+git push fork <branch_name>
Suggestion importance[1-10]: 5

__

Why: Adding git diff --check before committing is a reasonable safeguard to detect unresolved merge conflicts before they get committed. This is a valid improvement to the harness workflow that could prevent committing broken state.

Low
Possible issue
Fix overly restrictive git push permission patterns

The git push -u permission pattern may not match git push -u fork commands as
intended, since the -u flag is part of the command arguments rather than the command
name. Consider using a broader git push pattern or verifying that the permission
syntax correctly matches the actual commands used in the harness. Additionally, git
push --force-with-lease is a separate pattern that may not cover all force-push
variants used in the follow-up harness.

.claude/settings.json [21-22]

-"Bash(git push -u:*)",
-"Bash(git push --force-with-lease:*)",
+"Bash(git push:*)",
Suggestion importance[1-10]: 4

__

Why: The concern about git push -u pattern matching is valid since permission patterns in Claude settings may need to match the full command string. However, broadening to Bash(git push:*) could be too permissive, and the current patterns may already work correctly depending on how the permission system parses them.

Low
Suggestions up to commit ce54987
CategorySuggestion                                                                                                                                    Impact
General
Scope post-conflict re-verification to affected tests

After resolving merge conflicts, the harness runs the full integration test suite
(integTest) without scoping it to the relevant test class. This can be very slow and
may time out. Consider scoping the re-verification to the affected test class first,
consistent with how Phase 3.1 in the main harness structures verification steps.

.claude/harness/ppl-bugfix-followup.md [81]

-./gradlew spotlessApply && ./gradlew test && ./gradlew :integ-test:integTest  # Re-verify
+./gradlew spotlessApply && ./gradlew test && ./gradlew :integ-test:integTest -Dtests.class="*<YourIT>"  # Re-verify affected IT
+./gradlew :integ-test:integTest  # Full IT regression if time permits
Suggestion importance[1-10]: 5

__

Why: Running the full integTest suite after every merge conflict resolution is indeed slow and could time out. Scoping to the affected test class first is a practical improvement consistent with the main harness's approach in Phase 3.1, making this a reasonable maintainability suggestion.

Low
Ensure all git push variants are permitted

The git push -u permission pattern may not match git push -u fork correctly
depending on how Claude Code parses permission patterns. Additionally, a plain git
push (without flags) is not covered, which could block legitimate push operations.
Consider adding "Bash(git push:*)" to ensure all push variants are permitted.

.claude/settings.json [21-22]

-"Bash(git push -u:*)",
-"Bash(git push --force-with-lease:*)",
+"Bash(git push:*)",
Suggestion importance[1-10]: 4

__

Why: The suggestion raises a valid concern about git push variants not being covered, but the existing patterns Bash(git push -u:*) and Bash(git push --force-with-lease:*) cover the specific push commands used in the harness. Adding a blanket Bash(git push:*) would be more permissive than needed, and the improved_code removes the existing specific patterns rather than adding to them.

Low
Add fork owner fallback in follow-up harness

The fork owner placeholder <fork_owner> is hardcoded as a template variable with no fallback
logic in the follow-up harness, but the main harness (ppl-bugfix-harness.md)
provides a fallback to "qianheng-aws". The follow-up harness should include the same
fallback instruction so agents don't get stuck when the fork owner cannot be
inferred.

.claude/harness/ppl-bugfix-followup.md [20]

 git remote add fork https://github.com/<fork_owner>/sql.git
+# Use the git user name to infer the fork owner, or fall back to "qianheng-aws"
Suggestion importance[1-10]: 4

__

Why: The suggestion correctly identifies an inconsistency between the follow-up harness and the main harness regarding the <fork_owner> fallback. However, the improved_code only adds a comment rather than substantive logic, making this a minor documentation/consistency improvement.

Low
Suggestions up to commit 6b29b60
CategorySuggestion                                                                                                                                    Impact
Possible issue
Include merged/closed PRs in issue resolution lookup

These gh pr list searches only look at open PRs by default. A PR that was already
merged or closed would not be found, causing the agent to incorrectly classify a
follow-up as an "Initial Fix" and create a duplicate PR. Add --state all to each
search to cover merged and closed PRs as well.

.claude/commands/ppl-bugfix.md [47-49]

-gh pr list --search "Resolves #<issue_number>" --json number,url,state --limit 5
-gh pr list --search "Fixes #<issue_number>" --json number,url,state --limit 5
-gh pr list --search "Closes #<issue_number>" --json number,url,state --limit 5
+gh pr list --search "Resolves #<issue_number>" --json number,url,state --state all --limit 5
+gh pr list --search "Fixes #<issue_number>" --json number,url,state --state all --limit 5
+gh pr list --search "Closes #<issue_number>" --json number,url,state --state all --limit 5
Suggestion importance[1-10]: 7

__

Why: This is a valid and important issue — without --state all, merged PRs won't be found, potentially causing the agent to create duplicate PRs for already-fixed issues. The improved_code correctly adds --state all to each search command.

Medium
Add missing plain git push permission

The permission Bash(git push -u:) only matches git push -u as a prefix, but git
push --force-with-lease and plain git push are separate patterns. More critically,
git push without -u or --force-with-lease is not covered, which may block the
subagent when pushing without those flags. Consider adding a plain Bash(git push:
)
entry or verifying the pattern matching behavior of the tool.

.claude/settings.json [21-22]

 "Bash(git push -u:*)",
 "Bash(git push --force-with-lease:*)",
+"Bash(git push:*)",
Suggestion importance[1-10]: 5

__

Why: The suggestion correctly identifies that git push without -u or --force-with-lease flags isn't covered, which could block the subagent in some scenarios. However, the harness files consistently use either git push -u or git push --force-with-lease, so the practical impact may be limited.

Low
General
Handle cherry-pick conflicts before amending commit

After git cherry-pick, the working tree may have conflicts that need resolution
before git commit --amend is valid. Additionally, if the cherry-pick produces a
merge commit or fails, the subsequent amend will operate on the wrong commit. Add a
conflict-check step or note between cherry-pick and amend to handle this case.

.claude/harness/ppl-bugfix-followup.md [52-55]

 git checkout -B clean-branch origin/main
 git cherry-pick <fix_commit_sha>
+# Resolve any conflicts before amending
 git commit --amend -s -m "<updated message>"
 git push <your_fork_remote> clean-branch:<pr_branch> --force-with-lease
Suggestion importance[1-10]: 2

__

Why: The suggestion only adds a comment to the existing code without any functional change, making the existing_code and improved_code effectively equivalent in behavior. This is documentation-level guidance rather than a real fix.

Low
Suggestions up to commit 638f0ce
CategorySuggestion                                                                                                                                    Impact
Possible issue
Include merged/closed PRs in issue resolution lookup

These searches only look for open PRs by default. A closed or merged PR that
resolves the issue would not be found, potentially causing the agent to create a
duplicate PR for an already-fixed issue. Add --state all to each search to catch
merged or closed PRs.

.claude/commands/ppl-bugfix.md [47-49]

-gh pr list --search "Resolves #<issue_number>" --json number,url,state --limit 5
-gh pr list --search "Fixes #<issue_number>" --json number,url,state --limit 5
-gh pr list --search "Closes #<issue_number>" --json number,url,state --limit 5
+gh pr list --search "Resolves #<issue_number>" --json number,url,state --state all --limit 5
+gh pr list --search "Fixes #<issue_number>" --json number,url,state --state all --limit 5
+gh pr list --search "Closes #<issue_number>" --json number,url,state --state all --limit 5
Suggestion importance[1-10]: 6

__

Why: This is a valid functional concern — without --state all, the agent could miss already-merged PRs and create duplicate work. The improved_code accurately reflects the change needed.

Low
Security
Restrict push permissions to fork remote only

The git push permissions are overly broad — git push -u: and git push
--force-with-lease:
allow pushing to any remote and branch. A compromised or
misbehaving subagent could force-push to upstream branches. Consider restricting to
known fork remotes or at minimum documenting that origin should never point to
upstream.

.claude/settings.json [21-22]

-"Bash(git push -u:*)",
-"Bash(git push --force-with-lease:*)",
+"Bash(git push -u <your_fork_remote>:*)",
+"Bash(git push --force-with-lease <your_fork_remote>:*)",
Suggestion importance[1-10]: 3

__

Why: While the security concern about overly broad push permissions is valid, the improved_code uses a literal placeholder <your_fork_remote> which is not a valid permission pattern in settings.json — the actual remote name would need to be hardcoded, which isn't practical for a shared config file.

Low
General
Handle cherry-pick conflicts before amending commit

After git cherry-pick, if the cherry-pick itself creates a commit, running git
commit --amend will amend that cherry-pick commit correctly. However, if the
cherry-pick fails due to conflicts, the subsequent git commit --amend will fail or
produce unexpected results. Add a conflict check or note that conflicts must be
resolved before amending.

.claude/harness/ppl-bugfix-followup.md [52-55]

 git checkout -B clean-branch origin/main
 git cherry-pick <fix_commit_sha>
-git commit --amend -s -m "<updated message>"
+# Resolve any conflicts before proceeding
+git commit --amend -s --no-edit -m "<updated message>"
 git push <your_fork_remote> clean-branch:<pr_branch> --force-with-lease
Suggestion importance[1-10]: 3

__

Why: The suggestion adds a comment about resolving conflicts, but the improved_code also changes git commit --amend -s -m to add --no-edit while still keeping -m, which is contradictory — --no-edit and -m together would conflict. The improvement is marginal and the code change is slightly inaccurate.

Low
Clarify remote naming to avoid push/sync confusion

The git merge origin/main step merges from origin, but the harness instructs pushing
to a personal fork remote (not origin). If origin points to the upstream repo, this
is correct for syncing, but the inconsistency with the push step (which uses <your_fork_remote>) may
confuse the agent. Clarify that origin here refers to the upstream remote, distinct
from the fork remote used for pushing.

.claude/harness/ppl-bugfix-harness.md [235-238]

-# Sync with main (merge, not rebase — PRs use squash-merge)
+# Sync with upstream main (origin = upstream; PRs use squash-merge, so merge not rebase)
 git fetch origin && git merge origin/main
 # Always re-run tests after merge — even a trivial merge can introduce regressions
 ./gradlew test && ./gradlew :integ-test:integTest -Dtests.class="*<YourIT>"
Suggestion importance[1-10]: 2

__

Why: The suggestion only adds a comment clarifying that origin is the upstream remote, which is a documentation-only change with minimal functional impact. The existing_code and improved_code differ only in a comment.

Low

- ppl-bugfix-harness.md: systematic bugfix SOP distilled from 15+
  historical commits, covering triage, TDD-style fix paths (A-E by
  bug layer), test templates (UT/IT/YAML), verification, PR creation,
  decision log, and post-PR follow-up
- .claude/commands/ppl-bugfix.md: slash command that auto-resolves
  issue↔PR, dispatches subagent with worktree isolation and scoped
  permissions, handles both initial fix and follow-up flows
- CLAUDE.md: reference to /ppl-bugfix as the entry point for PPL bugs
- .gitignore: allow .claude/commands/ and settings.json to be tracked

Signed-off-by: Heng Qian <qianheng@amazon.com>
… reorg

- Add --safe/--yolo permission mode flags (default: bypassPermissions)
- Support multiple issue/PR references for parallel processing
- Move harness files to .claude/harness/ directory
- Add ppl-bugfix-followup.md for post-PR follow-up workflow
- Add .claude/settings.json with pre-approved tool permissions
- Add CLAUDE_GUIDE.md documenting all slash commands

Signed-off-by: Heng Qian <qianheng@amazon.com>
@qianheng-aws qianheng-aws added the maintenance Improves code quality, but not the product label Apr 3, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 3, 2026

Persistent review updated to latest commit d4b9b01

The Reconstruct Context section now explicitly loads all PR comments
(bot and human) and categorizes bot suggestions as actionable review
feedback, not just CI checks and human reviews.

Signed-off-by: Heng Qian <qianheng@amazon.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 3, 2026

Persistent review updated to latest commit b9584ee

- Add missing git merge permission to settings.json (used in harness merge steps)
- Search all PR closing keyword variants (Resolves/Fixes/Closes) for issue-PR linkage
- Replace grep -oP with portable grep -oE for macOS compatibility
- Use --force-with-lease instead of --force in cherry-pick cleanup flow
- Make post-merge test re-run mandatory with explicit command
- Add --repo opensearch-project/sql to gh pr create for correct targeting

Signed-off-by: Heng Qian <qianheng@amazon.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 3, 2026

Persistent review updated to latest commit 57bfdb0

- Replace broad `git push:*` with `git push -u:*` and `git push --force-with-lease:*`
  to prevent accidental force-pushes to upstream
- Restrict `git reset:*` to `git reset --soft:*` since the harness only uses soft reset
- Update followup harness push commands to use `-u` flag consistently

Signed-off-by: Heng Qian <qianheng@amazon.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 3, 2026

Persistent review updated to latest commit 638f0ce

Followup agent now reflects on each feedback comment to identify
gaps in the harness that should have prevented the issue, and
fixes them in the same commit.

Signed-off-by: Heng Qian <qianheng@amazon.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 3, 2026

Persistent review updated to latest commit 6b29b60

- Add fork remote discovery step to both harness and followup
  (replace <your_fork_remote> placeholder with concrete instructions)
- Remove redundant follow-up type from command dispatch — subagent
  discovers it from PR state via followup harness categorization table
- Fix Phase 0.1: remove fake REST API call, use integration test only
- Add no-coauthor rule to followup harness (was only in harness)
- Add --repo flag to gh pr checkout in followup for worktree context

Signed-off-by: Heng Qian <qianheng@amazon.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 3, 2026

Persistent review updated to latest commit ce54987

Signed-off-by: Heng Qian <qianheng@amazon.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 3, 2026

Persistent review updated to latest commit 45bbb51

Subagent was ignoring the "stop and report back" instruction and still
creating regression tests + PRs for bugs that no longer reproduce.

- Harness: clarify definition of "not reproducible", replace soft
  "stop and report back" with explicit HARD STOP + forbidden actions
- Skill: add CRITICAL instruction in subagent dispatch prompt so it
  sees the constraint before even reading the harness

Signed-off-by: Heng Qian <qianheng@amazon.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 3, 2026

Persistent review updated to latest commit 42caa29

Copy link
Copy Markdown
Collaborator

@penghuo penghuo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!
Questions, How do we evaluate ppl-bugfix skills? Leverage skills-creator evaluation?

Comment on lines +57 to +58
| Issue exists, no PR | **Initial Fix** (Step 2A) |
| Issue exists, open PR found | **Follow-up** (Step 2B) |
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Include "agent-team review" before publish PR?

penghuo
penghuo previously approved these changes Apr 3, 2026
Harness: 369→151 lines. Move fix paths, test templates, symptom
table, and case index to ppl-bugfix-reference.md (loaded on demand).
Add completion gate with 8 mandatory deliverables. Add guardrails
for runaway agents. Merge overlapping checklists.

CLAUDE.md: trim build commands, remove v2 engine references,
simplify ppl-bugfix section.

Command: add bypassPermissions allow-list note.

Signed-off-by: Heng Qian <qianheng@amazon.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 4, 2026

Persistent review updated to latest commit e2ff864

@LantaoJin
Copy link
Copy Markdown
Member

LantaoJin commented Apr 7, 2026

will this /ppl-bugfix #1234 #5678 fires multiple PRs or single one?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

maintenance Improves code quality, but not the product

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants