Skip to content

Comments

feat: triage and detect duplicates via github actions#1422

Open
geolessel wants to merge 6 commits intomainfrom
geo/ai-triage-and-dupe-detection
Open

feat: triage and detect duplicates via github actions#1422
geolessel wants to merge 6 commits intomainfrom
geo/ai-triage-and-dupe-detection

Conversation

@geolessel
Copy link

@geolessel geolessel commented Feb 12, 2026

As issues are created, this will, in parallel

  • attempt to triage the issue into a product team (or needs-triage if one can't be specified reliably) by applying a label onto the issue. Uses Haiku for cost and speed
  • look for potential duplicate issues from the past 18 months. If ones are found, it will then generate a reason why it may be a duplicate, generate a summary of the key discussion points, and provide a summary of the resolution or outcome. This will hopefully catch some duplicates or otherwise point users to past issues that may solve their particular issue faster than we can respond to them personally. Uses Sonnet for a bit more ability but keeping costs less than Opus.

On a local test run on the data from issue #1399, it correctly labeled the issue as Publishing and output this from its duplicate detection (everything after [DRY RUN] would be added as an issue comment).

Searching for duplicates of issue #1399: "Attaching uploaded image to Episode fails with 500 error"
Using model: claude-sonnet-4-5-20250929
Search keywords: attaching, uploaded, image, episode, fails, 500, error, affected
Found 14 candidate issues to compare
Claude identified 1 potential matches
Fetching full context for 1 matched issues...
Fetched context for 1 of 1 matches
Enriching match summaries with full context...
Enriched 1 match summaries
[DRY RUN] Would post the following comment:

Potentially Related Issues

I found some existing issues that may be related to this one. Please check if any of them address your question before waiting for a response:

🟣 #1350 — Publishing Episode Creation – sermonaudio and art parameters "hash" value (Closed)

Summary: This issue describes the same core problem: attempting to attach uploaded files (audio or artwork) to Publishing episodes results in 500 errors. The user discovered that the documented file upload process returns MD5 hashes or file IDs that cannot be used with episode creation/updates, and that a special 'signed_identifier' value from an undocumented endpoint is actually required.

Key discussion points:

  • The File Upload API returns MD5 hashes and file IDs in format 'us#-UUID', but these cannot be used with the sermon_audio or art parameters
  • The actual required value is a 'signed_identifier' from an undocumented endpoint '/rails/active_storage/sermon_audio_direct_uploads' in format 'base64--SHA1'
  • User provided detailed reproduction steps showing that using the documented file upload approach results in HTTP 500 errors
  • An undocumented '/publishing/v2/files/{file_id}' endpoint exists that returns the proper signed_identifier value
  • Staff member confirmed that attaching files uploaded via upload.planningcenteronline.com to episodes is not currently supported via API
  • Workaround suggested: host files elsewhere (like S3) and use the 'library_audio_url' attribute instead
  • A follow-up comment from another user confirms the issue persists, with 500 errors occurring when trying to use documented file upload IDs with episode creation

Resolution: Staff acknowledged this is not currently supported via the API and added it to the engineering backlog. The issue was closed, but the functionality remains unavailable. The recommended workaround is to host files externally and reference them via URL rather than uploading through the API.


This comment was generated automatically. If none of these match your issue, no action is needed — a team member will review your issue soon.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds automated AI-powered issue triage capabilities to the repository through GitHub Actions. When new issues are opened, the system automatically attempts to classify them by product and detect potential duplicates, improving response times and reducing duplicate work.

Changes:

  • Added GitHub Actions workflow that triggers on new issues to perform automated triage and duplicate detection in parallel
  • Implemented AI-powered product labeling using Claude Haiku to automatically tag issues with appropriate product labels or "needs-triage"
  • Implemented duplicate detection system using Claude Sonnet that searches past 18 months of issues, identifies potential duplicates, and posts informative comments with related issue context
  • Updated issue templates to include new product categories (API, Calendar, Home, Publishing, Registrations) for consistency with automated labeling

Reviewed changes

Copilot reviewed 7 out of 9 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
package.json Defines project dependencies for Anthropic SDK and GitHub Actions toolkit
package-lock.json Locks dependency versions for reproducible builds
.gitignore Excludes node_modules, environment files, and logs from version control
.github/workflows/issue-triage.yml Defines two parallel jobs for product labeling and duplicate detection on issue creation
.github/scripts/label-product.js Uses Claude AI to classify issues by product and apply appropriate labels
.github/scripts/detect-duplicates.js Searches for duplicate issues using keyword extraction and AI comparison, then posts findings as comments
.github/ISSUE_TEMPLATE/general_api_question.md Updated product list to match valid labels used by automation
.github/ISSUE_TEMPLATE/feature_request.md Updated product list to match valid labels used by automation
.github/ISSUE_TEMPLATE/bug_report.md Updated product list to match valid labels used by automation

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

"People",
"Publishing",
"Registrations",
"Resources",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw we removed "Resources" from the template in a commit. Do we also want to remove it here?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, good call.

Copy link

@zanebliss zanebliss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is cool! Outside of a couple of questions/observations, I wonder about cost tracking. In Natural Language Lists, we utilized cost tracking through AWS Bedrock. In this situation, are we okay with it living in the Anthropic console?

@geolessel
Copy link
Author

This is cool! Outside of a couple of questions/observations, I wonder about cost tracking. In Natural Language Lists, we utilized cost tracking through AWS Bedrock. In this situation, are we okay with it living in the Anthropic console?

Yes, I believe so. There is a cap on the spend on the specific key and if we need to modify it we can go from there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants