Skip to content

chore(workflows): Simplify sync-docs-from-node workflow #257

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dohernandez merged 18 commits into from
Jul 26, 2025
Merged

chore(workflows): Simplify sync-docs-from-node workflow #257

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
4e1b926
chore(workflows): Update sync-docs-from-node workflow to use version …
dohernandez Jul 25, 2025
7213b7a
chore(workflows): Simplify sync-docs-from-node workflow to use single…
dohernandez Jul 25, 2025
8056410
chore(workflows): Rename tag retrieval step to version and update ref…
dohernandez Jul 25, 2025
e99f5b5
chore(workflows): Add validation for missing version in repository_di…
dohernandez Jul 25, 2025
2b57909
chore(workflows): Sanitize version string for branch name in sync-doc…
dohernandez Jul 25, 2025
3d2624c
chore(workflows): Add branch existence check and force push with leas…
dohernandez Jul 25, 2025
a546dc4
chore(workflows): Replace echo with printf for improved formatting in…
dohernandez Jul 25, 2025
5f8b4a6
chore(workflows): Refactor regex matching to use a dedicated function…
dohernandez Jul 25, 2025
fcc8deb
chore(workflows): Improve error message clarity by showing relative p…
dohernandez Jul 25, 2025
3f5eba7
chore(workflows): Use printf for improved formatting of missing sourc…
dohernandez Jul 25, 2025
973d808
chore(workflows): Use printf with explicit format for improved consis…
dohernandez Jul 25, 2025
51718c9
chore(workflows): Update API paths in workflows and documentation for…
dohernandez Jul 26, 2025
713085e
chore(workflows): Add debug output for config file processing in sync…
dohernandez Jul 26, 2025
a9e00ed
chore(workflows): Enhance debug output for config sanitization and co…
dohernandez Jul 26, 2025
fe559f1
chore(workflows): Enhance debug output for config sanitization and co…
dohernandez Jul 26, 2025
fb48029
chore(workflows): Improve debug output and error handling in config s…
dohernandez Jul 26, 2025
493a9db
chore(workflows): Enhance PR creation logic to check for open and clo…
dohernandez Jul 26, 2025
1e91472
chore(workflows): Skip handling of _meta.json to prevent unintended m…
dohernandez Jul 26, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 25 additions & 11 deletions .github/scripts/sanitize-config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,25 +7,39 @@
"""

import sys
import yaml
import re


def sanitize_config(config_file_path):
"""Remove node.dev and node.admin sections from config file."""
"""Remove node.dev and node.admin sections from config file using regex."""
print(f"Sanitizing config file: {config_file_path}")

# Read the YAML file
with open(config_file_path, 'r') as f:
config = yaml.safe_load(f)
content = f.read()

print(f"Original file size: {len(content)} bytes")

# Remove node.dev and node.admin if they exist
if 'node' in config:
if 'dev' in config['node']:
del config['node']['dev']
if 'admin' in config['node']:
del config['node']['admin']
# Remove node.admin section
# This regex matches the admin: line and all indented content that follows
admin_pattern = r'(\n\s+admin:\s*\n(?:\s+(?:port:\s*\d+|\S.*)\s*\n)*)'
if re.search(admin_pattern, content):
content = re.sub(admin_pattern, '\n', content)
print("Removed node.admin section")

# Write back to file preserving the structure
# Remove node.dev section
# This regex matches the dev: line and all indented content that follows
dev_pattern = r'(\n\s+dev:\s*\n(?:\s+\S.*\s*\n)*)'
if re.search(dev_pattern, content):
content = re.sub(dev_pattern, '\n', content)
print("Removed node.dev section")

Comment on lines +23 to +36
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot Jul 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Regex patterns may be unreliable for YAML parsing.

The regex patterns have several potential issues:

  1. Restrictive admin pattern: The pattern port:\s*\d+|\S.* is too specific and may miss other admin configuration fields
  2. Missing edge cases: Patterns require leading newlines, potentially missing sections at file start
  3. YAML structure assumptions: The patterns assume specific indentation which may vary

Consider more robust patterns:

-admin_pattern = r'(\n\s+admin:\s*\n(?:\s+(?:port:\s*\d+|\S.*)\s*\n)*)'
+admin_pattern = r'(\n?\s*admin:\s*\n(?:\s+.*\n)*)'
-dev_pattern = r'(\n\s+dev:\s*\n(?:\s+\S.*\s*\n)*)'
+dev_pattern = r'(\n?\s*dev:\s*\n(?:\s+.*\n)*)'

Or consider keeping the YAML parsing approach for reliability, as structured data parsing is inherently more robust than regex for this use case.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
# Remove node.admin section
# This regex matches the admin: line and all indented content that follows
admin_pattern = r'(\n\s+admin:\s*\n(?:\s+(?:port:\s*\d+|\S.*)\s*\n)*)'
if re.search(admin_pattern, content):
content = re.sub(admin_pattern, '\n', content)
print("Removed node.admin section")
# Write back to file preserving the structure
# Remove node.dev section
# This regex matches the dev: line and all indented content that follows
dev_pattern = r'(\n\s+dev:\s*\n(?:\s+\S.*\s*\n)*)'
if re.search(dev_pattern, content):
content = re.sub(dev_pattern, '\n', content)
print("Removed node.dev section")
# Remove node.admin section
# This regex matches the admin: line and all indented content that follows
admin_pattern = r'(\n?\s*admin:\s*\n(?:\s+.*\n)*)'
if re.search(admin_pattern, content):
content = re.sub(admin_pattern, '\n', content)
print("Removed node.admin section")
# Remove node.dev section
# This regex matches the dev: line and all indented content that follows
dev_pattern = r'(\n?\s*dev:\s*\n(?:\s+.*\n)*)'
if re.search(dev_pattern, content):
content = re.sub(dev_pattern, '\n', content)
print("Removed node.dev section")
🤖 Prompt for AI Agents 
In .github/scripts/sanitize-config.py around lines 23 to 36, the current regex
patterns for removing the node.admin and node.dev sections are too restrictive,
rely on specific indentation and leading newlines, and may miss valid YAML
structures. To fix this, replace the regex-based removal with a YAML parsing
approach: load the content as YAML, remove the 'admin' and 'dev' keys from the
'node' section if present, then dump the YAML back to string. This ensures
reliable and accurate removal regardless of formatting or indentation
variations.

# Write back to file
with open(config_file_path, 'w') as f:
yaml.dump(config, f, default_flow_style=False, sort_keys=False, allow_unicode=True)
f.write(content)

print(f"Sanitized file size: {len(content)} bytes")
print("Config sanitization completed")


def main():
Expand Down
12 changes: 6 additions & 6 deletions .github/workflows/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,8 @@ Add this to a workflow in the genlayer-node repository:
{
"source_branch": "${{ github.ref_name }}",
"changelog_path": "docs/changelog",
"api_gen_path": "docs/api/rpc/gen",
"api_debug_path": "docs/api/rpc/debug",
"api_gen_path": "docs/api/rpc",
"api_debug_path": "docs/api/rpc",
"api_gen_regex": "gen_(?!dbg_).*",
"api_debug_regex": "gen_dbg_.*"
}
Expand Down Expand Up @@ -81,8 +81,8 @@ From the Actions tab:
3. Specify parameters:
- Tag for branch naming (required, e.g., v0.3.5)
- Source branch (optional, default: main)
- API gen path (optional, default: `docs/api/rpc/gen`)
- API debug path (optional, default: `docs/api/rpc/debug`)
- API gen path (optional, default: `docs/api/rpc`)
- API debug path (optional, default: `docs/api/rpc`)
- API gen regex filter (optional, default: `gen_(?!dbg_).*`)
- API debug regex filter (optional, default: `gen_dbg_.*`)

Expand Down Expand Up @@ -114,8 +114,8 @@ The source paths and filters can be customized in the event payload:

#### Paths
- `changelog_path`: Path to changelog files (default: `docs/changelog`)
- `api_gen_path`: Path to API gen methods (default: `docs/api/rpc/gen`)
- `api_debug_path`: Path to API debug methods (default: `docs/api/rpc/debug`)
- `api_gen_path`: Path to API gen methods (default: `docs/api/rpc`)
- `api_debug_path`: Path to API debug methods (default: `docs/api/rpc`)

#### Regex Filters
- `api_gen_regex`: Regex pattern to filter gen API files (default: `gen_(?!dbg_).*`)
Expand Down
24 changes: 11 additions & 13 deletions .github/workflows/example-trigger-from-node.yml.example
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,16 +22,16 @@ jobs:
with:
fetch-depth: 0 # Fetch all history for tags

- name: Get latest tag
id: get_tag
- name: Get version
id: get_version
run: |
# Get the latest tag
LATEST_TAG=$(git describe --tags --abbrev=0 2>/dev/null || echo "")
if [ -z "$LATEST_TAG" ]; then
# Get the latest tag as the version
VERSION=$(git describe --tags --abbrev=0 2>/dev/null || echo "")
if [ -z "$VERSION" ]; then
# Fallback to commit SHA if no tags found
LATEST_TAG="commit-$(git rev-parse --short HEAD)"
VERSION="commit-$(git rev-parse --short HEAD)"
fi
echo "tag=$LATEST_TAG" >> $GITHUB_OUTPUT
echo "version=$VERSION" >> $GITHUB_OUTPUT

- name: Trigger documentation sync
uses: peter-evans/repository-dispatch@v2
Expand All @@ -43,11 +43,10 @@ jobs:
event-type: sync-docs
client-payload: |
{
"source_branch": "${{ github.ref_name }}",
"tag": "${{ steps.get_tag.outputs.tag }}",
"version": "${{ steps.get_version.outputs.version }}",
"changelog_path": "docs/changelog",
"api_gen_path": "docs/api/rpc/gen",
"api_debug_path": "docs/api/rpc/debug",
"api_gen_path": "docs/api/rpc",
"api_debug_path": "docs/api/rpc",
"api_gen_regex": "gen_(?!dbg_).*",
"api_debug_regex": "gen_dbg_.*"
}
Expand All @@ -56,5 +55,4 @@ jobs:
run: |
echo "✅ Triggered documentation sync" >> $GITHUB_STEP_SUMMARY
echo "- Target repository: genlayerlabs/genlayer-docs" >> $GITHUB_STEP_SUMMARY
echo "- Source branch: ${{ github.ref_name }}" >> $GITHUB_STEP_SUMMARY
echo "- Tag: ${{ steps.get_tag.outputs.tag }}" >> $GITHUB_STEP_SUMMARY
echo "- Version: ${{ steps.get_version.outputs.version }}" >> $GITHUB_STEP_SUMMARY
Loading