Skip to content

Feat/ga checks#127

Merged
CIGbalance merged 21 commits intofeat/ga_consistencyfrom
feat/ga_checks
Apr 8, 2026
Merged

Feat/ga checks#127
CIGbalance merged 21 commits intofeat/ga_consistencyfrom
feat/ga_checks

Conversation

@CIGbalance
Copy link
Copy Markdown
Collaborator

@CIGbalance CIGbalance commented Nov 19, 2025

Adds the GitHub Action responsible for triggering the validation of the new problem

Note that this check is only supposed to be triggered when the template file "new_problem.yaml" is changed. It will mark a failure if the name is not changed, that's why the test currently fails here.

Closes #124

@CIGbalance CIGbalance marked this pull request as ready for review November 19, 2025 10:25
@CIGbalance CIGbalance changed the base branch from main to feat/ga_consistency November 19, 2025 10:28
Copy link
Copy Markdown
Collaborator

@kvdblom kvdblom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very small change, otherwise looks good!

@kvdblom kvdblom self-requested a review November 27, 2025 10:30
@kvdblom
Copy link
Copy Markdown
Collaborator

kvdblom commented Nov 27, 2025

Looks good, only merge after: #114

Copilot AI review requested due to automatic review settings April 8, 2026 08:53
@CIGbalance CIGbalance merged commit 7dc8d9f into feat/ga_consistency Apr 8, 2026
2 of 4 checks passed
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a GitHub Actions workflow intended to validate updates to utils/new_problem.yaml (consistency/novelty checks), and additionally updates the YAML→HTML generation pipeline and regenerated docs/content.

Changes:

  • Add New Problem Check GitHub Action to run utils/validate_yaml.py when utils/new_problem.yaml changes.
  • Update yaml_to_html.py to linkify URLs in cells and adjust HTML rendering options; regenerate docs HTML accordingly.
  • Add multiple new entries to problems.yaml and update contribution instructions (README + site header/index) to include a Google Form.

Reviewed changes

Copilot reviewed 9 out of 10 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
.github/workflows/new_problem_check.yml New workflow to validate utils/new_problem.yaml on PRs/pushes.
yaml_to_html.py Adds URL linkification and changes HTML escaping/link rendering behavior.
utils/new_problem.yaml Template edited (currently triggers failing validation).
problems.yaml Appends many new problems/suites (some data quality issues found).
docs/problems.html Regenerated table output reflecting new linkification + new problems.
docs/index.html Updates submission text + regenerated table output + DataTables config change.
docs/header.html Adds Google Form submission option.
docs/javascript.html Enables DataTables search widget in the layout.
README.md Adds Google Form link for submissions.
.gitignore Expanded ignore rules (Python-focused template).
Comments suppressed due to low confidence (1)

yaml_to_html.py:34

  • This file executes its YAML→HTML generation at import time (top-level code). Since utils/validate_yaml.py imports default_columns from this module, running the validation workflow will also regenerate docs and require pandas, which is an unexpected side effect for a validator. Consider moving the generation logic into a main() guarded by if __name__ == "__main__":, and keep only constants/helpers importable.
yaml_file = "problems.yaml"

html_dir = "docs/"
html_table = f"{html_dir}problems.html"
html_header = f"{html_dir}header.html"
html_scripts = f"{html_dir}javascript.html"
html_footer = f"{html_dir}footer.html"
html_index = f"{html_dir}index.html"

# Load data
with open(yaml_file) as in_file:
    data = pd.json_normalize(yaml.safe_load(in_file))

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +56 to +60
data = data.map(linkify_cell)

# Generate plain table
table = data.to_html(render_links=True,
table = data.to_html(render_links=False,
escape=False, # Don't escape HTML in cells (to allow links)
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

escape=False combined with inserting raw <a> tags means any HTML present in problems.yaml will be rendered directly into the generated docs, which is an XSS/vector for script injection (and also produces invalid HTML for URLs containing & since they won't be escaped to &amp;). Safer options: keep HTML escaping enabled and post-process the generated table HTML to linkify URLs, or ensure linkify_cell HTML-escapes non-URL text and properly escapes attribute values (e.g., &, <, >).

Copilot uses AI. Check for mistakes.
@@ -1,6 +1,6 @@
- name: template
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As written, this change will make the new workflow fail: utils/validate_yaml.py errors if name is still template, and this PR modifies utils/new_problem.yaml without changing the name. Either revert this template edit before merging, or update the entry to a non-template name so the check passes in this PR.

Suggested change
- name: template
- name: new_problem

Copilot uses AI. Check for mistakes.
implementation: https://github.com/ShuaiqunPan/Transfer_Random_forests_BBOB_Real_world
source (real-world/artificial): 'real-world'
textual description: ''
- name: VehicleDynamics
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name value has leading whitespace (" VehicleDynamics"), which will be treated as part of the name string in YAML and can break uniqueness checks / lookups (and render inconsistently). Please remove the extra spaces so the name is exactly VehicleDynamics.

Suggested change
- name: VehicleDynamics
- name: VehicleDynamics

Copilot uses AI. Check for mistakes.
- name: MECHBench
suite/generator/single: Problem Suite
variable type: Continuous
dimensionality: scalable'
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dimensionality: scalable' includes a stray trailing quote. This will propagate into the generated HTML (and any downstream processing) as scalable' rather than scalable. Please correct the value.

Suggested change
dimensionality: scalable'
dimensionality: scalable

Copilot uses AI. Check for mistakes.
Comment on lines +22 to +27
uses: actions/checkout@v3

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.x'
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actions/checkout@v3 and actions/setup-python@v4 are older major versions; GitHub periodically deprecates old majors. Consider upgrading to the current majors and pinning the Python version to a specific minor (instead of 3.x) for more reproducible validation behavior.

Suggested change
uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.x'
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.12'

Copilot uses AI. Check for mistakes.
Comment on lines +1 to +5
name: New Problem Check

on:
push:
paths:
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR description is focused on adding a GitHub Action for validating utils/new_problem.yaml, but this PR also includes unrelated changes (URL linkification logic, docs HTML regeneration, README updates, and a large problems.yaml content addition). If possible, split these into separate PRs or update the PR description to reflect the additional scope so reviewers know what to validate.

Copilot uses AI. Check for mistakes.

## Contributing
Contributions and corrections are very welcome, through pull requests, issue reporting, or email.
Contributions and corrections are very welcome, through pull requests, issue reporting, or email.
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line has trailing whitespace after the period. Please remove it to avoid unnecessary diffs and keep Markdown formatting clean.

Suggested change
Contributions and corrections are very welcome, through pull requests, issue reporting, or email.
Contributions and corrections are very welcome, through pull requests, issue reporting, or email.

Copilot uses AI. Check for mistakes.
CIGbalance added a commit that referenced this pull request Apr 8, 2026
* Add PorkchopPlot, KinematicsRobotArm, VehicleDynamics real-world problems

* Add global search box, and update webpage.

* Update reference links to dois.

* Update README with form link

Added a link to a form for submitting problems, suites, or generators.

* Remove 'now' from added form text

* github action that triggers checks

* fixing syntax

* only execute when the file changes

* Add link on the website to the google form for problem/benchmark submission. (#136)

* Add problems from Tailoring RSP workshop

* Minor fixes and website generation.

* Fix field value consistency for searchability.

* Change links processing for html conversion

* Also process links starting with www

* testing tests and gitignore

---------

Co-authored-by: ShuaiqunPan <s.pan@liacs.leidenuniv.nl>
Co-authored-by: Shuaiqun (Daniel) Pan <122439592+ShuaiqunPan@users.noreply.github.com>
Co-authored-by: Koen van der Blom <kvdb@cwi.nl>
Co-authored-by: Diederick Vermetten <44877568+Dvermetten@users.noreply.github.com>
Co-authored-by: Koen van der Blom <5031234+kvdblom@users.noreply.github.com>
Co-authored-by: Dvermetten <d.vermetten@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a GitHub Action that runs consistency checks on PR

5 participants