Skip to content

Expose remaining unbound Python APIs for evaluation and benchmarking#14

Merged
Rahuldrabit merged 6 commits intomainfrom
copilot/expose-python-functionality
Mar 28, 2026
Merged

Expose remaining unbound Python APIs for evaluation and benchmarking#14
Rahuldrabit merged 6 commits intomainfrom
copilot/expose-python-functionality

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 28, 2026

The issue asks to expose functionality that existed in C++ but was still unavailable in Python bindings. This PR closes the remaining practical gaps by binding evaluation utilities and benchmark APIs, and updating the guide to reflect the actual Python surface.

  • Parallel/distributed evaluation bindings

    • Added Python bindings for:
      • ga.ParallelEvaluator
      • ga.LocalDistributedExecutor
    • Bound Python callables safely for threaded execution (GIL-acquire in callback path, GIL-release during batch execution).
  • Benchmark suite bindings

    • Added Python bindings for benchmark types:
      • ga.BenchmarkConfig
      • ga.BenchmarkResult
      • ga.OperatorBenchmark
      • ga.FunctionBenchmark
      • ga.GABenchmark
    • Exposed benchmark execution/report APIs and result accessors:
      • run_all_benchmarks, run_operator_benchmarks, run_function_benchmarks, run_scalability_benchmarks
      • generate_report, export_to_csv
      • operator_results, function_results, scalability_results
  • C++ surface support for Python result access

    • Added const getters on GABenchmark for operator/function/scalability result vectors to enable direct Python retrieval.
  • Build/link integration

    • Updated Python module target to include benchmark/ga_benchmark.cc, ensuring benchmark symbols are linked into ga_python_module.
  • Docs + binding sanity coverage

    • Updated USER_GUIDE.md coverage table and sections 19/25 to document newly exposed Python APIs.
    • Extended python/bindings_sanity.py with focused checks for new evaluator and benchmark bindings.
import ga

def sphere(x): return 1000.0 / (1.0 + sum(v*v for v in x))

pe = ga.ParallelEvaluator(sphere, threads=4)
scores = pe.evaluate([[0.1, 0.2], [0.3, 0.4]])

cfg = ga.BenchmarkConfig()
cfg.warmup_iterations = 1
cfg.benchmark_iterations = 3
cfg.verbose = False

bench = ga.GABenchmark(cfg)
bench.run_operator_benchmarks()
rows = bench.operator_results()
bench.export_to_csv("benchmark_results.csv")

💬 Send tasks to Copilot coding agent from Slack and Teams to turn conversations into code. Copilot posts an update in your thread when it's finished.

Copilot AI changed the title [WIP] Expose necessary Python functions in user guide Correct USER_GUIDE Python exposure matrix and document threaded optimizer usage Mar 28, 2026
Copilot AI requested a review from Rahuldrabit March 28, 2026 12:24
Copilot AI changed the title Correct USER_GUIDE Python exposure matrix and document threaded optimizer usage Expose Python selection APIs and align USER_GUIDE with newly bound selection helpers Mar 28, 2026
Copilot AI changed the title Expose Python selection APIs and align USER_GUIDE with newly bound selection helpers Expose remaining unbound Python APIs for evaluation and benchmarking Mar 28, 2026
@Rahuldrabit
Copy link
Copy Markdown
Owner

@copilot review

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Mar 28, 2026

@copilot review

Reviewed the latest branch state at ecc89b3; there are no additional code changes needed from this comment, and the current implementation remains valid as submitted.

@Rahuldrabit Rahuldrabit marked this pull request as ready for review March 28, 2026 13:42
Copilot AI review requested due to automatic review settings March 28, 2026 13:42
@Rahuldrabit Rahuldrabit merged commit 6d11fb9 into main Mar 28, 2026
7 checks passed
@Rahuldrabit Rahuldrabit deleted the copilot/expose-python-functionality branch March 28, 2026 13:43
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Exposes previously C++-only evaluation and benchmarking functionality through the Python bindings, and updates documentation/sanity checks to reflect the expanded Python surface.

Changes:

  • Added Python bindings for threaded evaluation utilities (ParallelEvaluator, LocalDistributedExecutor) and benchmark suite types/APIs (BenchmarkConfig, GABenchmark, result structs).
  • Added const result getters to GABenchmark to enable Python access to benchmark results.
  • Updated build integration, user guide, and Python bindings sanity checks for the new APIs.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
python/ga_bindings.cpp Adds pybind11 bindings for evaluation helpers, benchmark suite, and selection helper functions.
python/bindings_sanity.py Extends sanity script to exercise new evaluator/selection/benchmark Python APIs and CSV export.
benchmark/ga_benchmark.h Adds const getters for benchmark result vectors to support Python retrieval.
USER_GUIDE.md Updates coverage table + adds Python usage docs for selection helpers, evaluation helpers, and benchmark suite.
CMakeLists.txt Links benchmark implementation into the Python extension module target.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +821 to +823
py::arg("fitness"),
py::arg("threads") = std::thread::hardware_concurrency())
.def("evaluate", &DoubleBatchEvaluator::evaluate, py::arg("batch"),
Copy link

Copilot AI Mar 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

std::thread::hardware_concurrency() is allowed to return 0; if that happens, the Python-visible default for threads becomes 0 (even though the C++ implementation later clamps to >=1). Consider setting the binding default to max(1, hardware_concurrency()) (or using None/omitting the default) so the Python signature and runtime behavior are not misleading.

Copilot uses AI. Check for mistakes.
Comment on lines +839 to +842
py::arg("workers") = std::thread::hardware_concurrency())
.def("execute", &ga::evaluation::LocalDistributedExecutor::execute, py::arg("batch"),
py::call_guard<py::gil_scoped_release>(),
"Execute a batch of candidate vectors and return fitness values");
Copy link

Copilot AI Mar 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue as threads: std::thread::hardware_concurrency() may return 0, which makes the Python-visible default workers=0 even though the executor clamps to >=1 internally. Prefer a default that is always at least 1 so Python users don’t accidentally run with an unexpected setting.

Copilot uses AI. Check for mistakes.
},
py::arg("fitness"),
py::arg("tournament_size") = 3u,
"Tournament selection helper: returns one winner index from the tournament");
Copy link

Copilot AI Mar 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TournamentSelection::selectIndices(...) returns a vector<unsigned int> containing a single winner index, so this binding returns a Python list (length 1), not a single integer. Either unwrap and return the scalar index, or update the docstring/name to make the list-of-one behavior explicit (to match the actual return type).

Suggested change
"Tournament selection helper: returns one winner index from the tournament");
"Tournament selection helper: returns a list containing one winner index from the tournament");

Copilot uses AI. Check for mistakes.
Comment on lines +552 to +556
- `ga.selection_tournament_indices(fitness, tournament_size=3)` (returns one index)
- `ga.selection_roulette_indices(fitness, count)`
- `ga.selection_rank_indices(fitness, count)`
- `ga.selection_sus_indices(fitness, count)` *(stochastic universal sampling)*
- `ga.selection_elitism_indices(fitness, elite_count)`
Copy link

Copilot AI Mar 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The guide says selection_tournament_indices(...) “returns one index”, but the Python API actually returns a list containing one index (consistent with the other *_indices helpers). Please clarify in the documentation (e.g., “returns a list of length 1”) or adjust the API to return an int.

Copilot uses AI. Check for mistakes.
- `ga.ParallelEvaluator(fitness, threads=...)`
- `ga.LocalDistributedExecutor(evaluator, workers=...)`
- plus optimizer-level threading via `ga.Optimizer.with_threads(...)`

Copy link

Copilot AI Mar 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since these evaluators call a Python callback, each callback invocation must hold the GIL; that means a pure-Python fitness function will not execute concurrently across threads (speedups generally require native code that releases the GIL, e.g. NumPy/Cython). Consider adding a short note here so users don’t assume CPU-parallel execution for pure Python callables.

Suggested change
> **Note:** These evaluators invoke a Python callback while holding the GIL, so a pure-Python `fitness` function will not execute CPU-bound work concurrently across threads. For real CPU parallelism, use native code that releases the GIL (e.g. NumPy, Cython), or a process-based executor.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants