Expose remaining unbound Python APIs for evaluation and benchmarking#14
Expose remaining unbound Python APIs for evaluation and benchmarking#14Rahuldrabit merged 6 commits intomainfrom
Conversation
Agent-Logs-Url: https://github.com/Rahuldrabit/Genetic_algorithm/sessions/bad4840b-a7ae-4f9c-992c-9395a4f5e2bb Co-authored-by: Rahuldrabit <104688569+Rahuldrabit@users.noreply.github.com>
Agent-Logs-Url: https://github.com/Rahuldrabit/Genetic_algorithm/sessions/492b49ef-b64f-4e9d-b328-125edae00a71 Co-authored-by: Rahuldrabit <104688569+Rahuldrabit@users.noreply.github.com>
…ions Agent-Logs-Url: https://github.com/Rahuldrabit/Genetic_algorithm/sessions/492b49ef-b64f-4e9d-b328-125edae00a71 Co-authored-by: Rahuldrabit <104688569+Rahuldrabit@users.noreply.github.com>
Agent-Logs-Url: https://github.com/Rahuldrabit/Genetic_algorithm/sessions/492b49ef-b64f-4e9d-b328-125edae00a71 Co-authored-by: Rahuldrabit <104688569+Rahuldrabit@users.noreply.github.com>
…mark suite Agent-Logs-Url: https://github.com/Rahuldrabit/Genetic_algorithm/sessions/89bbeb2e-117f-49e8-833e-da867e70114c Co-authored-by: Rahuldrabit <104688569+Rahuldrabit@users.noreply.github.com>
|
@copilot review |
Reviewed the latest branch state at |
There was a problem hiding this comment.
Pull request overview
Exposes previously C++-only evaluation and benchmarking functionality through the Python bindings, and updates documentation/sanity checks to reflect the expanded Python surface.
Changes:
- Added Python bindings for threaded evaluation utilities (
ParallelEvaluator,LocalDistributedExecutor) and benchmark suite types/APIs (BenchmarkConfig,GABenchmark, result structs). - Added const result getters to
GABenchmarkto enable Python access to benchmark results. - Updated build integration, user guide, and Python bindings sanity checks for the new APIs.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| python/ga_bindings.cpp | Adds pybind11 bindings for evaluation helpers, benchmark suite, and selection helper functions. |
| python/bindings_sanity.py | Extends sanity script to exercise new evaluator/selection/benchmark Python APIs and CSV export. |
| benchmark/ga_benchmark.h | Adds const getters for benchmark result vectors to support Python retrieval. |
| USER_GUIDE.md | Updates coverage table + adds Python usage docs for selection helpers, evaluation helpers, and benchmark suite. |
| CMakeLists.txt | Links benchmark implementation into the Python extension module target. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| py::arg("fitness"), | ||
| py::arg("threads") = std::thread::hardware_concurrency()) | ||
| .def("evaluate", &DoubleBatchEvaluator::evaluate, py::arg("batch"), |
There was a problem hiding this comment.
std::thread::hardware_concurrency() is allowed to return 0; if that happens, the Python-visible default for threads becomes 0 (even though the C++ implementation later clamps to >=1). Consider setting the binding default to max(1, hardware_concurrency()) (or using None/omitting the default) so the Python signature and runtime behavior are not misleading.
| py::arg("workers") = std::thread::hardware_concurrency()) | ||
| .def("execute", &ga::evaluation::LocalDistributedExecutor::execute, py::arg("batch"), | ||
| py::call_guard<py::gil_scoped_release>(), | ||
| "Execute a batch of candidate vectors and return fitness values"); |
There was a problem hiding this comment.
Same issue as threads: std::thread::hardware_concurrency() may return 0, which makes the Python-visible default workers=0 even though the executor clamps to >=1 internally. Prefer a default that is always at least 1 so Python users don’t accidentally run with an unexpected setting.
| }, | ||
| py::arg("fitness"), | ||
| py::arg("tournament_size") = 3u, | ||
| "Tournament selection helper: returns one winner index from the tournament"); |
There was a problem hiding this comment.
TournamentSelection::selectIndices(...) returns a vector<unsigned int> containing a single winner index, so this binding returns a Python list (length 1), not a single integer. Either unwrap and return the scalar index, or update the docstring/name to make the list-of-one behavior explicit (to match the actual return type).
| "Tournament selection helper: returns one winner index from the tournament"); | |
| "Tournament selection helper: returns a list containing one winner index from the tournament"); |
| - `ga.selection_tournament_indices(fitness, tournament_size=3)` (returns one index) | ||
| - `ga.selection_roulette_indices(fitness, count)` | ||
| - `ga.selection_rank_indices(fitness, count)` | ||
| - `ga.selection_sus_indices(fitness, count)` *(stochastic universal sampling)* | ||
| - `ga.selection_elitism_indices(fitness, elite_count)` |
There was a problem hiding this comment.
The guide says selection_tournament_indices(...) “returns one index”, but the Python API actually returns a list containing one index (consistent with the other *_indices helpers). Please clarify in the documentation (e.g., “returns a list of length 1”) or adjust the API to return an int.
| - `ga.ParallelEvaluator(fitness, threads=...)` | ||
| - `ga.LocalDistributedExecutor(evaluator, workers=...)` | ||
| - plus optimizer-level threading via `ga.Optimizer.with_threads(...)` | ||
|
|
There was a problem hiding this comment.
Since these evaluators call a Python callback, each callback invocation must hold the GIL; that means a pure-Python fitness function will not execute concurrently across threads (speedups generally require native code that releases the GIL, e.g. NumPy/Cython). Consider adding a short note here so users don’t assume CPU-parallel execution for pure Python callables.
| > **Note:** These evaluators invoke a Python callback while holding the GIL, so a pure-Python `fitness` function will not execute CPU-bound work concurrently across threads. For real CPU parallelism, use native code that releases the GIL (e.g. NumPy, Cython), or a process-based executor. |
The issue asks to expose functionality that existed in C++ but was still unavailable in Python bindings. This PR closes the remaining practical gaps by binding evaluation utilities and benchmark APIs, and updating the guide to reflect the actual Python surface.
Parallel/distributed evaluation bindings
ga.ParallelEvaluatorga.LocalDistributedExecutorBenchmark suite bindings
ga.BenchmarkConfigga.BenchmarkResultga.OperatorBenchmarkga.FunctionBenchmarkga.GABenchmarkrun_all_benchmarks,run_operator_benchmarks,run_function_benchmarks,run_scalability_benchmarksgenerate_report,export_to_csvoperator_results,function_results,scalability_resultsC++ surface support for Python result access
GABenchmarkfor operator/function/scalability result vectors to enable direct Python retrieval.Build/link integration
benchmark/ga_benchmark.cc, ensuring benchmark symbols are linked intoga_python_module.Docs + binding sanity coverage
USER_GUIDE.mdcoverage table and sections 19/25 to document newly exposed Python APIs.python/bindings_sanity.pywith focused checks for new evaluator and benchmark bindings.💬 Send tasks to Copilot coding agent from Slack and Teams to turn conversations into code. Copilot posts an update in your thread when it's finished.