diff --git a/.github/workflows/new_problem_check.yml b/.github/workflows/new_problem_check.yml new file mode 100644 index 0000000..5a13ee6 --- /dev/null +++ b/.github/workflows/new_problem_check.yml @@ -0,0 +1,37 @@ +name: New Problem Check + +on: + push: + paths: + - "utils/new_problem.yaml" + pull_request: + paths: + - "utils/new_problem.yaml" + workflow_dispatch: # allow triggering workflow manually + inputs: + reason: + description: "Reason for Running" + required: false + default: "Manually trigger problem check" + +jobs: + check_new_problems: + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@v3 + + - name: Set up Python + uses: actions/setup-python@v4 + with: + python-version: '3.x' + + - name: Install dependencies + run: | + python -m pip install --upgrade pip + pip install -r utils/requirements.txt + + - name: Run New Problem Check + run: | + python utils/validate_yaml.py utils/new_problem.yaml + diff --git a/.gitignore b/.gitignore index f7275bb..e15106e 100644 --- a/.gitignore +++ b/.gitignore @@ -1 +1,216 @@ +# Byte-compiled / optimized / DLL files +__pycache__/ +*.py[codz] +*$py.class + +# C extensions +*.so + +# Distribution / packaging +.Python +build/ +develop-eggs/ +dist/ +downloads/ +eggs/ +.eggs/ +lib/ +lib64/ +parts/ +sdist/ +var/ +wheels/ +share/python-wheels/ +*.egg-info/ +.installed.cfg +*.egg +MANIFEST + +# PyInstaller +# Usually these files are written by a python script from a template +# before PyInstaller builds the exe, so as to inject date/other infos into it. +*.manifest +*.spec + +# Installer logs +pip-log.txt +pip-delete-this-directory.txt + +# Unit test / coverage reports +htmlcov/ +.tox/ +.nox/ +.coverage +.coverage.* +.cache +nosetests.xml +coverage.xml +*.cover +*.py.cover +.hypothesis/ +.pytest_cache/ +cover/ + +# Translations +*.mo +*.pot + +# Django stuff: +*.log +local_settings.py +db.sqlite3 +db.sqlite3-journal + +# Flask stuff: +instance/ +.webassets-cache + +# Scrapy stuff: +.scrapy + +# Sphinx documentation +docs/_build/ + +# PyBuilder +.pybuilder/ +target/ + +# Jupyter Notebook +.ipynb_checkpoints + +# IPython +profile_default/ +ipython_config.py + +# pyenv +# For a library or package, you might want to ignore these files since the code is +# intended to run in multiple environments; otherwise, check them in: +# .python-version + +# pipenv +# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. +# However, in case of collaboration, if having platform-specific dependencies or dependencies +# having no cross-platform support, pipenv may install dependencies that don't work, or not +# install all needed dependencies. +# Pipfile.lock + +# UV +# Similar to Pipfile.lock, it is generally recommended to include uv.lock in version control. +# This is especially recommended for binary packages to ensure reproducibility, and is more +# commonly ignored for libraries. +# uv.lock + +# poetry +# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control. +# This is especially recommended for binary packages to ensure reproducibility, and is more +# commonly ignored for libraries. +# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control +# poetry.lock +# poetry.toml + +# pdm +# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control. +# pdm recommends including project-wide configuration in pdm.toml, but excluding .pdm-python. +# https://pdm-project.org/en/latest/usage/project/#working-with-version-control +# pdm.lock +# pdm.toml +.pdm-python +.pdm-build/ + +# pixi +# Similar to Pipfile.lock, it is generally recommended to include pixi.lock in version control. +# pixi.lock +# Pixi creates a virtual environment in the .pixi directory, just like venv module creates one +# in the .venv directory. It is recommended not to include this directory in version control. +.pixi + +# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm +__pypackages__/ + +# Celery stuff +celerybeat-schedule +celerybeat.pid + +# Redis +*.rdb +*.aof +*.pid + +# RabbitMQ +mnesia/ +rabbitmq/ +rabbitmq-data/ + +# ActiveMQ +activemq-data/ + +# SageMath parsed files +*.sage.py + +# Environments +.env +.envrc +.venv +env/ venv/ +ENV/ +env.bak/ +venv.bak/ + +# Spyder project settings +.spyderproject +.spyproject + +# Rope project settings +.ropeproject + +# mkdocs documentation +/site + +# mypy +.mypy_cache/ +.dmypy.json +dmypy.json + +# Pyre type checker +.pyre/ + +# pytype static type analyzer +.pytype/ + +# Cython debug symbols +cython_debug/ + +# PyCharm +# JetBrains specific template is maintained in a separate JetBrains.gitignore that can +# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore +# and can be added to the global gitignore or merged into this file. For a more nuclear +# option (not recommended) you can uncomment the following to ignore the entire idea folder. +# .idea/ + +# Abstra +# Abstra is an AI-powered process automation framework. +# Ignore directories containing user credentials, local state, and settings. +# Learn more at https://abstra.io/docs +.abstra/ + +# Visual Studio Code +# Visual Studio Code specific template is maintained in a separate VisualStudioCode.gitignore +# that can be found at https://github.com/github/gitignore/blob/main/Global/VisualStudioCode.gitignore +# and can be added to the global gitignore or merged into this file. However, if you prefer, +# you could uncomment the following to ignore the entire vscode folder +# .vscode/ + +# Ruff stuff: +.ruff_cache/ + +# PyPI configuration file +.pypirc + +# Marimo +marimo/_static/ +marimo/_lsp/ +__marimo__/ + +# Streamlit +.streamlit/secrets.toml diff --git a/README.md b/README.md index 6f37f53..24de4d3 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,9 @@ Collection of optimisation problems and benchmark suites with information about high level properties like the type of decision variables, number of objectives, and the presence of constraints. Where available references to papers describing the problem and implementations are included. ## Contributing -Contributions and corrections are very welcome, through pull requests, issue reporting, or email. +Contributions and corrections are very welcome, through pull requests, issue reporting, or email. + +If you want to provide a problem/suite/generator, you can also do so via this [form](https://docs.google.com/forms/d/e/1FAIpQLSehQp24AuFAH2j9jizDhq8K_BYgNGMKXWTMu6s-2RwEJrK59Q/viewform?usp=sharing&ouid=107462254722022409950). ### Details to provide It is often far easier for the creator of a problem/suite/generator, or someone working with it, to provide details about it than for someone new to it. Currently, the following specific details are being collected: diff --git a/docs/header.html b/docs/header.html index 5d1adbe..62df510 100644 --- a/docs/header.html +++ b/docs/header.html @@ -22,5 +22,5 @@

OPL – Optimisation problem library

-Submit problems and corrections on GitHub with pull requests / issues, or by email: koen.van.der.blom@cwi.nl +Submit problems and corrections on GitHub with pull requests / issues, through the Google form, or by email: koen.van.der.blom@cwi.nl

diff --git a/docs/index.html b/docs/index.html index ad3eef9..c3a5882 100644 --- a/docs/index.html +++ b/docs/index.html @@ -22,7 +22,7 @@

OPL – Optimisation problem library

-Submit problems and corrections on GitHub with pull requests / issues, or by email: koen.van.der.blom@cwi.nl +Submit problems and corrections on GitHub with pull requests / issues, through the Google form, or by email: koen.van.der.blom@cwi.nl

@@ -55,8 +55,8 @@

OPL – Optimisation problem library

- - + + @@ -70,8 +70,8 @@

OPL – Optimisation problem library

- - + + @@ -85,8 +85,8 @@

OPL – Optimisation problem library

- - + + @@ -100,8 +100,8 @@

OPL – Optimisation problem library

- - + + @@ -115,8 +115,8 @@

OPL – Optimisation problem library

- - + + @@ -130,8 +130,8 @@

OPL – Optimisation problem library

- - + + @@ -145,8 +145,8 @@

OPL – Optimisation problem library

- - + + @@ -161,7 +161,7 @@

OPL – Optimisation problem library

- + @@ -175,8 +175,8 @@

OPL – Optimisation problem library

- - + + @@ -190,8 +190,8 @@

OPL – Optimisation problem library

- - + + @@ -205,8 +205,8 @@

OPL – Optimisation problem library

- - + + @@ -220,7 +220,7 @@

OPL – Optimisation problem library

- + @@ -235,7 +235,7 @@

OPL – Optimisation problem library

- + @@ -250,7 +250,7 @@

OPL – Optimisation problem library

- + @@ -265,7 +265,7 @@

OPL – Optimisation problem library

- + @@ -280,7 +280,7 @@

OPL – Optimisation problem library

- + @@ -295,8 +295,8 @@

OPL – Optimisation problem library

- - + + @@ -310,8 +310,8 @@

OPL – Optimisation problem library

- - + + @@ -325,8 +325,8 @@

OPL – Optimisation problem library

- - + + @@ -340,7 +340,7 @@

OPL – Optimisation problem library

- + @@ -355,8 +355,8 @@

OPL – Optimisation problem library

- - + + @@ -370,8 +370,8 @@

OPL – Optimisation problem library

- - + + @@ -385,8 +385,8 @@

OPL – Optimisation problem library

- - + + @@ -400,8 +400,8 @@

OPL – Optimisation problem library

- - + + @@ -415,8 +415,8 @@

OPL – Optimisation problem library

- - + + @@ -430,8 +430,8 @@

OPL – Optimisation problem library

- - + + @@ -445,8 +445,8 @@

OPL – Optimisation problem library

- - + + @@ -460,7 +460,7 @@

OPL – Optimisation problem library

- + @@ -475,7 +475,7 @@

OPL – Optimisation problem library

- + @@ -490,8 +490,8 @@

OPL – Optimisation problem library

- - + + @@ -505,8 +505,8 @@

OPL – Optimisation problem library

- - + + @@ -520,8 +520,8 @@

OPL – Optimisation problem library

- - + + @@ -535,8 +535,8 @@

OPL – Optimisation problem library

- - + + @@ -565,8 +565,8 @@

OPL – Optimisation problem library

- - + + @@ -580,8 +580,8 @@

OPL – Optimisation problem library

- - + + @@ -595,8 +595,8 @@

OPL – Optimisation problem library

- - + + @@ -610,7 +610,7 @@

OPL – Optimisation problem library

- + @@ -625,7 +625,7 @@

OPL – Optimisation problem library

- + @@ -640,7 +640,7 @@

OPL – Optimisation problem library

- + @@ -655,7 +655,7 @@

OPL – Optimisation problem library

- + @@ -670,7 +670,7 @@

OPL – Optimisation problem library

- + @@ -685,7 +685,7 @@

OPL – Optimisation problem library

- + @@ -700,7 +700,7 @@

OPL – Optimisation problem library

- + @@ -715,7 +715,7 @@

OPL – Optimisation problem library

- + @@ -730,7 +730,7 @@

OPL – Optimisation problem library

- + @@ -745,12 +745,12 @@

OPL – Optimisation problem library

- - + + - + @@ -760,8 +760,8 @@

OPL – Optimisation problem library

- - + + @@ -775,12 +775,12 @@

OPL – Optimisation problem library

- - + + - + @@ -790,8 +790,8 @@

OPL – Optimisation problem library

- - + + @@ -805,8 +805,8 @@

OPL – Optimisation problem library

- - + + @@ -820,8 +820,8 @@

OPL – Optimisation problem library

- - + + @@ -835,8 +835,8 @@

OPL – Optimisation problem library

- - + + @@ -850,8 +850,8 @@

OPL – Optimisation problem library

- - + + @@ -865,12 +865,12 @@

OPL – Optimisation problem library

- - + + - + @@ -880,12 +880,12 @@

OPL – Optimisation problem library

- - + + - + @@ -895,8 +895,8 @@

OPL – Optimisation problem library

- - + + @@ -910,7 +910,7 @@

OPL – Optimisation problem library

- + @@ -925,7 +925,7 @@

OPL – Optimisation problem library

- + @@ -940,16 +940,211 @@

OPL – Optimisation problem library

- + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
no no https://doi.org/10.1080/10556788.2020.1808977https://github.com/numbbo/cocohttps://doi.org/10.1080/10556788.2020.1808977https://github.com/numbbo/coco
BBOB-biobj no no https://doi.org/10.48550/arXiv.1604.00359https://github.com/numbbo/cocohttps://doi.org/10.48550/arXiv.1604.00359https://github.com/numbbo/coco
BBOB-noisy yes no https://hal.inria.fr/inria-00369466https://web.archive.org/web/20210416065610/https://coco.gforge.inria.fr/doku.php?id=downloadshttps://hal.inria.fr/inria-00369466https://web.archive.org/web/20210416065610/https://coco.gforge.inria.fr/doku.php?id=downloads
BBOB-largescale no no https://doi.org/10.48550/arXiv.1903.06396https://github.com/numbbo/cocohttps://doi.org/10.48550/arXiv.1903.06396https://github.com/numbbo/coco
BBOB-mixint no no https://doi.org/10.1145/3321707.3321868https://github.com/numbbo/cocohttps://doi.org/10.1145/3321707.3321868https://github.com/numbbo/coco
BBOB-biobj-mixint no no https://doi.org/10.1145/3321707.3321868https://github.com/numbbo/cocohttps://doi.org/10.1145/3321707.3321868https://github.com/numbbo/coco
BBOB-constrained no no http://numbbo.github.io/coco-doc/bbob-constrained/https://github.com/numbbo/cocohttp://numbbo.github.io/coco-doc/bbob-constrained/https://github.com/numbbo/coco
MOrepo no https://github.com/MCDMSociety/MOrepohttps://github.com/MCDMSociety/MOrepo
ZDT no no https://doi.org/10.1162/106365600568202https://github.com/anyoptimization/pymoohttps://doi.org/10.1162/106365600568202https://github.com/anyoptimization/pymoo
DTLZ no no https://doi.org/10.1109/CEC.2002.1007032https://pymoo.org/problems/many/dtlz.htmlhttps://doi.org/10.1109/CEC.2002.1007032https://pymoo.org/problems/many/dtlz.html
WFG no no https://doi.org/10.1109/TEVC.2005.861417https://pymoo.org/problems/many/wfg.htmlhttps://doi.org/10.1109/TEVC.2005.861417https://pymoo.org/problems/many/wfg.html
CDMP ? no https://doi.org/10.1145/3321707.3321878https://doi.org/10.1145/3321707.3321878 ?
? no https://doi.org/10.1109/TCYB.2019.2896021https://doi.org/10.1109/TCYB.2019.2896021 ?
? no https://doi.org/10.1016/j.swevo.2019.02.003https://doi.org/10.1016/j.swevo.2019.02.003 ?
? no https://doi.org/10.1109/CEC.2019.8790277https://doi.org/10.1109/CEC.2019.8790277 ?
optional no https://doi.org/10.1016/j.asoc.2020.106139https://doi.org/10.1016/j.asoc.2020.106139 ?
no no https://doi.org/10.48550/arXiv.2110.08033https://github.com/songbai-liu/etmohttps://doi.org/10.48550/arXiv.2110.08033https://github.com/songbai-liu/etmo
MMOPP no no http://www5.zzu.edu.cn/system/_content/download.jsp?urltype=news.DownloadAttachUrl&owner=1327567121&wbfileid=4764412http://www5.zzu.edu.cn/ecilab/info/1036/1251.htmhttp://www5.zzu.edu.cn/system/_content/download.jsp?urltype=news.DownloadAttachUrl&owner=1327567121&wbfileid=4764412http://www5.zzu.edu.cn/ecilab/info/1036/1251.htm
CFD no no real worldhttps://doi.org/10.1007/978-3-319-99259-4_24https://bitbucket.org/arahat/cfd-test-problem-suitehttps://doi.org/10.1007/978-3-319-99259-4_24https://bitbucket.org/arahat/cfd-test-problem-suite
GBEA yes no real worldhttps://doi.org/10.1145/3321707.3321805https://doi.org/10.1145/3321707.3321805 ?
no no real worldhttps://doi.org/10.1145/3205651.3205702http://ladse.eng.isas.jaxa.jp/benchmark/https://doi.org/10.1145/3205651.3205702http://ladse.eng.isas.jaxa.jp/benchmark/
EMO2017 no no real worldhttps://www.ini.rub.de/PEOPLE/glasmtbl/projects/bbcomp/https://www.ini.rub.de/PEOPLE/glasmtbl/projects/bbcomp/downloads/realworld-problems-bbcomp-EMO-2017.ziphttps://www.ini.rub.de/PEOPLE/glasmtbl/projects/bbcomp/https://www.ini.rub.de/PEOPLE/glasmtbl/projects/bbcomp/downloads/realworld-problems-bbcomp-EMO-2017.zip
JSEC2019 no no real worldhttp://www.jpnsec.org/files/competition2019/EC-Symposium-2019-Competition-English.htmlhttp://www.jpnsec.org/files/competition2019/EC-Symposium-2019-Competition-English.htmlhttp://www.jpnsec.org/files/competition2019/EC-Symposium-2019-Competition-English.htmlhttp://www.jpnsec.org/files/competition2019/EC-Symposium-2019-Competition-English.html
RE no no real world likehttps://doi.org/10.1016/j.asoc.2020.106078https://github.com/ryojitanabe/reproblemshttps://doi.org/10.1016/j.asoc.2020.106078https://github.com/ryojitanabe/reproblems
CRE no no real world likehttps://doi.org/10.1016/j.asoc.2020.106078https://github.com/ryojitanabe/reproblemshttps://doi.org/10.1016/j.asoc.2020.106078https://github.com/ryojitanabe/reproblems
Radar waveform no no real worldhttps://doi.org/10.1007/978-3-540-70928-2_53http://code.evanhughes.org/https://doi.org/10.1007/978-3-540-70928-2_53http://code.evanhughes.org/
MF2 no yes https://doi.org/10.21105/joss.02049https://github.com/sjvrijn/mf2https://doi.org/10.21105/joss.02049https://github.com/sjvrijn/mf2
AMVOP no no https://doi.org/10.1109/TEVC.2013.2281531https://doi.org/10.1109/TEVC.2013.2281531 ?
no no real worldhttps://doi.org/10.1109/TEVC.2013.2281531https://doi.org/10.1109/TEVC.2013.2281531 ?
no no https://doi.org/10.48550/arXiv.2305.12221https://github.com/IOHprofiler/IOHexperimenter/https://doi.org/10.48550/arXiv.2305.12221https://github.com/IOHprofiler/IOHexperimenter/
ρMNK-Landscapes no no https://doi.org/10.1016/j.ejor.2012.12.019https://gitlab.com/aliefooghe/mocobench/https://doi.org/10.1016/j.ejor.2012.12.019https://gitlab.com/aliefooghe/mocobench/
mUBQP no no https://doi.org/10.1016/j.asoc.2013.11.008https://gitlab.com/aliefooghe/mocobench/https://doi.org/10.1016/j.asoc.2013.11.008https://gitlab.com/aliefooghe/mocobench/
ρmTSP no no https://doi.org/10.1007/978-3-319-45823-6_40https://gitlab.com/aliefooghe/mocobench/https://doi.org/10.1007/978-3-319-45823-6_40https://gitlab.com/aliefooghe/mocobench/
CEC2015-DMOO no optional Real-world-likehttps://doi.org/10.1145/3638530.3654299https://github.com/qrenau/Ealainhttps://doi.org/10.1145/3638530.3654299https://github.com/qrenau/Ealain
MA-BBOB no no artificialhttps://doi.org/10.1145/3673908https://github.com/IOHprofiler/IOHexperimenter/blob/master/example/Competitions/MA-BBOB/Example_MABBOB.ipynbhttps://doi.org/10.1145/3673908https://github.com/IOHprofiler/IOHexperimenter/blob/master/example/Competitions/MA-BBOB/Example_MABBOB.ipynb
MPM2 no no https://ls11-www.cs.tu-dortmund.de/_media/techreports/tr15-01.pdfhttps://github.com/jakobbossek/smoof/blob/master/inst/mpm2.pyhttps://ls11-www.cs.tu-dortmund.de/_media/techreports/tr15-01.pdfhttps://github.com/jakobbossek/smoof/blob/master/inst/mpm2.py
Convex DTLZ2 no no https://doi.org/10.1109/TEVC.2013.2281535https://doi.org/10.1109/TEVC.2013.2281535 ?
no no https://doi.org/10.1109/TEVC.2013.2281534https://doi.org/10.1109/TEVC.2013.2281534 ?
no no https://doi.org/10.1109/TEVC.2016.2587749https://doi.org/10.1109/TEVC.2016.2587749 ?
no no https://doi.org/10.1109/TEVC.2016.2587749https://doi.org/10.1109/TEVC.2016.2587749 ?
no no https://doi.org/10.1145/1143997.1144179https://doi.org/10.1145/1143997.1144179 ?
no no https://doi.org/10.1145/1143997.1144179https://doi.org/10.1145/1143997.1144179 ?
no no https://doi.org/10.1145/1143997.1144179https://doi.org/10.1145/1143997.1144179 ?
no no https://doi.org/10.1145/1143997.1144179https://doi.org/10.1145/1143997.1144179 ?
no no https://doi.org/10.1145/1143997.1144179https://doi.org/10.1145/1143997.1144179 ?
no no artificialhttps://www.academia.edu/download/94499025/TR-CEC2018-DMOP-Competition.pdfhttps://pymoo.org/problems/dynamic/df.htmlhttps://www.academia.edu/download/94499025/TR-CEC2018-DMOP-Competition.pdfhttps://pymoo.org/problems/dynamic/df.html
MODAct - multiobjective design of actuatorsRealistic Constrained Multi-Objective Optimization Benchmark Problems from Design. Need the https://github.com/epfl-lamd/modact package installed; evaluation times around 20msRealistic Constrained Multi-Objective Optimization Benchmark Problems from Design. Need the https://github.com/epfl-lamd/modact package installed; evaluation times around 20ms suite 2 3 4 or 5 20 no no real-worldhttps://doi.org/10.1109/TEVC.2020.3020046https://pymoo.org/problems/constrained/modact.htmlhttps://doi.org/10.1109/TEVC.2020.3020046https://pymoo.org/problems/constrained/modact.html
IOHClustering no no artificial, but based on real datahttps://arxiv.org/pdf/2505.09233https://github.com/IOHprofiler/IOHClusteringhttps://arxiv.org/pdf/2505.09233https://github.com/IOHprofiler/IOHClustering
GNBG-IIGeneralized Numerical Benchmark Generator (version 2). Also in IOH https://github.com/IOHprofiler/IOHGNBGGeneralized Numerical Benchmark Generator (version 2). Also in IOH https://github.com/IOHprofiler/IOHGNBG suite; generator 1 scalable no no artificialhttps://dl.acm.org/doi/pdf/10.1145/3712255.3734271https://github.com/rohitsalgotra/GNBG-IIhttps://dl.acm.org/doi/pdf/10.1145/3712255.3734271https://github.com/rohitsalgotra/GNBG-II
GNBG no no artificialhttps://arxiv.org/abs/2312.07083https://github.com/Danial-Yazdani/GNBG-Generatorhttps://arxiv.org/abs/2312.07083https://github.com/Danial-Yazdani/GNBG-Generator
DynamicBinVal no no artificialhttps://arxiv.org/pdf/2404.15837https://github.com/IOHprofiler/IOHexperimenterhttps://arxiv.org/pdf/2404.15837https://github.com/IOHprofiler/IOHexperimenter
PBO no no artificialhttps://dl.acm.org/doi/pdf/10.1145/3319619.3326810https://github.com/IOHprofiler/IOHexperimenterhttps://dl.acm.org/doi/pdf/10.1145/3319619.3326810https://github.com/IOHprofiler/IOHexperimenter
W-model no no artificialhttps://dl.acm.org/doi/abs/10.1145/3205651.3208240?casa_token=S4U_Pi9f6MwAAAAA:U9ztNTPwmupT8K3GamWZfBL7-8fqjxPtr_kprv51vdwA-REsp0EyOFGa99BtbANb0XbqyrVg795hIwhttps://github.com/thomasWeise/BBDOB_W_Modelhttps://dl.acm.org/doi/abs/10.1145/3205651.3208240?casa_token=S4U_Pi9f6MwAAAAA:U9ztNTPwmupT8K3GamWZfBL7-8fqjxPtr_kprv51vdwA-REsp0EyOFGa99BtbANb0XbqyrVg795hIwhttps://github.com/thomasWeise/BBDOB_W_Model
Submodular Optimitzation no no artificialhttps://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=10254181https://github.com/IOHprofiler/IOHexperimenterhttps://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=10254181https://github.com/IOHprofiler/IOHexperimenter
CEC2013suite used for cec2013 competition. Also in IOH https://github.com/IOHprofiler/IOHexperimentersuite used for cec2013 competition. Also in IOH https://github.com/IOHprofiler/IOHexperimenter suite 1 scalable no no artificialhttps://peerj.com/articles/cs-2671/CEC2013.pdfhttps://github.com/P-N-Suganthan/CEC2013https://peerj.com/articles/cs-2671/CEC2013.pdfhttps://github.com/P-N-Suganthan/CEC2013
CEC2022suite used for cec2022 competition. Also in IOH https://github.com/IOHprofiler/IOHexperimentersuite used for cec2022 competition. Also in IOH https://github.com/IOHprofiler/IOHexperimenter suite 1 scalable no no artificialhttps://github.com/P-N-Suganthan/2022-SO-BO/blob/main/CEC2022%20TR.pdfhttps://github.com/P-N-Suganthan/2022-SO-BOhttps://github.com/P-N-Suganthan/2022-SO-BO/blob/main/CEC2022%20TR.pdfhttps://github.com/P-N-Suganthan/2022-SO-BO
Onemax+Sphere / Zeromax+Sphere no no artificialhttps://doi.org/10.1145/3449726.3459521https://doi.org/10.1145/3449726.3459521 None
no no artificialhttps://doi.org/10.1145/3449726.3459521https://doi.org/10.1145/3449726.3459521 None
no no artificialhttps://doi.org/10.1145/3449726.3459521https://doi.org/10.1145/3449726.3459521 None
PorkchopPlotInterplanetaryTrajectorysuite12continuousnonononoreal-worldhttps://doi.org/10.1109/CEC65147.2025.11042973https://github.com/ShuaiqunPan/Transfer_Random_forests_BBOB_Real_world
KinematicsRobotArmsuite121continuousnonononoreal-worldhttps://doi.org/10.1023/A:1013258808932https://github.com/ShuaiqunPan/Transfer_Random_forests_BBOB_Real_world
VehicleDynamicssuite12continuousnonononoreal-worldhttps://www.scitepress.org/Papers/2023/121580/121580.pdfhttps://zenodo.org/records/8307853
MECHBenchThis is a set of problems with inspiration from Structural Mechanics Design Optimization. The suite comprises three physical models, from which the user may define different kind of problems which impact the final design output.Problem Suite1scalable'ContinuousyesnononoReal-World Applicationhttps://arxiv.org/abs/2511.10821https://github.com/BayesOptApp/MECHBench
EXPObenchWind farm layout optimization, gas filter design, pipe shape optimization, hyperparameter tuning, and hospital simulationProblem Suite110 to 135Continuous, Integer, Categorical, ConditionalyesnoyesnoReal-World Applicationhttps://doi.org/10.1016/j.asoc.2023.110744https://github.com/AlgTUDelft/ExpensiveOptimBenchmark
Gasoline direct injection engine designA multi-objective optimization problem seeking to minimize fuel consumption and NOx emissions over a two-minute dynamic duty cycle, subject to five constraints (turbine inlet temperature, number of knock occurrences, peak cylinder pressure, peak cylinder pressure rise, total work). Seven decision variables are defined: four define the hardware choices of cylinder compression ratio, turbo machinery and EGR cooler sizing; three relate to control variables that parameterise the engine control logic.Single Problem27Continuous, OrdinalyesnonoyesReal-World Applicationhttps://doi.org/10.1016/j.ejor.2022.08.032
BEACONGenerator for bi-objective benchmark problems with explicitly controlled correlations in continuous spaces.Generator2scalableContinuousnonononoArtificially Generatedhttps://dl.acm.org/doi/10.1145/3712255.3734303https://github.com/Stebbet/BEACON/
TulipaEnergyDetermine the optimal investment and operation decisions for different types of assets in the energy system (production, consumption, conversion, storage, and transport), while minimizing loss of load.Problem Suite1scalableContinuousyesnoyesyesReal-World Applicationhttps://tulipaenergy.github.io/TulipaEnergyModel.jl/stable/40-scientific-foundation/45-scientific-referenceshttps://tulipaenergy.github.io/TulipaEnergyModel.jl/stable/
ATOParameters of the Modules of the Automatic Train Operation should be optimized. The parameters are continuous with different ranges. There are two objectives (minimizing energy consumption, minimizing driving duration.Single Problem210ContinuousnonononoReal-World Application-
Brachytherapy treatment planningTreatment planning for internal radiation therapyProblem Suite2-3100-500ContinuousyesnonoyesReal-World Applicationhttps://www.sciencedirect.com/science/article/pii/S1538472123016781
FleetOptHealthcare organisation in the UK provided data about their current fleet of vehicles to conduct non-emergency heathcare trips in the Argyll and Bute region of Scotland, UK. They also provided historical data about the trips the vehicles took and about the bases which the vehicles return to. The aim is to reduce the existing fleet of vehicles while still ensuring all trips can be covered. Moving a vehicle from one base to another to help cover trips is OK as long as the original base can still cover its trips. Link to paper with more details: https://dl.acm.org/doi/abs/10.1145/3638530.3664137Single Problem1Upper level: 54; lower level: 13208IntegeryesnononoReal-World Applicationhttps://dl.acm.org/doi/abs/10.1145/3638530.3664137Not public: was done for real client with their private data
Building spatial designOptimise the spatial layout of a building to: minimise energy consumption for climate control, and minimise the strain on the structureSingle Problem2scalable depending on problem size (e.g. 90 for)Continuous, BooleanyesnononoReal-World Applicationhttps://hdl.handle.net/1887/81789https://github.com/TUe-excellent-buildings/BSO-toolbox
Electric Motor Design OptimizationThe goal is to find a design of a synchronous electric motor for power steering systems that minimizes costs and satisfies all constraints.Single Problem113Continuous, IntegeryesnoyesnoReal-World Applicationhttps://dis.ijs.si/tea/Publications/Tusar23Multistep.pdf (paper in Slovene)Implementation not freely available
name textual description suite/generator/single objectives dimensionality variable type constraints dynamic noise multi-fidelity source (real-world/artificial) reference implementation