3D Maze Environment Design Doc

[helenlu66]

Decide on a candidate repo that supports procedural generation of 3D grid world maze. It needs to not be deprecated, support continuous action spaces, and easy to install.

[helenlu66]

editable link

Name	Open Source	Overall	Procedural Gen	Ease of Installation	Action Space	Limitation	Simulator	Software Maintenance	Community	MVP	Key Differentiator	Limitations	Paper	Github	Website
MiniWorld	true	3	High – built-in procedural maze and indoor navigation tasks	Very easy – pip install Python environment	Discrete navigation actions (forward, turn, etc.), though physics is continuous internally	Lacks native continuous control and is deprecated, which limits scalability and long-term use despite strong built-in support for procedural maze generation and easy installation.
Godot RL Agents	true	4	Moderate – procedural environments easy to script in Godot scenes but grid maze generators must be implemented manually	Easy–Moderate – install Godot + Python package; works on macOS, Linux, Windows	Continuous, discrete, or mixed action spaces supported natively	Supports continuous control and is relatively easy to install, but requires implementing most procedural maze generation logic	Godot Engine (open-source 3D game engine with physics)	Actively maintained with updates in 2024–2025; documentation and examples maintained in repo	~1k ⭐ / ~200 forks; RL + Godot dev community; GitHub issues and Discord discussions	Train RL agents in continuous or discrete control tasks inside Godot scenes; supports navigation tasks and custom environments	Fully open-source engine; easy to script procedural environments and continuous control via Python/Gym interface; lightweight compared to Unity	Smaller ecosystem and fewer ready-made RL benchmarks; users must implement their own procedural generators and tasks	Godot RL Agents: Deep Reinforcement Learning in the Godot Engine	https://github.com/edbeeching/godot_rl_agents	https://github.com/edbeeching/godot_rl_agents
Unity ML-Agents	true	4	Moderate – procedural generation supported through Unity scripts/prefabs but maze generator not built-in	Moderate – requires Unity Hub, Unity Editor, Python ML-Agents package	Continuous or discrete action vectors supported	Provides excellent continuous control and scalability, but procedural grid maze generation must be built from scratch and installation/setup is heavier.	Unity Engine	Actively maintained by Unity Technologies; frequent releases (latest updates in 2025)	~18k ⭐ / ~4.5k forks; large global community; extensive tutorials and forums	Train RL agents with continuous or discrete actions in Unity environments; supports robotics, navigation, and physics-based tasks	Mature RL toolkit with strong ecosystem, excellent physics simulation, and easy procedural environment scripting via Unity	Heavy install (Unity Hub + engine version management); large disk footprint; Unity engine itself is not open-source	Unity ML-Agents Toolkit: An Open-Source Framework for Game AI	https://github.com/Unity-Technologies/ml-agents	https://unity.com/products/machine-learning-agents
DeepMind Lab	true	3	High – includes procedurally generated labyrinth environments and standalone maze generator (LabMaze)	Hard – requires Bazel build and engine compilation	Mostly discrete navigation actions; some parameters (camera turning) can be analog	Provides built-in procedural maze generation and good scalability, but its mostly discrete action interface and difficult installation make it less suitable for continuous-control gridworld research.	Custom Quake III–based 3D engine	Stable project; development slower than newer engines; maintained for research use	~2k ⭐ / ~400 forks; widely cited in RL research (1000+ citations)	First-person navigation tasks and procedurally generated maze environments (e.g., DMLab-30)	Built-in procedural 3D maze environments designed specifically for RL research; classic benchmark used in many RL papers	Harder installation (Bazel build); older engine and tooling; environment modification more difficult; mostly discrete navigation actions	DeepMind Lab: A 3D Learning Environment for Agents	https://github.com/google-deepmind/lab	https://deepmind.google/blog/deepmind-lab