Refactor python code, add publish workflows, add docs website

[JohnCHarrington]

Apologies, lots going on here ;)

• Refactored python code so we can share bits between the web and desktop interfaces
• Moved all python code to a dedicated python folder
• Added uv for managing python dependencies and build
• Moved all documentation to /documentation folder
• Added github actions pipeline to publish open_echo python package on release
• Added github actions pipeline to publish documentation to github pages on merge to main
• Added github actions pipeline to run python linting and tests on PRs

[Copilot]

Pull request overview

This PR restructures the repository around a publishable open_echo Python package (shared by desktop + web UIs), adds CI/release automation, and migrates documentation to a Jekyll/Just-the-Docs site for GitHub Pages.

Changes:

• Introduces a packaged Python implementation (python/src/open_echo) with shared readers/settings/output logic plus a CLI entrypoint.
• Adds Python CI (tests/lint/typecheck) and PyPI publish-on-release workflows.
• Migrates docs into /documentation and adds a GitHub Pages publishing workflow + Jekyll config.

Reviewed changes

Copilot reviewed 30 out of 43 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
reverse_engineering/live_waterfall.py	Minor import reordering.
reverse_engineering/images/.DS_Store	Adds a macOS metadata file (should not be versioned).
python/tests/test_settings.py	Adds unit tests for Settings parsing/validation and save/load roundtrip.
python/tests/test_echo.py	Adds tests for packet parsing, checksum handling, and reader behaviors.
python/tests/test_depth_output.py	Adds tests for SignalK/NMEA outputs and OutputManager lifecycle.
python/src/open_echo/web.py	Adds FastAPI web server + websocket streaming using shared readers/settings.
python/src/open_echo/settings.py	Moves/updates Settings model and validators into package.
python/src/open_echo/py.typed	Marks package as typed for type checkers.
python/src/open_echo/echo.py	Adds shared Echo packet parsing + Serial/UDP async readers.
python/src/open_echo/desktop.py	Refactors desktop UI to use shared readers/settings and qasync loop integration.
python/src/open_echo/depth_output.py	Refactors depth output (SignalK + NMEA0183) into the package.
python/src/open_echo/cli.py	Adds openecho CLI to run desktop or web.
python/src/open_echo/assets/templates/spectrogram.js	Adds client-side rendering for live spectrogram with cursor/zoom UI.
python/src/open_echo/assets/templates/frontend.html	Adds web frontend template for the spectrogram view.
python/src/open_echo/assets/templates/config.html	Adds web config form for connection/output/display settings.
python/src/open_echo/assets/static/style.css	Adds styling for the web spectrogram UI.
pyproject.toml	Defines packaged project metadata, dependencies, scripts, and tooling (ruff/mypy/pytest).
documentation/lucky_fishfinder/lucky_fishfinder.md	Adds Jekyll frontmatter and updates image paths.
documentation/getting_started/web_interface.md	Updates install/run instructions for the new CLI-based web flow.
documentation/getting_started/index.md	Adds Getting Started landing page/navigation.
documentation/getting_started/desktop_interface.md	Updates desktop install/run instructions and references new code location.
documentation/getting_started/TUSS4470_hardware.md	Adds frontmatter and updates ordering + image references.
documentation/getting_started/TUSS4470_firmware.md	Adds frontmatter and updates firmware references/links.
documentation/contributing.md	Adds contribution guide including uv workflow and git hooks.
_config.yml	Adds Jekyll configuration for GitHub Pages + just-the-docs.
README.md	Adds Jekyll frontmatter and updates documentation links/media.
Gemfile	Adds Ruby dependencies for building the docs site.
.gitignore	Updates ignore rules (including lockfiles and Jekyll build output).
.github/workflows/python-ci.yml	Adds CI for pytest + ruff + mypy using uv.
.github/workflows/pypi-publish.yml	Adds release workflow to build and publish to PyPI.
.github/workflows/docs.yml	Adds workflow to build and deploy docs to GitHub Pages.
.githooks/pre-commit	Adds optional local pre-commit hook running ruff/pytest/mypy via uv.
TUSS4470_shield_002/web/uv.lock	Removes legacy web interface lockfile.
TUSS4470_shield_002/web/requirements.txt	Removes legacy web interface requirements export.
TUSS4470_shield_002/web/pyproject.toml	Removes legacy web interface project definition.
TUSS4470_shield_002/web/echo.py	Removes legacy web echo reader implementation (replaced by package).
TUSS4470_shield_002/web/app.py	Removes legacy FastAPI app (replaced by open_echo.web).
TUSS4470_shield_002/requirements.txt	Removes legacy top-level Python requirements file.

Comments suppressed due to low confidence (7)

python/src/open_echo/depth_output.py:49

• Output scheduling logic is incorrect here: (1) the interval comparison multiplies output_interval (seconds) by 1000 even though asyncio.get_event_loop().time() is already in seconds, which makes outputs run ~1000x less frequently than configured; (2) the or last_output_time is None branch can call output() even when current_value is None, causing outputs to send null depth. Use a seconds-based comparison (no *1000) and gate outputs on current_value is not None (or explicitly handle an initial/heartbeat output separately).
  documentation/lucky_fishfinder/lucky_fishfinder.md:20
• These image src values include documentation/lucky_fishfinder/... even though the page itself already lives under that path, so they will resolve to a duplicated path at render time. Use a path relative to the markdown file (e.g., images/...) or Jekyll’s relative_url filter so the images render correctly on the docs site.
  python/src/open_echo/depth_output.py:59
• _run() currently loops without any sleep/backoff when settings is set, which will busy-spin and consume 100% CPU. Re-introduce a sleep (or await a condition/event) to control the output frequency (e.g., await asyncio.sleep(...) based on the smallest configured output interval).
  python/src/open_echo/desktop.py:780
• Qt override must be named closeEvent to run on window close. With close_event, the async reader task may not be cancelled/cleaned up when the window closes. Rename this back to closeEvent (and consider awaiting reader shutdown if needed).
  documentation/getting_started/desktop_interface.md:42
• This image path is likely broken when rendered under /documentation/getting_started/... because it’s a relative URL including documentation/ again. Prefer using Jekyll’s relative_url/{{ site.baseurl }} helpers (or a path relative to the page, e.g. ../images/...) so it resolves correctly on GitHub Pages.
  documentation/getting_started/TUSS4470_hardware.md:34
• The src="documentation/images/..." paths are relative to the current page URL, so on pages under /documentation/getting_started/... this will typically resolve to a duplicated path and break images. Use Jekyll helpers like {{ '/documentation/images/shield_pinout.png' | relative_url }} (or ../images/...) for stable links under the configured baseurl.
  python/src/open_echo/desktop.py:663
• Qt overrides must be named keyPressEvent (camel-case) to be invoked by Qt. Renaming it to key_press_event means keyboard shortcuts (Q/C) will no longer work. Restore the method name to keyPressEvent (keeping the body) or explicitly install an event filter to handle key presses.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

@JohnCHarrington I've opened a new pull request, #21, to work on those changes. Once the pull request is ready, I'll request review from you.

[Copilot]

@JohnCHarrington I've opened a new pull request, #22, to work on those changes. Once the pull request is ready, I'll request review from you.

[Copilot]

@JohnCHarrington I've opened a new pull request, #23, to work on those changes. Once the pull request is ready, I'll request review from you.

[Copilot]

Pull request overview

Copilot reviewed 30 out of 43 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (3)

python/src/open_echo/desktop.py:772

• Method name close_event doesn't follow PyQt5 naming convention. PyQt5 uses closeEvent (camelCase) as the method name that gets called when the window closes. This snake_case version won't be triggered. Rename to closeEvent to properly override the parent method.
  python/src/open_echo/depth_output.py:46
• The time comparison logic is incorrect. output_interval is in seconds (as indicated by line 74 comment), but it's being multiplied by 1000 and compared against time in seconds from get_event_loop().time(). This will cause output to happen every millisecond instead of every second. Remove the * 1000 multiplication.
  python/src/open_echo/desktop.py:653
• Method name key_press_event doesn't follow PyQt5 naming convention. PyQt5 uses keyPressEvent (camelCase) as the method name that gets called by the framework. This snake_case version won't be triggered by keyboard events. Rename to keyPressEvent to properly override the parent method.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 31 out of 44 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (4)

python/src/open_echo/depth_output.py:48

• The conditional logic for determining whether to output is incorrect. The condition output_class.current_value is not None or (...) will always trigger output if current_value is not None, ignoring the time interval check. Additionally, the time comparison uses milliseconds incorrectly - output_interval is in seconds but is multiplied by 1000 when compared to a difference in seconds. This will cause output to occur much less frequently than intended (e.g., 1 second interval becomes 1000 seconds).
  python/src/open_echo/depth_output.py:58
• The loop removed the await asyncio.sleep(1.0) which was throttling the output rate. Now the loop runs as fast as possible, constantly calling await self.output(). This will cause excessive CPU usage and network traffic. The sleep should be restored or the loop should wait on an event/signal.
  python/src/open_echo/desktop.py:653
• The method name key_press_event follows snake_case but should be keyPressEvent (camelCase) to properly override the Qt event handler. The current name will not receive keyboard events from Qt. This appears to be a renaming error - the old code had keyPressEvent which was correct.
  python/src/open_echo/desktop.py:772
• The method name close_event follows snake_case but should be closeEvent (camelCase) to properly override the Qt event handler. The current name will not be called when the window is closed. This appears to be a renaming error - the old code had closeEvent which was correct.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 35 out of 48 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (8)

python/src/open_echo/UART_UDP_relay.py:88

• The condition if not header or header[0] != START_BYTE is incorrect. ser.read(1) returns an empty bytes object (b'') when no data is available, not None. An empty bytes object is falsy, so not header works, but then header[0] will raise IndexError if header is empty. The check should be if len(header) == 0 or header[0] != START_BYTE or split into two conditions to avoid the IndexError.
  python/src/open_echo/settings.py:96
• The mode parameter in open() was removed on line 96. While Python defaults to "r" mode, it's better to be explicit when reading files, especially since the write mode is explicitly specified in save(). Add mode="r" for clarity and consistency.
  documentation/getting_started/TUSS4470_hardware.md:25
• Typo on line 25: "direclty" should be "directly".
  python/src/open_echo/depth_output.py:50
• The output timing logic is flawed. Line 42 checks if current_value is not None OR if enough time has passed, but then always updates last_output_time and calls output(). This means output() will be called even when current_value is None if enough time has passed. The logic should be: only output if current_value is not None AND (last_output_time is None OR enough time has passed).
  python/src/open_echo/depth_output.py:58
• The _run() loop no longer has a sleep, so it will spin at maximum CPU usage. The old code had await asyncio.sleep(1.0) at line 58 which was removed. Without any delay, this tight loop will consume unnecessary CPU resources. Add back a small sleep (e.g., await asyncio.sleep(0.01)) to prevent CPU spinning.
  documentation/lucky_fishfinder/lucky_fishfinder.md:21
• Typo in line 21: "brigter" should be "brighter".
  python/src/open_echo/depth_output.py:35
• All logging has been replaced with print() statements. This is a regression from proper logging practices. The code should use Python's logging module (as it did before) for proper log level control, formatting, and integration with application logging frameworks. Replace print() calls with appropriate log.info(), log.debug(), log.error(), etc.
  python/src/open_echo/depth_output.py:135
• The description field was changed from "OpenEcho Depth Sounder" to "open_echo Depth Sounder". This creates an inconsistency - the project is called "Open Echo" (with capital letters and space) everywhere else in the codebase. Use "Open Echo Depth Sounder" for consistency with the project branding.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The password field references a secret that may not exist. Ensure that PYPI_TOKEN is configured in the repository secrets before this workflow runs, otherwise the publish will fail. Consider adding a note in the PR description or documentation about required secrets.

[Copilot]

Typo on line 35: "direclty" should be "directly".

Suggested change

	They can also be bought as a complete and tested set direclty from Elecrow: https://www.elecrow.com/open-echo-tuss4470-development-shield.html
	They can also be bought as a complete and tested set directly from Elecrow: https://www.elecrow.com/open-echo-tuss4470-development-shield.html

[Copilot]

The dependency pyserial<3.5 pins to versions older than 3.5, which may be unnecessarily restrictive. Unless there's a specific incompatibility with 3.5+, consider using pyserial>=3.4,<4.0 to allow bug fixes and minor updates while preventing breaking changes.

Suggested change

	"pyserial<3.5",
	"pyserial>=3.4,<4.0",

[Copilot]

Pull request overview

Copilot reviewed 35 out of 48 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

python/src/open_echo/depth_output.py:61

• The _run() loop removed the await asyncio.sleep(1.0) call after calling output(), which means this tight loop will continuously call output() at maximum speed without yielding control. This can cause high CPU usage and starve other async tasks. The rate limiting should happen in the loop itself, not just within the output interval check. Consider adding await asyncio.sleep(0.01) or similar small delay after await self.output() to yield control to the event loop.
  python/src/open_echo/depth_output.py:154
• The get_token() method has an infinite polling loop without timeout (line 151-154). If the SignalK server never transitions from PENDING state or becomes unreachable during polling, this will hang indefinitely. Add a timeout mechanism or maximum retry count to prevent hanging the application.
  documentation/getting_started/TUSS4470_firmware.md:34
• The link to documentation/getting_started/desktop_interface.md appears to be a relative path from the repository root, but in a Jekyll site with baseurl, this may not resolve correctly. Since this file is already in documentation/getting_started/, consider using a relative link like desktop_interface.md or an absolute Jekyll link. Additionally, verify this link works when the site is deployed to GitHub Pages with the /open_echo baseurl.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 35 out of 48 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

python/src/open_echo/depth_output.py:61

• The _run() method runs an infinite loop that calls await self.output() with no sleep between iterations. This will cause a busy loop consuming 100% CPU. Add a small sleep (e.g., await asyncio.sleep(0.1)) after await self.output() to prevent CPU spinning.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.