Remove opendal dependency from QEMU driver#535
Remove opendal dependency from QEMU driver#535ambient-code[bot] wants to merge 4 commits intomainfrom
Conversation
❌ Deploy Preview for jumpstarter-docs failed. Why did it fail? →
|
|
Important Review skippedBot user detected. To trigger a single review, invoke the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Fixed the CI failures by adding The new The version requirement |
| @@ -0,0 +1,286 @@ | |||
| """ | |||
| Simplified FlasherClient that does not depend on opendal. | |||
There was a problem hiding this comment.
I think it's reasonable to start accumulating common driver interfaces in the core jumpstarter, as, for example, this one is broadly used around Jumpstarter, and lived on opendal initially.
There was a problem hiding this comment.
Agreed! The FlasherInterface ABC is purely abstract with no external dependencies, so it makes sense for it to live in core. This way any driver can implement it without pulling in opendal as a transitive dependency.
| local_path, url = _parse_path(path) | ||
|
|
||
| if url is not None: | ||
| with _http_url_adapter(client=self, url=url, mode="wb") as handle: |
There was a problem hiding this comment.
Are we testing all paths in here: https://github.com/jumpstarter-dev/jumpstarter/blob/main/python/packages/jumpstarter-driver-qemu/jumpstarter_driver_qemu/driver_test.py#L79 ?
I see it uses the Operator, can we make sure that we take that and interpret that for compatibility purposes without really relying/using opendal?
There was a problem hiding this comment.
Good question. The current test covers two paths:
- Local file path (
cached_image.resolve()) -- this exercises the_local_file_adaptercode path in the newFlasherClient, streaming the file viaresource_async. - HTTP URL (the Fedora download URL) -- this was previously using
Operator("http", endpoint=...)and is now a directhttps://URL, which exercises the_http_url_adaptercode path usingPresignedRequestResource.
Both paths in the new FlasherClient are covered by the integration test.
Regarding backward compatibility with opendal.Operator: the opendal package's FlasherClient (in jumpstarter_driver_opendal.client) is unchanged and still accepts the operator= parameter. Drivers that depend on jumpstarter-driver-opendal (esp32, pi-pico, dutlink, sdwire) continue to use that client via their own FlasherInterface.client() classmethod which points to the opendal version.
The QEMU driver now uses the core FlasherInterface which points to the simplified FlasherClient -- this one intentionally does not accept an operator= parameter since the goal is to avoid the opendal dependency entirely. The Operator usage in the old test was only needed because the old flasher client required it for HTTP downloads; the new client handles HTTP URLs natively.
So the API surface for QEMU users changes from:
# Old (required opendal)
qemu.flasher.flash("path/on/server", operator=Operator("http", endpoint="https://example.com"))
# New (no opendal needed)
qemu.flasher.flash("https://example.com/path/on/server")The new API is simpler and doesn't break any existing QEMU usage since the operator= parameter was only used in this test, not in production QEMU workflows.
There was a problem hiding this comment.
Ok, but if that test is there, is not executing. Should we make sure to have a test that executes the full url download?
There was a problem hiding this comment.
You're right -- when the cached image exists (which it does in CI), the HTTP URL branch is never exercised.
I'll add a dedicated unit test for the _http_url_adapter and _parse_path helpers in the core jumpstarter package to make sure the URL code path is covered without needing a full Fedora image download. This will verify that:
_parse_pathcorrectly distinguishes local paths from HTTP/HTTPS URLs_http_url_adapterproduces a properPresignedRequestResourcefor HTTP URLs- The
FlasherClient._flash_singlemethod routes HTTP URLs through the presigned request path
For a full end-to-end HTTP download test, that would involve downloading a large image in CI which is expensive. I think a unit test covering the routing logic is the right trade-off here. Does that work for you?
|
Added unit tests for The new tests in
This ensures the URL download logic is exercised in CI without needing to download a full Fedora image. |
|
Fixed the
The netlify deploy preview failure is unrelated to this PR's changes. |
|
Rebase the pr please. |
Move FlasherInterface to core jumpstarter package (jumpstarter.driver.flasher) and create a simplified FlasherClient (jumpstarter.client.flasher) that streams local files via resource_async and passes HTTP URLs as PresignedRequestResource for exporter-side download, eliminating the need for the opendal library. Update QEMU driver to import from the new core location and remove jumpstarter-driver-opendal from its dependencies. Update tests to use direct HTTP URLs instead of opendal Operator. Closes #441 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The new FlasherClient in jumpstarter/client/flasher.py imports click but it was not listed in the dependencies, causing import errors in tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Ensures the HTTP URL code path in FlasherClient is covered by tests, since the integration test only exercises this path when no cached image is available. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove unused `pathlib.Path` and `PresignedRequestResource` imports that were flagged by ruff (F401). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
6c8eee9 to
bffbd65
Compare
|
Rebased onto main (d7a65e0). The branch now has a clean history on top of the latest main. |
Summary
FlasherInterfaceABC to corejumpstarterpackage (jumpstarter.driver.flasher) — it's a pure abstract class with no opendal importsFlasherClientin core (jumpstarter.client.flasher) that handles local files viaresource_asyncand HTTP URLs viaPresignedRequestResource, without requiring the opendal libraryFlasherInterfacefromjumpstarter.driverinstead ofjumpstarter_driver_opendal.driverjumpstarter-driver-opendalfrom QEMU'spyproject.tomldependenciesopendal.OperatorBackward Compatibility
FlasherInterfaceis unchanged — drivers that depend on opendal (esp32, pi-pico, dutlink, sdwire) continue to work without modificationFlasherInterface.client()points tojumpstarter.client.flasher.FlasherClient(simplified, no opendal)FlasherInterface.client()continues pointing tojumpstarter_driver_opendal.client.FlasherClient(full opendal support)Test plan
test_driver_qemu) fails only due to missingqemu-imgbinary in CI environment (pre-existing)Related #441
🤖 Generated with Claude Code