From 466766a7ae5af809c0e500e6934327cf1dd82f26 Mon Sep 17 00:00:00 2001 From: Victor Garcia Date: Thu, 12 Feb 2026 08:52:20 -0500 Subject: [PATCH 1/9] Add platform setup guides for macOS and Linux --- README.md | 23 ++++++++++++- docs/linux.md | 89 ++++++++++++++++++++++++++++++++++++++++++++++++ docs/macos.md | 94 +++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 205 insertions(+), 1 deletion(-) create mode 100644 docs/linux.md create mode 100644 docs/macos.md diff --git a/README.md b/README.md index 9e459d9..9c88d2c 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # docker2vm -`docker2vm` converts OCI container images (or Dockerfiles via BuildKit) into VM-compatible outputs. Today, the runtime materialization target is Gondolin. +`docker2vm` converts OCI container images (or Dockerfiles via BuildKit) into VM-compatible outputs. Today, the runtime materialization target is [Gondolin](https://github.com/earendil-works/gondolin). It follows an OCI-first flow inspired by "Docker without Docker": @@ -53,6 +53,11 @@ Ubuntu helpers: sudo apt-get install -y e2fsprogs qemu-system-x86 ``` +## Platform setup guides + +- [macOS guide](./docs/macos.md) +- [Linux guide](./docs/linux.md) + ## Install ```bash @@ -75,6 +80,21 @@ bun run build bun run test:integration ``` +### Choosing the build platform (`--platform`) + +`--platform` selects which OCI image variant to convert, and should match the architecture you plan to run in Gondolin. + +- Apple Silicon / arm64 Linux hosts: `linux/arm64` +- Intel / amd64 hosts: `linux/amd64` +- If omitted, `oci2gondolin` defaults from host arch (`x64 -> linux/amd64`, `arm64 -> linux/arm64`). + +For helper scripts: + +- `e2e:smoke` uses `PLATFORM` +- integration tests use `INTEGRATION_PLATFORM` + +Cross-arch builds are possible at image-selection time, but for reliable runtime execution you should use a platform that matches the runtime guest architecture. + ### 2) Convert image -> assets ```bash @@ -176,4 +196,5 @@ dockerfile2gondolin --file PATH --context PATH --out PATH [options] ## Repo notes - This repo is standalone; Gondolin core is not modified. +- Gondolin upstream repository: https://github.com/earendil-works/gondolin - `out/` is generated output and ignored by git. diff --git a/docs/linux.md b/docs/linux.md new file mode 100644 index 0000000..30215c2 --- /dev/null +++ b/docs/linux.md @@ -0,0 +1,89 @@ +# Linux setup guide + +This guide is for running `docker2vm` on Linux hosts. + +## 1) Install required tools + +### Ubuntu / Debian + +```bash +sudo apt-get update +sudo apt-get install -y curl unzip e2fsprogs qemu-system-x86 +``` + +Install Bun: + +```bash +curl -fsSL https://bun.sh/install | bash +source ~/.bashrc +``` + +If you want Dockerfile conversion (`dockerfile2gondolin`), install Docker and Buildx. + +## 2) Verify toolchain + +```bash +bun --version +qemu-system-x86_64 --version +mke2fs -V +debugfs -V +``` + +## 3) Install dependencies and validate + +```bash +bun install +bun run test +bun run typecheck +bun run build +``` + +## 4) Choose the build platform + +Use a platform that matches the architecture you will run in Gondolin. + +- `uname -m` => `x86_64` or `amd64`: use `linux/amd64` +- `uname -m` => `aarch64` or `arm64`: use `linux/arm64` + +`oci2gondolin` defaults automatically from host arch if `--platform` is omitted, but passing it explicitly is recommended. + +Example: + +```bash +bun run oci2gondolin -- \ + --image busybox:latest \ + --platform linux/amd64 \ + --mode assets \ + --out ./out/busybox-assets +``` + +## 5) Run integration + smoke checks + +amd64 host: + +```bash +INTEGRATION_PLATFORM=linux/amd64 bun run test:integration +PLATFORM=linux/amd64 bun run e2e:smoke +``` + +arm64 host: + +```bash +INTEGRATION_PLATFORM=linux/arm64 bun run test:integration +PLATFORM=linux/arm64 bun run e2e:smoke +``` + +## Notes on virtualization performance + +- If `/dev/kvm` is available, QEMU can use hardware acceleration. +- If `/dev/kvm` is unavailable (common in CI), the project falls back to TCG emulation (slower but functional). + +## Troubleshooting + +### `sandbox_stopped` / VM exits quickly + +Confirm QEMU is installed and that you are using a platform that matches your host and assets. + +### `mke2fs` or `debugfs` missing + +Reinstall `e2fsprogs` and verify the commands are available in your shell. diff --git a/docs/macos.md b/docs/macos.md new file mode 100644 index 0000000..355af2d --- /dev/null +++ b/docs/macos.md @@ -0,0 +1,94 @@ +# macOS setup guide + +This guide is for running `docker2vm` on macOS (Apple Silicon or Intel). + +## 1) Install required tools + +```bash +brew install bun qemu e2fsprogs +``` + +If you want Dockerfile conversion (`dockerfile2gondolin`), also install Docker Desktop. + +## 2) Ensure `mke2fs` and `debugfs` are on `PATH` + +`e2fsprogs` is often keg-only on macOS. + +### Apple Silicon (`/opt/homebrew`) + +```bash +echo 'export PATH="/opt/homebrew/opt/e2fsprogs/sbin:$PATH"' >> ~/.zshrc +source ~/.zshrc +``` + +### Intel (`/usr/local`) + +```bash +echo 'export PATH="/usr/local/opt/e2fsprogs/sbin:$PATH"' >> ~/.zshrc +source ~/.zshrc +``` + +## 3) Verify toolchain + +```bash +bun --version +qemu-system-aarch64 --version || qemu-system-x86_64 --version +mke2fs -V +debugfs -V +``` + +## 4) Install dependencies and validate + +```bash +bun install +bun run test +bun run typecheck +bun run build +``` + +## 5) Choose the build platform + +Use a platform that matches the architecture you will run in Gondolin. + +- Apple Silicon (`uname -m` => `arm64`): use `linux/arm64` +- Intel Mac (`uname -m` => `x86_64`): use `linux/amd64` + +`oci2gondolin` defaults automatically from host arch if `--platform` is omitted, but passing it explicitly is recommended. + +Example: + +```bash +bun run oci2gondolin -- \ + --image busybox:latest \ + --platform linux/arm64 \ + --mode assets \ + --out ./out/busybox-assets +``` + +## 6) Run integration + smoke checks + +Apple Silicon: + +```bash +INTEGRATION_PLATFORM=linux/arm64 bun run test:integration +PLATFORM=linux/arm64 bun run e2e:smoke +``` + +Intel Mac: + +```bash +INTEGRATION_PLATFORM=linux/amd64 bun run test:integration +PLATFORM=linux/amd64 bun run e2e:smoke +``` + +## Troubleshooting + +### `mke2fs` / `debugfs` not found + +Confirm PATH includes the `e2fsprogs` `sbin` directory shown above. + +### Case-sensitive filename conflicts during conversion + +Some images include paths that conflict on case-insensitive filesystems. + +Use a case-sensitive APFS location for temporary work/output (for example, a case-sensitive volume) and retry. From 28883c07c6e567d5487068ead57358e8c99626b9 Mon Sep 17 00:00:00 2001 From: Victor Garcia Date: Thu, 12 Feb 2026 08:55:31 -0500 Subject: [PATCH 2/9] Run integration tests as distro matrix in CI --- .github/workflows/ci.yml | 22 ++++++++++++-- README.md | 8 +++++ .../oci2gondolin.integration.test.ts | 30 +++++++++++-------- 3 files changed, 45 insertions(+), 15 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index cecf76c..f4808f1 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -33,7 +33,22 @@ jobs: integration-tests: runs-on: ubuntu-latest needs: build-and-test - timeout-minutes: 40 + timeout-minutes: 50 + strategy: + fail-fast: false + matrix: + include: + - distro: alpine + image: alpine:3.20 + - distro: debian + image: debian:bookworm-slim + - distro: ubuntu + image: ubuntu:24.04 + - distro: fedora + image: fedora:41 + - distro: archlinux + image: archlinux:latest + name: integration-tests (${{ matrix.distro }}) steps: - name: Checkout @@ -55,7 +70,10 @@ jobs: - name: Run integration tests env: INTEGRATION_PLATFORM: linux/amd64 - INTEGRATION_IMAGE: busybox:latest + INTEGRATION_IMAGE: ${{ matrix.image }} + INTEGRATION_ROOTFS_CHECK_PATH: /bin/sh + INTEGRATION_VM_CHECK_COMMAND: echo integration-vm-ok + INTEGRATION_VM_CHECK_EXPECT: integration-vm-ok run: bun run test:integration e2e-smoke: diff --git a/README.md b/README.md index 9c88d2c..eafc91e 100644 --- a/README.md +++ b/README.md @@ -80,6 +80,14 @@ bun run build bun run test:integration ``` +The CI integration matrix currently validates: + +- `alpine:3.20` +- `debian:bookworm-slim` +- `ubuntu:24.04` +- `fedora:41` +- `archlinux:latest` + ### Choosing the build platform (`--platform`) `--platform` selects which OCI image variant to convert, and should match the architecture you plan to run in Gondolin. diff --git a/test/integration/oci2gondolin.integration.test.ts b/test/integration/oci2gondolin.integration.test.ts index 86dc1d1..83e8954 100644 --- a/test/integration/oci2gondolin.integration.test.ts +++ b/test/integration/oci2gondolin.integration.test.ts @@ -22,17 +22,21 @@ type CommandOptions = { const REPO_ROOT = process.cwd(); const IMAGE = process.env.INTEGRATION_IMAGE ?? "busybox:latest"; const PLATFORM = resolveIntegrationPlatform(process.env.INTEGRATION_PLATFORM ?? process.arch); +const ROOTFS_CHECK_PATH = process.env.INTEGRATION_ROOTFS_CHECK_PATH ?? "/bin/sh"; +const VM_CHECK_COMMAND = process.env.INTEGRATION_VM_CHECK_COMMAND ?? "echo integration-vm-ok"; +const VM_CHECK_EXPECT = process.env.INTEGRATION_VM_CHECK_EXPECT ?? "integration-vm-ok"; -const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), "docker2vm-integration-")); -const rootfsOutDir = path.join(tempRoot, "busybox-rootfs"); -const assetsOutDir = path.join(tempRoot, "busybox-assets"); +const imageSlug = IMAGE.replace(/[^a-z0-9._-]+/gi, "-").toLowerCase(); +const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), `docker2vm-integration-${imageSlug}-`)); +const rootfsOutDir = path.join(tempRoot, `${imageSlug}-rootfs`); +const assetsOutDir = path.join(tempRoot, `${imageSlug}-assets`); afterAll(() => { fs.rmSync(tempRoot, { recursive: true, force: true }); }); describe("oci2gondolin integration", () => { - it("materializes a busybox rootfs image", async () => { + it(`materializes a rootfs image for ${IMAGE}`, async () => { const debugfsBinary = requireBinary("debugfs", [ "/opt/homebrew/opt/e2fsprogs/sbin/debugfs", "/usr/local/opt/e2fsprogs/sbin/debugfs", @@ -52,7 +56,7 @@ describe("oci2gondolin integration", () => { "--out", rootfsOutDir, ], - { cwd: REPO_ROOT, timeoutMs: 180_000 }, + { cwd: REPO_ROOT, timeoutMs: 300_000 }, ); assertSuccess(result, "oci2gondolin rootfs conversion"); @@ -76,16 +80,16 @@ describe("oci2gondolin integration", () => { const debugfsResult = await runCommand( debugfsBinary, - ["-R", "stat /bin/busybox", rootfsPath], + ["-R", `stat ${ROOTFS_CHECK_PATH}`, rootfsPath], { timeoutMs: 30_000 }, ); - assertSuccess(debugfsResult, "debugfs stat /bin/busybox"); + assertSuccess(debugfsResult, `debugfs stat ${ROOTFS_CHECK_PATH}`); const debugText = `${debugfsResult.stdout}\n${debugfsResult.stderr}`.toLowerCase(); expect(debugText).not.toContain("file not found by ext2_lookup"); - }, 240_000); + }, 420_000); - it("materializes busybox assets and executes inside a VM", async () => { + it(`materializes assets for ${IMAGE} and executes inside a VM`, async () => { requireBinary(process.arch === "arm64" ? "qemu-system-aarch64" : "qemu-system-x86_64"); const result = await runCommand( @@ -102,7 +106,7 @@ describe("oci2gondolin integration", () => { "--out", assetsOutDir, ], - { cwd: REPO_ROOT, timeoutMs: 180_000 }, + { cwd: REPO_ROOT, timeoutMs: 300_000 }, ); assertSuccess(result, "oci2gondolin assets conversion"); @@ -141,9 +145,9 @@ describe("oci2gondolin integration", () => { process.env.GONDOLIN_GUEST_DIR = assetsOutDir; vm = await VM.create({ sandbox: vmSandbox }); - const execResult = await vm.exec(["/bin/busybox", "echo", "integration-vm-ok"]); + const execResult = await vm.exec(["/bin/sh", "-lc", VM_CHECK_COMMAND]); expect(execResult.exitCode).toBe(0); - expect(execResult.stdout).toContain("integration-vm-ok"); + expect(execResult.stdout).toContain(VM_CHECK_EXPECT); } finally { await vm?.close().catch(() => { // ignore close errors in test teardown @@ -155,7 +159,7 @@ describe("oci2gondolin integration", () => { process.env.GONDOLIN_GUEST_DIR = originalGuestDir; } } - }, 300_000); + }, 420_000); }); function resolveIntegrationPlatform(raw: string): "linux/amd64" | "linux/arm64" { From b96360289b2675df3a1c31382e85e2a731f2cce6 Mon Sep 17 00:00:00 2001 From: Victor Garcia Date: Thu, 12 Feb 2026 09:09:07 -0500 Subject: [PATCH 3/9] Assert distro-specific probes in integration matrix tests --- .github/workflows/ci.yml | 21 ++++++++-- README.md | 2 + .../oci2gondolin.integration.test.ts | 41 +++++++++++++++++-- 3 files changed, 57 insertions(+), 7 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index f4808f1..d883067 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -40,14 +40,29 @@ jobs: include: - distro: alpine image: alpine:3.20 + probe_path: /etc/alpine-release + probe_command: cat /etc/alpine-release + probe_expect: "3.20" - distro: debian image: debian:bookworm-slim + probe_path: /etc/debian_version + probe_command: cat /etc/debian_version + probe_expect: "12" - distro: ubuntu image: ubuntu:24.04 + probe_path: /etc/os-release + probe_command: . /etc/os-release; echo "$ID:$VERSION_ID" + probe_expect: ubuntu:24.04 - distro: fedora image: fedora:41 + probe_path: /etc/fedora-release + probe_command: cat /etc/fedora-release + probe_expect: "Fedora release 41" - distro: archlinux image: archlinux:latest + probe_path: /etc/os-release + probe_command: . /etc/os-release; echo "$ID" + probe_expect: arch name: integration-tests (${{ matrix.distro }}) steps: @@ -71,9 +86,9 @@ jobs: env: INTEGRATION_PLATFORM: linux/amd64 INTEGRATION_IMAGE: ${{ matrix.image }} - INTEGRATION_ROOTFS_CHECK_PATH: /bin/sh - INTEGRATION_VM_CHECK_COMMAND: echo integration-vm-ok - INTEGRATION_VM_CHECK_EXPECT: integration-vm-ok + INTEGRATION_ROOTFS_CHECK_PATH: ${{ matrix.probe_path }} + INTEGRATION_VM_CHECK_COMMAND: ${{ matrix.probe_command }} + INTEGRATION_VM_CHECK_EXPECT: ${{ matrix.probe_expect }} run: bun run test:integration e2e-smoke: diff --git a/README.md b/README.md index eafc91e..87e0894 100644 --- a/README.md +++ b/README.md @@ -88,6 +88,8 @@ The CI integration matrix currently validates: - `fedora:41` - `archlinux:latest` +For each distro row, tests run a distro-specific probe command (for example `/etc/debian_version`, `/etc/fedora-release`, etc.) and also assert that probe does **not** match on the base Gondolin guest image. + ### Choosing the build platform (`--platform`) `--platform` selects which OCI image variant to convert, and should match the architecture you plan to run in Gondolin. diff --git a/test/integration/oci2gondolin.integration.test.ts b/test/integration/oci2gondolin.integration.test.ts index 83e8954..64eca2b 100644 --- a/test/integration/oci2gondolin.integration.test.ts +++ b/test/integration/oci2gondolin.integration.test.ts @@ -20,11 +20,11 @@ type CommandOptions = { }; const REPO_ROOT = process.cwd(); -const IMAGE = process.env.INTEGRATION_IMAGE ?? "busybox:latest"; +const IMAGE = process.env.INTEGRATION_IMAGE ?? "debian:bookworm-slim"; const PLATFORM = resolveIntegrationPlatform(process.env.INTEGRATION_PLATFORM ?? process.arch); -const ROOTFS_CHECK_PATH = process.env.INTEGRATION_ROOTFS_CHECK_PATH ?? "/bin/sh"; -const VM_CHECK_COMMAND = process.env.INTEGRATION_VM_CHECK_COMMAND ?? "echo integration-vm-ok"; -const VM_CHECK_EXPECT = process.env.INTEGRATION_VM_CHECK_EXPECT ?? "integration-vm-ok"; +const ROOTFS_CHECK_PATH = process.env.INTEGRATION_ROOTFS_CHECK_PATH ?? "/etc/debian_version"; +const VM_CHECK_COMMAND = process.env.INTEGRATION_VM_CHECK_COMMAND ?? "cat /etc/debian_version"; +const VM_CHECK_EXPECT = process.env.INTEGRATION_VM_CHECK_EXPECT ?? "12"; const imageSlug = IMAGE.replace(/[^a-z0-9._-]+/gi, "-").toLowerCase(); const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), `docker2vm-integration-${imageSlug}-`)); @@ -140,6 +140,8 @@ describe("oci2gondolin integration", () => { const originalGuestDir = process.env.GONDOLIN_GUEST_DIR; const vmSandbox = resolveVmSandboxOptions(); + await assertProbeMissingFromBaseImage(vmSandbox, originalGuestDir); + let vm: VM | null = null; try { process.env.GONDOLIN_GUEST_DIR = assetsOutDir; @@ -194,6 +196,37 @@ function resolveVmSandboxOptions(): { accel?: "tcg"; cpu?: "max" } | undefined { } } +async function assertProbeMissingFromBaseImage( + vmSandbox: { accel?: "tcg"; cpu?: "max" } | undefined, + originalGuestDir: string | undefined, +): Promise { + let vm: VM | null = null; + + try { + if (originalGuestDir === undefined) { + delete process.env.GONDOLIN_GUEST_DIR; + } else { + process.env.GONDOLIN_GUEST_DIR = originalGuestDir; + } + + vm = await VM.create({ sandbox: vmSandbox }); + const baseResult = await vm.exec(["/bin/sh", "-lc", VM_CHECK_COMMAND]); + + expect(baseResult.stdout).not.toContain(VM_CHECK_EXPECT); + expect(baseResult.stderr).not.toContain(VM_CHECK_EXPECT); + } finally { + await vm?.close().catch(() => { + // ignore close errors in test teardown + }); + + if (originalGuestDir === undefined) { + delete process.env.GONDOLIN_GUEST_DIR; + } else { + process.env.GONDOLIN_GUEST_DIR = originalGuestDir; + } + } +} + function requireBinary(binary: string, fallbacks: string[] = []): string { const candidates = [binary, ...fallbacks]; From 4720c6cb78b76f00df53809d0e50755daf469dc4 Mon Sep 17 00:00:00 2001 From: Victor Garcia Date: Thu, 12 Feb 2026 09:33:55 -0500 Subject: [PATCH 4/9] Decouple gondolin runtime assets and refresh setup docs --- .github/workflows/ci.yml | 6 + README.md | 31 +++- bun.lock | 4 +- docs/linux.md | 33 +++- docs/macos.md | 46 +++-- package.json | 5 +- src/oci2gondolin/materialize/index.ts | 5 +- .../materialize/runtime-injection.ts | 7 +- src/shared/gondolin-assets.ts | 175 ++++++++++++++++++ 9 files changed, 267 insertions(+), 45 deletions(-) create mode 100644 src/shared/gondolin-assets.ts diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index d883067..188b74d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -82,6 +82,9 @@ jobs: sudo apt-get update sudo apt-get install -y qemu-system-x86 e2fsprogs + - name: Prime Gondolin guest assets + run: bunx gondolin exec -- /bin/true + - name: Run integration tests env: INTEGRATION_PLATFORM: linux/amd64 @@ -113,6 +116,9 @@ jobs: sudo apt-get update sudo apt-get install -y qemu-system-x86 e2fsprogs + - name: Prime Gondolin guest assets + run: bunx gondolin exec -- /bin/true + - name: Run end-to-end smoke test env: PLATFORM: linux/amd64 diff --git a/README.md b/README.md index 87e0894..584afde 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ `docker2vm` converts OCI container images (or Dockerfiles via BuildKit) into VM-compatible outputs. Today, the runtime materialization target is [Gondolin](https://github.com/earendil-works/gondolin). -It follows an OCI-first flow inspired by "Docker without Docker": +It follows an OCI-first flow inspired by ["Docker without Docker"](https://fly.io/blog/docker-without-docker/): - resolve/pull an OCI image - apply layers to a root filesystem @@ -16,6 +16,7 @@ Docker containers share the host kernel. Gondolin runs workloads inside a VM, so ## Current features +- zero runtime npm dependencies (`dependencies: {}`) - `oci2gondolin` core converter - input: `--image`, `--oci-layout`, `--oci-tar` (exactly one) - platform: `linux/amd64`, `linux/arm64` @@ -39,6 +40,7 @@ Docker containers share the host kernel. Gondolin runs workloads inside a VM, so - Bun >= 1.2 - `e2fsprogs` (`mke2fs`, `debugfs`) - QEMU (for runtime smoke checks via `gondolin exec`) +- Gondolin CLI installed separately (tested with `@earendil-works/gondolin@0.2.1`) - Docker (only required for `dockerfile2gondolin`) macOS helpers: @@ -53,17 +55,25 @@ Ubuntu helpers: sudo apt-get install -y e2fsprogs qemu-system-x86 ``` -## Platform setup guides +Install Gondolin CLI (tested version): -- [macOS guide](./docs/macos.md) -- [Linux guide](./docs/linux.md) +```bash +bun add -g @earendil-works/gondolin@0.2.1 +``` -## Install +Prime guest assets once: ```bash -bun install +gondolin exec -- /bin/true ``` +> On macOS, `docker2vm` checks common Homebrew `e2fsprogs` locations automatically; updating `PATH` is usually optional. + +## Platform setup guides + +- [macOS guide](./docs/macos.md) +- [Linux guide](./docs/linux.md) + ## Quickstart ### 1) Validate @@ -92,11 +102,12 @@ For each distro row, tests run a distro-specific probe command (for example `/et ### Choosing the build platform (`--platform`) -`--platform` selects which OCI image variant to convert, and should match the architecture you plan to run in Gondolin. +`--platform` selects which OCI image variant to convert, and should match the architecture you plan to run in Gondolin. This applies to both `oci2gondolin` and `dockerfile2gondolin`. - Apple Silicon / arm64 Linux hosts: `linux/arm64` - Intel / amd64 hosts: `linux/amd64` -- If omitted, `oci2gondolin` defaults from host arch (`x64 -> linux/amd64`, `arm64 -> linux/arm64`). +- If omitted, both commands default from host arch (`x64 -> linux/amd64`, `arm64 -> linux/arm64`). +- You can always override manually with `--platform linux/amd64` or `--platform linux/arm64`. For helper scripts: @@ -118,7 +129,7 @@ bun run oci2gondolin -- \ ### 3) Run with Gondolin ```bash -GONDOLIN_GUEST_DIR=./out/busybox-assets bunx gondolin exec -- /bin/busybox echo hello +GONDOLIN_GUEST_DIR=./out/busybox-assets gondolin exec -- /bin/busybox echo hello ``` ## Dockerfile flow @@ -143,7 +154,7 @@ bun run dockerfile2gondolin -- \ Then run: ```bash -GONDOLIN_GUEST_DIR=./out/demo-assets bunx gondolin exec -- /usr/games/cowsay "hello" +GONDOLIN_GUEST_DIR=./out/demo-assets gondolin exec -- /usr/games/cowsay "hello" ``` ## End-to-end smoke test diff --git a/bun.lock b/bun.lock index d023471..46da84a 100644 --- a/bun.lock +++ b/bun.lock @@ -4,10 +4,8 @@ "workspaces": { "": { "name": "docker2vm", - "dependencies": { - "@earendil-works/gondolin": "^0.2.1", - }, "devDependencies": { + "@earendil-works/gondolin": "0.2.1", "@types/node": "^22.13.10", "typescript": "^5.7.3", }, diff --git a/docs/linux.md b/docs/linux.md index 30215c2..b876dd2 100644 --- a/docs/linux.md +++ b/docs/linux.md @@ -2,6 +2,8 @@ This guide is for running `docker2vm` on Linux hosts. +`docker2vm` itself has **0 runtime npm dependencies**; system/runtime tools are installed separately. + ## 1) Install required tools ### Ubuntu / Debian @@ -15,30 +17,49 @@ Install Bun: ```bash curl -fsSL https://bun.sh/install | bash -source ~/.bashrc +export BUN_INSTALL="$HOME/.bun" +export PATH="$BUN_INSTALL/bin:$PATH" ``` If you want Dockerfile conversion (`dockerfile2gondolin`), install Docker and Buildx. -## 2) Verify toolchain +## 2) Install Gondolin CLI separately (tested version) + +`docker2vm` is tested with: + +- `@earendil-works/gondolin@0.2.1` + +Install (global): + +```bash +bun add -g @earendil-works/gondolin@0.2.1 +``` + +Prime guest assets once: + +```bash +gondolin exec -- /bin/true +``` + +## 3) Verify toolchain ```bash bun --version qemu-system-x86_64 --version mke2fs -V debugfs -V +gondolin --help >/dev/null ``` -## 3) Install dependencies and validate +## 4) Validate from source checkout ```bash -bun install bun run test bun run typecheck bun run build ``` -## 4) Choose the build platform +## 5) Choose the build platform Use a platform that matches the architecture you will run in Gondolin. @@ -57,7 +78,7 @@ bun run oci2gondolin -- \ --out ./out/busybox-assets ``` -## 5) Run integration + smoke checks +## 6) Run integration + smoke checks amd64 host: diff --git a/docs/macos.md b/docs/macos.md index 355af2d..cea7202 100644 --- a/docs/macos.md +++ b/docs/macos.md @@ -2,6 +2,8 @@ This guide is for running `docker2vm` on macOS (Apple Silicon or Intel). +`docker2vm` itself has **0 runtime npm dependencies**; system/runtime tools are installed separately. + ## 1) Install required tools ```bash @@ -10,43 +12,55 @@ brew install bun qemu e2fsprogs If you want Dockerfile conversion (`dockerfile2gondolin`), also install Docker Desktop. -## 2) Ensure `mke2fs` and `debugfs` are on `PATH` +## 2) Optional: add `e2fsprogs` binaries to `PATH` + +With Homebrew, `e2fsprogs` is often keg-only. `docker2vm` checks common Homebrew locations automatically, so a PATH change is usually **not required** for normal usage. + +If you want to run `mke2fs` / `debugfs` manually in your shell: + +```bash +export PATH="$(brew --prefix e2fsprogs)/sbin:$PATH" +``` + +To persist it, add that `export PATH=...` line to your shell profile (`~/.zshrc`, `~/.bashrc`, `~/.profile`, etc.). + +## 3) Install Gondolin CLI separately (tested version) + +`docker2vm` is tested with: -`e2fsprogs` is often keg-only on macOS. +- `@earendil-works/gondolin@0.2.1` -### Apple Silicon (`/opt/homebrew`) +Install (global): ```bash -echo 'export PATH="/opt/homebrew/opt/e2fsprogs/sbin:$PATH"' >> ~/.zshrc -source ~/.zshrc +bun add -g @earendil-works/gondolin@0.2.1 ``` -### Intel (`/usr/local`) +Prime guest assets once: ```bash -echo 'export PATH="/usr/local/opt/e2fsprogs/sbin:$PATH"' >> ~/.zshrc -source ~/.zshrc +gondolin exec -- /bin/true ``` -## 3) Verify toolchain +## 4) Verify toolchain ```bash bun --version qemu-system-aarch64 --version || qemu-system-x86_64 --version -mke2fs -V -debugfs -V +"$(brew --prefix e2fsprogs)/sbin/mke2fs" -V +"$(brew --prefix e2fsprogs)/sbin/debugfs" -V +gondolin --help >/dev/null ``` -## 4) Install dependencies and validate +## 5) Validate from source checkout ```bash -bun install bun run test bun run typecheck bun run build ``` -## 5) Choose the build platform +## 6) Choose the build platform Use a platform that matches the architecture you will run in Gondolin. @@ -65,7 +79,7 @@ bun run oci2gondolin -- \ --out ./out/busybox-assets ``` -## 6) Run integration + smoke checks +## 7) Run integration + smoke checks Apple Silicon: @@ -85,7 +99,7 @@ PLATFORM=linux/amd64 bun run e2e:smoke ### `mke2fs` / `debugfs` not found -Confirm PATH includes the `e2fsprogs` `sbin` directory shown above. +`docker2vm` should find common Homebrew locations automatically. If your install uses a custom prefix, either add it to `PATH` or point `GONDOLIN_GUEST_DIR` to prepared assets and verify `e2fsprogs` binaries are installed. ### Case-sensitive filename conflicts during conversion diff --git a/package.json b/package.json index fa3bc54..2b884ae 100644 --- a/package.json +++ b/package.json @@ -19,6 +19,7 @@ "dockerfile2gondolin": "bun run src/bin/dockerfile2gondolin.ts" }, "devDependencies": { + "@earendil-works/gondolin": "0.2.1", "@types/node": "^22.13.10", "typescript": "^5.7.3" }, @@ -26,7 +27,5 @@ "bun": ">=1.2.0" }, "packageManager": "bun@1.3.6", - "dependencies": { - "@earendil-works/gondolin": "^0.2.1" - } + "dependencies": {} } diff --git a/src/oci2gondolin/materialize/index.ts b/src/oci2gondolin/materialize/index.ts index ae21e61..208a622 100644 --- a/src/oci2gondolin/materialize/index.ts +++ b/src/oci2gondolin/materialize/index.ts @@ -1,10 +1,9 @@ import fs from "node:fs"; import path from "node:path"; -import { ensureGuestAssets } from "@earendil-works/gondolin"; - import type { AppliedRootfs, MaterializedOutput, Oci2GondolinOptions } from "../types"; import { sha256File } from "../utils/digest"; +import { resolveGondolinGuestAssets } from "../../shared/gondolin-assets"; import { ensureDirectory } from "../utils/fs"; import { createExt4FromDirectory } from "./ext4"; import { injectGondolinRuntime } from "./runtime-injection"; @@ -51,7 +50,7 @@ export async function materializeOutput( let assetManifestPath: string | undefined; if (options.mode === "assets") { - const baseAssets = await ensureGuestAssets(); + const baseAssets = resolveGondolinGuestAssets(); const kernelPath = path.join(outDir, KERNEL_FILENAME); const initramfsPath = path.join(outDir, INITRAMFS_FILENAME); diff --git a/src/oci2gondolin/materialize/runtime-injection.ts b/src/oci2gondolin/materialize/runtime-injection.ts index e2d2271..36d7073 100644 --- a/src/oci2gondolin/materialize/runtime-injection.ts +++ b/src/oci2gondolin/materialize/runtime-injection.ts @@ -2,9 +2,8 @@ import fs from "node:fs"; import path from "node:path"; import { spawnSync } from "node:child_process"; -import { ensureGuestAssets } from "@earendil-works/gondolin"; - import { CliUsageError } from "../../shared/cli-errors"; +import { resolveGondolinGuestAssets } from "../../shared/gondolin-assets"; type RuntimeFileSpec = { sourcePathInRootfs: string; @@ -89,7 +88,7 @@ export interface RuntimeInjectionResult { } export async function extractBaseRootfsTree(destinationDir: string): Promise { - const guestAssets = await ensureGuestAssets(); + const guestAssets = resolveGondolinGuestAssets(); const baseRootfsPath = guestAssets.rootfsPath; if (!fs.existsSync(baseRootfsPath)) { @@ -119,7 +118,7 @@ export async function extractBaseRootfsTree(destinationDir: string): Promise { - const guestAssets = await ensureGuestAssets(); + const guestAssets = resolveGondolinGuestAssets(); const baseRootfsPath = guestAssets.rootfsPath; if (!fs.existsSync(baseRootfsPath)) { diff --git a/src/shared/gondolin-assets.ts b/src/shared/gondolin-assets.ts new file mode 100644 index 0000000..9403fd5 --- /dev/null +++ b/src/shared/gondolin-assets.ts @@ -0,0 +1,175 @@ +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; + +import { CliUsageError } from "./cli-errors"; + +export const TESTED_GONDOLIN_VERSION = "0.2.1"; + +export interface GondolinGuestAssets { + assetDir: string; + kernelPath: string; + initrdPath: string; + rootfsPath: string; +} + +type AssetFileNames = { + kernel: string; + initramfs: string; + rootfs: string; +}; + +const DEFAULT_FILE_NAMES: AssetFileNames = { + kernel: "vmlinuz-virt", + initramfs: "initramfs.cpio.lz4", + rootfs: "rootfs.ext4", +}; + +export function resolveGondolinGuestAssets(): GondolinGuestAssets { + const explicitDir = process.env.GONDOLIN_GUEST_DIR; + if (explicitDir && explicitDir.trim().length > 0) { + return loadAssetsFromDirectory(path.resolve(explicitDir), "GONDOLIN_GUEST_DIR"); + } + + for (const candidateDir of discoverCachedGuestAssetDirectories()) { + const loaded = tryLoadAssetsFromDirectory(candidateDir); + if (loaded) { + return loaded; + } + } + + throw new CliUsageError("Gondolin guest assets were not found.", [ + "Install gondolin CLI separately (tested with @earendil-works/gondolin@0.2.1).", + "Run once to populate guest assets: gondolin exec -- /bin/true", + "Or set GONDOLIN_GUEST_DIR to a directory containing: vmlinuz-virt, initramfs.cpio.lz4, rootfs.ext4.", + "Expected cache location: ~/.cache/gondolin//", + ]); +} + +function discoverCachedGuestAssetDirectories(): string[] { + const cacheBase = process.env.XDG_CACHE_HOME ?? path.join(os.homedir(), ".cache"); + const gondolinCacheRoot = path.resolve(cacheBase, "gondolin"); + + if (!fs.existsSync(gondolinCacheRoot)) { + return []; + } + + const dirs = fs + .readdirSync(gondolinCacheRoot, { withFileTypes: true }) + .filter((entry) => entry.isDirectory()) + .map((entry) => path.join(gondolinCacheRoot, entry.name)); + + dirs.sort((a, b) => compareCacheDirectoryPriority(path.basename(a), path.basename(b))); + + return dirs; +} + +function compareCacheDirectoryPriority(a: string, b: string): number { + const parsedA = parseSemverTag(a); + const parsedB = parseSemverTag(b); + + if (parsedA && parsedB) { + for (let i = 0; i < parsedA.length; i += 1) { + if (parsedA[i] !== parsedB[i]) { + return parsedB[i] - parsedA[i]; + } + } + return 0; + } + + if (parsedA) { + return -1; + } + + if (parsedB) { + return 1; + } + + return b.localeCompare(a); +} + +function parseSemverTag(value: string): [number, number, number] | null { + const match = /^v?(\d+)\.(\d+)\.(\d+)$/.exec(value.trim()); + if (!match) { + return null; + } + + return [Number(match[1]), Number(match[2]), Number(match[3])]; +} + +function tryLoadAssetsFromDirectory(candidateDir: string): GondolinGuestAssets | null { + try { + return loadAssetsFromDirectory(candidateDir, "cache"); + } catch { + return null; + } +} + +function loadAssetsFromDirectory(assetDir: string, source: "GONDOLIN_GUEST_DIR" | "cache"): GondolinGuestAssets { + const fileNames = resolveAssetFileNames(assetDir); + + const kernelPath = path.join(assetDir, fileNames.kernel); + const initrdPath = path.join(assetDir, fileNames.initramfs); + const rootfsPath = path.join(assetDir, fileNames.rootfs); + + const missing = [ + [fileNames.kernel, kernelPath], + [fileNames.initramfs, initrdPath], + [fileNames.rootfs, rootfsPath], + ] + .filter(([, filePath]) => !fs.existsSync(filePath)) + .map(([name]) => name); + + if (missing.length > 0) { + const hint = + source === "GONDOLIN_GUEST_DIR" + ? "Verify GONDOLIN_GUEST_DIR points to a valid gondolin guest asset directory." + : "Run 'gondolin exec -- /bin/true' to download guest assets into the cache."; + + throw new CliUsageError("Gondolin guest assets are incomplete.", [ + `Directory: ${assetDir}`, + `Missing files: ${missing.join(", ")}`, + hint, + ]); + } + + return { + assetDir, + kernelPath, + initrdPath, + rootfsPath, + }; +} + +function resolveAssetFileNames(assetDir: string): AssetFileNames { + const manifestPath = path.join(assetDir, "manifest.json"); + if (!fs.existsSync(manifestPath)) { + return DEFAULT_FILE_NAMES; + } + + try { + const raw = fs.readFileSync(manifestPath, "utf8"); + const parsed = JSON.parse(raw) as { + assets?: { + kernel?: unknown; + initramfs?: unknown; + rootfs?: unknown; + }; + }; + + const kernel = typeof parsed.assets?.kernel === "string" ? parsed.assets.kernel : DEFAULT_FILE_NAMES.kernel; + const initramfs = + typeof parsed.assets?.initramfs === "string" + ? parsed.assets.initramfs + : DEFAULT_FILE_NAMES.initramfs; + const rootfs = typeof parsed.assets?.rootfs === "string" ? parsed.assets.rootfs : DEFAULT_FILE_NAMES.rootfs; + + return { + kernel, + initramfs, + rootfs, + }; + } catch { + return DEFAULT_FILE_NAMES; + } +} From 3d92f5cab592bd724f094b9954dbeaf3194ae861 Mon Sep 17 00:00:00 2001 From: Victor Garcia Date: Thu, 12 Feb 2026 09:38:31 -0500 Subject: [PATCH 5/9] Prime Gondolin assets without booting VM in CI --- .github/workflows/ci.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 188b74d..b5ae06c 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -83,7 +83,7 @@ jobs: sudo apt-get install -y qemu-system-x86 e2fsprogs - name: Prime Gondolin guest assets - run: bunx gondolin exec -- /bin/true + run: bun --eval 'import { ensureGuestAssets } from "@earendil-works/gondolin"; await ensureGuestAssets();' - name: Run integration tests env: @@ -117,7 +117,7 @@ jobs: sudo apt-get install -y qemu-system-x86 e2fsprogs - name: Prime Gondolin guest assets - run: bunx gondolin exec -- /bin/true + run: bun --eval 'import { ensureGuestAssets } from "@earendil-works/gondolin"; await ensureGuestAssets();' - name: Run end-to-end smoke test env: From cf091915b45a79b0d09f76c47de06a079542ccdd Mon Sep 17 00:00:00 2001 From: Victor Garcia Date: Thu, 12 Feb 2026 09:53:12 -0500 Subject: [PATCH 6/9] Keep Gondolin runtime dependency and auto-resolve guest assets --- README.md | 23 +-- bun.lock | 4 +- docs/linux.md | 16 +- docs/macos.md | 16 +- package.json | 5 +- src/oci2gondolin/materialize/index.ts | 2 +- .../materialize/runtime-injection.ts | 8 +- src/shared/gondolin-assets.ts | 172 ++---------------- 8 files changed, 44 insertions(+), 202 deletions(-) diff --git a/README.md b/README.md index 584afde..0b9c5a4 100644 --- a/README.md +++ b/README.md @@ -16,7 +16,7 @@ Docker containers share the host kernel. Gondolin runs workloads inside a VM, so ## Current features -- zero runtime npm dependencies (`dependencies: {}`) +- pinned Gondolin runtime dependency (`@earendil-works/gondolin@0.2.1`) for guest-asset retrieval - `oci2gondolin` core converter - input: `--image`, `--oci-layout`, `--oci-tar` (exactly one) - platform: `linux/amd64`, `linux/arm64` @@ -39,32 +39,27 @@ Docker containers share the host kernel. Gondolin runs workloads inside a VM, so - Bun >= 1.2 - `e2fsprogs` (`mke2fs`, `debugfs`) -- QEMU (for runtime smoke checks via `gondolin exec`) -- Gondolin CLI installed separately (tested with `@earendil-works/gondolin@0.2.1`) +- QEMU (for runtime smoke checks) - Docker (only required for `dockerfile2gondolin`) -macOS helpers: +`docker2vm` uses `@earendil-works/gondolin@0.2.1` as a runtime dependency and resolves/downloads guest assets automatically during conversion. -```bash -brew install e2fsprogs qemu -``` - -Ubuntu helpers: +If you also want to run generated assets with `gondolin exec`, install the CLI separately: ```bash -sudo apt-get install -y e2fsprogs qemu-system-x86 +bun add -g @earendil-works/gondolin@0.2.1 ``` -Install Gondolin CLI (tested version): +macOS helpers: ```bash -bun add -g @earendil-works/gondolin@0.2.1 +brew install e2fsprogs qemu ``` -Prime guest assets once: +Ubuntu helpers: ```bash -gondolin exec -- /bin/true +sudo apt-get install -y e2fsprogs qemu-system-x86 ``` > On macOS, `docker2vm` checks common Homebrew `e2fsprogs` locations automatically; updating `PATH` is usually optional. diff --git a/bun.lock b/bun.lock index 46da84a..9c0ee29 100644 --- a/bun.lock +++ b/bun.lock @@ -4,8 +4,10 @@ "workspaces": { "": { "name": "docker2vm", - "devDependencies": { + "dependencies": { "@earendil-works/gondolin": "0.2.1", + }, + "devDependencies": { "@types/node": "^22.13.10", "typescript": "^5.7.3", }, diff --git a/docs/linux.md b/docs/linux.md index b876dd2..2db1cd5 100644 --- a/docs/linux.md +++ b/docs/linux.md @@ -2,7 +2,7 @@ This guide is for running `docker2vm` on Linux hosts. -`docker2vm` itself has **0 runtime npm dependencies**; system/runtime tools are installed separately. +`docker2vm` includes a pinned Gondolin runtime dependency (`@earendil-works/gondolin@0.2.1`) to resolve guest assets during conversion. ## 1) Install required tools @@ -23,24 +23,16 @@ export PATH="$BUN_INSTALL/bin:$PATH" If you want Dockerfile conversion (`dockerfile2gondolin`), install Docker and Buildx. -## 2) Install Gondolin CLI separately (tested version) +## 2) Optional: install Gondolin CLI (for running generated assets) -`docker2vm` is tested with: +`docker2vm` is tested with `@earendil-works/gondolin@0.2.1` and can fetch guest assets automatically during conversion. -- `@earendil-works/gondolin@0.2.1` - -Install (global): +Install the CLI globally if you want to execute generated assets via `gondolin exec`: ```bash bun add -g @earendil-works/gondolin@0.2.1 ``` -Prime guest assets once: - -```bash -gondolin exec -- /bin/true -``` - ## 3) Verify toolchain ```bash diff --git a/docs/macos.md b/docs/macos.md index cea7202..48575ec 100644 --- a/docs/macos.md +++ b/docs/macos.md @@ -2,7 +2,7 @@ This guide is for running `docker2vm` on macOS (Apple Silicon or Intel). -`docker2vm` itself has **0 runtime npm dependencies**; system/runtime tools are installed separately. +`docker2vm` includes a pinned Gondolin runtime dependency (`@earendil-works/gondolin@0.2.1`) to resolve guest assets during conversion. ## 1) Install required tools @@ -24,24 +24,16 @@ export PATH="$(brew --prefix e2fsprogs)/sbin:$PATH" To persist it, add that `export PATH=...` line to your shell profile (`~/.zshrc`, `~/.bashrc`, `~/.profile`, etc.). -## 3) Install Gondolin CLI separately (tested version) +## 3) Optional: install Gondolin CLI (for running generated assets) -`docker2vm` is tested with: +`docker2vm` is tested with `@earendil-works/gondolin@0.2.1` and can fetch guest assets automatically during conversion. -- `@earendil-works/gondolin@0.2.1` - -Install (global): +Install the CLI globally if you want to execute generated assets via `gondolin exec`: ```bash bun add -g @earendil-works/gondolin@0.2.1 ``` -Prime guest assets once: - -```bash -gondolin exec -- /bin/true -``` - ## 4) Verify toolchain ```bash diff --git a/package.json b/package.json index 2b884ae..e2e111b 100644 --- a/package.json +++ b/package.json @@ -19,7 +19,6 @@ "dockerfile2gondolin": "bun run src/bin/dockerfile2gondolin.ts" }, "devDependencies": { - "@earendil-works/gondolin": "0.2.1", "@types/node": "^22.13.10", "typescript": "^5.7.3" }, @@ -27,5 +26,7 @@ "bun": ">=1.2.0" }, "packageManager": "bun@1.3.6", - "dependencies": {} + "dependencies": { + "@earendil-works/gondolin": "0.2.1" + } } diff --git a/src/oci2gondolin/materialize/index.ts b/src/oci2gondolin/materialize/index.ts index 208a622..e76fb12 100644 --- a/src/oci2gondolin/materialize/index.ts +++ b/src/oci2gondolin/materialize/index.ts @@ -50,7 +50,7 @@ export async function materializeOutput( let assetManifestPath: string | undefined; if (options.mode === "assets") { - const baseAssets = resolveGondolinGuestAssets(); + const baseAssets = await resolveGondolinGuestAssets(); const kernelPath = path.join(outDir, KERNEL_FILENAME); const initramfsPath = path.join(outDir, INITRAMFS_FILENAME); diff --git a/src/oci2gondolin/materialize/runtime-injection.ts b/src/oci2gondolin/materialize/runtime-injection.ts index 36d7073..fea1aac 100644 --- a/src/oci2gondolin/materialize/runtime-injection.ts +++ b/src/oci2gondolin/materialize/runtime-injection.ts @@ -88,13 +88,13 @@ export interface RuntimeInjectionResult { } export async function extractBaseRootfsTree(destinationDir: string): Promise { - const guestAssets = resolveGondolinGuestAssets(); + const guestAssets = await resolveGondolinGuestAssets(); const baseRootfsPath = guestAssets.rootfsPath; if (!fs.existsSync(baseRootfsPath)) { throw new CliUsageError("Gondolin base rootfs.ext4 was not found.", [ `Expected path: ${baseRootfsPath}`, - "Run a gondolin command once to download guest assets, then retry.", + "Guest assets should be downloaded automatically via @earendil-works/gondolin; verify network access and retry.", ]); } @@ -118,13 +118,13 @@ export async function extractBaseRootfsTree(destinationDir: string): Promise { - const guestAssets = resolveGondolinGuestAssets(); + const guestAssets = await resolveGondolinGuestAssets(); const baseRootfsPath = guestAssets.rootfsPath; if (!fs.existsSync(baseRootfsPath)) { throw new CliUsageError("Gondolin base rootfs.ext4 was not found.", [ `Expected path: ${baseRootfsPath}`, - "Run a gondolin command once to download guest assets, then retry.", + "Guest assets should be downloaded automatically via @earendil-works/gondolin; verify network access and retry.", ]); } diff --git a/src/shared/gondolin-assets.ts b/src/shared/gondolin-assets.ts index 9403fd5..82499fe 100644 --- a/src/shared/gondolin-assets.ts +++ b/src/shared/gondolin-assets.ts @@ -1,7 +1,7 @@ -import fs from "node:fs"; -import os from "node:os"; import path from "node:path"; +import { ensureGuestAssets } from "@earendil-works/gondolin"; + import { CliUsageError } from "./cli-errors"; export const TESTED_GONDOLIN_VERSION = "0.2.1"; @@ -13,163 +13,23 @@ export interface GondolinGuestAssets { rootfsPath: string; } -type AssetFileNames = { - kernel: string; - initramfs: string; - rootfs: string; -}; - -const DEFAULT_FILE_NAMES: AssetFileNames = { - kernel: "vmlinuz-virt", - initramfs: "initramfs.cpio.lz4", - rootfs: "rootfs.ext4", -}; - -export function resolveGondolinGuestAssets(): GondolinGuestAssets { - const explicitDir = process.env.GONDOLIN_GUEST_DIR; - if (explicitDir && explicitDir.trim().length > 0) { - return loadAssetsFromDirectory(path.resolve(explicitDir), "GONDOLIN_GUEST_DIR"); - } - - for (const candidateDir of discoverCachedGuestAssetDirectories()) { - const loaded = tryLoadAssetsFromDirectory(candidateDir); - if (loaded) { - return loaded; - } - } - - throw new CliUsageError("Gondolin guest assets were not found.", [ - "Install gondolin CLI separately (tested with @earendil-works/gondolin@0.2.1).", - "Run once to populate guest assets: gondolin exec -- /bin/true", - "Or set GONDOLIN_GUEST_DIR to a directory containing: vmlinuz-virt, initramfs.cpio.lz4, rootfs.ext4.", - "Expected cache location: ~/.cache/gondolin//", - ]); -} - -function discoverCachedGuestAssetDirectories(): string[] { - const cacheBase = process.env.XDG_CACHE_HOME ?? path.join(os.homedir(), ".cache"); - const gondolinCacheRoot = path.resolve(cacheBase, "gondolin"); - - if (!fs.existsSync(gondolinCacheRoot)) { - return []; - } - - const dirs = fs - .readdirSync(gondolinCacheRoot, { withFileTypes: true }) - .filter((entry) => entry.isDirectory()) - .map((entry) => path.join(gondolinCacheRoot, entry.name)); - - dirs.sort((a, b) => compareCacheDirectoryPriority(path.basename(a), path.basename(b))); - - return dirs; -} - -function compareCacheDirectoryPriority(a: string, b: string): number { - const parsedA = parseSemverTag(a); - const parsedB = parseSemverTag(b); - - if (parsedA && parsedB) { - for (let i = 0; i < parsedA.length; i += 1) { - if (parsedA[i] !== parsedB[i]) { - return parsedB[i] - parsedA[i]; - } - } - return 0; - } - - if (parsedA) { - return -1; - } - - if (parsedB) { - return 1; - } - - return b.localeCompare(a); -} - -function parseSemverTag(value: string): [number, number, number] | null { - const match = /^v?(\d+)\.(\d+)\.(\d+)$/.exec(value.trim()); - if (!match) { - return null; - } - - return [Number(match[1]), Number(match[2]), Number(match[3])]; -} - -function tryLoadAssetsFromDirectory(candidateDir: string): GondolinGuestAssets | null { - try { - return loadAssetsFromDirectory(candidateDir, "cache"); - } catch { - return null; - } -} - -function loadAssetsFromDirectory(assetDir: string, source: "GONDOLIN_GUEST_DIR" | "cache"): GondolinGuestAssets { - const fileNames = resolveAssetFileNames(assetDir); - - const kernelPath = path.join(assetDir, fileNames.kernel); - const initrdPath = path.join(assetDir, fileNames.initramfs); - const rootfsPath = path.join(assetDir, fileNames.rootfs); - - const missing = [ - [fileNames.kernel, kernelPath], - [fileNames.initramfs, initrdPath], - [fileNames.rootfs, rootfsPath], - ] - .filter(([, filePath]) => !fs.existsSync(filePath)) - .map(([name]) => name); - - if (missing.length > 0) { - const hint = - source === "GONDOLIN_GUEST_DIR" - ? "Verify GONDOLIN_GUEST_DIR points to a valid gondolin guest asset directory." - : "Run 'gondolin exec -- /bin/true' to download guest assets into the cache."; - - throw new CliUsageError("Gondolin guest assets are incomplete.", [ - `Directory: ${assetDir}`, - `Missing files: ${missing.join(", ")}`, - hint, - ]); - } - - return { - assetDir, - kernelPath, - initrdPath, - rootfsPath, - }; -} - -function resolveAssetFileNames(assetDir: string): AssetFileNames { - const manifestPath = path.join(assetDir, "manifest.json"); - if (!fs.existsSync(manifestPath)) { - return DEFAULT_FILE_NAMES; - } - +export async function resolveGondolinGuestAssets(): Promise { try { - const raw = fs.readFileSync(manifestPath, "utf8"); - const parsed = JSON.parse(raw) as { - assets?: { - kernel?: unknown; - initramfs?: unknown; - rootfs?: unknown; - }; - }; - - const kernel = typeof parsed.assets?.kernel === "string" ? parsed.assets.kernel : DEFAULT_FILE_NAMES.kernel; - const initramfs = - typeof parsed.assets?.initramfs === "string" - ? parsed.assets.initramfs - : DEFAULT_FILE_NAMES.initramfs; - const rootfs = typeof parsed.assets?.rootfs === "string" ? parsed.assets.rootfs : DEFAULT_FILE_NAMES.rootfs; + const assets = await ensureGuestAssets(); return { - kernel, - initramfs, - rootfs, + assetDir: path.dirname(assets.rootfsPath), + kernelPath: assets.kernelPath, + initrdPath: assets.initrdPath, + rootfsPath: assets.rootfsPath, }; - } catch { - return DEFAULT_FILE_NAMES; + } catch (error) { + const message = error instanceof Error ? error.message : String(error); + + throw new CliUsageError("Failed to resolve Gondolin guest assets.", [ + message, + `Ensure @earendil-works/gondolin@${TESTED_GONDOLIN_VERSION} is installed.`, + "To use custom assets, set GONDOLIN_GUEST_DIR to a directory containing kernel/initramfs/rootfs assets.", + ]); } } From d5ccda84bec577252f86cfa5020bdb5ad511c0b9 Mon Sep 17 00:00:00 2001 From: Victor Garcia Date: Thu, 12 Feb 2026 10:31:10 -0500 Subject: [PATCH 7/9] Restore busybox smoke assertion in integration tests --- .../oci2gondolin.integration.test.ts | 48 +++++++++++++++++++ 1 file changed, 48 insertions(+) diff --git a/test/integration/oci2gondolin.integration.test.ts b/test/integration/oci2gondolin.integration.test.ts index 64eca2b..ad9594f 100644 --- a/test/integration/oci2gondolin.integration.test.ts +++ b/test/integration/oci2gondolin.integration.test.ts @@ -25,11 +25,13 @@ const PLATFORM = resolveIntegrationPlatform(process.env.INTEGRATION_PLATFORM ?? const ROOTFS_CHECK_PATH = process.env.INTEGRATION_ROOTFS_CHECK_PATH ?? "/etc/debian_version"; const VM_CHECK_COMMAND = process.env.INTEGRATION_VM_CHECK_COMMAND ?? "cat /etc/debian_version"; const VM_CHECK_EXPECT = process.env.INTEGRATION_VM_CHECK_EXPECT ?? "12"; +const BUSYBOX_IMAGE = process.env.INTEGRATION_BUSYBOX_IMAGE ?? "busybox:latest"; const imageSlug = IMAGE.replace(/[^a-z0-9._-]+/gi, "-").toLowerCase(); const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), `docker2vm-integration-${imageSlug}-`)); const rootfsOutDir = path.join(tempRoot, `${imageSlug}-rootfs`); const assetsOutDir = path.join(tempRoot, `${imageSlug}-assets`); +const busyboxAssetsOutDir = path.join(tempRoot, "busybox-assets-smoke"); afterAll(() => { fs.rmSync(tempRoot, { recursive: true, force: true }); @@ -162,6 +164,52 @@ describe("oci2gondolin integration", () => { } } }, 420_000); + + it("keeps the busybox VM smoke check", async () => { + requireBinary(process.arch === "arm64" ? "qemu-system-aarch64" : "qemu-system-x86_64"); + + const result = await runCommand( + "bun", + [ + "run", + "src/bin/oci2gondolin.ts", + "--image", + BUSYBOX_IMAGE, + "--platform", + PLATFORM, + "--mode", + "assets", + "--out", + busyboxAssetsOutDir, + ], + { cwd: REPO_ROOT, timeoutMs: 300_000 }, + ); + + assertSuccess(result, "oci2gondolin busybox assets conversion"); + + const originalGuestDir = process.env.GONDOLIN_GUEST_DIR; + const vmSandbox = resolveVmSandboxOptions(); + + let vm: VM | null = null; + try { + process.env.GONDOLIN_GUEST_DIR = busyboxAssetsOutDir; + vm = await VM.create({ sandbox: vmSandbox }); + + const busyboxResult = await vm.exec(["/bin/busybox", "echo", "integration-busybox-ok"]); + expect(busyboxResult.exitCode).toBe(0); + expect(busyboxResult.stdout).toContain("integration-busybox-ok"); + } finally { + await vm?.close().catch(() => { + // ignore close errors in test teardown + }); + + if (originalGuestDir === undefined) { + delete process.env.GONDOLIN_GUEST_DIR; + } else { + process.env.GONDOLIN_GUEST_DIR = originalGuestDir; + } + } + }, 420_000); }); function resolveIntegrationPlatform(raw: string): "linux/amd64" | "linux/arm64" { From 45548b0e331dc2d9b661271c820dc68e165b331c Mon Sep 17 00:00:00 2001 From: Victor Garcia Date: Thu, 12 Feb 2026 11:02:39 -0500 Subject: [PATCH 8/9] Keep busybox runtime check in matrix integration test --- .../materialize/runtime-injection.ts | 5 ++ .../oci2gondolin.integration.test.ts | 48 +------------------ 2 files changed, 7 insertions(+), 46 deletions(-) diff --git a/src/oci2gondolin/materialize/runtime-injection.ts b/src/oci2gondolin/materialize/runtime-injection.ts index fea1aac..0342e44 100644 --- a/src/oci2gondolin/materialize/runtime-injection.ts +++ b/src/oci2gondolin/materialize/runtime-injection.ts @@ -27,6 +27,11 @@ const REQUIRED_RUNTIME_FILES: RuntimeFileSpec[] = [ targetRelativePath: "bin/kmod", mode: 0o755, }, + { + sourcePathInRootfs: "/bin/busybox", + targetRelativePath: "bin/busybox", + mode: 0o755, + }, { sourcePathInRootfs: "/usr/lib/libcrypto.so.3", targetRelativePath: "usr/lib/libcrypto.so.3", diff --git a/test/integration/oci2gondolin.integration.test.ts b/test/integration/oci2gondolin.integration.test.ts index ad9594f..94af73a 100644 --- a/test/integration/oci2gondolin.integration.test.ts +++ b/test/integration/oci2gondolin.integration.test.ts @@ -25,13 +25,11 @@ const PLATFORM = resolveIntegrationPlatform(process.env.INTEGRATION_PLATFORM ?? const ROOTFS_CHECK_PATH = process.env.INTEGRATION_ROOTFS_CHECK_PATH ?? "/etc/debian_version"; const VM_CHECK_COMMAND = process.env.INTEGRATION_VM_CHECK_COMMAND ?? "cat /etc/debian_version"; const VM_CHECK_EXPECT = process.env.INTEGRATION_VM_CHECK_EXPECT ?? "12"; -const BUSYBOX_IMAGE = process.env.INTEGRATION_BUSYBOX_IMAGE ?? "busybox:latest"; const imageSlug = IMAGE.replace(/[^a-z0-9._-]+/gi, "-").toLowerCase(); const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), `docker2vm-integration-${imageSlug}-`)); const rootfsOutDir = path.join(tempRoot, `${imageSlug}-rootfs`); const assetsOutDir = path.join(tempRoot, `${imageSlug}-assets`); -const busyboxAssetsOutDir = path.join(tempRoot, "busybox-assets-smoke"); afterAll(() => { fs.rmSync(tempRoot, { recursive: true, force: true }); @@ -152,52 +150,10 @@ describe("oci2gondolin integration", () => { const execResult = await vm.exec(["/bin/sh", "-lc", VM_CHECK_COMMAND]); expect(execResult.exitCode).toBe(0); expect(execResult.stdout).toContain(VM_CHECK_EXPECT); - } finally { - await vm?.close().catch(() => { - // ignore close errors in test teardown - }); - - if (originalGuestDir === undefined) { - delete process.env.GONDOLIN_GUEST_DIR; - } else { - process.env.GONDOLIN_GUEST_DIR = originalGuestDir; - } - } - }, 420_000); - - it("keeps the busybox VM smoke check", async () => { - requireBinary(process.arch === "arm64" ? "qemu-system-aarch64" : "qemu-system-x86_64"); - - const result = await runCommand( - "bun", - [ - "run", - "src/bin/oci2gondolin.ts", - "--image", - BUSYBOX_IMAGE, - "--platform", - PLATFORM, - "--mode", - "assets", - "--out", - busyboxAssetsOutDir, - ], - { cwd: REPO_ROOT, timeoutMs: 300_000 }, - ); - - assertSuccess(result, "oci2gondolin busybox assets conversion"); - - const originalGuestDir = process.env.GONDOLIN_GUEST_DIR; - const vmSandbox = resolveVmSandboxOptions(); - - let vm: VM | null = null; - try { - process.env.GONDOLIN_GUEST_DIR = busyboxAssetsOutDir; - vm = await VM.create({ sandbox: vmSandbox }); - const busyboxResult = await vm.exec(["/bin/busybox", "echo", "integration-busybox-ok"]); + const busyboxResult = await vm.exec(["/bin/busybox", "echo", "integration-vm-ok"]); expect(busyboxResult.exitCode).toBe(0); - expect(busyboxResult.stdout).toContain("integration-busybox-ok"); + expect(busyboxResult.stdout).toContain("integration-vm-ok"); } finally { await vm?.close().catch(() => { // ignore close errors in test teardown From 7d9ecded6b69da7ac9b4c35d971e5872c1a73771 Mon Sep 17 00:00:00 2001 From: Victor Garcia Date: Thu, 12 Feb 2026 11:10:51 -0500 Subject: [PATCH 9/9] Point install guidance to official docs pages --- README.md | 30 ++++++++---------------------- docs/linux.md | 27 ++++++++++----------------- docs/macos.md | 19 +++++++++++-------- 3 files changed, 29 insertions(+), 47 deletions(-) diff --git a/README.md b/README.md index 0b9c5a4..cb96aee 100644 --- a/README.md +++ b/README.md @@ -37,30 +37,16 @@ Docker containers share the host kernel. Gondolin runs workloads inside a VM, so ## Requirements -- Bun >= 1.2 -- `e2fsprogs` (`mke2fs`, `debugfs`) -- QEMU (for runtime smoke checks) -- Docker (only required for `dockerfile2gondolin`) +- Bun >= 1.2 — https://bun.com/ +- `e2fsprogs` (`mke2fs`, `debugfs`) — https://e2fsprogs.sourceforge.net/ +- QEMU (for runtime smoke checks) — https://www.qemu.org/download/ +- Docker (only required for `dockerfile2gondolin`) — https://docs.docker.com/get-docker/ `docker2vm` uses `@earendil-works/gondolin@0.2.1` as a runtime dependency and resolves/downloads guest assets automatically during conversion. -If you also want to run generated assets with `gondolin exec`, install the CLI separately: - -```bash -bun add -g @earendil-works/gondolin@0.2.1 -``` - -macOS helpers: - -```bash -brew install e2fsprogs qemu -``` - -Ubuntu helpers: - -```bash -sudo apt-get install -y e2fsprogs qemu-system-x86 -``` +If you also want to run generated assets with `gondolin exec`, install the Gondolin CLI: +- CLI docs: https://earendil-works.github.io/gondolin/cli/ +- Package: https://www.npmjs.com/package/@earendil-works/gondolin > On macOS, `docker2vm` checks common Homebrew `e2fsprogs` locations automatically; updating `PATH` is usually optional. @@ -93,7 +79,7 @@ The CI integration matrix currently validates: - `fedora:41` - `archlinux:latest` -For each distro row, tests run a distro-specific probe command (for example `/etc/debian_version`, `/etc/fedora-release`, etc.) and also assert that probe does **not** match on the base Gondolin guest image. +For each distro row, tests run a distro-specific probe command (for example `/etc/debian_version`, `/etc/fedora-release`, etc.), assert that probe does **not** match on the base Gondolin guest image, and verify `/bin/busybox` executes inside the converted image. ### Choosing the build platform (`--platform`) diff --git a/docs/linux.md b/docs/linux.md index 2db1cd5..3ee833a 100644 --- a/docs/linux.md +++ b/docs/linux.md @@ -6,32 +6,25 @@ This guide is for running `docker2vm` on Linux hosts. ## 1) Install required tools -### Ubuntu / Debian +Install tools using their official docs/download pages: -```bash -sudo apt-get update -sudo apt-get install -y curl unzip e2fsprogs qemu-system-x86 -``` +- Bun: https://bun.com/ +- QEMU: https://www.qemu.org/download/ +- e2fsprogs: https://e2fsprogs.sourceforge.net/ -Install Bun: +If you want Dockerfile conversion (`dockerfile2gondolin`), also install Docker + Buildx: -```bash -curl -fsSL https://bun.sh/install | bash -export BUN_INSTALL="$HOME/.bun" -export PATH="$BUN_INSTALL/bin:$PATH" -``` - -If you want Dockerfile conversion (`dockerfile2gondolin`), install Docker and Buildx. +- Docker: https://docs.docker.com/get-docker/ +- Buildx: https://docs.docker.com/build/buildx/install/ ## 2) Optional: install Gondolin CLI (for running generated assets) `docker2vm` is tested with `@earendil-works/gondolin@0.2.1` and can fetch guest assets automatically during conversion. -Install the CLI globally if you want to execute generated assets via `gondolin exec`: +Use Gondolin CLI install docs: -```bash -bun add -g @earendil-works/gondolin@0.2.1 -``` +- https://earendil-works.github.io/gondolin/cli/ +- Package page: https://www.npmjs.com/package/@earendil-works/gondolin ## 3) Verify toolchain diff --git a/docs/macos.md b/docs/macos.md index 48575ec..e837011 100644 --- a/docs/macos.md +++ b/docs/macos.md @@ -6,11 +6,14 @@ This guide is for running `docker2vm` on macOS (Apple Silicon or Intel). ## 1) Install required tools -```bash -brew install bun qemu e2fsprogs -``` +Install tools using their official docs/download pages: + +- Bun: https://bun.com/ +- QEMU: https://www.qemu.org/download/ +- e2fsprogs: https://e2fsprogs.sourceforge.net/ -If you want Dockerfile conversion (`dockerfile2gondolin`), also install Docker Desktop. +If you want Dockerfile conversion (`dockerfile2gondolin`), also install Docker Desktop: +- https://docs.docker.com/desktop/setup/install/mac-install/ ## 2) Optional: add `e2fsprogs` binaries to `PATH` @@ -28,11 +31,11 @@ To persist it, add that `export PATH=...` line to your shell profile (`~/.zshrc` `docker2vm` is tested with `@earendil-works/gondolin@0.2.1` and can fetch guest assets automatically during conversion. -Install the CLI globally if you want to execute generated assets via `gondolin exec`: +Gondolin CLI docs: +- https://earendil-works.github.io/gondolin/cli/ -```bash -bun add -g @earendil-works/gondolin@0.2.1 -``` +Package page: +- https://www.npmjs.com/package/@earendil-works/gondolin ## 4) Verify toolchain