Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 41 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,37 @@ jobs:
integration-tests:
runs-on: ubuntu-latest
needs: build-and-test
timeout-minutes: 40
timeout-minutes: 50
strategy:
fail-fast: false
matrix:
include:
- distro: alpine
image: alpine:3.20
probe_path: /etc/alpine-release
probe_command: cat /etc/alpine-release
probe_expect: "3.20"
- distro: debian
image: debian:bookworm-slim
probe_path: /etc/debian_version
probe_command: cat /etc/debian_version
probe_expect: "12"
- distro: ubuntu
image: ubuntu:24.04
probe_path: /etc/os-release
probe_command: . /etc/os-release; echo "$ID:$VERSION_ID"
probe_expect: ubuntu:24.04
- distro: fedora
image: fedora:41
probe_path: /etc/fedora-release
probe_command: cat /etc/fedora-release
probe_expect: "Fedora release 41"
- distro: archlinux
image: archlinux:latest
probe_path: /etc/os-release
probe_command: . /etc/os-release; echo "$ID"
probe_expect: arch
name: integration-tests (${{ matrix.distro }})

steps:
- name: Checkout
Expand All @@ -52,10 +82,16 @@ jobs:
sudo apt-get update
sudo apt-get install -y qemu-system-x86 e2fsprogs

- name: Prime Gondolin guest assets
run: bun --eval 'import { ensureGuestAssets } from "@earendil-works/gondolin"; await ensureGuestAssets();'

- name: Run integration tests
env:
INTEGRATION_PLATFORM: linux/amd64
INTEGRATION_IMAGE: busybox:latest
INTEGRATION_IMAGE: ${{ matrix.image }}
INTEGRATION_ROOTFS_CHECK_PATH: ${{ matrix.probe_path }}
INTEGRATION_VM_CHECK_COMMAND: ${{ matrix.probe_command }}
INTEGRATION_VM_CHECK_EXPECT: ${{ matrix.probe_expect }}
run: bun run test:integration

e2e-smoke:
Expand All @@ -80,6 +116,9 @@ jobs:
sudo apt-get update
sudo apt-get install -y qemu-system-x86 e2fsprogs

- name: Prime Gondolin guest assets
run: bun --eval 'import { ensureGuestAssets } from "@earendil-works/gondolin"; await ensureGuestAssets();'

- name: Run end-to-end smoke test
env:
PLATFORM: linux/amd64
Expand Down
65 changes: 44 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# docker2vm

`docker2vm` converts OCI container images (or Dockerfiles via BuildKit) into VM-compatible outputs. Today, the runtime materialization target is Gondolin.
`docker2vm` converts OCI container images (or Dockerfiles via BuildKit) into VM-compatible outputs. Today, the runtime materialization target is [Gondolin](https://github.com/earendil-works/gondolin).

It follows an OCI-first flow inspired by "Docker without Docker":
It follows an OCI-first flow inspired by ["Docker without Docker"](https://fly.io/blog/docker-without-docker/):

- resolve/pull an OCI image
- apply layers to a root filesystem
Expand All @@ -16,6 +16,7 @@ Docker containers share the host kernel. Gondolin runs workloads inside a VM, so

## Current features

- pinned Gondolin runtime dependency (`@earendil-works/gondolin@0.2.1`) for guest-asset retrieval
- `oci2gondolin` core converter
- input: `--image`, `--oci-layout`, `--oci-tar` (exactly one)
- platform: `linux/amd64`, `linux/arm64`
Expand All @@ -36,28 +37,23 @@ Docker containers share the host kernel. Gondolin runs workloads inside a VM, so

## Requirements

- Bun >= 1.2
- `e2fsprogs` (`mke2fs`, `debugfs`)
- QEMU (for runtime smoke checks via `gondolin exec`)
- Docker (only required for `dockerfile2gondolin`)
- Bun >= 1.2 — https://bun.com/
- `e2fsprogs` (`mke2fs`, `debugfs`) — https://e2fsprogs.sourceforge.net/
- QEMU (for runtime smoke checks) — https://www.qemu.org/download/
- Docker (only required for `dockerfile2gondolin`) — https://docs.docker.com/get-docker/

macOS helpers:
`docker2vm` uses `@earendil-works/gondolin@0.2.1` as a runtime dependency and resolves/downloads guest assets automatically during conversion.

```bash
brew install e2fsprogs qemu
```
If you also want to run generated assets with `gondolin exec`, install the Gondolin CLI:
- CLI docs: https://earendil-works.github.io/gondolin/cli/
- Package: https://www.npmjs.com/package/@earendil-works/gondolin

Ubuntu helpers:

```bash
sudo apt-get install -y e2fsprogs qemu-system-x86
```
> On macOS, `docker2vm` checks common Homebrew `e2fsprogs` locations automatically; updating `PATH` is usually optional.

## Install
## Platform setup guides

```bash
bun install
```
- [macOS guide](./docs/macos.md)
- [Linux guide](./docs/linux.md)

## Quickstart

Expand All @@ -75,6 +71,32 @@ bun run build
bun run test:integration
```

The CI integration matrix currently validates:

- `alpine:3.20`
- `debian:bookworm-slim`
- `ubuntu:24.04`
- `fedora:41`
- `archlinux:latest`

For each distro row, tests run a distro-specific probe command (for example `/etc/debian_version`, `/etc/fedora-release`, etc.), assert that probe does **not** match on the base Gondolin guest image, and verify `/bin/busybox` executes inside the converted image.

### Choosing the build platform (`--platform`)

`--platform` selects which OCI image variant to convert, and should match the architecture you plan to run in Gondolin. This applies to both `oci2gondolin` and `dockerfile2gondolin`.

- Apple Silicon / arm64 Linux hosts: `linux/arm64`
- Intel / amd64 hosts: `linux/amd64`
- If omitted, both commands default from host arch (`x64 -> linux/amd64`, `arm64 -> linux/arm64`).
- You can always override manually with `--platform linux/amd64` or `--platform linux/arm64`.

For helper scripts:

- `e2e:smoke` uses `PLATFORM`
- integration tests use `INTEGRATION_PLATFORM`

Cross-arch builds are possible at image-selection time, but for reliable runtime execution you should use a platform that matches the runtime guest architecture.

### 2) Convert image -> assets

```bash
Expand All @@ -88,7 +110,7 @@ bun run oci2gondolin -- \
### 3) Run with Gondolin

```bash
GONDOLIN_GUEST_DIR=./out/busybox-assets bunx gondolin exec -- /bin/busybox echo hello
GONDOLIN_GUEST_DIR=./out/busybox-assets gondolin exec -- /bin/busybox echo hello
```

## Dockerfile flow
Expand All @@ -113,7 +135,7 @@ bun run dockerfile2gondolin -- \
Then run:

```bash
GONDOLIN_GUEST_DIR=./out/demo-assets bunx gondolin exec -- /usr/games/cowsay "hello"
GONDOLIN_GUEST_DIR=./out/demo-assets gondolin exec -- /usr/games/cowsay "hello"
```

## End-to-end smoke test
Expand Down Expand Up @@ -176,4 +198,5 @@ dockerfile2gondolin --file PATH --context PATH --out PATH [options]
## Repo notes

- This repo is standalone; Gondolin core is not modified.
- Gondolin upstream repository: https://github.com/earendil-works/gondolin
- `out/` is generated output and ignored by git.
2 changes: 1 addition & 1 deletion bun.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

95 changes: 95 additions & 0 deletions docs/linux.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# Linux setup guide

This guide is for running `docker2vm` on Linux hosts.

`docker2vm` includes a pinned Gondolin runtime dependency (`@earendil-works/gondolin@0.2.1`) to resolve guest assets during conversion.

## 1) Install required tools

Install tools using their official docs/download pages:

- Bun: https://bun.com/
- QEMU: https://www.qemu.org/download/
- e2fsprogs: https://e2fsprogs.sourceforge.net/

If you want Dockerfile conversion (`dockerfile2gondolin`), also install Docker + Buildx:

- Docker: https://docs.docker.com/get-docker/
- Buildx: https://docs.docker.com/build/buildx/install/

## 2) Optional: install Gondolin CLI (for running generated assets)

`docker2vm` is tested with `@earendil-works/gondolin@0.2.1` and can fetch guest assets automatically during conversion.

Use Gondolin CLI install docs:

- https://earendil-works.github.io/gondolin/cli/
- Package page: https://www.npmjs.com/package/@earendil-works/gondolin

## 3) Verify toolchain

```bash
bun --version
qemu-system-x86_64 --version
mke2fs -V
debugfs -V
gondolin --help >/dev/null
```

## 4) Validate from source checkout

```bash
bun run test
bun run typecheck
bun run build
```

## 5) Choose the build platform

Use a platform that matches the architecture you will run in Gondolin.

- `uname -m` => `x86_64` or `amd64`: use `linux/amd64`
- `uname -m` => `aarch64` or `arm64`: use `linux/arm64`

`oci2gondolin` defaults automatically from host arch if `--platform` is omitted, but passing it explicitly is recommended.

Example:

```bash
bun run oci2gondolin -- \
--image busybox:latest \
--platform linux/amd64 \
--mode assets \
--out ./out/busybox-assets
```

## 6) Run integration + smoke checks

amd64 host:

```bash
INTEGRATION_PLATFORM=linux/amd64 bun run test:integration
PLATFORM=linux/amd64 bun run e2e:smoke
```

arm64 host:

```bash
INTEGRATION_PLATFORM=linux/arm64 bun run test:integration
PLATFORM=linux/arm64 bun run e2e:smoke
```

## Notes on virtualization performance

- If `/dev/kvm` is available, QEMU can use hardware acceleration.
- If `/dev/kvm` is unavailable (common in CI), the project falls back to TCG emulation (slower but functional).

## Troubleshooting

### `sandbox_stopped` / VM exits quickly

Confirm QEMU is installed and that you are using a platform that matches your host and assets.

### `mke2fs` or `debugfs` missing

Reinstall `e2fsprogs` and verify the commands are available in your shell.
103 changes: 103 additions & 0 deletions docs/macos.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# macOS setup guide

This guide is for running `docker2vm` on macOS (Apple Silicon or Intel).

`docker2vm` includes a pinned Gondolin runtime dependency (`@earendil-works/gondolin@0.2.1`) to resolve guest assets during conversion.

## 1) Install required tools

Install tools using their official docs/download pages:

- Bun: https://bun.com/
- QEMU: https://www.qemu.org/download/
- e2fsprogs: https://e2fsprogs.sourceforge.net/

If you want Dockerfile conversion (`dockerfile2gondolin`), also install Docker Desktop:
- https://docs.docker.com/desktop/setup/install/mac-install/

## 2) Optional: add `e2fsprogs` binaries to `PATH`

With Homebrew, `e2fsprogs` is often keg-only. `docker2vm` checks common Homebrew locations automatically, so a PATH change is usually **not required** for normal usage.

If you want to run `mke2fs` / `debugfs` manually in your shell:

```bash
export PATH="$(brew --prefix e2fsprogs)/sbin:$PATH"
```

To persist it, add that `export PATH=...` line to your shell profile (`~/.zshrc`, `~/.bashrc`, `~/.profile`, etc.).

## 3) Optional: install Gondolin CLI (for running generated assets)

`docker2vm` is tested with `@earendil-works/gondolin@0.2.1` and can fetch guest assets automatically during conversion.

Gondolin CLI docs:
- https://earendil-works.github.io/gondolin/cli/

Package page:
- https://www.npmjs.com/package/@earendil-works/gondolin

## 4) Verify toolchain

```bash
bun --version
qemu-system-aarch64 --version || qemu-system-x86_64 --version
"$(brew --prefix e2fsprogs)/sbin/mke2fs" -V
"$(brew --prefix e2fsprogs)/sbin/debugfs" -V
gondolin --help >/dev/null
```

## 5) Validate from source checkout

```bash
bun run test
bun run typecheck
bun run build
```

## 6) Choose the build platform

Use a platform that matches the architecture you will run in Gondolin.

- Apple Silicon (`uname -m` => `arm64`): use `linux/arm64`
- Intel Mac (`uname -m` => `x86_64`): use `linux/amd64`

`oci2gondolin` defaults automatically from host arch if `--platform` is omitted, but passing it explicitly is recommended.

Example:

```bash
bun run oci2gondolin -- \
--image busybox:latest \
--platform linux/arm64 \
--mode assets \
--out ./out/busybox-assets
```

## 7) Run integration + smoke checks

Apple Silicon:

```bash
INTEGRATION_PLATFORM=linux/arm64 bun run test:integration
PLATFORM=linux/arm64 bun run e2e:smoke
```

Intel Mac:

```bash
INTEGRATION_PLATFORM=linux/amd64 bun run test:integration
PLATFORM=linux/amd64 bun run e2e:smoke
```

## Troubleshooting

### `mke2fs` / `debugfs` not found

`docker2vm` should find common Homebrew locations automatically. If your install uses a custom prefix, either add it to `PATH` or point `GONDOLIN_GUEST_DIR` to prepared assets and verify `e2fsprogs` binaries are installed.

### Case-sensitive filename conflicts during conversion

Some images include paths that conflict on case-insensitive filesystems.

Use a case-sensitive APFS location for temporary work/output (for example, a case-sensitive volume) and retry.
Loading