WIP Migrate zig compiler #413

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft

lukewilliamboswell wants to merge 36 commits into main from migrate-zig-compiler

.github/workflows/ci.yml

-Original file line number
+Diff line change
@@ -1,3 +1,5 @@
+    name: CI
     on:
       pull_request:
       workflow_dispatch:
@@ Expand All / @@ -8,71 +10,44 @@ concurrency: @@
       cancel-in-progress: true
     # Do not add permissions here! Configure them at the job level!
-    permissions: {}
+    permissions:
+      contents: read
     jobs:
-      build-and-test-native:
+      build-and-test:
+        name: test (${{ matrix.os }})
         runs-on: ${{ matrix.os }}
         strategy:
+          fail-fast: false
           matrix:
-            os: [ubuntu-22.04, ubuntu-24.04, ubuntu-24.04-arm, macos-15]
-        steps:
-          - uses: actions/checkout@v4
+            os:
+              - macos-15        # Apple Silicon
+              - macos-15-intel  # Intel Mac
+              - ubuntu-22.04    # Linux x86_64
+              - ubuntu-24.04-arm # Linux ARM64
+        defaults:
+          run:
+            shell: bash
-          - name: Does init() in platform/src/lib.rs contain all roc_fx functions? (Imperfect check)
-            run: cat platform/src/lib.rs | grep -oP 'roc_fx_[^(\s]*' | sort | uniq -u | grep -q . && exit 1 || exit 0
+        steps:
+          - name: Checkout
+            uses: actions/checkout@v4
-          - uses: roc-lang/setup-roc@39c354a6a838a0089eea9068a0414f49b62c5c08
+          - name: Install Zig
+            uses: mlugg/setup-zig@8d6198c65fb0feaa111df26e6b467fea8345e46f # ratchet:mlugg/setup-zig@v2.0.5
             with:
-              # Note: nightly hashes are not verified because they are updated regularly.
-              version: nightly
+              version: 0.15.2
-          - run: roc version
-          - name: Install dependencies (Ubuntu)
+          - name: Install expect (Ubuntu)
             if: startsWith(matrix.os, 'ubuntu-')
-            run: |
-              sudo apt install -y expect ncat ripgrep
+            run: sudo apt-get install -y expect
-          - name: Install dependencies (macOS)
+          - name: Install expect (macOS)
             if: startsWith(matrix.os, 'macos-')
-            run: |
-              brew install expect # expect for testing
-              brew install nmap  # includes ncat, for tcp-client example
-              brew install ripgrep # ripgrep for ci/check_all_exposed_funs_tested.roc
-          - run: expect -v
+            run: brew install expect
           - name: Run all tests
-            run: ROC=roc EXAMPLES_DIR=./examples/ ./ci/all_tests.sh
-          - name: Install dependencies for musl build
-            if: startsWith(matrix.os, 'ubuntu-')
-            run: |
-              sudo apt-get install -y musl-tools
-              if [[ "${{ matrix.os }}" == *"-arm" ]]; then
-                # TODO re-enable once TODO below is done: rustup target add aarch64-unknown-linux-musl
-                echo "no-op"
-              else
-                rustup target add x86_64-unknown-linux-musl
-              fi
-          - name: Test building with musl target
-            if: startsWith(matrix.os, 'ubuntu-')
-            env:
-              ROC: roc
-            run: |
-              if [[ "${{ matrix.os}}" == *"-arm" ]]; then
-                # TODO debug this: CARGO_BUILD_TARGET=aarch64-unknown-linux-musl $ROC build.roc
-                echo "no-op"
-              else
-                CARGO_BUILD_TARGET=x86_64-unknown-linux-musl $ROC build.roc
-              fi
-          - name: Test using musl build
-            if: startsWith(matrix.os, 'ubuntu-')
-            run: |
-              # TODO remove `if` when above TODOs are done
-              if [[ "${{ matrix.os }}" != *"-arm" ]]; then
-                NO_BUILD=1 IS_MUSL=1 ROC=roc EXAMPLES_DIR=./examples/ ./ci/all_tests.sh
-              fi
+            uses: roc-lang/roc/.github/actions/flaky-retry@main
+            with:
+              command: ./ci/all_tests.sh
+              error_string_contains: "error: unable"

.github/workflows/ci_nix.yml

This file was deleted.

.github/workflows/release.yml

-Original file line number
+Diff line change
@@ -0,0 +1,226 @@
+    name: Release
+    on:
+      workflow_dispatch:
+        inputs:
+          release_tag:
+            description: 'Release tag (e.g., v0.1.0)'
+            required: true
+            type: string
+      pull_request:
+    # Ensure only one release workflow runs at a time
+    concurrency:
+      group: ${{ github.workflow }}-${{ github.ref }}
+      cancel-in-progress: true
+    # Do not add permissions here! Configure them at the job level!
+    permissions:
+      contents: read
+    env:
+      ZIG_VERSION: 0.15.2
+    jobs:
+      build-and-bundle:
+        name: Build and Bundle Platform
+        runs-on: macos-15  # macOS can cross-compile to all targets
+        outputs:
+          bundle_filename: ${{ steps.bundle.outputs.bundle_filename }}
+        defaults:
+          run:
+            shell: bash
+        steps:
+          - name: Checkout
+            uses: actions/checkout@v4
+          - name: Install Zig
+            uses: mlugg/setup-zig@8d6198c65fb0feaa111df26e6b467fea8345e46f # ratchet:mlugg/setup-zig@v2.0.5
+            with:
+              version: ${{ env.ZIG_VERSION }}
+          - name: Install Rust cross-compilation targets
+            run: |
+              rustup target add x86_64-apple-darwin
+              rustup target add aarch64-apple-darwin
+              rustup target add x86_64-unknown-linux-musl
+              rustup target add aarch64-unknown-linux-musl
+          - name: Download Roc nightly
+            run: |
+              source ci/get_roc_nightly_url.sh
+              echo "Downloading $ROC_NIGHTLY_URL"
+              curl -fOL "$ROC_NIGHTLY_URL"
+              tar -xzf "$ROC_NIGHTLY_ARCHIVE"
+              ROC_DIR=$(ls -d roc_nightly-*/ | head -1)
+              echo "$(pwd)/${ROC_DIR}" >> "$GITHUB_PATH"
+          - name: Build platform for all targets
+            run: ./build.sh --all
+          - name: Bundle platform
+            id: bundle
+            run: |
+              # Run bundle.sh and capture output (disable set -e to ensure output is printed)
+              set +e
+              BUNDLE_OUTPUT=$(./bundle.sh 2>&1)
+              EXIT_CODE=$?
+              set -e
+              echo "$BUNDLE_OUTPUT"
+              if [ $EXIT_CODE -ne 0 ]; then
+                echo "bundle.sh failed with exit code $EXIT_CODE"
+                exit $EXIT_CODE
+              fi
+              # Extract the tar.zst filename from "Created: /path/to/HASH.tar.zst" line
+              BUNDLE_PATH=$(echo "$BUNDLE_OUTPUT" | grep "^Created:" | awk '{print $2}')
+              BUNDLE_FILENAME=$(basename "$BUNDLE_PATH")
+              if [ -z "$BUNDLE_FILENAME" ]; then
+                echo "Error: Could not extract bundle filename from output"
+                echo "Bundle output was:"
+                echo "$BUNDLE_OUTPUT"
+                exit 1
+              fi
+              echo "Bundle created: $BUNDLE_FILENAME"
+              echo "bundle_filename=$BUNDLE_FILENAME" >> "$GITHUB_OUTPUT"
+          - name: Upload bundle artifact
+            uses: actions/upload-artifact@v4
+            with:
+              name: platform-bundle
+              path: ${{ steps.bundle.outputs.bundle_filename }}
+              retention-days: 30
+      test-bundle:
+        name: Test Bundle (${{ matrix.os }})
+        needs: build-and-bundle
+        strategy:
+          fail-fast: false
+          matrix:
+            os:
+              - macos-15          # Apple Silicon
+              - macos-15-intel    # Intel Mac
+              - ubuntu-22.04      # Linux x86_64
+              - ubuntu-24.04-arm  # Linux ARM64
+        runs-on: ${{ matrix.os }}
+        defaults:
+          run:
+            shell: bash
+        steps:
+          - name: Checkout
+            uses: actions/checkout@v4
+          - name: Install Zig
+            uses: mlugg/setup-zig@8d6198c65fb0feaa111df26e6b467fea8345e46f # ratchet:mlugg/setup-zig@v2.0.5
+            with:
+              version: ${{ env.ZIG_VERSION }}
+          - name: Install expect (Ubuntu)
+            if: startsWith(matrix.os, 'ubuntu-')
+            run: sudo apt-get install -y expect
+          - name: Install expect (macOS)
+            if: startsWith(matrix.os, 'macos-')
+            run: brew install expect
+          - name: Download Roc nightly
+            run: |
+              source ci/get_roc_nightly_url.sh
+              echo "Downloading $ROC_NIGHTLY_URL"
+              curl -fOL "$ROC_NIGHTLY_URL"
+              tar -xzf "$ROC_NIGHTLY_ARCHIVE"
+              ROC_DIR=$(ls -d roc_nightly-*/ | head -1)
+              echo "$(pwd)/${ROC_DIR}" >> "$GITHUB_PATH"
+          - name: Download bundle artifact
+            uses: actions/download-artifact@v4
+            with:
+              name: platform-bundle
+          - name: Start HTTP server for bundle
+            run: |
+              # Start Python HTTP server in background on port 8000
+              python3 -m http.server 8000 &
+              HTTP_SERVER_PID=$!
+              echo "HTTP_SERVER_PID=$HTTP_SERVER_PID" >> "$GITHUB_ENV"
+              # Wait for server to start
+              sleep 5
+              # Verify server is running and bundle is accessible
+              if ! curl -f -I "http://localhost:8000/${{ needs.build-and-bundle.outputs.bundle_filename }}" > /dev/null 2>&1; then
+                echo "Error: HTTP server not accessible or bundle not found"
+                kill $HTTP_SERVER_PID || true
+                exit 1
+              fi
+              echo "HTTP server started on port 8000 (PID: $HTTP_SERVER_PID)"
+              echo "Bundle accessible at: http://localhost:8000/${{ needs.build-and-bundle.outputs.bundle_filename }}"
+          - name: Modify examples to use bundled platform
+            run: |
+              # Replace local platform path with HTTP URL in all examples
+              for example in examples/*.roc; do
+                sed -i.bak "s|platform \"../platform/main.roc\"|platform \"http://localhost:8000/${{ needs.build-and-bundle.outputs.bundle_filename }}\"|g" "$example"
+              done
+              # Show diff for verification
+              echo "Modified examples:"
+              git diff examples/ | head -50 || true
+          # Skip native build since we're using the bundle
+          # The test script will detect all targets exist and use bundle testing
+          # Run all tests against the bundled platform
+          - name: Run tests with bundled platform
+            uses: roc-lang/roc/.github/actions/flaky-retry@main
+            with:
+              command: bash ci/all_tests.sh
+              error_string_contains: "error: unable"
+            env:
+              NO_BUILD: "1"
+      create-release:
+        name: Create GitHub Release
+        needs: [build-and-bundle, test-bundle]
+        runs-on: ubuntu-24.04
+        # Only run on manual trigger
+        if: github.event_name == 'workflow_dispatch'
+        permissions:
+          contents: write
+        steps:
+          - name: Checkout
+            uses: actions/checkout@v4
+          - name: Download bundle artifact
+            uses: actions/download-artifact@v4
+            with:
+              name: platform-bundle
+          - name: Create GitHub Release
+            env:
+              GH_TOKEN: ${{ github.token }}
+            run: |
+              BUNDLE_FILE=$(ls *.tar.zst | head -1)
+              if [ -z "$BUNDLE_FILE" ]; then
+                echo "Error: No bundle file found"
+                exit 1
+              fi
+              source ci/get_roc_nightly_url.sh
+              ROC_COMMIT="$ROC_NIGHTLY_COMMIT"
+              # Create release with auto-generated notes
+              gh release create "${{ github.event.inputs.release_tag }}" \
+                "$BUNDLE_FILE" \
+                --title "${{ github.event.inputs.release_tag }}" \
+                --generate-notes \
+                --notes "Platform bundle built with Rust (stable) and Roc $ROC_COMMIT"
+              echo "Release created: ${{ github.event.inputs.release_tag }}"
+              echo "Bundle uploaded: $BUNDLE_FILE"

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP Migrate zig compiler #413

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

WIP Migrate zig compiler #413

Are you sure you want to change the base?

Uh oh!

WIP Migrate zig compiler #413

Uh oh!

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!