Skip to content

Open-API: Replace shadow JAR with distribution plugin for REST fixture#16214

Open
kevinjqliu wants to merge 16 commits intoapache:mainfrom
kevinjqliu:kevinjqliu/open-api
Open

Open-API: Replace shadow JAR with distribution plugin for REST fixture#16214
kevinjqliu wants to merge 16 commits intoapache:mainfrom
kevinjqliu:kevinjqliu/open-api

Conversation

@kevinjqliu
Copy link
Copy Markdown
Contributor

@kevinjqliu kevinjqliu commented May 5, 2026

Changes

  • Replace shadow JAR with Gradle distribution plugin, modelled after
    kafka-connect-runtime. Uses installDist/distZip instead of shadowJar,
    producing a lib/ directory of individual JARs (66 JARs, 59MB total vs 201MB
    shadow JAR).
  • Exclude cloud-specific thin modules (iceberg-aws, iceberg-gcp, iceberg-azure,
    iceberg-bigquery) and JUnit from the distribution. Cloud bundles are copied as
    separate shadow JARs in the Dockerfile.
  • Update Dockerfile to use java -cp "lib/*" with the installDist output
    instead of java -jar with the shadow JAR. Remove EXPOSE since it is
    build-time metadata that cannot track runtime port overrides.
  • Fix REST_PORT env var — renamed to CATALOG_REST_PORT so the server reads
    it through the CATALOG_* environment mapping convention.
  • Add LICENSE, NOTICE, and runtime-deps.txt for the open-api distribution.
  • Update CI workflow to build installDist plus cloud bundle shadowJar tasks.

Motivation

The shadow JAR bundled all transitive dependencies into a single 201MB JAR that was
unintentionally published to Maven Central.
Replacing it with individual JARs in a lib/ directory:

  • Simplifies license compliance — each JAR retains its own META-INF/LICENSE
  • Reduces build artifact size (201MB → 59MB)
  • Follows the same pattern as kafka-connect-runtime
  • Allows cloud bundles to be included/excluded independently

Testing

  • ./gradlew :iceberg-open-api:installDist produces 66 JARs (no JUnit, no cloud thin modules)
  • ./gradlew :iceberg-open-api:checkRuntimeDeps passes against baseline
  • java -cp "lib/*" org.apache.iceberg.rest.RESTCatalogServer starts, /v1/config and
    /v1/namespaces respond correctly
  • Docker image builds and passes healthcheck with full REST API functional

kevinjqliu and others added 3 commits May 6, 2026 15:06
@kevinjqliu kevinjqliu force-pushed the kevinjqliu/open-api branch from 9261ef1 to f2ee703 Compare May 6, 2026 22:06
Co-authored-by: Copilot <copilot@github.com>
@kevinjqliu kevinjqliu changed the title Open-API: Slim down REST fixture shadow JAR and externalize cloud bundles Open-API: Replace shadow JAR with distribution plugin for REST fixture May 6, 2026
kevinjqliu and others added 12 commits May 6, 2026 15:40
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
@kevinjqliu kevinjqliu marked this pull request as ready for review May 7, 2026 01:49
@kevinjqliu
Copy link
Copy Markdown
Contributor Author

this is ready for first round of review.

Please take a look at build.gradle, .github/workflows/publish-iceberg-rest-fixture-docker.yml, docker/iceberg-rest-fixture/Dockerfile
and then open-api/runtime-deps.txt and runtime-deps.gradle

once those are good, please check open-api/LICENSE and open-api/NOTICE

@manuzhang
Copy link
Copy Markdown
Member

I'm wondering whether we need all those jars and bundles for docker image and integration tests in iceberg-python and iceberg-rust.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants