Skip to content

script and data to build the combined collection of docs#72

Open
heckj wants to merge 30 commits intoswiftlang:mainfrom
heckj:build-docs
Open

script and data to build the combined collection of docs#72
heckj wants to merge 30 commits intoswiftlang:mainfrom
heckj:build-docs

Conversation

@heckj
Copy link
Copy Markdown
Member

@heckj heckj commented Mar 11, 2026

Summary

  • Internal and external data for existing Swift docs repositories in JSON
  • script that clones them, builds them individually - either through package.swift & the docc plugin or directly with DocC in the case of "bare catalogs"

Related Issue

Closes: #24

Additional Notes

  • combines some of the repositories together at the tail end of the process
  • uses DOCC_JSON_PRETTYPRINT=YES to make the output a bit more deterministic (diffs are easier if we need to debug or merge in changes in source control)
  • uses some custom build flags:
    --experimental-enable-custom-templates w/ header & footer from common/
    --enable-mentioned-in
    --enable-experimental-external-link-support (completely arbitrary- can drop)
  • no current hosting base path is set, so we'll need/want to tweak that before we're ready to publish

Script CLI:

Usage: build-docs.sh [OPTIONS]

Build and export DocC documentation archives from multiple sources.

Options:
  --output-dir <path>   Where to export built archives (default: .build-output/)
  --workspace <path>    Where to clone external repos (default: .build-workspace/)
  --clean               Remove workspace and re-clone everything
  --only <id>           Build only a specific source by id
  -h, --help            Show this help message

Most of the time in this script is cloning and (for the docs that need it) resolving the packages before building the docs (SwiftPM & swiftly). Once it's all cloned and resolved, doing a rebuild of the content is pretty quick:

time scripts/build-docs.sh:

./scripts/build-docs.sh  31.04s user 16.22s system 127% cpu 37.168 total

Once built, see the result locally:

python3 -m http.server -d .build-output/Combined.doccarchive
Screenshot 2026-03-11 at 11 09 43 AM

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@heckj heckj requested a review from shahmishal March 11, 2026 18:10
@heckj heckj self-assigned this Mar 11, 2026
@heckj heckj added the task Internal work or maintenance task label Mar 11, 2026
@heckj heckj requested review from a team as code owners March 12, 2026 00:09
@heckj
Copy link
Copy Markdown
Member Author

heckj commented Mar 26, 2026

The swift-book repo, in particular, needs some specific prep work for it's catalog prior to running docc convert on it - so I've opened swiftlang/swift-book#454 to support being able to specify a script to run as part of this updated scripting.

heckj added a commit to swiftlang/swift-book that referenced this pull request Apr 3, 2026
This lets us run this setup script with an external repository's
(swiftlang/docs) build script to do the expected prep work for the DocC
catalog in order to combine it with other DocC catalogs for a combined
archive of multiple doc sets together.

work needed to support swiftlang/docs#72,
ultimately to support swiftlang/docs#24
heckj added 5 commits April 3, 2026 11:19
Add Apache 2.0 license headers to all _empty.swift files and
build-docs.sh. Add .licenseignore to exclude files that cannot
have license headers (Package.swift, .gitkeep, .html, README).
Configure license_header_check_project_name in CI workflow.
Replace build-docs.sh with build_docs.py, removing the --branch CLI
option, fallback_branch field, and latest-release auto-detection.
Each git source now has an explicit "ref" field in sources.json,
making builds fully deterministic. The combined archive is named
<version>.doccarchive (e.g. main.doccarchive) and individual
archives are intermediates cleaned up after merge.
Runs the build_docs.py script on PRs and pushes to main using the
swift:6.2-noble container, caches .workspace/ between runs, and
uploads the merged .doccarchive as a CI artifact.
github-actions bot added a commit to ekassos/swift-book-archive that referenced this pull request Apr 4, 2026
extracts the grammar generation step into it's own script (#454)

This lets us run this setup script with an external repository's
(swiftlang/docs) build script to do the expected prep work for the DocC
catalog in order to combine it with other DocC catalogs for a combined
archive of multiple doc sets together.

work needed to support swiftlang/docs#72,
ultimately to support swiftlang/docs#24

Upstream commit: swiftlang/swift-book@2306b38
heckj added 7 commits April 5, 2026 11:08
The swift:6.2-noble image does not include python3, causing the build
step to fail with exit code 127.
The swift:6.2-noble release image fails to build swift-testing docs.
The swift-testing CI itself uses nightly-main images, so align with that.
The shared swiftlang/github-workflows default to jammy (Ubuntu 22.04).
nightly-main-noble crashed in libdispatch; 6.2-noble failed on
swift-testing. Try the release toolchain on jammy.
swift-testing requires Swift tools version 6.3.0, so 6.2.x images
can't build it. Use nightly-main on jammy (Ubuntu 22.04) which is
the platform swift-testing CI actually runs on.
Use --output-dir to write to a known path under GITHUB_WORKSPACE,
and add a debug ls step to verify output before upload.
The swift-book preflight (bin/generate-grammar) fails if its output
file already exists from a previous cached run. Adding git clean -fdx
ensures untracked files from prior builds are removed.
DocC archives contain colons in filenames (e.g. when(configuration:).json)
which upload-artifact rejects. Package as a tarball instead.

Also add explanatory comment on the git clean step.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

task Internal work or maintenance task

Projects

None yet

Development

Successfully merging this pull request may close these issues.

set up archive publishing

1 participant