feat: serve pages as markdown and update llms.txt (#2249)#2289
feat: serve pages as markdown and update llms.txt (#2249)#2289
Conversation
kaankacar
commented
Mar 2, 2026
kaankacar
commented
- Installed docusaurus-markdown-source-plugin to inject "Open Markdown" dropdown on every /docs/ page linking to currentPath + .md
- Added nginx location block to intercept .md URLs, strip the extension, and proxy to developers.stellar.org with Accept: text/markdown header
- Updated static/llms.txt to document the .md URL scheme and Accept header
- Install docusaurus-markdown-source-plugin to inject "Open Markdown" dropdown on every /docs/ page linking to currentPath + .md - Add nginx location block to intercept .md URLs, strip the extension, and proxy to developers.stellar.org with Accept: text/markdown header - Update static/llms.txt to document the .md URL scheme and Accept header
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
There was a problem hiding this comment.
Pull request overview
Adds a “markdown view” pathway for Stellar developer docs so tooling/LLMs can retrieve page content as Markdown via a .md URL pattern (and documents the behavior in llms.txt).
Changes:
- Add
docusaurus-markdown-source-pluginand enable it in Docusaurus config. - Add an nginx rule to intercept
*.mdrequests and proxy to the corresponding non-.mdpath withAccept: text/markdown. - Update
static/llms.txtto describe the.mdURL scheme and header-based access.
Reviewed changes
Copilot reviewed 4 out of 5 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
package.json |
Adds docusaurus-markdown-source-plugin dependency. |
yarn.lock |
Locks the new plugin and its dependency tree. |
docusaurus.config.ts |
Enables the markdown-source plugin in the Docusaurus plugins list. |
nginx/nginx.conf |
Proxies *.md requests to the corresponding page with Accept: text/markdown. |
static/llms.txt |
Documents the Markdown access pattern for consumers. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
nginx/nginx.conf
Outdated
| location ~* ^(.+)\.md$ { | ||
| resolver 1.1.1.1 valid=300s; | ||
| proxy_pass https://developers.stellar.org$1; | ||
| proxy_set_header Accept "text/markdown"; | ||
| proxy_set_header Host "developers.stellar.org"; |
There was a problem hiding this comment.
This regex location will be considered against all requests ending in .md (including under /assets/, /img/, etc.) and can override the existing prefix locations because they aren't marked ^~. To avoid unexpected proxying of static paths, scope the match to the intended docs routes (e.g., only /docs/...\.md$) and/or mark the static prefix locations as ^~ so they always win.
There was a problem hiding this comment.
scoped the regex to /docs/ only in a follow-up commit
| add_header 'Cache-Control' 'no-store' always; | ||
| } | ||
| location ~* ^(.+)\.md$ { | ||
| resolver 1.1.1.1 valid=300s; |
There was a problem hiding this comment.
resolver 1.1.1.1 hard-codes a public DNS resolver. This can break in restricted networks and may violate infra/security expectations; prefer using the environment's DNS (or a configurable internal resolver) and consider defining it at the http {} level to keep resolver policy centralized.
| resolver 1.1.1.1 valid=300s; |
There was a problem hiding this comment.
Resolver is required here, nginx won't resolve the hostname in proxy_pass without an explicit resolver directive. Removing it would break the proxy entirely. 1.1.1.1 is intentional for this public-facing deployment is what i see. @ElliotFriend thoughts?
| Each documentation page is available as clean markdown by appending `.md` to the URL | ||
| (e.g. `https://developers.stellar.org/docs/build/smart-contracts/overview.md`). | ||
| Alternatively, send an `Accept: text/markdown` request header to any page URL to receive markdown without modifying the URL. |
There was a problem hiding this comment.
llms.txt claims that sending Accept: text/markdown to any page URL returns markdown, but the nginx config in this repo only forces that header when a .md URL is requested (and otherwise serves static HTML). Either adjust this wording to match the actual behavior, or add server/CDN logic to honor Accept: text/markdown on normal page URLs.
There was a problem hiding this comment.
The accept: text/markdown behavior is handled by Cloudflare in front of this nginx. Tt already performs the HTML-to-markdown conversion when it sees that header, for any page URL. The nginx config in this repo only needs to handle the .md URL scheme (which it does by stripping .md and setting the header before passing to Cloudflare)
|
Preview is available here: |
|
Preview is available here: |
1 similar comment
|
Preview is available here: |
|
Preview is available here: |
