Skip to content

feat(docs): add llms.txt feed and optional description generation for better LLM routing#3629

Open
benjaminach wants to merge 48 commits intomasterfrom
feat/llms.txt
Open

feat(docs): add llms.txt feed and optional description generation for better LLM routing#3629
benjaminach wants to merge 48 commits intomasterfrom
feat/llms.txt

Conversation

@benjaminach
Copy link
Copy Markdown
Contributor

@benjaminach benjaminach commented Mar 26, 2026

Summary

This PR introduces a generated llms.txt feed for doc.scalingo.com and improves LLM routing quality by adding optional, high-signal page descriptions.

Changes

  • Added src/llms.txt (Jekyll-generated) to expose a machine-readable list of documentation pages.
  • Updated _config.yml to include llms.txt in the generated site.
  • Filtered out non-target content from the feed (e.g. changelog entries, directory/index-like pages, explicitly excluded pages).
  • Added a new Codex skill: .codex/skills/scalingo-generate-description/SKILL.md.
  • Added its OpenAI agent config: .codex/skills/scalingo-generate-description/agents/openai.yaml.
  • Added/updated description front matter on selected user-management pages.
  • Standardized description formatting with double-quoted YAML values.

Why

description fields are optional, but they significantly help LLMs understand page intent and route queries to the right documentation page when scanning llms.txt.

Impact

  • New machine-readable endpoint: /llms.txt
  • Better semantic retrieval/routing for doc-focused LLM workflows
  • No expected user-facing behavior changes in normal site navigation

Validation

  • Verified Jekyll generation includes llms.txt.
  • Checked feed formatting and URL output.
  • Checked YAML front matter validity on updated pages.

@benjaminach benjaminach marked this pull request as draft March 27, 2026 11:03
@benjaminach benjaminach changed the title add files to generates an llms.txt file feat(docs): add llms.txt feed and optional description generation for better LLM routing Mar 27, 2026
@benjaminach benjaminach marked this pull request as ready for review March 30, 2026 15:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants