Skip to content

Commit 336749e

Browse files
MasuRiiMasuRii
committed
docs: add documentation stack research notes
Add research notes documenting the evaluation of documentation frameworks that led to the Astro Starlight selection. - Evaluation criteria for docs-as-code solutions - Comparison of VitePress, Docusaurus, and Starlight - Decision rationale for Astro Starlight
1 parent 8c22d77 commit 336749e

File tree

1 file changed

+395
-0
lines changed

1 file changed

+395
-0
lines changed

‎docs/research/2026-documentation-stacks-research-2026-03-20.md‎

Lines changed: 395 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,395 @@
1+
# Research Report: 2026 Documentation Stacks and AI-Agentic Documentation Patterns
2+
3+
> Generated: 2026-03-20 | Researcher Agent | Task: research2026
4+
5+
## Executive Summary
6+
7+
This research evaluates modern documentation stacks for ReCursor's plain-HTML docs-site migration. The current docs-site uses vanilla HTML/CSS/JS with client-side rendering, which lacks modern features like search, versioning, and AI-friendly structured content.
8+
9+
**Key Finding**: In 2026, documentation has evolved beyond static sites to "agentic documentation" - structured content optimized for both human readers and AI/LLM consumption. The Model Context Protocol (MCP) and `llms.txt` standard are emerging as critical patterns for AI-friendly documentation.
10+
11+
**Primary Recommendation**: **Astro Starlight** for immediate migration, with a future path to **Fumadocs** (Next.js-based) when ReCursor needs deeper React ecosystem integration or API documentation features.
12+
13+
---
14+
15+
## Source Validation
16+
17+
| Source | Tier | Date | Relevance |
18+
|--------|------|------|-----------|
19+
| Naturaily SSG Comparison 2026 | 1 | 2026-01 | Framework comparison |
20+
| Docusaurus Official Docs | 1 | Current | React docs framework |
21+
| Fumadocs Official | 1 | Current | Next.js docs framework |
22+
| DevTools Guide: Docusaurus vs Starlight vs Mintlify | 2 | 2025-12 | Feature comparison |
23+
| PrimeDev.Tools 2026 Workflow | 2 | 2026-01 | Cost/feature analysis |
24+
| MCP Comprehensive Guide (Dysnix) | 1 | 2025 | AI protocol patterns |
25+
| Red Hat MCP Article | 1 | 2026-01 | Enterprise MCP adoption |
26+
| Cisco MCP/A2A Article | 1 | 2025 | Agent protocol comparison |
27+
| LLM-Friendly Documentation Guide (Aronhack) | 2 | 2025 | AI-readable docs |
28+
| GitBook LLM Optimization | 1 | Current | Platform LLM features |
29+
| Vuetify0 AI Tools | 2 | 2026-02 | `llms.txt` implementation |
30+
| Addy Osmani LLM Workflow 2026 | 2 | 2026-01 | Industry best practices |
31+
32+
---
33+
34+
## Part 1: 2026 Documentation Stack Comparison
35+
36+
### 1.1 Self-Hosted Frameworks
37+
38+
| Framework | Runtime | Bundle Size | Build Time | Best For | Learning Curve |
39+
|-----------|---------|-------------|------------|----------|----------------|
40+
| **Astro Starlight** | Node.js | ~50KB | ~4s (100 pages) | Performance-first docs | Low-Medium |
41+
| **Docusaurus** | React/Node | ~200KB | ~15s (100 pages) | Feature-rich, versioning | Medium |
42+
| **Fumadocs** | Next.js/React | ~180KB | Varies | Next.js ecosystem | Medium |
43+
| **VitePress** | Vue/Node | ~80KB | Fast | Vue ecosystem | Medium |
44+
| **Nextra** | Next.js/React | ~150KB | Fast | Next.js docs | Medium |
45+
| **MkDocs + Material** | Python | Minimal | Very fast | Python projects | Low |
46+
47+
*Sources: DevTools Guide [^1], PrimeDev.Tools [^2], Naturaily [^3]*
48+
49+
### 1.2 Managed/Platform Options
50+
51+
| Platform | Cost | Self-Host | Best For | AI Features |
52+
|----------|------|-----------|----------|-------------|
53+
| **Mintlify** | Free → $99+/mo | No | API-first, beautiful | Built-in AI assistant |
54+
| **GitBook** | Free tier → Paid | No | Collaboration | Auto `llms.txt`, MCP |
55+
| **ReadMe** | $99+/mo | No | API documentation | Limited |
56+
| **CloudCannon** | $55+/mo | Yes | Git-based CMS | Limited |
57+
58+
*Source: PrimeDev.Tools [^2], ToolQuestor [^4]*
59+
60+
### 1.3 Detailed Framework Analysis
61+
62+
#### Astro Starlight (Recommended for ReCursor)
63+
64+
**Strengths:**
65+
- Minimal JavaScript by default (~50KB initial bundle)
66+
- Fastest build times among React-based options
67+
- Built-in search via Pagefind (no external service needed)
68+
- Type-safe frontmatter via Astro content collections
69+
- Supports React/Vue/Svelte components via islands architecture
70+
- Excellent Core Web Vitals scores
71+
- Native MDX support
72+
73+
**Weaknesses:**
74+
- Versioning requires plugin (not built-in)
75+
- Smaller ecosystem than Docusaurus
76+
- Less customizable theming out of the box
77+
78+
**Verdict for ReCursor:** Ideal match. ReCursor is docs-first with Flutter/TypeScript stack - no strong React coupling. Starlight's performance aligns with mobile-first philosophy.
79+
80+
#### Docusaurus (Established Alternative)
81+
82+
**Strengths:**
83+
- Mature ecosystem with extensive plugins
84+
- Built-in versioning and i18n
85+
- Algolia DocSearch integration
86+
- Strong React community support
87+
- Purpose-built for documentation
88+
89+
**Weaknesses:**
90+
- Heavy bundle size (~200KB)
91+
- Slower build times
92+
- Full SPA architecture (slower page loads)
93+
- Overkill for smaller doc sets
94+
95+
**Verdict for ReCursor:** Over-engineered for current needs. Better suited when ReCursor needs complex versioning or extensive custom React components.
96+
97+
#### Fumadocs (Next.js-Native)
98+
99+
**Strengths:**
100+
- Deep Next.js App Router integration
101+
- React Server Components support
102+
- Highly composable (Content → Core → UI)
103+
- Framework agnostic (works with TanStack Start, Waku, React Router)
104+
- Modern architecture for 2026
105+
- Used by Vercel, Unkey, Orama
106+
107+
**Weaknesses:**
108+
- Requires Next.js knowledge
109+
- Newer framework (less battle-tested)
110+
- Heavier than Starlight
111+
112+
**Verdict for ReCursor:** Excellent future option if ReCursor adopts Next.js for other properties or needs API documentation with interactive playgrounds.
113+
114+
---
115+
116+
## Part 2: AI-Agentic Documentation Patterns (2026)
117+
118+
### 2.1 The Shift to Agent-Readable Documentation
119+
120+
Documentation in 2026 must serve two audiences: humans and AI agents. The emergence of AI coding assistants (Claude Code, Cursor, Copilot) has created a new requirement: **Agent Experience (AX)** is as important as Developer Experience (DX) [^5].
121+
122+
> "You can't have great DX if you don't have great AX." — Theneo API Documentation Guide [^5]
123+
124+
### 2.2 Key AI-Agentic Patterns
125+
126+
#### Pattern 1: `llms.txt` Standard
127+
128+
The `llms.txt` standard (proposed by Anthropic) provides a machine-readable index of documentation:
129+
130+
```
131+
# ReCursor Documentation
132+
> Mobile-first companion UI for AI coding workflows
133+
134+
## Getting Started
135+
- [Overview](docs/README.md): Project introduction
136+
- [Quickstart](docs/quickstart.md): First-time setup
137+
138+
## Core Documentation
139+
- [Architecture](docs/architecture/overview.md): System design
140+
- [Bridge Protocol](docs/bridge-protocol.md): WebSocket protocol
141+
- [Integration](docs/integration/): Claude Code Hooks, Agent SDK
142+
143+
## API Reference
144+
- [Bridge HTTP API](docs/bridge-http-api.md): REST endpoints
145+
- [Type Mapping](docs/type-mapping.md): Dart↔TypeScript contracts
146+
```
147+
148+
**Implementation:** Auto-generate at build time from doc structure [^6].
149+
150+
#### Pattern 2: `llms-full.txt` for Complete Context
151+
152+
A concatenated version of all documentation for deep AI understanding:
153+
154+
| File | Size | Purpose |
155+
|------|------|---------|
156+
| `llms.txt` | ~20KB | Quick context, navigation |
157+
| `llms-full.txt` | ~400KB | Complete documentation |
158+
| `SKILL.md` | ~5KB | Patterns & anti-patterns for coding agents |
159+
160+
*Source: Vuetify0 AI Tools [^6]*
161+
162+
#### Pattern 3: Model Context Protocol (MCP) Integration
163+
164+
MCP (introduced by Anthropic November 2024) is becoming the standard for AI-tool integration [^7]:
165+
166+
**MCP for Documentation Sites:**
167+
- Expose documentation as MCP resources
168+
- Enable AI assistants to query docs programmatically
169+
- Self-describing capabilities via MCP manifest
170+
171+
**Enterprise Adoption:**
172+
- Microsoft integrated MCP into Semantic Kernel
173+
- Red Hat OpenShift AI supports MCP
174+
- Google A2A (Agent-to-Agent) protocol complements MCP for multi-agent systems [^8]
175+
176+
#### Pattern 4: LLM-Friendly Content Structure
177+
178+
Best practices for AI-readable documentation [^9]:
179+
180+
1. **Atomic pages** - One clear intent per page
181+
2. **Descriptive headings** - H1, H2, H3 with semantic meaning
182+
3. **Section length** - 200-400 words per section for clean chunking
183+
4. **Self-contained code snippets** - Runnable examples with known inputs/outputs
184+
5. **Plain language** - Avoid marketing copy, use precise technical terms
185+
6. **Semantic HTML** - `<article>`, `<section>`, `<nav>` over generic `<div>`
186+
187+
### 2.3 Documentation as Code (2026 Best Practices)
188+
189+
From Microsoft, Google, and GitHub patterns [^10]:
190+
191+
| Practice | Implementation |
192+
|----------|----------------|
193+
| Version Control | Store docs in `/docs` alongside code |
194+
| Linting | Use `markdownlint` for style consistency |
195+
| CI/CD | Automated build/deploy on PR merge |
196+
| Review Process | PR-based doc reviews with code owners |
197+
| Line Breaks | Break at sentence boundaries for clean diffs |
198+
| Format | GitHub Flavored Markdown (GFM) |
199+
200+
---
201+
202+
## Part 3: ReCursor-Specific Analysis
203+
204+
### 3.1 Current State Assessment
205+
206+
**Current docs-site:**
207+
- Plain HTML/CSS/JS
208+
- Client-side SPA navigation
209+
- 12 pages in `docs-site/pages/`
210+
- Manual search implementation
211+
- No versioning
212+
- No AI-specific optimizations
213+
214+
**Current docs/ folder:**
215+
- 20+ Markdown documents
216+
- Well-organized structure
217+
- Mermaid diagrams
218+
- Cross-referenced
219+
- **Already AI-friendly** (Markdown source)
220+
221+
### 3.2 ReCursor Requirements Analysis
222+
223+
| Requirement | Priority | Notes |
224+
|-------------|----------|-------|
225+
| Search | High | Currently manual/lacking |
226+
| Mobile-friendly | Critical | ReCursor is mobile-first app |
227+
| Dark mode | High | Current site has it |
228+
| Versioning | Medium | For post-v1.0 releases |
229+
| API docs | Medium | Bridge HTTP API needs docs |
230+
| AI-friendly | High | Agent SDK integration focus |
231+
| Self-hosted | High | No external dependencies preferred |
232+
| Fast builds | Medium | CI/CD integration |
233+
234+
### 3.3 What's Missing from Current Setup
235+
236+
1. **Search functionality** - No full-text search
237+
2. **Structured data** - No `llms.txt` or MCP exposure
238+
3. **Auto-generated API docs** - Bridge HTTP API manually documented
239+
4. **Versioning** - No doc versioning for releases
240+
5. **SEO optimization** - Basic meta tags only
241+
6. **Social sharing** - No Open Graph images
242+
7. **Analytics** - No usage tracking
243+
244+
---
245+
246+
## Part 4: Recommendations
247+
248+
### 4.1 Immediate Recommendation: Astro Starlight
249+
250+
**Rationale:**
251+
1. **Aligns with ReCursor's stack** - No React/Vue framework lock-in
252+
2. **Performance-first** - Matches mobile app philosophy
253+
3. **Markdown-native** - Leverages existing `docs/` content
254+
4. **Built-in search** - Pagefind requires no external service
255+
5. **Type-safe** - Content collections catch errors at build time
256+
6. **AI-ready** - Easy to add `llms.txt` generation
257+
7. **Low migration cost** - Copy Markdown files, minimal config
258+
259+
**Migration Path:**
260+
```bash
261+
# Step 1: Create Starlight project
262+
npm create astro@latest -- --template starlight
263+
264+
# Step 2: Copy existing docs
265+
cp -r docs/* src/content/docs/
266+
267+
# Step 3: Add frontmatter to existing docs
268+
# (title, description, sidebar position)
269+
270+
# Step 4: Configure sidebar navigation
271+
# Match current docs-site structure
272+
273+
# Step 5: Add llms.txt generation script
274+
# Auto-generate from content structure
275+
276+
# Step 6: Deploy to Vercel/Netlify
277+
```
278+
279+
**Cost:** Free (open source + free hosting tier)
280+
281+
### 4.2 Future Enhancement Path: Fumadocs
282+
283+
**When to migrate:**
284+
- ReCursor adopts Next.js for web dashboard
285+
- Need interactive API documentation with try-it features
286+
- Require advanced React component showcases
287+
- Need deep Vercel ecosystem integration
288+
289+
**Migration complexity:** Medium (2-3 days for medium site) [^1]
290+
291+
### 4.3 Optional Enhancement: Mintlify (Managed)
292+
293+
**When to consider:**
294+
- Team wants web-based editing (non-technical contributors)
295+
- Need AI assistant built into docs
296+
- Budget allows $99+/mo for advanced features
297+
- Don't require self-hosting
298+
299+
**Trade-off:** Lose self-hosting, gain polished UX
300+
301+
### 4.4 AI-Agentic Enhancements (All Stacks)
302+
303+
Regardless of framework choice, implement:
304+
305+
1. **`/llms.txt` endpoint** - Auto-generated index
306+
2. **`/llms-full.txt` endpoint** - Complete documentation dump
307+
3. **`SKILL.md` file** - Coding assistant patterns for ReCursor
308+
4. **MCP server** - Expose docs via Model Context Protocol
309+
5. **Structured frontmatter** - Consistent metadata for AI parsing
310+
311+
---
312+
313+
## Part 5: Comparison Matrix
314+
315+
### Framework Comparison for ReCursor
316+
317+
| Criteria | Astro Starlight | Docusaurus | Fumadocs | Mintlify |
318+
|----------|-----------------|------------|----------|----------|
319+
| **Self-hosted** | ✅ Yes | ✅ Yes | ✅ Yes | ❌ No |
320+
| **Cost** | Free | Free | Free | $99+/mo |
321+
| **Bundle Size** | ~50KB | ~200KB | ~180KB | N/A |
322+
| **Build Time** | ~4s | ~15s | Medium | N/A |
323+
| **Search** | Built-in (Pagefind) | Algolia plugin | Built-in | Built-in |
324+
| **Versioning** | Plugin | Built-in | Plugin | Built-in |
325+
| **React Components** | Via islands | Native | Native | Limited |
326+
| **Mobile Performance** | ⭐⭐⭐ Excellent | ⭐⭐ Good | ⭐⭐ Good | ⭐⭐⭐ Excellent |
327+
| **AI-friendly** | ⭐⭐⭐ Excellent | ⭐⭐ Good | ⭐⭐⭐ Excellent | ⭐⭐⭐ Excellent |
328+
| **Learning Curve** | Low | Medium | Medium | Very Low |
329+
| **ReCursor Fit** | ⭐⭐⭐ Perfect | ⭐⭐ Good | ⭐⭐⭐ Perfect | ⭐⭐ Good |
330+
331+
### AI-Agentic Feature Support
332+
333+
| Feature | Implementation Difficulty | Impact |
334+
|---------|--------------------------|--------|
335+
| `llms.txt` | Low (build script) | High |
336+
| `llms-full.txt` | Low (concatenation) | High |
337+
| `SKILL.md` | Medium (content curation) | Medium |
338+
| MCP server | Medium (TypeScript SDK) | High |
339+
| Structured frontmatter | Low (convention) | Medium |
340+
341+
---
342+
343+
## References
344+
345+
[^1]: DevTools Guide. "Modern Documentation Systems: Docusaurus, Starlight, and Mintlify Comparison." https://devtoolsguide.com/developer-documentation-systems/
346+
347+
[^2]: PrimeDev.Tools. "Docs & Technical Writing Workflow 2026." https://primedev.tools/workflows/technical-writer.html
348+
349+
[^3]: Naturaily. "Best Static Site Generators in 2026: Top SSGs Compared." https://naturaily.com/blog/best-static-site-generators
350+
351+
[^4]: ToolQuestor. "Best 20+ Tools for Managing Documentation Content in 2026." https://toolquestor.com/use-case/manage-documentation-content
352+
353+
[^5]: Theneo. "The Ultimate Guide to API Documentation Best Practices (2025-2026)." https://www.theneo.io/blog/api-documentation-best-practices-guide-2025
354+
355+
[^6]: Vuetify0. "AI Tools - LLM-Friendly Documentation." https://0.vuetifyjs.com/guide/tooling/ai-tools
356+
357+
[^7]: Dysnix. "Model Context Protocol (MCP) Comprehensive Guide for 2025." https://dysnix.com/blog/model-context-protocol
358+
359+
[^8]: Cisco Blogs. "MCP and A2A: A Network Engineer's Mental Model for Agentic AI." https://blogs.cisco.com/ai/mcp-and-a2a-a-network-engineers-mental-model-for-agentic-ai
360+
361+
[^9]: Aronhack. "LLM-Friendly Documentation: Creating Content That AI Can Understand." https://aronhack.com/llm-friendly-documentation-creating-content-that-ai-can-understand-and-process-effectively/
362+
363+
[^10]: Kong HQ. "What is Docs as Code? Guide to Modern Technical Documentation." https://konghq.com/blog/learning-center/what-is-docs-as-code
364+
365+
[^11]: GitBook. "How to optimize your docs for AI search and LLM ingestion." https://gitbook.com/docs/guides/seo-and-llm-optimization/geo-guide
366+
367+
[^12]: Addy Osmani. "My LLM coding workflow going into 2026." https://medium.com/@addyosmani/my-llm-coding-workflow-going-into-2026-52fe1681325e
368+
369+
[^13]: Fumadocs. Official Documentation. https://www.fumadocs.dev/
370+
371+
[^14]: Docusaurus. Official Documentation. https://docusaurus.io/docs/next
372+
373+
[^15]: Red Hat Developers. "Building effective AI agents with Model Context Protocol (MCP)." https://developers.redhat.com/articles/2026/01/08/building-effective-ai-agents-mcp
374+
375+
---
376+
377+
## Appendix: Quick Decision Tree
378+
379+
```
380+
Is self-hosting required?
381+
├── No → Consider Mintlify (managed) or GitBook
382+
└── Yes → Continue
383+
384+
Is React/Next.js already in your stack?
385+
├── Yes → Consider Fumadocs (Next.js-native)
386+
└── No → Continue
387+
388+
Is performance the top priority?
389+
├── Yes → Astro Starlight (recommended)
390+
└── No → Consider Docusaurus (features) or Fumadocs (flexibility)
391+
```
392+
393+
---
394+
395+
*End of Research Report*

0 commit comments

Comments
 (0)