Add cache + Etag on response

[flovntp]

• cache key with computation of the file mimetime + optionnal params
• CACHE_TTL envVar added everywhere (default to 300ms)
• send corresponding headers
• send Etag to the response
• envVar CACHE_TTL already added to the Meta Registry Upsun project

ETag Implementation Test Report
Test Summary
All API routes have been tested successfully. The ETag implementation is working correctly across all endpoints with proper caching, 304 Not Modified responses, and unique ETags for filtered queries.

Test Results

1. /images - Container Images
   Status: ✅ Pass
   HTTP Code: 200 OK
   ETag: "1773391306186-305001"
   Last-Modified: Fri, 13 Mar 2026 08:41:46 GMT
   Cache-Control: public, max-age=300, must-revalidate
   Content-Length: 15,605 bytes
2. /images - 304 Not Modified Test
   Status: ✅ Pass
   HTTP Code: 304 Not Modified
   If-None-Match: "1773391306186-305001"
   Response: Empty body (bandwidth saved)
   Headers: Same ETag returned, confirming client cache validity
3. /composable - Composable Images
   Status: ✅ Pass
   HTTP Code: 200 OK
   ETag: "1773391306185-893"
   Last-Modified: Fri, 13 Mar 2026 08:41:45 GMT
   Cache-Control: public, max-age=300, must-revalidate
4. /regions - All Regions (unfiltered)
   Status: ✅ Pass
   HTTP Code: 200 OK
   ETag: "1773391306182-99083"
   Last-Modified: Fri, 13 Mar 2026 08:41:46 GMT
   Cache-Control: public, max-age=300, must-revalidate
   Content: Full regions list (15+ regions, multiple providers)
5. /regions?provider=AWS - Filtered Regions
   Status: ✅ Pass
   HTTP Code: 200 OK
   ETag: "1773391306182-99083-qa7394265" (with query hash suffix)
   Last-Modified: Fri, 13 Mar 2026 08:41:46 GMT
   Cache-Control: public, max-age=300, must-revalidate
   Content: AWS regions only (6 regions)
   Query Hash: qa7394265 uniquely identifies the provider=AWS filter
6. /regions?provider=Azure - Filtered Regions (different filter)
   Status: ✅ Pass
   HTTP Code: 200 OK
   ETag: "1773391306182-99083-qf4ba701f" (different query hash)
   Last-Modified: Fri, 13 Mar 2026 08:41:46 GMT
   Cache-Control: public, max-age=300, must-revalidate
   Content: Azure regions only (3 regions)
   Query Hash: qf4ba701f uniquely identifies the provider=Azure filter
   Validation: ✅ Different hash from AWS filter confirms proper query parameter differentiation
7. /openapi-spec - OpenAPI Specification
   Status: ✅ Pass
   HTTP Code: 200 OK
   ETag: "1773307872197-972812"
   Last-Modified: Thu, 12 Mar 2026 09:31:12 GMT
   Cache-Control: public, max-age=300, must-revalidate
   Content-Length: 972,812 bytes (large spec file)
8. /extensions/php - PHP Extensions
   Status: ✅ Pass
   HTTP Code: 200 OK
   ETag: "1773332487904-144236"
   Last-Modified: Thu, 12 Mar 2026 16:21:27 GMT
   Cache-Control: public, max-age=300, must-revalidate
   Content-Length: 78,473 bytes
9. /schema/upsun - Upsun Validation Schema
   Status: ✅ Pass
   HTTP Code: 200 OK
   ETag: "1773307872204-60016"
   Last-Modified: Thu, 12 Mar 2026 09:31:12 GMT
   Cache-Control: public, max-age=300, must-revalidate
   Content-Length: 39,819 bytes
   Key Findings
   ✅ Successful Implementation
   All routes support ETags: Every API endpoint returns proper ETag and Last-Modified headers
   304 Not Modified works: Validated with If-None-Match request header
   Query parameter hashing functional: Filtered routes generate unique ETags
   Base file ETag: 1773391306182-99083
   AWS filtered ETag: 1773391306182-99083-qa7394265
   Azure filtered ETag: 1773391306182-99083-qf4ba701f
   Consistent cache TTL: All routes use 300-second (5-minute) cache duration from CACHE_TTL environment variable
   Proper cache headers: Cache-Control: public, max-age=300, must-revalidate on all responses
   Cache Collision Prevention
   The query parameter hashing successfully prevents cache collisions:

Same base file (regions.json) generates different ETags based on query filters
Clients can cache filtered and unfiltered responses independently
No risk of serving AWS data when requesting Azure data
Performance Benefits
Bandwidth savings: 304 responses send only headers (no body)
Client-side caching: Mintlify and other consumers can cache for up to 5 minutes
CDN compatibility: public cache-control allows intermediate caching
Conditional requests: must-revalidate ensures freshness checks after expiry
Configuration
Cache TTL: Configurable via CACHE_TTL environment variable (default: 300 seconds)
ETag format:
Unfiltered: "{timestamp}-{size}"
Filtered: "{timestamp}-{size}-q{md5_hash}"
Query hash: MD5 (truncated to 8 characters) of sorted, normalized query parameters

[Copilot]

Pull request overview

This PR introduces HTTP caching support for the backend’s resource-backed endpoints by adding ETag/Last-Modified metadata to resources and using it to emit cache headers + 304 responses, with a configurable TTL via CACHE_TTL.

Changes:

• Add ResourceManager APIs that return resource data along with ETag/Last-Modified metadata (local + GitHub).
• Introduce cache.manager.ts helpers to generate/query-aware ETags, set cache headers, and send 304 responses.
• Update multiple API routes to use these helpers and add CACHE_TTL to runtime config and .env.example.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
backend/src/utils/resource.manager.ts	Adds metadata-aware resource fetch methods (local + GitHub) and exposes ETag/Last-Modified.
backend/src/utils/cache.manager.ts	New utility for ETag generation (incl. query params), cache header setting, and 304 responses.
backend/src/utils/index.ts	Re-exports new cache utilities and metadata types.
backend/src/routes/validation.routes.ts	Uses metadata-aware resource fetch + conditional 304 + cache headers.
backend/src/routes/region.routes.ts	Adds query-aware ETags for filtered list endpoint + cache headers/304.
backend/src/routes/openapi.upsun.routes.ts	Adds cache headers/304 for serving OpenAPI spec files.
backend/src/routes/image.routes.ts	Adds cache headers/304 for image registry endpoints.
backend/src/routes/extension.routes.ts	Adds cache headers/304 for PHP extensions endpoints.
backend/src/routes/composable.routes.ts	Adds cache headers/304 for composable endpoint.
backend/src/middleware/httpLogger.middleware.ts	Skips logging for /socket.io polling requests.
backend/src/config/env.config.ts	Adds config.cache.TTL sourced from CACHE_TTL.
backend/.env.example	Documents and adds CACHE_TTL example value.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Adds HTTP caching support (Cache-Control + ETag/Last-Modified) for resource-backed endpoints, with a new cache utility layer and metadata-aware resource fetching to enable conditional requests (especially in GitHub-backed mode).

Changes:

• Introduce ResourceManager metadata fetch APIs (etag, last-modified, upstream 304 handling).
• Add cache.manager utilities to extract conditional headers, generate query-aware ETags, and set caching headers.
• Wire caching behavior + CACHE_TTL config through major routes (images, regions, extensions, composable, openapi, validation schemas).

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
backend/src/utils/resource.manager.ts	Adds metadata-aware resource reads and GitHub conditional fetching paths.
backend/src/utils/cache.manager.ts	New helpers for conditional headers, ETag generation, 304 responses, and cache headers.
backend/src/utils/index.ts	Re-exports new caching utilities and metadata types.
backend/src/routes/validation.routes.ts	Uses metadata fetch + conditional/ETag headers for validation schema endpoints.
backend/src/routes/region.routes.ts	Adds conditional fetch and query-aware ETag for filtered/unfiltered regions.
backend/src/routes/openapi.upsun.routes.ts	Adds conditional fetch + caching headers for serving OpenAPI files.
backend/src/routes/image.routes.ts	Adds conditional fetch + caching headers for images endpoints.
backend/src/routes/extension.routes.ts	Adds conditional fetch + caching headers for extensions endpoints; fixes a missing return on 404 path.
backend/src/routes/composable.routes.ts	Adds conditional fetch + caching headers for composable endpoint.
backend/src/middleware/httpLogger.middleware.ts	Skips request logging for /socket.io polling requests.
backend/src/config/env.config.ts	Adds config.cache.TTL sourced from CACHE_TTL (seconds).
backend/doc/CONDITIONAL_FETCH_OPTIMIZATION.md	Documents the conditional fetch optimization and expected behavior.
backend/.env.example	Documents/introduces CACHE_TTL example value.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

This PR adds HTTP caching support across the backend API by introducing resource metadata (ETag/Last-Modified), propagating conditional request headers upstream (GitHub raw), and emitting cache headers on API responses to enable 304 responses and reduce bandwidth/CPU.

Changes:

• Introduce ResourceWithMetadata retrieval (raw + parsed) with upstream conditional fetching and 304 short-circuiting.
• Add centralized cache helpers (extractConditionalHeaders, ETag-with-query hashing, cache headers, 304 helper) and wire them into all routes.
• Add configurable cache TTL via CACHE_TTL env var and document the conditional-fetch optimization.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 17 comments.

Show a summary per file

File	Description
backend/src/utils/resource.manager.ts	Adds metadata-aware resource fetch APIs (local + GitHub) with upstream conditional requests and 304 handling.
backend/src/utils/cache.manager.ts	New cache utilities for conditional headers, ETag generation w/ query params, response cache headers, and 304 responses.
backend/src/utils/index.ts	Exports new cache utilities and metadata types.
backend/src/routes/validation.routes.ts	Uses metadata-aware resource loads; returns 304 on upstream 304; sets cache headers with TTL.
backend/src/routes/region.routes.ts	Adds query-param-aware ETag handling for filtered regions; uses metadata-aware loads + cache headers.
backend/src/routes/image.routes.ts	Uses metadata-aware loads + cache headers for image registry endpoints.
backend/src/routes/extension.routes.ts	Uses metadata-aware loads + cache headers for extension endpoints; fixes missing return on 404 path.
backend/src/routes/composable.routes.ts	Uses metadata-aware loads + cache headers for composable endpoint.
backend/src/routes/openapi.upsun.routes.ts	Uses raw metadata-aware loads + cache headers for OpenAPI spec endpoint.
backend/src/config/env.config.ts	Adds config.cache.TTL derived from CACHE_TTL env var.
backend/src/middleware/httpLogger.middleware.ts	Skips HTTP logging for /socket.io polling requests.
backend/doc/CONDITIONAL_FETCH_OPTIMIZATION.md	Documents the upstream-conditional-fetch optimization and new APIs.
backend/.env.example	Documents CACHE_TTL env var.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.