Skip to content

TASK: IGDB ETL – Dead Letter Handling Strategy (placeholder table or log) #32

@AndersonGACFilho

Description

@AndersonGACFilho

Summary

Implement a basic dead-letter handling strategy for records that fail to be processed after multiple retries.

Details

  • For now, this can be as simple as logging the failed record to a specific log level (e.g., ERROR) with all its details, or inserting it into a dedicated ingestion_dlq table in the database.

Acceptance Criteria

  • Records that fail consistently are not dropped silently.
  • Failed records are either logged in detail or saved to a DLQ table.
  • Tests updated/added (unit/integration)
  • No linter / build errors

Dependencies

  • IGDB ETL – Retry & Backoff Strategy Implementation

Out of Scope

  • Automated reprocessing of the dead-letter queue.

Test Plan

  • Write a test that simulates a record that always fails validation and verify that it ends up in the designated dead-letter location.

Checklist

  • Labels added (task, component/igdb-etl)
  • CHANGELOG updated if user-visible
  • No secrets committed

Metadata

Metadata

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions