TASK: IGDB ETL – Watermark Read/Write (ingestion_watermark)

[AndersonGACFilho]

Summary

Implement the logic to read the last successful watermark from a dedicated ingestion_watermark table and write the new watermark after a successful batch.

Details

• A new table, ingestion_watermark, will be needed to store the timestamp or ID of the last processed record.
• The ETL must read from this table at the start of a run and update it at the end.

Acceptance Criteria

• The ETL correctly reads the last watermark value.
• The ETL updates the watermark value upon successful completion of a run.
• Tests updated/added (unit/integration)
• No linter / build errors

Dependencies

• IGDB ETL – Config & OAuth Token Retrieval

Out of Scope

• The actual data fetching based on the watermark.

Test Plan

• Write an integration test that connects to a test database, seeds a watermark value, and verifies that the ETL reads it and then updates it correctly.

Checklist

• Labels added (task, component/igdb-etl, priority/P1)
• CHANGELOG updated if user-visible
• No secrets committed