Summary
Instrument the ETL process with key operational metrics.
Details
fetched_count: A counter for the number of game records fetched from the API.
upsert_count: A counter for the number of game records successfully upserted into the database.
lag_seconds: A gauge to measure the time difference between now() and the last successful watermark timestamp.
Acceptance Criteria
Dependencies
Out of Scope
- Setting up dashboards or alerts.
Test Plan
- Write a unit test that runs a mock ETL process and verifies that the metric collectors are called with the expected values.
Checklist
Summary
Instrument the ETL process with key operational metrics.
Details
fetched_count: A counter for the number of game records fetched from the API.upsert_count: A counter for the number of game records successfully upserted into the database.lag_seconds: A gauge to measure the time difference betweennow()and the last successful watermark timestamp.Acceptance Criteria
Dependencies
IGDB ETL – Game UpsertOut of Scope
Test Plan
Checklist