Postgres Data Freshness Monitoring with Email Alerts
Go to WorkflowDescription
Monitor Postgres Data Freshness and Email Alert If Stale
This template monitors a set of tables inside a Postgres database to ensure they're getting updated.
If the table hasn't been updated in 3 days (configurable), an email alert is sent containing the tables that are stale.
Requirements
You must have a Postgres database containing one or more tables that you'd like to monitor.
Each table to monitor must have a date or timestamp column that tracks when data was pushed.
For example, this might be:
A timestamp column if your table holds event/timeseries data
A last_updated column if your rows are expected to be modified
Usage
Use this template
Add your Postgres and email credentials
Adjust the Produce tables + date columns node to produce pairs of [table, date_column] that should be monitored for freshness
💁♂️ Note that a timestamp column also works
(Optional) Adjust the Remove fresh tables node for your desired staleness window (default is 3 days, but you can adjust as you please)
(Optional) Customize the Send alerts node to call whichever alerting workflow you please (I recommend my alerting workflow for easiest plug-and-play)
How it works
This template works by:
Pulling the most recent row for each table
Calculating how out-of-date each table is, in days
Dropping fresh tables that have been updated within the past 3 days
Sending an email alert with the stale tables that haven't been updated within the past 3 days