Data engineering · Jan 2026
Air quality monitor for Valencia
Ingest Valencia's air-quality stations in real time, transform the data with dbt and alert when thresholds are exceeded.
- My role
- Second contributor on the team. Ingestion, dbt transformation models and integration of all services in Docker Compose.
- Team
- Team of 5 · Data Project 1
- Outcome
- Recognised as the top team of the cohort. A complete Docker pipeline with dbt every five minutes, Grafana dashboards and Telegram alerts.
Context
First team project of the master’s, closing the data-processing block. The case: build an air-quality monitoring system for Valencia that would serve both to check the current state and to analyse history, with a team of five and a two-week deadline.
Problem
Station data arrives continuously, with readings that are sometimes missing or out of range. Three things had to be clearly separated: receiving and storing raw data, transforming it into clean and aggregated tables, and consuming it from dashboards and alerts. And everything had to come up on any machine with a single command.
What I did
- Part of the ingestion of readings into PostgreSQL and its historical backfill.
- dbt models turning raw readings into intermediate tables and marts ready for visualisation, run every five minutes.
- Integration of all services in Docker Compose: database, API, transformations, Grafana and the alert bot, with environment variables and API keys generated by script.
Architecture
Stations → Python ingestion → PostgreSQL → dbt (staging, intermediate, marts) → Grafana for dashboards and a Telegram bot for alerts. A FastAPI service exposes current state and active alerts, protected with API keys.
Outcome
A system that comes up with docker compose up, transforms continuously and alerts when a pollutant crosses its threshold. The panel recognised it as the best project of the cohort.
The repository is being prepared for publication.
What I take with me
dbt changed how I write SQL: small, named models with tests and documentation next to them. It is the same pattern I later applied with Dataform in the thesis.