# Glue + Athena Serverless Pipeline Checklist (July 2026)

## Layout

- [ ] Buckets: `raw/` · `curated/` (Parquet) · `athena-results/`
- [ ] Glue Data Catalog DB + tables (crawler or explicit DDL)
- [ ] Glue Spark job on a supported Glue version (prefer current **Glue 5.x** line — verify console)
- [ ] Schedule + SNS/EventBridge on FAILED

## Cost controls

- [ ] Write **Parquet + Snappy** (or ZSTD) — not CSV for Athena hot path
- [ ] Partition by query filters (`year`/`month`/`day` or `dt=`)
- [ ] Athena: always filter on partition columns; avoid `SELECT *` in prod reports
- [ ] Optional: Athena workgroup with bytes-scanned limit / Capacity Reservation for steady BI

## When not Glue batch

- Streaming / sub-minute: Kinesis / Firehose / Flink / Lambda — not a daily Glue job alone
- Heavy transforms with dbt-on-warehouse: see Glue vs dbt companion

## Smoke test

1. Drop 1 sample day of raw data  
2. Run job → curated Parquet  
3. `MSCK REPAIR TABLE` or Glue partition update  
4. Athena query with partition predicate; note data scanned in query stats