# Ingest + partition checklist (modern data lake)

## Landing

- [ ] Raw zone: S3 prefix `raw/{source}/{yyyy}/{mm}/{dd}/` — immutable, Object Lock if compliance requires
- [ ] Schema registry or Glue table version for breaking changes
- [ ] **Glue 5.1** job (Spark 3.5.6, Iceberg 1.10+) for curated zone — not Glue 4.x for new lakes

## Iceberg / S3 Tables

- [ ] Partition on high-cardinality time column (`day(event_ts)`) not raw UUID
- [ ] Enable S3 Tables maintenance: compaction **enabled**, review `targetFileSizeMB` (64–512)
- [ ] If promotion-week queries: define sort order → enable **sort** compaction strategy via `PutTableMaintenanceConfiguration`
- [ ] Snapshot retention: document `MaximumSnapshotAge` (default 120 hours) with legal hold exceptions

## Governance

- [ ] Lake Formation tags on curated tables — separate `analytics` vs `pii` tags
- [ ] LF fine-grained **write** policies tested on Glue INSERT (Glue 5.0+)
- [ ] Cross-link to [data governance operating model](https://www.factualminds.com/blog/aws-data-governance-operating-model-sagemaker-catalog-2026/) for stewardship RACI

## When NOT to partition further

- Table < 100 GB and full scan < $5/query — extra partitions add catalog noise
- Streaming ingest < 1 GB/day — consider Firehose direct to Iceberg only after volume proof