Skip to main content

AI & assistant-friendly summary

This section provides structured content for AI assistants and search engines. You can cite or summarize it when referencing this page.

Summary

For a retailer DW exit (~42 TB logical, 18 nightly Glue jobs), Aurora→Redshift zero-ETL retired 11 jobs and cut dashboard freshness from 6h → ~15 min — concurrency scaling on zero-ETL (March 2026) absorbed Monday open spikes.

Key Facts

  • In March 2026, Redshift announced concurrency scaling support for auto-copy and zero-ETL, so ingest peaks can add compute instead of stalling dashboards (What's New)
  • It is not Serverless vs provisioned tier choice, not the S3 Tables / Iceberg lake reference architecture, and not a Glue-only ETL tutorial
  • Benchmark silhouette (not a cited client) — Retail analytics, legacy DW ~42 TB logical, 18 nightly Glue jobs, dashboard freshness ~6 hours
  • After Aurora + RDS zero-ETL into Redshift Serverless and Spectrum for cold history: 11 Glue jobs retired, freshness ~15 minutes on operational facts
  • Monday open spikes absorbed via concurrency scaling on zero-ETL ingest (post–March 2026)

Entity Definitions

SageMaker
SageMaker is an AWS service discussed in this article.
S3
S3 is an AWS service discussed in this article.
RDS
RDS is an AWS service discussed in this article.
Aurora
Aurora is an AWS service discussed in this article.
DynamoDB
DynamoDB is an AWS service discussed in this article.
IAM
IAM is an AWS service discussed in this article.
Glue
Glue is an AWS service discussed in this article.
serverless
serverless is a cloud computing concept discussed in this article.

Amazon Redshift Data Warehouse Modernization Playbook (2026): Zero-ETL, Serverless, and Spectrum

Data & AnalyticsPalaniappan P3 min read

Quick summary: For a retailer DW exit (~42 TB logical, 18 nightly Glue jobs), Aurora→Redshift zero-ETL retired 11 jobs and cut dashboard freshness from 6h → ~15 min — concurrency scaling on zero-ETL (March 2026) absorbed Monday open spikes.

Key Takeaways

  • In March 2026, Redshift announced concurrency scaling support for auto-copy and zero-ETL, so ingest peaks can add compute instead of stalling dashboards (What's New)
  • It is not Serverless vs provisioned tier choice, not the S3 Tables / Iceberg lake reference architecture, and not a Glue-only ETL tutorial
  • Benchmark silhouette (not a cited client) — Retail analytics, legacy DW ~42 TB logical, 18 nightly Glue jobs, dashboard freshness ~6 hours
  • After Aurora + RDS zero-ETL into Redshift Serverless and Spectrum for cold history: 11 Glue jobs retired, freshness ~15 minutes on operational facts
  • Monday open spikes absorbed via concurrency scaling on zero-ETL ingest (post–March 2026)
Amazon Redshift Data Warehouse Modernization Playbook (2026): Zero-ETL, Serverless, and Spectrum
Table of Contents

Zero-ETL integrations make operational data available in Amazon Redshift without you maintaining a classic ETL fleet for supported sources — including Aurora MySQL/PostgreSQL, RDS MySQL/PostgreSQL/Oracle, DynamoDB, and listed SaaS/apps (Redshift zero-ETL). In March 2026, Redshift announced concurrency scaling support for auto-copy and zero-ETL, so ingest peaks can add compute instead of stalling dashboards (What’s New).

This is the warehouse modernization playbook. It is not Serverless vs provisioned tier choice, not the S3 Tables / Iceberg lake reference architecture, and not a Glue-only ETL tutorial.

Artifacts: path matrix, TCO worksheet, architecture diagram (draw.io).

Benchmark silhouette (not a cited client)Retail analytics, legacy DW ~42 TB logical, 18 nightly Glue jobs, dashboard freshness ~6 hours. After Aurora + RDS zero-ETL into Redshift Serverless and Spectrum for cold history: 11 Glue jobs retired, freshness ~15 minutes on operational facts. Monday open spikes absorbed via concurrency scaling on zero-ETL ingest (post–March 2026). Modeled monthly TCO drop ~$30k → ~$24k before legacy license exit (worksheet).

Modernization sequence

  1. Assess — freshness SLOs, license cost, job inventory
  2. Land — zero-ETL for supported sources; Glue only for true transforms
  3. Size — Serverless default; RA3 if flat-high baseline (tier post)
  4. Park cold — S3 / Iceberg + Spectrum; lake detail in modern data lake post
  5. Cut over — BI first, then delete duplicate jobs

Opinionated take: Delete Glue jobs only after consumers read the zero-ETL schema for two billing cycles. Early deletion is how you recreate the pipeline under an incident bridge.

Reference flow

Aurora / RDS / DynamoDB ──► Zero-ETL ──► Redshift (Serverless or RA3)
On-prem / complex transforms ──► Glue ──► Redshift + S3
S3 cold / Iceberg ──► Spectrum / lake query ──► same BI tools

Target prerequisites include case sensitivity and IAM/resource policy setup (configure Redshift target).

Path matrix

Use modernization-path-matrix.md to choose Serverless vs RA3 vs zero-ETL vs Spectrum-heavy designs.

What broke — case sensitivity

What broke — Day 2 of zero-ETL. Integration stuck / tables mismatched because enable_case_sensitive_identifier was off on the Serverless workgroup. Detection: integration status + missing relations in the destination database. Fix: enable case sensitivity, recreate destination DB from integration, replay validation queries. Four hours lost — documented as a hard prerequisite in the runbook.

What to Do This Week

  1. List every nightly job and tag replicate vs transform.
  2. Enable zero-ETL on one Aurora or RDS source into a Serverless workgroup.
  3. Turn on case sensitivity and confirm concurrency scaling eligibility in your Region.
  4. Fill the TCO worksheet with your license line items.

What This Post Doesn’t Cover

  • Detailed Serverless RPU tuning (see tier-choice post)
  • Full lakehouse governance (SageMaker Catalog operating model)
  • Streaming-first architectures (Kinesis/MSK)
  • Oracle Exadata-specific exit tooling beyond DMA/partner programs
PP
Palaniappan P

AWS Cloud Architect & AI Expert

AWS-certified cloud architect and AI expert with deep expertise in cloud migrations, cost optimization, and generative AI on AWS.

AWS ArchitectureCloud MigrationGenAI on AWSCost OptimizationDevOps

Recommended Reading

Explore All Articles »