Snowflake on AWS

Snowflake on AWS

Snowflake is a cloud-native data warehouse that runs on AWS. It separates compute (query processing) from storage (data), allowing independent scaling and cost optimization.

Why Snowflake on AWS?

Simplicity

• No infrastructure management
• Automatic scaling: grows with data
• Zero administration operational overhead

Performance

• Queries execute instantly via distributed compute
• Caching accelerates repeated queries
• Data sharing enables instant access across teams

Cost Efficiency

• Pay for compute only when queries run
• Automatic resource suspend when idle
• Storage is cheap; compute is expensive (optimize queries)

Architecture: Compute Separate from Storage

Traditional data warehouse: storage and compute tightly coupled

• Add data → must add compute capacity
• Fixed costs

Snowflake: storage and compute independent

• Add data without adding compute
• Scale compute up/down independently
• Pay only for resources used

Key Snowflake + AWS Features

Snowflake Warehouses

• Compute clusters for query execution
• Auto-scale up for large queries
• Auto-suspend when idle (save cost)
• Multiple warehouses for concurrency

External Tables (S3 Integration)

• Query S3 data without loading to Snowflake
• Join Snowflake tables with S3 files
• Cost-effective for infrequently accessed data

Data Sharing

• Share tables with other Snowflake accounts securely
• Reader accounts access data in real-time
• Shared data is read-only

Continuous Data Pipeline

• Snowpipe: continuous loading from S3
• Stream: track data changes
• Task: scheduled transformations

Data Loading from AWS

From S3 (most common)

COPY INTO my_table
FROM @my_s3_stage/data.csv
FILE_FORMAT = (TYPE = 'CSV', SKIP_HEADER = 1);

From RDS/DynamoDB

• AWS Glue jobs can extract from RDS
• Write to S3, then load to Snowflake
• Or use third-party ETL tools (Fivetran, Talend)

Real-time Streaming

• Snowflake Connector for Kafka ingests streaming data
• Ideal for IoT, event tracking, real-time analytics

Snowflake Pricing

Compute Credits

• Per-second billing: $2-4 per credit depending on region/edition
• 1 credit ≈ 1 compute hour
• Typical query: 0.1-5 credits
• Warehouse with 4 nodes suspended: $0

Storage

• $25-40 per TB per month (Standard edition)
• Includes 1-month data retention
• Data cloning, backup: additional cost

Example Costs

• Small warehouse (2 nodes, 4 hours/day): ~$200/month compute + $100/month storage
• Large warehouse (8 nodes, 20 hours/day): ~$2,000/month compute + $500/month storage

Snowflake vs Redshift vs BigQuery

Best Practices

Performance

• Choose right warehouse size (small for dev, medium for prod queries)
• Cluster keys on frequent filters
• Archive old data to S3 to reduce storage

Cost

• Auto-suspend after 10 minutes of inactivity
• Use query cache (free if result exists)
• Archive infrequently used data

Data Quality

• Monitor data freshness (when was S3 last updated?)
• Implement data validation in loading pipelines
• Use change tracking (Streams) for incremental loads

Related Services

• Analytics on AWS
• AWS Data Consulting