Amazon S3

Definition

Amazon Simple Storage Service (S3) is AWS’s object storage service — store and retrieve any amount of data, at any time, from anywhere. S3 stores data as objects (files + metadata) inside buckets. It provides 99.999999999% (11 nines) data durability and is used by 90% of Fortune 100 companies for backups, data lakes, static website hosting, and application data.

How S3 Works

Objects and Buckets:

• Bucket: Container for objects; globally unique name; tied to a region
• Object: File + metadata stored in a bucket; up to 5 TB per object
• Key: Unique identifier for each object within a bucket (behaves like a file path)

Objects are accessed via HTTPS URL: https://bucket-name.s3.region.amazonaws.com/key

Storage Classes

Choose storage class based on access frequency to optimize cost:

S3 Intelligent-Tiering automatically moves objects between tiers based on access patterns — ideal when access is unpredictable. Objects not accessed for 30 days move to infrequent access tier automatically at lower cost.

Key Features

S3 Versioning: Keep multiple versions of each object. Protects against accidental deletion and overwrites. Required for compliance use cases.

S3 Lifecycle Policies: Automatically transition objects to cheaper storage classes or delete them after a set period. Example: Move objects to Glacier after 90 days, delete after 7 years.

S3 Replication: Replicate objects to another bucket — same region (SRR) or cross-region (CRR). Used for disaster recovery, compliance data residency, or reducing latency.

S3 Object Lock: Write-once-read-many (WORM) storage. Prevents deletion or modification for a defined retention period. Required for SEC 17a-4, FINRA, and HIPAA retention compliance.

S3 Event Notifications: Trigger Lambda, SQS, or SNS on object create/delete events. Foundation for event-driven data pipelines.

S3 Vectors (New 2025)

Amazon S3 Vectors is a purpose-built vector storage tier within S3:

• Supports up to 2 billion vectors per index
• ~90% cheaper than specialized vector databases
• Sub-100ms query latencies
• No infrastructure to manage
• Native integration with Amazon Bedrock Knowledge Bases

Use S3 Vectors for RAG pipelines and semantic search workloads instead of managing a separate vector database.

S3 Tables (New 2025)

Amazon S3 Tables provides native Apache Iceberg table support directly in S3:

• Automatic cost optimization via intelligent storage tiering
• Simplified table replication across AWS Regions
• Eliminates need for a separate data catalog for structured analytics

Common Mistakes

Mistake 1: Leaving all objects in S3 Standard. Most data is rarely accessed after 30 days; implement lifecycle policies or enable Intelligent-Tiering to reduce costs by 40–70%.

Mistake 2: Not enabling versioning for critical data. Without versioning, an accidental DELETE or overwrite is permanent. Enable versioning on all buckets storing important data.

Mistake 3: Public bucket access. Enable S3 Block Public Access at the account level; grant access through IAM policies or pre-signed URLs, not public bucket policies.

Related AWS Services

• Amazon CloudFront: Serve S3 objects via CDN for low-latency global delivery
• AWS Glue / Amazon Athena: Query S3 data lakes with SQL
• AWS Backup: Centralized backup policy for S3 versioned buckets
• Amazon S3 Vectors: Vector storage tier for AI/ML workloads (new 2025)

Related FactualMinds Content

• Cloud Cost Optimization Services
• FinOps Consulting
• AWS Data Analytics