amazon-kinesis

On a composite ingest workload (~8k ordered TPS, 1 KB payloads), staying on SQS FIFO without high-throughput mode capped effective throughput near 300 TPS/API and modeled queue backlog cost near $95/mo before ops time — enabling high-throughput FIFO or switching to Kinesis on-demand changed the ceiling, not the consumer code.

Kinesis Data Streams and Amazon MSK both handle real-time streaming on AWS, but they serve different architectures. Here is how to choose between them for your workload.

Kinesis Data Streams combined with Lambda and DynamoDB is the simplest path to a real-time data pipeline on AWS. Here is the complete architecture, code patterns, and operational guidance.