High-Concurrency Server I/O: epoll, Syscalls, and Zero-Copy on AWS EC2
Quick summary: C10k is solved until syscall overhead and context switches eat your Graviton cores. epoll, sendfile, and SO_REUSEPORT behaviors on EC2, and why Lambda caps concurrency differently.
Key Takeaways
- C10k is solved until syscall overhead and context switches eat your Graviton cores
- epoll, sendfile, and SO_REUSEPORT behaviors on EC2, and why Lambda caps concurrency differently
- As of June 2026, Linux on EC2 serves most self-hosted APIs
- On AWS, ALB plus ENA instances with epoll edge-triggering handles most concurrency without kernel bypass
- Benchmark pattern (hypothetical workload): c6gn
Table of Contents
As of June 2026, Linux on EC2 serves most self-hosted APIs. Event loops use epoll (edge-triggered) to watch thousands of sockets per thread. BSD and macOS dev laptops use kqueue instead, so production parity tests matter.
Symptoms, mechanisms, and AWS controls
| Production symptom | Mechanism | AWS control |
|---|---|---|
| CPU pegged at moderate QPS | Excessive syscall churn (read/write per chunk) | sendfile/splice zero-copy, larger socket buffers |
| Connection accept backlog | Single-threaded accept bottleneck | SO_REUSEPORT multi-listener, ALB pre-connection scaling |
| epoll thundering herd | All workers wake on single event | EPOLLEXCLUSIVE or SO_REUSEPORT + per-core accept threads |
My recommendation: profile syscalls before rewriting in Rust. On AWS, ALB plus ENA instances with epoll edge-triggering handles most concurrency without kernel bypass.
Benchmark pattern (hypothetical workload): c6gn.2xlarge (ENA enhanced) running epoll edge-triggered server, 48K concurrent connections, 0.8ms p99 accept-to-response, sendfile zero-copy reduces CPU 22% vs buffered I/O at 12K req/sec.
Syscall and context switch tax
Each read/write crossing the user/kernel boundary costs CPU. Zero-copy sendfile() moves a file to a socket without user-space buffers, which suits static assets behind NGINX on EC2 (often superseded by CloudFront).
| Pattern | Benefit on AWS |
|---|---|
| epoll + non-blocking I/O | Few threads, many connections on c7g Graviton |
| SO_REUSEPORT | Accept queue spread across workers |
| sendfile | Static file serving from EBS |
Lambda avoids your epoll tuning: concurrency is execution-environment reuse with platform limits, and CPU-bound work still belongs on EC2/ECS.
AWS services map
| Need | Service | Skip when |
|---|---|---|
| High connection fan-in | ALB (L7) or NLB (L4) | <1K connections on single instance |
| Enhanced networking | ENA-enabled instance types (c6gn, m6in) | Bursty Lambda with no persistent connections |
| Kernel bypass alternatives | Consider Nitro + DPDK only at 10Gbps+ | Standard web API under 1Gbps |
When this advice breaks
- TLS termination on instance: crypto dominates, so offload to ALB/CloudFront first.
- Tiny payloads: syscall cost is noise next to business logic.
What to do this week
- Compare
nginx/envoyworker count to CPU cores on origin ASG. - Profile with
perf topduring a load test and look forsyscallhotspots. - Move static assets to S3+CloudFront; keep dynamic on epoll stack.
More in this track
Part of the Engineering Guides library (June 2026).
- Previous: Part 2
- Next: Part 4
- Browse tracks: Engineering Guides hub
What this guide doesn’t cover
CPU cache false sharing is covered in part 4 of this track.
Frequently asked questions
Should I rewrite the origin in Rust to fix syscall overhead?
Does Lambda need epoll tuning?
When is kernel bypass (DPDK) the wrong next step?
What could go wrong with a single accept thread?
AWS Cloud Architect & AI Expert
AWS-certified cloud architect and AI expert with deep expertise in cloud migrations, cost optimization, and generative AI on AWS.
