C10K to C10M: from thread-per-connection model to event-driven architectures

The C10K problem is the challenge of optimizing network software to handle 10,000 concurrent client connections on a single server. Coined in 1999 by software engineer Dan Kegel, it became the defining scalability benchmark for modern web servers, load balancers, and network architecture. [1, 2, 3]

Why the Problem Exists

In the 1990s, web servers like older versions of Apache used a "thread-per-connection" model. Whenever a client connected, the operating system spawned a new thread or process to handle it. This caused servers to fail at 10,000 connections for two main reasons: [1, 2]

• Memory Exhaustion: Each OS thread required a significant chunk of memory (e.g., 512 KB to 2 MB for the thread stack). Supporting 10,000 connections required gigabytes of RAM just to manage the sleeping, waiting threads. [1]
• CPU Thrashing: With thousands of threads, the CPU spent most of its time "context switching" between them, leaving little compute power to execute application code. [1]

Architectural Solutions

To solve the C10K problem, the industry underwent a massive shift in two areas:

• Event-Driven Architecture: Servers moved away from thread-per-connection to single-threaded or multi-threaded event loops. Using non-blocking I/O, an application processes only active connections and ignores idle ones. [1, 2]
• Modern Kernel APIs: Older notification mechanisms like select and poll forced the kernel to scan all connections, scaling poorly (O(N)). Modern asynchronous APIs—such as epoll in Linux, kqueue in BSD/macOS, and IOCP in Windows—only alert the application to active connections (O(1) scaling). [1]

The Modern Landscape

Pioneering event-driven servers like Nginx and runtime environments like Node.js successfully solved the C10K problem. Today, a single commodity server can manage millions of concurrent connections, pushing the limit of scalability much further into the C10M (10 million connections) era. [1, 2, 3]

For a visual walkthrough of the evolution of server concurrency, from the traditional thread-per-connection model to modern event-driven architectures.

 

 

 

References 

• https://www.systemdesignhandbook.com/guides/c10k-problem/ 
• https://en.wikipedia.org/wiki/C10k_problem
• Dan Kegel: https://www.kegel.com/c10k.html 
• C10k problem: https://www.youtube.com/watch?v=ChYf-xV6B4o
• From C10K to C10M: The Evolution of Server Concurrency: https://www.youtube.com/watch?v=Udvw6-L8QCU 
• From C10K to io_uring: The Evolution of High-Performance Server Concurrency: https://www.youtube.com/watch?v=JR0vJeth_1k&t=218s
• https://www.linkedin.com/pulse/why-one-thread-per-connection-doesnt-scale-deekshith-b-t1rcc/
• https://medium.com/beyond-localhost/when-one-thread-per-connection-breaks-building-i-o-that-scales-to-millions-1af3e61fb14d