Concurrent Servers: Part 1 - Introduction - Eli Bendersky's website

Toggle navigation

Eli Bendersky's website

About

Projects

Archives

This is the first post in a series about concurrent network servers. My plan is<br>to examine several popular concurrency models for network servers that handle<br>multiple clients simultaneously, and judge those models on scalability and ease<br>of implementation. All servers will listen for socket connections and implement<br>a simple protocol to interact with clients.

All posts in the series:

Part 1 - Introduction

Part 2 - Threads

Part 3 - Event-driven

Part 4 - libuv

Part 5 - Redis case study

Part 6 - Callbacks, Promises and async/await

The protocol

The protocol used throughout this series is very simple, but should be<br>sufficient to demonstrate many interesting aspects of concurrent server design.<br>Notably, the protocol is stateful - the server changes internal state based on<br>the data clients send, and its behavior depends on that internal state.<br>Not all protocols all stateful - in fact, many protocols over HTTP these days<br>are stateless - but stateful protocols are sufficiently common to warrant a<br>serious discussion.

Here's the protocol, from the server's point of view:

Concurrent server protocol, state machine<br>In words: the server waits for a new client to connect; when a client connects,<br>the server sends it a * character and enters a "wait for message state". In<br>this state, the server ignores everything the client sends until it sees a ^<br>character that signals that a new message begins. At this point it moves to the<br>"in message" state, where it echoes back everything the client sends,<br>incrementing each byte [1]. When the client sends a $, the server goes back<br>to waiting for a new message. The ^ and $ characters are only used to<br>delimit messages - they are not echoed back.

An implicit arrow exists from each state back to the "wait for client" state, in<br>case the client disconnects. By corollary, the only way for a client to signal<br>"I'm done" is to simply close its side of the connection.

Obviously, this protocol is a simplification of more realistic protocols that<br>have complicated headers, escape sequences (to support $ inside a message<br>body, for example) and additional state transitions, but for our goals this will<br>do just fine.

Another note: this series is introductory, and assumes clients are generally<br>well behaved (albeit potentially slow); therefore there are no timeouts and no<br>special provisions made to ensure that the server doesn't end up being blocked<br>indefinitely by rogue (or buggy) clients.

A sequential server

Our first server in this series is a simple "sequential" server, written in C<br>without using any libraries beyond standard POSIX fare for sockets. The server<br>is sequential because it can only handle a single client at any given time; when<br>a client connects, the server enters the state machine shown above and won't<br>even listen on the socket for new clients until the current client is done.<br>Obviously this isn't concurrent and doesn't scale beyond very light loads, but<br>it's helpful to discuss since we need a simple-to-understand baseline.

The full code for this server is here;<br>in what follows, I'll focus on some highlights. The outer loop in main<br>listens on the socket for new clients to connect. Once a client connects, it<br>calls serve_connection which runs through the protocol until the client<br>disconnects.

To accept new connections, the sequential server calls accept on a listening<br>socket in a loop:

while (1) {<br>struct sockaddr_in peer_addr;<br>socklen_t peer_addr_len = sizeof(peer_addr);

int newsockfd =<br>accept(sockfd, (struct sockaddr*)&peer_addr, &peer_addr_len);

if (newsockfd 0) {<br>perror_die("ERROR on accept");

report_peer_connected(&peer_addr, peer_addr_len);<br>serve_connection(newsockfd);<br>printf("peer done\n");

Each time accept returns a new connected socket, the server calls<br>serve_connection; note that this is a blocking call - until<br>serve_connection returns, accept is not called again; the server blocks<br>until one client is done before accepting a new client. In other words, clients<br>are serviced sequentially.

Here's serve_connection:

typedef enum { WAIT_FOR_MSG, IN_MSG } ProcessingState;

void serve_connection(int sockfd) {<br>if (send(sockfd, "*", 1, 0) 1) {<br>perror_die("send");

ProcessingState state = WAIT_FOR_MSG;

while (1) {<br>uint8_t buf[1024];<br>int len = recv(sockfd, buf, sizeof buf, 0);<br>if (len 0) {<br>perror_die("recv");<br>} else if (len == 0) {<br>break;

for (int i = 0; i len; ++i) {<br>switch (state) {<br>case WAIT_FOR_MSG:<br>if (buf[i] == '^') {<br>state = IN_MSG;<br>break;<br>case IN_MSG:<br>if (buf[i] == '$') {<br>state = WAIT_FOR_MSG;<br>} else {<br>buf[i] += 1;<br>if (send(sockfd, &buf[i], 1, 0) 1) {<br>perror("send error");<br>close(sockfd);<br>return;<br>break;

close(sockfd);

It pretty much follows the protocol state machine. Each time around the loop,<br>the server attempts to receive data from the client. Receiving 0 bytes means the<br>client disconnected, and the loop exits....