Concurrent Servers: Part 8 - Go - Eli Bendersky's website

Toggle navigation

Eli Bendersky's website

About

Projects

Archives

This is part 8 in a series of posts on writing concurrent network servers. In<br>this part, we'll switch to Go and see how it tackles the challenges described<br>earlier in the series.

All posts in the series:

Part 1 - Introduction

Part 2 - Threads

Part 3 - Event-driven

Part 4 - libuv

Part 5 - Redis case study

Part 6 - Callbacks, Promises and async/await

Part 7 - Rust

Part 8 - Go (this part)

This post assumes a basic familiarity with the Go programming language.

Sequential state machine server

As before, we'll start with a sequential server for the basic state machine<br>protocol presented in part 1.

This is the main function:

func main() {<br>port := "9090"<br>if len(os.Args) >= 2 {<br>port = os.Args[1]<br>log.Println("Serving on port", port)

listener, err := net.Listen("tcp", ":"+port)<br>if err != nil {<br>log.Fatal("Error listening:", err)<br>defer listener.Close()

for {<br>conn, err := listener.Accept()<br>if err != nil {<br>log.Printf("Error accepting connection: %v", err)<br>continue<br>log.Println("connection received from", conn.RemoteAddr())

if err := server.ServeSerialProtocol(conn); err != nil {<br>log.Printf("Error serving %v: %v", conn.RemoteAddr(), err)<br>} else {<br>log.Println("peer done", conn.RemoteAddr())

As in the previous parts, the server is "infinite"; it never stops serving new<br>connections until it's explicitly killed.

This is the function implementing the protocol for a single client; it takes<br>a net.Conn value that represents a socket with a client connected on the<br>other end:

type processingState int

const (<br>waitForMsg processingState = iota<br>inMsg

// ServeSerialProtocol serves our serial protocol to a single TCP connection.<br>func ServeSerialProtocol(conn net.Conn) error {<br>defer conn.Close()

if _, err := conn.Write([]byte{'*'}); err != nil {<br>return err

var state processingState = waitForMsg<br>buf := make([]byte, 1024)<br>for {<br>n, err := conn.Read(buf)<br>for _, b := range buf[:n] {<br>switch state {<br>case waitForMsg:<br>if b == '^' {<br>state = inMsg<br>case inMsg:<br>if b == '$' {<br>state = waitForMsg<br>} else {<br>var bb byte = byte(b) + 1<br>if _, err := conn.Write([]byte{bb}); err != nil {<br>return err

// Check error after processing the bytes received along with it.<br>if err != nil {<br>if errors.Is(err, io.EOF) || errors.Is(err, net.ErrClosed) {<br>return nil<br>} else {<br>return err

One goroutine per client

Rather than directly exposing OS threads, the Go runtime implements its own<br>M:N scheduling of lightweight goroutines on top of OS threads. Using<br>goroutines in Go is cheap - both in terms of syntax and developer effort, and in<br>terms of system resources.

Here's a version of our serial protocol server that serves clients concurrently<br>by launching a goroutine for each client. The part of the code that's different<br>from the previous sample is highlighted:

func main() {<br>port := "9090"<br>if len(os.Args) >= 2 {<br>port = os.Args[1]<br>log.Println("Serving on port", port)

listener, err := net.Listen("tcp", ":"+port)<br>if err != nil {<br>log.Fatal("Error listening:", err)<br>defer listener.Close()

for {<br>conn, err := listener.Accept()<br>if err != nil {<br>log.Printf("Error accepting connection: %v", err)<br>continue<br>log.Println("connection received from", conn.RemoteAddr())

go func() {<br>if err := server.ServeSerialProtocol(conn); err != nil {<br>log.Printf("Error serving %v: %v", conn.RemoteAddr(), err)<br>} else {<br>log.Println("peer done", conn.RemoteAddr())<br>}()

The concurrent modification in this case is particularly simple because the<br>server is infinite; there's no point waiting for these goroutines to finish<br>(and hence no need for a sync.WaitGroup).<br>The parameters for server.ServeSerialProtocol are lexically captured from<br>the enclosing scope and its return value is handled by the surrounding closure.

Because goroutines are very cheap, this server is very unlikely to run out<br>of resources due to launching too many goroutines; in fact, it will probably<br>run out of something else - like file descriptors for sockets - first. However,<br>sometimes it's still useful to limit the degree of concurrency - even<br>in Go, and we'll discuss some approaches to do so in the following sections.

Limiting concurrency with a semaphore

Here are some scenarios in which it makes sense to limit the degree of<br>concurrency in Go programs, even though goroutines are cheap to launch and<br>operate:

Tasks may be compute intensive, and the CPU capacity of any server is inherently<br>limited. If too many concurrent goroutines compete for limited CPUs, they<br>will all make very little progress. It may make more sense to have fewer tasks<br>that complete in a reasonable time.

Protecting potentially limited downstream resources, such as concurrent<br>DB connections or other services. For example, if the server has to send<br>requests to other services for each task, and these are rate-limited,<br>concurrency will have to be carefully managed.

Security reasons when work is dictated by clients; malicious...