Why the Hardest Concept for Python Devs Is Concurrency: asyncio vs threading vs multiprocessing · Home&darr;<br>Skip to main content

Home

Table of Contents<br>Table of Contents

Introduction<br>In technology, something has been doubling every two years. If you think it&rsquo;s the number of transistors on a chip, also known as Moore&rsquo;s Law, think again. Despite incredible engineering efforts, scaling has slowed, and we have fallen behind Moore&rsquo;s schedule.<br>But people also had other ideas to make computers faster (pipelining, caches, out-of-order execution, branch prediction, speculative execution, SIMD, and many more). These techniques were designed to be hidden from the average developer, and for the most part, they still are. However, with threads and multicore processors, another Moore&rsquo;s Law started to emerge, this time in the amount of complexity and confusion developers face.

The concurrency lobby quickly discovered they could trade developer sanity for performance. So today we live in a world dominated by concurrency and parallelism, which are loosely defined as handling and doing many things at once. We can also think of concurrency as an umbrella term that includes parallelism.<br>Going back to the confusion levels, developers in high-level languages like Python have it worst. The important ideas surrounding concurrency already assume you understand some lower-level concepts: the operating system and its kernel, system calls, processes, threads, intra- and inter-process communication. Yet, you will also need higher-level Python concepts like coroutines and event loops, pickling data between processes, and the infamous Global Interpreter Lock, or GIL. The GIL gets a lot of hate, but it was probably a trade worth making. It made parallelism harder for Python users, but it kept CPython simple for the core developers who grew the language, and it kept single-threaded code fast.<br>This is a lot, but fear not. Today we are going to learn how to conquer concurrency. We will see how computers run our code, sequentially and in parallel, and when to use Python&rsquo;s asyncio, threading, and multiprocessing modules.<br>How the Operating System Works<br>The fundamental computing model is a sequential machine that runs one instruction at a time and understands only 0s and 1s. As users, our goal is to run programs on computer hardware: the CPU, memory, GPU, and input/output (I/O) devices like the disk, network card, keyboard, and mouse. But talking to hardware directly is hard, and a mistake can even cause damage. So there&rsquo;s another software layer between the user and the hardware, called the operating system.<br>At the heart of an operating system, there is a part that is closest to the hardware, called the kernel. Every time the user needs to interact with the hardware, the request goes through the kernel. The kernel is also just software. For example, the Linux kernel is written mostly in C (you&rsquo;ll also see some other languages on the Linux GitHub repository like Python, but Python in there is a collection of development infrastructure; the kernel itself does not embed or require Python). Operating systems built on the Linux kernel are called Linux distributions (distros for short) like Ubuntu, Debian, Arch, and Kali Linux. Apple and Microsoft operating systems like macOS and Windows have their own kernels, also written mostly in C and C++.

Telling users only to use the kernel to interact with the hardware is not enough; the users can still try to be sneaky. That&rsquo;s why, for protection, the CPUs operate in two execution modes: user mode and kernel mode. When the CPU runs in user mode, it can only execute instructions that can&rsquo;t do anything dangerous. This includes copying data, doing arithmetic and logical operations, and control flow. When the CPU runs in kernel mode, it can execute anything in its instruction set including things like managing memory and communicating with I/O devices.<br>To let users run privileged instructions, the kernel provides an API. The set of such functionality is called the system call interface. Making a system call changes the CPU into kernel mode, the kernel does the work, and control comes back to the user. Reading and writing data (read, write), and asking which connections are ready (select), are all system calls. So, in Python, every time you open a file or read from a socket, a system call is performed underneath.<br>With that, we are ready to talk about two of the most fundamental ideas in computing: processes and threads.<br>Processes vs. Threads<br>A program in execution is called a process. Whenever you click and launch an application, or you execute some code, the OS creates a process for you and assigns a process ID. Each process gets its own address space, divided into text, data, stack, and heap sections. Text holds your program&rsquo;s code. Data holds your global variables and constants. Stack and heap are where your local and dynamic data lives. A...