Async: What is blocking? – Alice Ryhl

The async/await feature in Rust is implemented using a mechanism known as cooperative<br>scheduling, and this has some important consequences for people who write asynchronous<br>Rust code.

The intended audience of this blog post is new users of async Rust. I will be using the<br>Tokio runtime for the examples, but the points raised here apply to any asynchronous<br>runtime.

If you remember only one thing from this article, this should be it:

Async code should never spend a long time without reaching an .await.

Translations: chinese

Blocking vs. non-blocking code

The naive way to write an application that works on many things at the same time is to<br>spawn a new thread for every task. If the number of tasks is small, this is a perfectly<br>fine solution, but as the number of tasks becomes large, you will eventually run into<br>problems due to the large number of threads. There are various solutions to this problem<br>in different programming languages, but they all boil down to the same thing: very<br>quickly swap out the currently running task on each thread, such that all of the tasks<br>get an opportunity to run. In Rust, this swapping happens when you .await something.

When writing async Rust, the phrase “blocking the thread” means “preventing the runtime<br>from swapping the current task”. This can be a major issue because it means that other<br>tasks on the same runtime will stop running until the thread is no longer being blocked.<br>To prevent this, we should write code that can be swapped quickly, which you do by never<br>spending a long time away from an .await.

Let's take an example:

▶︎<br>use std::time::Duration;

#[tokio::main]<br>async fn main() {<br>println!("Hello World!");

// No .await here!<br>std::thread::sleep(Duration::from_secs(5));

println!("Five seconds later...");

The above code looks correct, and if you run it, it will appear to work. But it has a<br>fatal flaw: it is blocking the thread. In this case, there are no other tasks, so it's<br>not a problem, but this wont be the case in real programs. To illustrate this point,<br>consider the following example:

▶︎<br>use std::time::Duration;

async fn sleep_then_print(timer: i32) {<br>println!("Start timer {}.", timer);

// No .await here!<br>std::thread::sleep(Duration::from_secs(1));

println!("Timer {} done.", timer);

#[tokio::main]<br>async fn main() {<br>// The join! macro lets you run multiple things concurrently.<br>tokio::join!(<br>sleep_then_print(1),<br>sleep_then_print(2),<br>sleep_then_print(3),<br>);

Start timer 1.<br>Timer 1 done.<br>Start timer 2.<br>Timer 2 done.<br>Start timer 3.<br>Timer 3 done.

The example will take three seconds to run, and the timers will run one after the other<br>with no concurrency whatsoever. The reason is simple: the Tokio runtime was not able to<br>swap one task for another, because such a swap can only happen at an .await. Since<br>there is no .await in sleep_then_print, no swapping can happen while it is running.

However if we instead use Tokio's sleep function, which uses an .await to sleep,<br>the function will behave correctly:

▶︎<br>use tokio::time::Duration;

async fn sleep_then_print(timer: i32) {<br>println!("Start timer {}.", timer);

tokio::time::sleep(Duration::from_secs(1)).await;<br>// ^ execution can be paused here

println!("Timer {} done.", timer);

#[tokio::main]<br>async fn main() {<br>// The join! macro lets you run multiple things concurrently.<br>tokio::join!(<br>sleep_then_print(1),<br>sleep_then_print(2),<br>sleep_then_print(3),<br>);

Start timer 1.<br>Start timer 2.<br>Start timer 3.<br>Timer 1 done.<br>Timer 2 done.<br>Timer 3 done.

The code runs in just one second, and properly runs all three functions at the same time<br>as desired.

Be aware that it is not always this obvious. By using tokio::join!, all three tasks<br>are guaranteed to run on the same thread, but if you replace it with tokio::spawn and<br>use a multi-threaded runtime, you will be able to run multiple blocking tasks until you<br>run out of threads. The default Tokio runtime spawns one thread per CPU core, and you<br>will typically have around 8 CPU cores. This is enough that you can miss the issue when<br>testing locally, but sufficiently few that you will very quickly run out of threads when<br>running the code for real.

To give a sense of scale of how much time is too much, a good rule of thumb is no more<br>than 10 to 100 microseconds between each .await. That said, this depends on the kind of<br>application you are writing.

What if I want to block?

Sometimes we just want to block the thread. This is completely normal. There are two<br>common reasons for this:

Expensive CPU-bound computation.

Synchronous IO.

In both cases, we are dealing with an operation that prevents the task from reaching an<br>.await for an extended period of time. To solve this issue, we must move the blocking<br>operation to a thread outside of Tokio's thread pool. There are three variations on<br>this:

Use the tokio::task::spawn_blocking function.

Use the rayon crate.

Spawn a dedicated thread with std::thread::spawn.

Let us go through each solution to...