Ch 10 — Creating Your Own Runtime
Creating Your Own Runtime In the last few chapters, we covered a lot of aspects that are relevant to asynchronous programming in Rust, but we did that by implementing alternative and simpler abstractions than what we have in Rust today. This last chapter will focus on bridging that gap by changing our runtime so that it works with Rust futures and async/await instead of our own futures and coroutine/wait. Since we’ve pretty much covered everything there is to know about coroutines, state machines, futures, wakers, runtimes, and pinning, adapting what we have now will be a relatively easy task. When we get everything working, we’ll do some experiments with our runtime to showcase and discuss some of the aspects that make asynchronous Rust somewhat difficult for newcomers today. We’ll also take some time to discuss what we might expect in the future with asynchronous Rust before we summarize what we’ve done and learned in this book. We’ll cover the following main topics:
• Creating our own runtime with futures and async/await • Experimenting with our runtime • Challenges with asynchronous Rust • The future of asynchronous Rust
Technical requirements The examples in this chapter will build on the code from the last chapter, so the requirements are the same. The example is cross-platform and will work on all platforms that Rust (https://doc.rustlang.org/beta/rustc/platform-support.html#tier-1-with-host-tools) and mio (https://github.com/tokio-rs/mio#platforms) support.
252 Creating Your Own Runtime
The only thing you need is Rust installed and the book’s repository downloaded locally. All the code
in this chapter can be found in the ch10 folder.
We’ll use delayserver in this example as well, so you need to open a separate terminal, enter the
delayserver folder at the root of the repository, and type cargo run so it’s ready and available
for the examples going forward.
Remember to change the ports in the code if for some reason you have to change what port delayserver
listens on.
Creating our own runtime with futures and async/await
Okay, so we’re in the home stretch; the last thing we’ll do is change our runtime so it uses the Rust
Future trait, Waker, and async/await. This will be a relatively easy task for us now that we’ve
pretty much covered the most complex aspects of asynchronous programming in Rust by building
everything up ourselves. We have even gone into quite some detail on the design decisions that Rust
had to make along the way.
The asynchronous programming model Rust has today is the result of an evolutionary process. Rust
started in its early stages with green threads, but this was before it reached version 1.0. At the point of
reaching version 1.0, Rust didn’t have the notion of futures or asynchronous operations in its standard
library at all. This space was explored on the side in the futures-rs crate (https://github.com/
rust-lang/futures-rs), which still serves as a nursery for async abstractions today. However,
it didn’t take long before Rust settled around a version of the Future trait similar to what we have
today, often referred to as futures 0.1. Supporting coroutines created by async/await was something
that was in the works already at that point but it took a few years before the design reached its final
stage and entered the stable version of the standard library.
So, many of the choices we had to make with our async implementation are real choices that Rust had
to make along the way. However, it all brings us to this point, so let’s get to it and start adapting our
runtime so it works with Rust futures.
Before we get to the example, let’s cover the things that are different from our current implementation:
• The Future trait Rust uses is slightly different from what we have now. The biggest difference
is that it takes something called Context instead of Waker. The other difference is that it
returns an enum called Poll instead of PollState.
• Context is a wrapper around Rust’s Waker type. Its only purpose is to future-proof the API
so it can hold additional data in the future without having to change anything related to Waker.
• The Poll enum returns one of two states, Ready(T) or Pending. This is slightly different
from what we have now with our PollState enum, but the two states mean the same as
Ready(T)/NotReady in our current implementation.
• Wakers in Rust is slightly more complex to create than what we’re used to with our current
Waker. We’ll go through how and why later in the chapter.
Technical requirements 253
Other than the differences outlined above, everything else can stay pretty much as is. For the most part, we’re renaming and refactoring this time. Now that we’ve got an idea of what we need to do, it’s time to set everything up so we can get our new example up and running.
Note Even though we create a runtime to run futures properly in Rust, we still try to keep this simple by avoiding error handling and not focusing on making our runtime more flexible. Improving our runtime is certainly possible, and while it can be a bit tricky at times to use the type system correctly and please the borrow checker, it has relatively little to do with async Rust and more to do with Rust being Rust.
Setting up our example
Tip You’ll find this example in the book’s repository in the ch10/a-rust-futures folder.
We’ll continue where we left off in the last chapter, so let’s copy everything we had over to a new project:
1. Create a new folder called a-rust-futures.
2. Copy everything from the example in the previous chapter. If you followed the naming I
suggested, it would be stored in the e-coroutines-pin folder.
3. You should now have a folder containing a copy of our previous example, so the last thing to
do is to change the project name in Cargo.toml to a-rust-futures.
Okay, so let’s start with the program we want to run. Open main.rs.
main.rs We’ll go back to the simplest version of our program and get it running before we try anything more complex. Open main.rs and replace all the code in that file with this:
ch10/a-rust-futures/src/main.rs mod http; mod runtime; use crate::http::Http;
254 Creating Your Own Runtime
fn main() {
let mut executor = runtime::init();
executor.block_on(async_main());
}
async fn async_main() {
println!("Program starting");
let txt = Http::get("/600/HelloAsyncAwait").await;
println!("{txt}");
let txt = Http::get("/400/HelloAsyncAwait").await;
println!("{txt}");
}
No need for corofy or anything special this time. The compiler will rewrite this for us.
Note
Notice that we’ve removed the declaration of the future module. That’s because we simply
don’t need it anymore. The only exception is if you want to retain and use the join_all
function we created to join multiple futures together. You can either try to rewrite that yourself
or take a look in the repository and locate the ch10/a-rust-futures-bonus/src/
future.rs file, where you’ll find the same version of our example, only this version retains
the future module with a join_all function that works with Rust futures.
future.rs
You can delete this file altogether as we don’t need our own Future trait anymore.
Let’s move right along to http.rs and see what we need to change there.
http.rs
The first thing we need to change is our dependencies. We’ll no longer rely on our own Future,
Waker, and PollState; instead, we’ll depend on Future, Context, and Poll from the standard
library. Our dependencies should look like this now:
ch10/a-rust-futures/src/http.rs
use crate::runtime::{self, reactor};
use mio::Interest;
use std::{
future::Future,
io::{ErrorKind, Read, Write},
Technical requirements 255
pin::Pin,
task::{Context, Poll},
};
We have to do some minor refactoring in the poll implementation for HttpGetFuture. First, we need to change the signature of the poll function so it complies with the new Future trait:
ch10/a-rust-futures/src/http.rs fn poll(mut self: Pin<&mut Self>, cx: &mut Context) -> Poll<Self::Output>
Since we named the new argument cx, we have to change what we pass in to set_waker with the following:
ch10/a-rust-futures/src/http.rs runtime::reactor().set_waker(cx, self.id);
Next, we need to change our future implementation so it returns Poll instead of PollState. To do that, locate the poll method and start by changing the signature so it matches the Future trait from the standard library:
ch10/a-rust-futures/src/http.rs fn poll(mut self: Pin<&mut Self>, cx: &mut Context) -> Poll<Self::Output>
Next, we need to change our return types wherever we return from the function (I’ve only presented the relevant part of the function body here):
ch10/a-rust-futures/src/http.rs
loop {
match self.stream.as_mut().unwrap().read(&mut buff) {
Ok(0) => {
let s = String::from_utf8_lossy(&self.buffer).
to_string();
runtime::reactor().deregister(self.stream.as_
mut().unwrap(), id);
break Poll::Ready(s.to_string());
}
Ok(n) => {
self.buffer.extend(&buff[0..n]);
256 Creating Your Own Runtime
continue;
}
Err(e) if e.kind() == ErrorKind::WouldBlock => {
// always store the last given Waker
runtime::reactor().set_waker(cx, self.id);
break Poll::Pending;
}
Err(e) => panic!("{e:?}"),
}
}
That’s it for this file. Not bad, huh? Let’s take a look at what we need to change in our executor and
open executor.rs.
executor.rs
The first thing we need to change in executor.rs is our dependencies. This time, we only rely on
types from the standard library, and our dependencies section should now look like this:
ch10/a-rust-futures/src/runtime/executor.rs
use std::{
cell::{Cell, RefCell},
collections::HashMap,
future::Future,
pin::Pin,
sync::{Arc, Mutex},
task::{Poll, Context, Wake, Waker},
thread::{self, Thread},
};
Our coroutines will no longer be limited to only output String, so we can safely use a more sensible
Output type for our top-level futures:
ch10/a-rust-futures/src/runtime/executor.rs
type Task = Pin<Box<dyn Future<Output = ()>>>;
The next thing we’ll dive straight into is Waker since the changes we make here will result in several
other changes to this file.
Technical requirements 257
Creating a waker in Rust can be quite a complex task since Rust wants to give us maximum flexibility on how we choose to implement wakers. The reason for this is twofold:
• Wakers must work just as well on a server as it does on a microcontroller • A waker must be a zero-cost abstraction
Realizing that most programmers never need to create their own wakers, the cost that the lack of ergonomics has was deemed acceptable. Until quite recently, the only way to construct a waker in Rust was to create something very similar to a trait object without being a trait object. To do so, you had to go through quite a complex process of constructing a v-table (a set of function pointers), combining that with a pointer to the data that the waker stored, and creating RawWaker. Fortunately, we don’t actually have to go through this process anymore as Rust now has the Wake trait. The Wake trait works if the Waker type we create is placed in Arc. Wrapping Waker in an Arc results in a heap allocation, but for most Waker implementations on the kind of systems we’re talking about in this book, that’s perfectly fine and what most production runtimes do. This simplifies things for us quite a bit.
Info This is an example of Rust adopting what turns out to be best practices from the ecosystem. For a long time, a popular way to construct wakers was by implementing a trait called ArcWake provided by the futures crate (https://github.com/rust-lang/futures-rs). The futures crate is not a part of the language but it’s in the rust-lang repository and can be viewed much like a toolbox and nursery for abstractions that might end up in the language at some point in the future.
To avoid confusion by having multiple things with the same name, let’s rename our concrete Waker type to MyWaker:
ch10/a-rust-futures/src/runtime/executor.rs
#[derive(Clone)]
pub struct MyWaker {
thread: Thread,
id: usize,
ready_queue: Arc<Mutex<Vec<usize>>>,
}
258 Creating Your Own Runtime
We can keep the implementation of wake pretty much the same, but we put it in the implementation
of the Wake trait instead of just having a wake function on MyWaker:
ch10/a-rust-futures/src/runtime/executor.rs
impl Wake for MyWaker {
fn wake(self: Arc<Self>) {
self.ready_queue
.lock()
.map(|mut q| q.push(self.id))
.unwrap();
self.thread.unpark();
}
}
You’ll notice that the wake function takes a self: Arc<Self> argument, much like we saw when
working with the Pin type. Writing the function signature this way means that wake is only callable
on MyWaker instances that are wrapped in Arc.
Since our waker has changed slightly, there are a few places we need to make some minor corrections.
The first is in the get_waker function:
ch10/a-rust-futures/src/runtime/executor.rs
fn get_waker(&self, id: usize) -> Arc<MyWaker> {
Arc::new(MyWaker {
id,
thread: thread::current(),
ready_queue: CURRENT_EXEC.with(|q| q.ready_queue.clone()),
})
}
So, not a big change here. The only difference is that we heap-allocate the waker by placing it in Arc.
The next place we need to make a change is in the block_on function.
First, we need to change its signature so that it matches our new definition of a top-level future:
ch10/a-rust-futures/src/runtime/executor.rs
pub fn block_on<F>(&mut self, future: F)
where
F: Future<Output = ()> + 'static,
{
Technical requirements 259
The next step is to change how we create a waker and wrap it in a Context struct in the block_ on function:
ch10/a-rust-futures/src/runtime/executor.rs // guard against false wakeups None => continue, };
let waker: Waker = self.get_waker(id).into(); let mut cx = Context::from_waker(&waker);
match future.as_mut().poll(&mut cx) {
This change is a little bit complex, so we’ll go through it step by step:
1. First, we get Arc<MyWaker> by calling the get_waker function just like we did before.
2. We convert MyWaker into a simple Waker by specifying the type we expect with let
waker: Waker and calling into() on MyWaker. Since every instance of MyWaker is
also a kind of Waker, this will convert it into the Waker type that’s defined in the standard
library, which is just what we need.
3. Since Future::poll expects Context and not Waker, we create a new Context struct
with a reference to the waker we just created.
The last place we need to make changes is to the signature of our spawn function so that it takes the new definition of top-level futures as well:
ch10/a-rust-futures/src/runtime/executor.rs pub fn spawn<F>(future: F) where F: Future<Output = ()> + 'static,
That was the last thing we needed to change in our executor, and we’re almost done. The last change we need to make to our runtime is in the reactor, so let’s go ahead and open reactor.rs.
reactor.rs The first thing we do is to make sure our dependencies are correct. We have to remove the dependency on our old Waker implementation and instead pull in these types from the standard library. The dependencies section should look like this:
260 Creating Your Own Runtime
ch10/a-rust-futures/src/runtime/reactor.rs
use mio::{net::TcpStream, Events, Interest, Poll, Registry, Token};
use std::{
collections::HashMap,
sync::{
atomic::{AtomicUsize, Ordering},
Arc, Mutex, OnceLock,
},
thread, task::{Context, Waker},
};
There are two minor changes we need to make. The first one is that our set_waker function now
accepts Context from which it needs to get a Waker object:
ch10/a-rust-futures/src/runtime/reactor.rs
pub fn set_waker(&self, cx: &Context, id: usize) {
let _ = self
.wakers
.lock()
.map(|mut w| w.insert(id, cx.waker().clone()).is_none())
.unwrap();
}
The last change is that we need to call a slightly different method when calling wake in the event_
loop function:
ch10/a-rust-futures/src/runtime/reactor.rs
if let Some(waker) = wakers.get(&id) {
waker.wake_by_ref();
}
Since calling wake now consumes self, we call the version that takes &self instead since we want
to hold on to that waker for later.
That’s it. Our runtime can now run and take advantage of the full power of asynchronous Rust. Let’s
try it out by typing cargo run in the terminal.
Experimenting with our runtime 261
We should get the same output as we’ve seen before:
Program starting FIRST POLL - START OPERATION main: 1 pending tasks. Sleep until notified. HTTP/1.1 200 OK content-length: 15 [==== ABBREVIATED ====] HelloAsyncAwait main: All tasks are finished
That’s pretty neat, isn’t it? So, now we have created our own async runtime that uses Rust’s Future, Waker, Context, and async/await. Now that we can pride ourselves on being runtime implementors, it’s time to do some experiments. I’ll choose a few that will also teach us a few things about runtimes and futures in Rust. We’re not done learning just yet.
Experimenting with our runtime
Note You’ll find this example in the book’s repository in the ch10/b-rust-futuresexperiments folder. The different experiments will be implemented as different versions of the async_main function numbered chronologically. I’ll indicate which function corresponds with which function in the repository example in the heading of the code snippet.
Before we start experimenting, let’s copy everything we have now to a new folder:
1. Create a new folder called b-rust-futures-experiments. 2. Copy everything from the a-rust-futures folder to the new folder. 3. Open Cargo.toml and change the name attribute to b-rust-futures-experiments.
The first experiment will be to exchange our very limited HTTP client with a proper one. The easiest way to do that is to simply pick another production-quality HTTP client library that supports async Rust and use that instead. So, when trying to find a suitable replacement for our HTTP client, we check the list of the most popular high-level HTTP client libraries and find reqwest at the top. That might work for our purposes, so let’s try that first.
262 Creating Your Own Runtime
The first thing we do is add reqwest as a dependency in Cargo.toml by typing the following:
cargo add reqwest@0.11
Next, let’s change our async_main function so we use reqwest instead of our own HTTP client:
ch10/b-rust-futures-examples/src/main.rs (async_main2)
async fn async_main() {
println!("Program starting");
let url = "http://127.0.0.1:8080/600/HelloAsyncAwait1";
let res = reqwest::get(url).await.unwrap();
let txt = res.text().await.unwrap();
println!("{txt}");
let url = "http://127.0.0.1:8080/400/HelloAsyncAwait2";
let res = reqwest::get(url).await.unwrap();
let txt = res.text().await.unwrap();
println!("{txt}");
}
Besides using the reqwest API, I also changed the message we send. Most HTTP clients don’t
return the raw HTTP response to us and usually only provide a convenient way to get the body of the
response, which up until now was similar for both our requests.
That should be all we need to change, so let’s try to run our program by writing cargo run:
Running `target\debug\a-rust-futures.exe`
Program starting
thread 'main' panicked at C:\Users\cf\.cargo\registry\src\index.
crates.io-6f17d22bba15001f\tokio-1.35.0\src\net\tcp\stream.rs:160:18:
there is no reactor running, must be called from the context of a
Tokio 1.x runtime
Okay, so the error tells us that there is no reactor running and that it must be called from the context
of a Tokio 1.x runtime. Well, we know there is a reactor running, just not the one reqwest expects,
so let’s see how we can fix this.
We obviously need to add Tokio to our program, and since Tokio is heavily feature-gated (meaning
that it has very few features enabled by default), we’ll make it easy on ourselves and enable all of them:
cargo add tokio@1 --features full
According to the documentation, we need to start a Tokio runtime and explicitly enter it to enable
the reactor. The enter function will return EnterGuard to us that we can hold on to it as long as
we need the reactor up and running.
Experimenting with our runtime 263
Adding this to the top of our async_main function should work:
ch10/b-rust-futures-examples/src/main.rs (async_main2)
use tokio::runtime::Runtime;
async fn async_main
let rt = Runtime::new().unwrap();
let _guard = rt.enter();
println!("Program starting");
let url = "http://127.0.0.1:8080/600/HelloAsyncAwait1";
Note Calling Runtime::new creates a multithreaded Tokio runtime, but Tokio also has a singlethreaded runtime that you can create by using the runtime builder like this: Builder::new_ current_thread().enable_all().build().unwrap(). If you do that, you end up with a peculiar problem: a deadlock. The reason for that is interesting and one that you should know about. Tokio’s single-threaded runtime uses only the thread it’s called on for both the executor and the reactor. This is very similar to what we did in the first version of our runtime in Chapter 8. We used the Poll instance to park our executor directly. When both our reactor and executor execute on the same thread, they must have the same mechanism to park themselves and wait for new events, which means there will be a tight coupling between them. When handling an event, the reactor has to wake up first to call Waker::wake, but the executor is the last one to park the thread. If the executor parked itself by calling thread::park (like we do), the reactor is parked as well and will never wake up since they’re running on the same thread. The only way for this to work is that the executor parks on something shared with the reactor (like we did with Poll). Since we’re not tightly integrated with Tokio, all we get is a deadlock.
Now, if we try to run our program once more, we get the following output:
Program starting main: 1 pending tasks. Sleep until notified. main: 1 pending tasks. Sleep until notified. main: 1 pending tasks. Sleep until notified. HelloAsyncAwait1 main: 1 pending tasks. Sleep until notified. main: 1 pending tasks. Sleep until notified. main: 1 pending tasks. Sleep until notified. HelloAsyncAwait2 main: All tasks are finished
264 Creating Your Own Runtime
Okay, so now everything works as expected. The only difference is that we get woken up a few extra
times, but the program finishes and produces the expected result.
Before we discuss what we just witnessed, let’s do one more experiment.
Isahc is an HTTP client library that promises to be executor agnostic, meaning that it doesn’t rely on
any specific executor. Let’s put that to the test.
First, we add a dependency on isahc by typing the following:
cargo add isahc@1.7
Then, we rewrite our main function so it looks like this:
ch10/b-rust-futures-examples/src/main.rs (async_main3)
use isahc::prelude::*;
async fn async_main() {
println!("Program starting");
let url = "http://127.0.0.1:8080/600/HelloAsyncAwait1";
let mut res = isahc::get_async(url).await.unwrap();
let txt = res.text().await.unwrap();
println!("{txt}");
let url = "http://127.0.0.1:8080/400/HelloAsyncAwait2";
let mut res = isahc::get_async(url).await.unwrap();
let txt = res.text().await.unwrap();
println!("{txt}");
}
Now, if we run our program by writing cargo run, we get the following output:
Program starting
main: 1 pending tasks. Sleep until notified.
main: 1 pending tasks. Sleep until notified.
main: 1 pending tasks. Sleep until notified.
HelloAsyncAwait1
main: 1 pending tasks. Sleep until notified.
main: 1 pending tasks. Sleep until notified.
main: 1 pending tasks. Sleep until notified.
HelloAsyncAwait2
main: All tasks are finished
So, we get the expected output without having to jump through any hoops.
Challenges with asynchronous Rust 265
Why does all this have to be so unintuitive? The answer to that brings us to the topic of common challenges that we all face when programming with async Rust, so let’s cover some of the most noticeable ones and explain the reason they exist so we can figure out how to best deal with them.
Challenges with asynchronous Rust So, while we’ve seen with our own eyes that the executor and reactor could be loosely coupled, which in turn means that you could in theory mix and match reactors and executors, the question is why do we encounter so much friction when trying to do just that? Most programmers that have used async Rust have experienced problems caused by incompatible async libraries, and we saw an example of the kind of error message you would get previously. To understand this, we have to dive a little bit deeper into the existing async runtimes in Rust, specifically those we typically use for desktop and server applications.
Explicit versus implicit reactor instantiation
Info The type of future we’ll talk about going forward is leaf futures, the kind that actually represents an I/O operation (for example, HttpGetFuture).
When you create a runtime in Rust, you also need to create non-blocking primitives of the Rust standard library. Mutexes, channels, timers, TcpStreams, and so on are all things that need an async equivalent. Most of these can be implemented as different kinds of reactors, but the question that then comes up is: how is that reactor started? In both our own runtime and in Tokio, the reactor is started as part of the runtime initialization. We have a runtime::init() function that calls reactor::start(), and Tokio has a Runtime::new() and Runtime::enter() function. If we try to create a leaf future (the only one we created ourselves is HttpGetFuture) without the reactor started, both our runtime and Tokio will panic. The reactor has to be instantiated explicitly. Isahc, on the other hand, brings its own kind of reactor. Isahc is built on libcurl, a highly portable C library for multiprotocol file transfer. The thing that’s relevant for us, however, is that libcurl accepts a callback that is called when an operation is ready. So, Isahc passes the waker it receives to this callback and makes sure that Waker::wake is called when the callback is executed. This is a bit oversimplified, but it’s essentially what happens.
266 Creating Your Own Runtime
In practice, that means that Isahc brings its own reactor since it comes with the machinery to store
wakers and call wake on them when an operation is ready. The reactor is started implicitly.
Incidentally, this is also one of the major differences between async_std and Tokio. Tokio requires
explicit instantiation, and async_std relies on implicit instantiation.
I’m not going into so much detail on this just for fun; while this seems like a minor difference, it has
a rather big impact on how intuitive asynchronous programming in Rust is.
This problem mostly arises when you start programming using a different runtime than Tokio and
then have to use a library that internally relies on a Tokio reactor being present.
Since you can’t have two Tokio instances running on the same thread, the library can’t implicitly start
a Tokio reactor. Instead, what often happens is that you try to use that library and get an error like
we did in the preceding example.
Now, you have to solve this by starting a Tokio reactor yourself, use some kind of compatibility wrapper
created by someone else, or seeing whether the runtime you use has a built-in mechanism for running
futures that rely on a Tokio reactor being present.
For most people who don’t know about reactors, executors, and different kinds of leaf futures, this
can be quite unintuitive and cause quite a bit of frustration.
Note
The problem we describe here is quite common, and it’s not helped by the fact that async libraries
rarely explain this well or even try to be explicit about what kind of runtime they use. Some
libraries might only mention that they’re built on top of Tokio somewhere in the README file,
and some might simply state that they’re built on top of Hyper, for example, assuming that you
know that Hyper is built on top of Tokio (at least by default).
But now, you know that you should check this to avoid any surprises, and if you encounter this
issue, you know exactly what the problem is.
Ergonomics versus efficiency and flexibility
Rust is good at being ergonomic and efficient, and that almost makes it difficult to remember that
when Rust is faced with the choice between being efficient or ergonomic, it will choose to be efficient.
Many of the most popular crates in the ecosystem echo these values, and that includes async runtimes.
Some tasks can be more efficient if they’re tightly integrated with the executor, and therefore, if you
use them in your library, you will be dependent on that specific runtime.
Let’s take timers as an example, but task notifications where Task A notifies Task B that it can continue
is another example with some of the same trade-offs.
Challenges with asynchronous Rust 267
Tasks We’ve used the terms tasks and futures without making the difference explicitly clear, so let’s clear that up here. We first covered tasks in Chapter 1, and they still retain the same general meaning, but when talking about runtimes in Rust, they have a more specific definition. A task is a top-level future, the one that we spawn onto our executor. The executor schedules between different tasks. Tasks in a runtime in many ways represent the same abstraction that threads do in an OS. Every task is a future in Rust, but every future is not a task by this definition.
You can think of thread::sleep as a timer, and we often need something like this in an asynchronous context, so our asynchronous runtime will therefore need to have a sleep equivalent that tells the executor to park this task for a specified duration. We could implement this as a reactor and have separate OS-thread sleep for a specified duration and then wake the correct Waker. That would be simple and executor agnostic since the executor is oblivious to what happens and only concern itself with scheduling the task when Waker::wake is called. However, it’s also not optimally efficient for all workloads (even if we used the same thread for all timers). Another, and more common, way to solve this is to delegate this task to the executor. In our runtime, this could be done by having the executor store an ordered list of instants and a corresponding Waker, which is used to determine whether any timers have expired before it calls thread::park. If none have expired, we can calculate the duration until the next timer expires and use something such as thread::park_timeout to make sure that we at least wake up to handle that timer. The algorithms used to store the timers can be heavily optimized and you avoid the need for one extra thread just for timers with the additional overhead of synchronization between these threads just to signal that a timer has expired. In a multithreaded runtime, there might even be contention when multiple executors frequently add timers to the same reactor. Some timers are implemented reactor-style as separate libraries, and for many tasks, that will suffice. The important point here is that by using the defaults, you end up being tied to one specific runtime, and you have to make careful considerations if you want to avoid your library being tightly coupled to a specific runtime.
Common traits that everyone agrees about The last topic that causes friction in async Rust is the lack of universally agreed-upon traits and interfaces for typical async operations. I want to preface this segment by pointing out that this is one area that’s improving day by day, and there is a nursery for the traits and abstractions for asynchronous Rust in the futures-rs crate (https://github.com/rust-lang/futures-rs). However, since it’s still early days for async Rust, it’s something worth mentioning in a book like this.
268 Creating Your Own Runtime
Let’s take spawning as an example. When you write a high-level async library in Rust, such as a web
server, you’ll likely want to be able to spawn new tasks (top-level futures). For example, each connection
to the server will most likely be a new task that you want to spawn onto the executor.
Now, spawning is specific to each executor, and Rust doesn’t have a trait that defines how to spawn a
task. There is a trait suggested for spawning in the future-rs crate, but creating a spawn trait that
is both zero-cost and flexible enough to support all kinds of runtimes turns out to be very difficult.
There are ways around this. The popular HTTP library Hyper (https://hyper.rs/), for
example, uses a trait to represent the executor and internally uses that to spawn new tasks. This
makes it possible for users to implement this trait for a different executor and hand it back to Hyper.
By implementing this trait for a different executor, Hyper will use a different spawner than its default
option (which is the one in Tokio’s executor). Here is an example of how this is used for async_std
with Hyper: https://github.com/async-rs/async-std-hyper.
However, since there is no universal way of making this work, most libraries that rely on executor-
specific functionality do one of two things:
1. Choose a runtime and stick with it.
2. Implement two versions of the library supporting different popular runtimes that users choose
by enabling the correct features.
Async drop
Async drop, or async destructors, is an aspect of async Rust that’s somewhat unresolved at the time
of writing this book. Rust uses a pattern called RAII, which means that when a type is created, so are
its resources, and when a type is dropped, the resources are freed as well. The compiler automatically
inserts a call to drop on objects when they go out of scope.
If we take our runtime as an example, when resources are dropped, they do so in a blocking manner.
This is normally not a big problem since a drop likely won’t block the executor for too long, but it
isn’t always so.
If we have a drop implementation that takes a long time to finish (for example, if the drop needs to
manage I/O, or makes a blocking call to the OS kernel, which is perfectly legal and sometimes even
unavoidable in Rust), it can potentially block the executor. So, an async drop would somehow be able
to yield to the scheduler in such cases, and this is not possible at the moment.
Now, this isn’t a rough edge of async Rust you’re likely to encounter as a user of async libraries, but
it’s worth knowing about since right now, the only way to make sure this doesn’t cause issues is to be
careful what you put in the drop implementation for types that are used in an async context.
So, while this is not an extensive list of everything that causes friction in async Rust, it’s some of the
points I find most noticeable and worth knowing about.
The future of asynchronous Rust 269
Before we round off this chapter, let’s spend a little time talking about what we should expect in the future when it comes to asynchronous programming in Rust.
The future of asynchronous Rust Some of the things that make async Rust different from other languages are unavoidable. Asynchronous Rust is very efficient, has low latency, and is backed by a very strong type system due to how the language is designed and its core values. However, much of the perceived complexity today has more to do with the ecosystem and the kind of issues that result from a lot of programmers having to agree on the best way to solve different problems without any formal structure. The ecosystem gets fragmented for a while, and together with the fact that asynchronous programming is a topic that’s difficult for a lot of programmers, it ends up adding to the cognitive load associated with asynchronous Rust. All the issues and pain points I’ve mentioned in this chapter are constantly getting better. Some points that would have been on this list a few years ago are not even worth mentioning today. More and more common traits and abstractions will end up in the standard library, making async Rust more ergonomic since everything that uses them will “just work.” As different experiments and designs gain more traction than others, they become the de facto standard, and even though you will still have a lot of choices when programming asynchronous Rust, there will be certain paths to choose that cause a minimal amount of friction for those that want something that “just works.” With enough knowledge about asynchronous Rust and asynchronous programming in general, the issues I’ve mentioned here are, after all, relatively minor, and since you know more about asynchronous Rust than most programmers, I have a hard time imagining that any of these issues will cause you a lot of trouble. That doesn’t mean it’s not something worth knowing about since chances are your fellow programmers will struggle with some of these issues at some point.
Summary So, in this chapter, we did two things. First, we made some rather minor changes to our runtime so it works as an actual runtime for Rust futures. We tested the runtime using two external HTTP client libraries to learn a thing or two about reactors, runtimes, and async libraries in Rust. The next thing we did was to discuss some of the things that make asynchronous Rust difficult for many programmers coming from other languages. In the end, we also talked about what to expect going forward.
270 Creating Your Own Runtime
Depending on how you’ve followed along and how much you’ve experimented with the examples
we created along the way, it’s up to you what project to take on yourself if you want to learn more.
There is an important aspect of learning that only happens when you experiment on your own. Pick
everything apart, see what breaks, and how to fix it. Improve the simple runtime we created to learn
new stuff.
There are enough interesting projects to pick from, but here are some suggestions:
• Change out the parker implementation where we used thread::park with a proper parker.
You can choose one from a library or create a parker yourself (I added a small bonus at the end
of the ch10 folder called parker-bonus where you get a simple parker implementation).
• Implement a simple delayserver using the runtime you’ve created yourself. To do this,
you have to be able to write some raw HTTP responses and create a simple server. If you went
through the free introductory book called The Rust Programming Language, you created a simple
server in one of the last chapters (https://doc.rust-lang.org/book/ch20-02-
multithreaded.html), which gives you the basics you need. You also need to create a
timer as we discussed above or use an existing crate for async timers.
• You can create a “proper” multithreaded runtime and explore the possibilities that come with
having a global task queue, or as an alternative, implement a work-stealing scheduler that can
steal tasks from other executors’ local queues when they’re done with their own.
Only your imagination sets the limits on what you can do. The important thing to note is that there is
a certain joy in doing something just because you can and just for fun, and I hope that you get some
of the same enjoyment from this as I do.
I’ll end this chapter with a few words on how to make your life as an asynchronous programmer as
easy as possible.
The first thing is to realize that an async runtime is not just another library that you use. It’s extremely
invasive and impacts almost everything in your program. It’s a layer that rewrites, schedules tasks,
and reorders the program flow from what you’re used to.
My clear recommendation if you’re not specifically into learning about runtimes, or have very specific
needs, is to pick one runtime and stick to it for a while. Learn everything about it – not necessarily
everything from the start, but as you need more and more functionality from it, you will learn everything
eventually. This is almost like getting comfortable with everything in Rust’s standard library.
Summary 271
What runtime you start with depends a bit on what crates you’re using the most. Smol and asyncstd share a lot of implementation details and will behave similarly. Their big selling point is that their API strives to stay as close as possible to the standard library. Combined with the fact that the reactors are instantiated implicitly, this can result in a slightly more intuitive experience and a more gentle learning curve. Both are production-quality runtimes and see a lot of use. Smol was originally created with the goal of having a code base that’s easy for programmers to understand and learn from, which I think is true today as well. With that said, the most popular alternative for users looking for a general-purpose runtime at the time of writing is Tokio (https://tokio.rs/). Tokio is one of the oldest async runtimes in Rust. It is actively developed and has a welcoming and active community. The documentation is excellent. Being one of the most popular runtimes also means there is a good chance that you’ll find a library that does exactly what you need with support for Tokio out of the box. Personally, I tend to reach for Tokio for the reasons mentioned, but you can’t really go wrong with either of these runtimes unless you have very specific requirements. Finally, let’s not forget to mention the futures-rs crate (https://github.com/rustlang/futures-rs). I mentioned this crate earlier, but it’s really useful to know about as it contains several traits, abstractions, and executors (https://docs.rs/futures/latest/futures/ executor/index.html) for async Rust. It serves the purpose of an async toolbox that comes in handy in many situations.
272 Creating Your Own Runtime
Epilogue
So, you have reached the end. First of all, congratulations! You’ve come to the end of quite a journey!
We started by talking about concurrency and parallelism in Chapter 1. We even covered a bit about
the history, CPUs and OSs, hardware, and interrupts. In Chapter 2, we discussed how programming
languages modeled asynchronous program flow. We introduced coroutines and how stackful and
stackless coroutines differ. We discussed OS threads, fibers/green threads, and callbacks and their
pros and cons.
Then, in Chapter 3, we took a look at OS-backed event queues such as epoll, kqueue, and IOCP.
We even took quite a deep dive into syscalls and cross-platform abstractions.
In Chapter 4, we hit some quite difficult terrain when implementing our own mio-like event queue using
epoll. We even had to learn about the difference between edge-triggered and level-triggered events.
If Chapter 4 was somewhat rough terrain, Chapter 5 was more like climbing Mount Everest. No one
expects you to remember everything covered there, but you read through it and have a working
example you can use to experiment with. We implemented our own fibers/green threads, and while
doing so, we learned a little bit about processor architectures, ISAs, ABIs, and calling conventions.
We even learned quite a bit about inline assembly in Rust. If you ever felt insecure about the stack
versus heap difference, you surely understand it now that you’ve created stacks that we made our
CPU jump to ourselves.
In Chapter 6, we got a high-level introduction to asynchronous Rust, before we took a deep dive from
Chapter 7 and onward, starting with creating our own coroutines and our own coroutine/wait
syntax. In Chapter 8, we created the first versions of our own runtime while discussing basic runtime
design. We also deep-dived into reactors, executors, and wakers.
In Chapter 9, we improved our runtime and discovered the dangers of self-referential structs in Rust.
We then took a thorough look at pinning in Rust and how that helped us solve the problems we got into.
Finally, in Chapter 10, we saw that by making some rather minor changes, our runtime became a fully
functioning runtime for Rust futures. We rounded everything off by discussing some well-known
challenges with asynchronous Rust and some expectations for the future.
The Rust community is very inclusive and welcoming, and we’d happily welcome you to engage and
contribute if you find this topic interesting and want to learn more. One of the ways asynchronous Rust
gets better is through contributions by people with all levels of experience. If you want to get involved,
then the async work group (https://rust-lang.github.io/wg-async/welcome.html)
is a good place to start. There is also a very active community centered around the Tokio project
(https://github.com/tokio-rs/tokio/blob/master/CONTRIBUTING.md), and
many, many more depending on what specific area you want to dive deeper into. Don’t be afraid to
join the different channels and ask questions.
Epilogue 273
Now that we’re at the end I want to thank you for reading all the way to the end. I wanted this book to feel like a journey we took together, not like a lecture. I wanted you to be the focus, not me. I hope I succeeded with that, and I genuinely hope that you learned something that you find useful and can take with you going forward. If you did, then I’m sincerely happy that my work was of value to you. I wish you the best of luck with your asynchronous programming going forward. Until the next time! Carl Fredrik