The buzzword of the 2015 edition of Rust was stability because version 1.0 promised to be compatible with the versions that followed.

The buzzword of the 2018 edition of Rust was productivity because version 1.31 offered a mature ecosystem of tools that allowed command-line developers for desktop operating systems (Linux, Windows, macOS) to be more productive.

There is an intent to have a new Rust edition in the coming years, but for this edition, neither its release date, nor its features, nor its buzzword is defined yet.

However, after the release of the 2018 edition, several needs of Rust developers are being targeted by Rust ecosystem developers around the world. It is probable that the new buzzword will come out of one of these development lines.

The most interesting lines of development are as follows:

• Integrated Development Environments (IDEs) and interactive programming
• Crate maturity
• Asynchronous programming
• Optimization
• Embedded systems

By the end of this chapter, we will see the most probable developments of the Rust ecosystem: the language, the tooling, and the available libraries. You will learn what to expect in the next few years.

Two of the most exciting new features of the Rust language are the asynchronous programming paradigm and the const generics language feature. At the end of 2019, the former was already added to the language, while the latter was still under development. This will be explained in this chapter using code examples, and so you will get a working knowledge about them.

IDEs and interactive programming

A lot of developers prefer to work inside a graphical application that contains or orchestrates all the development tools, instead of using terminal command lines. Such graphical applications are usually named Development Environments—or DEs for short.

At present, the most popular IDEs are probably the following ones:

• Eclipse: This is used mainly for development in the Java language.
• Visual Studio: This is used mainly for development in the C# and Visual Basic languages.
• Visual Studio Code: This is used mainly for development in the JavaScript language.

In the 20th century, it was typical to create an IDE from scratch for a single programming language. That was a major task, though. Therefore, in the last decades, it has become more typical to create customizable IDEs, and then to add extensions (or plugins) to support specific programming languages. For most programming languages, there is at least one mature extension for a popular IDE. However, in 2018, Rust had very limited IDE support, meaning that there were some extensions to use Rust in a pair of IDEs but they offered few features, bad performance, and were also rather buggy.

In addition, many programmers prefer an interactive development style. When creating a new feature of a software system, they do not like to write a lot of software and then compile and test all of it. Instead, they prefer to write a single line or a bunch of few lines and test such snippets of code right away. After testing that snippet of code successfully, they integrate it into the rest of the system. This is typical of developers using interpreted languages such as JavaScript or Python.

The tools that are able to run snippets of code are language interpreters or fast in-memory compilers. Such interpreters read a command from the user, evaluate it, print the result, and go back to the first step. Therefore, they are usually named read-eval-print loop, or REPL for short. For all interpreted programming languages, and for some compiled languages, there are mature REPLs. In 2018, the Rust ecosystem was missing a mature REPL.

Here, the IDE issue and the REPL issue are presented together because they share the following common problem. The main feature of modern IDEs is to analyze source code as it is edited, with the following goals:

• To highlight the code containing invalid syntax, and to display a compilation error message in a popup window that appears near the invalid code
• To suggest the completion of identifiers, to be chosen among the already declared identifiers
• To show the synopsis documentation of an identifier selected in the editor
• To jump in the editor from the definition of an identifier to its uses, or vice versa
• In a debugging session, to evaluate an expression inside the current context, or to change the memory contents owned by a variable

Such operations require very fast parsing of Rust code, and this is also what is required by a Rust REPL. An attempt to address such issues is a project named the Rust Language Server (https://github.com/rust-lang/rls) that is developed by the Rust language team. Another attempt is the project named Rust Analyzer (https://github.com/rust-analyzer/rust-analyzer) that is developed by the Ferrous Systems company, supported by several partners. Hopefully, before the next Rust edition, there will be a fast and powerful Rust language analyzer to support smart programmers' editors, source-level debuggers, and REPL tools, just as many other programming languages have.

Crate maturity

A crate becomes mature when it reaches version 1.0. That milestone means that the following versions 1.x will be compatible with it. Instead, for versions 0.x, there is no such guarantee, and any version can have an application programming interface (API) that's quite different from the previous one.

Having a mature version is important for several reasons, listed as follows:

• When you upgrade your dependency to a newer version of a crate (to use new features of that library), you are guaranteed that your existing code won't get broken—that is, it will continue to behave in a previous way, or in a better way. Without such a guarantee, you typically need to review all your code using that crate and fix all the incompatibilities.
• Your investment in know-how is preserved. You need to neitherretrain yourself nor your coworkers and not even update your documentation.
• Typically, software quality is improved. If a version of an API remains unchanged for a long time, and many people use it in different corner cases, untested bugs and real-world performance issues can emerge and be fixed. Instead, a quickly changing version is usually bug-ridden and inefficient in many application cases.

Of course, there is an advantage to iterating through several improvement steps of the API, and APIs created in a few weeks are usually badly designed. Although there are still many crates that have been in a 0.x version for several years, the time is coming to stabilize them.

This is a reinterpretation of the buzzword stability. In 2015, it meant thestability of the language and of the standard library. Now, the rest of the mature ecosystem must stabilize to be accepted in real-world projects.

Asynchronous programming

A major innovation was introduced in stable Rust in November 2019—with release 1.39—it is the async-await syntax, to support asynchronous programming.

Asynchronous programming is a programming paradigm that is very useful in many application areas, mainly in multiuser servers, so that many programming languages—such as JavaScript, C#, Go, and Erlang—support it in the language. Other languages, such as C++ and Java, support asynchronous programming through the standard library.

Around 2016, it was very hard to do asynchronous programming in Rust because neither the language nor the available crates supported it in an easy and stable way. Then, some crates supporting asynchronous programming were developed, such as futures, mio, and tokio, though they were not much easier to use, and remained at a version before 1, meaning instability of their API.

After having seen the difficulty of creating convenient support for asynchronous programming using only libraries, it appeared clear that a language extension was needed.

The new syntax, similar to that of C#, includes the new async and await language keywords. The stabilization of this syntax means that the previous asynchronous crates should now be considered obsolete until they migrate to use the new syntax.

The new syntax—announced on the https://blog.rust-lang.org/2019/11/07/Async-await-stable.html web page—is described on the https://rust-lang.github.io/async-book/ web page.

For those who never felt the need for asynchronous programming, here is a quick example of it. Create a new Cargo project, with the following dependencies:

async-std = "1.5"
futures = "0.3"

Prepare in the root folder of that project a file named file.txt that contains only five Hello characters. Using a Unix-like command-line, you can do this using the following command:

echo -n "Hello" >file.txt

Put the following content into the src/main.rs file:

use async_std::fs::File;
use async_std::prelude::*;
use futures::executor::block_on;
use futures::try_join;

fn main() {
    block_on(parallel_read_file()).unwrap();
}

async fn parallel_read_file() -> std::io::Result<()> {
    print_file(1).await?;
    println!();
    print_file(2).await?;
    println!();
    print_file(3).await?;
    println!();
    try_join!(print_file(1), print_file(2), print_file(3))?;
    println!();
    Ok(())
}

async fn print_file(instance: u32) -> std::io::Result<()> {
    let mut file = File::open("file.txt").await?;
    let mut byte = [0u8];
    while file.read(&mut byte).await? > 0 {
        print!("{}:{} ", instance, byte[0] as char);
    }
    Ok(())
}

If you run this project, the output is not quite deterministic. The possible output is the following one:

1:H 1:e 1:l 1:l 1:o 
2:H 2:e 2:l 2:l 2:o 
3:H 3:e 3:l 3:l 3:o 
1:H 2:H 3:H 1:e 2:e 3:e 1:l 1:l 3:l 1:o 2:l 3:l 2:l 3:o 2:o 

The first three lines are deterministic. Instead, the last line can be shuffled a bit.

In a first reading, pretend it is synchronous code, ignoring the words async, await, block_on, and join!. With this simplification, the flow is easy to follow.

The main function calls the parallel_read_file function. The first six lines of the parallel_read_file function call the print_file function three times, with the arguments 1, 2, and 3, in different lines, each followed by a call to println!. The seventh line of the parallel_read_file functionagain calls the print_file function three times, with the same three arguments.

The print_file function uses the File::open function call to open a file, and then uses the file.read function call to read a byte at a time from that file. Any byte read is printed, preceded by the argument of the function (instance).

So, we obtain the information that the first call toprint_file prints 1:H 1:e 1:l 1:l 1:o. They are the five characters read from the file, preceded by the number 1, received as an argument.

The fourth line prints the same contents of the first three lines, mixing the characters. First, the three H characters are printed, then the three e characters, then the three l characters, and then something weird happens: an o is printed before all the l characters have been printed.

What is happening is that the first three lines are printed by three sequential invocations of theprint_file function, while the last line is printed by three parallel invocations of the same function. In any parallel invocation, all the letters printed by one invocation are in the correct order, but the other invocations may interleave their output.

If you think that this is similar to multithreading, you are not far from the truth. There is an important difference, though. Using threads, the operating system may interrupt the threads and pass control to another thread at any time, with the effect that the output may be broken at undesirable points.

To avoid such interruptions, critical regions or other synchronization mechanisms must be used. Instead, with asynchronous programming, functions are never interrupted except when a specific asynchronous operation is performed. Typically, such operations are an invocation of external services, such as accessing the filesystem, which could cause a wait. Instead of waiting, another asynchronous operation is activated.

Now, let's see the code from the beginning, implementing asynchronous operations. It uses the async_std crate. It is an asynchronous version of the standard library. The standard library is still available, but its functions are synchronous. The code can be seen in the following snippet:

use async_std::fs::File;
use async_std::prelude::*;

To have an asynchronous behavior, the functions of this crate must be used. In particular, we will use the functions of the File data type. In addition, some features of the not-yet-stabilized futures crate are used. The code can be seen in the following snippet:

use futures::executor::block_on;
use futures::try_join;

Then, there is themainfunction, whose body contains only the following line:

    block_on(parallel_read_file()).unwrap();

Here, the parallel_read_file function is called first.

This is an asynchronous function. When you call an asynchronous function using the normal function-call syntax, as in theparallel_read_file() expression, the body of that function is not actually executed, as a normal and synchronous function would be. Instead, such a call just returns an object, called a future. A future is similar to a closure, as it encapsulates a function and the arguments used to invoke such a function. The function encapsulated in the returned future is the body of the function we were calling.

To actually run the function encapsulated in the future, a particular kind of function is needed, called an executor. The block_on function is an executor. When an executor is invoked, passing a future to it, the body of the function encapsulated in that future is run, and the value returned by such a function is then returned by the executor itself.

So, when the block_on function is called, the body of parallel_read_file is run, and when it terminates, block_on also terminates, returning the same value returned byparallel_read_file. As this last function has a Result value type, it should be unwrapped.

Then, a function is defined whose signature is as follows:

async fn parallel_read_file() -> std::io::Result<()>

The async keyword marks that function as asynchronous. It is also fallible, and so a Result value is returned.

Asynchronous functions can be invoked only by other asynchronous functions or by executors, such as block_on and try_join. The main function is not asynchronous, and so there, we needed an executor.

The first line of the body of the function is added in the following code snippet. It is an invocation of theprint_file function, passing the value 1 to it. As the print_file function is asynchronous too, to invoke it from inside an asynchronous function, the .await clause must be used. Such a function is fallible, and so a ? operator is added, like this:

    print_file(1).await?;

When an asynchronous function is invoked using .await, the execution of the body of that function starts right away, but as soon as it yields control because it executes a blocking function, such as an operating system call, another ready asynchronous function may proceed. However, the flow of control does not proceed beyond the .await clause until the body of the called function is complete.

The second line of the body of the function is an invocation of a synchronous function, and so .await is neither needed nor allowed, as can be seen in the following code snippet:

    println!();

We can be sure that it is run after the previous statement because that statement ended with a .await clause.

This pattern is repeated three times, and then the seventh line consists of a set of three invocations in parallel with the same asynchronous function, as illustrated in the following code snippet:

    try_join!(print_file(1), print_file(2), print_file(3))?;

Even the try_join! macro is an executor. It runs all the three futures generated by the three calls to print_file. Only one thread is used by asynchronous programming, and so, in fact, one of the three futures is run first. If it never has to wait, it ends before the other futures have the opportunity to start.

Instead, as this function will have to wait, at any wait the context is switched to another running future, starting from the statement that had put it on wait. So, the executions of the three futures are interleaved.

Now, let's see the definition of such an invoked function. Its signature is shown in the following code snippet:

async fn print_file(instance: u32) -> std::io::Result<()> {

It is an asynchronous function, receiving an integer argument and returning an empty Result value.

The first line of its body opens a file using the File data type of the asynchronous standard library, as illustrated in the following code snippet:

    let mut file = File::open("file.txt").await?;

As such, the open function is asynchronous too, and it must be followed by .await, as illustrated in the following code snippet:

    let mut byte = [0u8];
    while file.read(&mut byte).await? > 0 {
        print!("{}:{} ", instance, byte[0] as char);
    }

The asynchronous read function is used to read bytes to fill the byte buffer. This buffer has length 1, and so just one byte at a time is read. The read function is fallible, and if it is successful, it returns the numbers of bytes read. This means that it returns 1 if a byte is read and 0 if the file is ended. If the call reads a byte, the loop continues.

The body of the loop is a synchronous output statement. It prints the identifier of the current instance of the file stream, and the byte just read.

So, the sequence of steps is as follows.

First, the print_file(1)future is started. When it executes the File::open call that is blocking, this future is put on hold, and a ready-to-run future is looked for. There are two ready futures: print_file(2) and print_file(3). The first one is chosen, and it is started. Also, it reaches the File::open call, and so it is put on hold, and the third future is started. When it reaches the File::open call, it is put on hold and a ready future is looked for. If there is no ready-to-run future, the thread itself waits for the first ready future.

The first future to complete the File::open call is the first one, which resumes its execution just after that call and starts to read a byte from the file. Even this one is a blocking operation, and so this future is put on hold, and control is moved to the second future, which starts to read one byte.

There is always a queue of ready futures. When a future has to wait for an operation, it yields control to the executor, which passes control to the first future in the queue of ready futures. When the blocking operation is complete, the waiting future is appended to the queue of ready futures and can be yielded control if no other future is running.

When all the bytes of a file have been read, the print_file function ends. When all the three calls to print_file are ended, the try_join! executor ends, and the parallel_read_file function can proceed. When it reaches its end, the block_on executor ends and, with it, the whole program.

As blocking operations take a variable amount of time, the sequence of steps is non-deterministic. Indeed, the last line of output of the example program seen before can be slightly different in different runs, swapping some portions of it.

As we have seen, asynchronous programming is similar to multithreaded programming but it is more efficient, saving both context-switch time and memory usage. It is appropriate primarily for input/output (I/O)-bound tasks as only one thread is used, and the flow of control is interrupted only when an I/O operation is performed. Instead, multithreading can allocate a different thread on any core, and so it is more appropriate for central processing unit (CPU)-bound operations.

After the addition of the async/await syntax extension, what is still needed is the development and stabilization of crates using and supporting such syntax.

Optimization

Usually, system programmers are quite interested in efficiency. In this regard, Rust shines as one of the most efficient languages, though there are still some issues with performance, as follows:

• A full build—in particular, an optimized release build—is quite slow, even more so if link-time optimization is enabled. For large projects, this can be quite a nuisance. At present, the Rust compiler is just a frontend that generates Low-Level Virtual Machine (LLVM) intermediate representation (IR) code and passes such code to the LLVM machine code generator. However, the Rust compiler generates a disproportionate amount of LLVM IR code, and so the LLVM backend must take a long time to optimize it. An improved Rust compiler would pass to LLVM a much more compact sequence of instructions. A refactoring of the compiler is in progress, and this could lead to a faster compiler.
• Since version 1.37, the Rust compiler supports profile-guided optimization (PGO), which can enhance performance for the typical processor workflows. However, such a feature is rather cumbersome to use. A graphical frontend or an IDE integration would make it easier to use.
• A development underway is an addition to the language of the const generics feature, described in the next section.
• In LLVM IR, any function argument of a pointer type can be tagged with the noalias attribute, meaning that the memory reference by this pointer will not be changed inside this function, except through this pointer. Using this information, LLVM can generate faster machine code. This attribute is similar to the restrict keyword in the C language. Yet in Rust, for every mutable reference (&mut), the noalias property is guaranteed by language ownership rules. Therefore, faster programs could be obtained that always generate LLVM IR code with the noalias attribute for every mutable reference. This has been done in versions 1.0 through 1.7 and in versions 1.28 and 1.29, although, because of bugs in the LLVM backend compiler, the resulting code was bugged. Therefore, until a correct LLVM implementation is released, the noalias optimization hint will not be used.

The const generics feature

At present, generic data types are parameterized only by types or lifetimes. It is useful to be able to also parameterize a generic data type by a constant expression. In a way, this feature is already available, but only for one kind of generic type: the arrays. You can have the [u32; 7] type that is an array parameterized by the u32 type and by the 7 constant, though you cannot define your own generic type parameterized by a constant.

This feature, already available in C++ language, is in development in the nightly build. It would allow variables to be replaced with constants in generic code, and this would surely improve performance. Here is an example program that uses as dependencies the num crate:

#![feature(const_generics)]
#![allow(incomplete_features)]

use num::Float;

struct Array2<T: Float, const WIDTH: usize, const HEIGHT: usize> {
    data: [[T; WIDTH]; HEIGHT],
}

impl<T: Float, const WIDTH: usize, const HEIGHT: usize>
Array2<T, WIDTH, HEIGHT> {
    fn new() -> Self {
        Self { data: [[T::zero(); WIDTH]; HEIGHT] }
    }
    fn width(&self) -> usize { WIDTH }
    fn height(&self) -> usize { HEIGHT }
}

fn main() {
    let matrix = Array2::<f64, 4, 3>::new();
    print!("{} {}", matrix.width(), matrix.height());
}

This program, to be compiled only using a nightly version of the compiler, creates a data type implementing a bidimensional array of floating-point numbers. Notice that the parameterization is as follows: T: Float, const WIDTH: usize, const HEIGHT: usize. The first parameter is the type of array items. The second and third parameters are the sizes of the array.

Having constant values instead of variables allows important code optimizations.

Embedded systems

Rust has been developed since when Mozilla started to sponsor it in 2009, with a specific goal: to create a web browser. Even after 2018, the core team of developers works for Mozilla Foundation, whose main business is to build client-side web applications. Such software is multiplatform, but oriented exclusively toward the following requirements:

• Random-access memory (RAM): At least 1 GB
• Supported CPUs: Initially only x86 and x86_64; later, also ARM and ARM64.
• Supported operating systems: Linux, Windows, macOS

These requirements excluded most microcontrollers as the Mozilla Foundation was not interested in such platforms, though the features of Rust appear to be a good match with the requirements of many embedded systems with more constrained requirements. Therefore, thanks to a worldwide group of volunteers, in 2018, the Embedded Working Group was created to develop the ecosystem needed to use Rust on embedded systems—that is, on bare-metal or on stripped-down operating systems, and with severe resource limitations.

Progress in this application area has been rather slow and directed mainly at a few architectures, but the future is promising, at least for 32-bit or 64-bit architectures, because any architecture supported by the LLVM backend is easily targetable by the Rust compiler.

Some specific improvements to the language, which ease the use of Rust for embedded systems, are listed as follows:

• The standard-library Pin generic class avoids moving objects in memory. This is needed when some external device is accessing a memory location.
• The cfg and cfg_attr attributes, which allow conditional compilation, have been extended. This feature is needed because trying to compile code for a wrong platform can create unacceptable code bloat, or even cause compilation errors.
• The allocator API has been made more customizable.
• The applicability of const fn has been extended. This construct allows a code base that is maintainable as normal algorithmic code, but as efficient as a wired constant.

Summary

In this chapter, we have seen the most probable development lines of the Rust ecosystem in the next few years—support for IDEs and for interactive programming; the maturity of the most popular crates; widespread support of the new asynchronous programming paradigm and its keywords (asyncandawait); further optimization of both the compiler and the generated machine code; and widespread support of embedded systems programming.

We have learned how to write asynchronous code and a possible way to define and use const generics (still unstable at the time of writing).

We have seen that there are quite a lot of application areas where Rust could really shine. Of course, if you are going to use it only for fun, the sky is the limit, but for real-world applications, the ecosystem of libraries and tools can really decide the viability of a programming system. Now, at last, the critical mass of high-quality libraries and tools is about to be reached.