Remote Message Passing

Throughout this book, we’ve been addressing your Elixir adoption one layer at a time. We started with functions and walked through how to organize code and think functionally. Next, we moved into concurrency. In Elixir, the fundamental constructs for concurrency are processes, and the OTP abstraction built upon them. We talked about building layered applications and a structure for sending messages between them.

Next, we’ll introduce the concept of nodes. A node is an abstract group of processes. They may be running on the same machine or different ones. When you’re using Elixir, you send messages between remote processes and local processes in exactly the same way. That means processes form the foundation of distributed applications. Elixir uses the same send/2 function for sending messages to processes running on the same node or on a separate node over the network. Throughout this chapter, we’re going to set up some nodes and do exactly that.

To start a node, you need to start it with a name. The name may be short, allowing connections only from the same machine, or long, allowing connections over the network. In both cases, nodes can only communicate if they share the same cookie. It is not a browser cookie; it is a unique identifier stored as an atom. You can find this cookie at ~/.erlang.cookie. Erlang creates one automatically when you start a named node, or you can pass the --cookie flag when starting the VM to specify your own. Keep in mind data sent between nodes is not encrypted out of the box. The security implications of running distributed Erlang in production are discussed in ​Security Guidelines​.

Start a new IEx session and give the node a short name of chip. You’ll only be able to access this node by name from other nodes running on the same machine, like this:

Now, start another node, named dale:

Your two nodes are now connected. Remember the examples here will likely have different node names when running on your machine and you need to adjust it accordingly.

When connected, Elixir maintains an open TCP connection between the nodes. If more nodes join the network, they’ll hold direct connections to each other. We call such a network a fully meshed network. If the TCP connection between two nodes drops or the node becomes unresponsive, those two nodes will then disconnect. You can do so explicitly via Node.disconnect, like this:

To send a message from a process running in chip to a process running in dale, you need to be able to identify and find processes across nodes. One option is to give the process a local name and ask the node to send a message to a process running locally with a given name.

Back in chip@macbook, give the IEx process the name of :my_iex:

Now in dale@macbook, let’s send a message to node chip, asking it to deliver that message to a local process named :my_iex:

Elixir used the tuple {process_name, node_name} to (a) open a connection to node chip@macbook if one does not yet exist, (b) serialize, and (c) send the message. The first point deserves special attention. Elixir will always attempt to connect the two nodes when sending remote messages, even if they’ve been explicitly disconnected.

Back on chip@macbook, you can run flush() and verify the IEx process has indeed received a message and the nodes are connected once again, like this:

So, you can send messages across nodes. You can monitor processes across nodes too. Back in dale@macbook, monitor the :my_iex process running on chip@macbook:

Now, if you terminate the chip@macbook node, the IEx session running on dale@macbook will receive a :DOWN message with a :noconnection reason, like this:

The same primitives we use for building concurrent and fault-tolerant applications are also available for building distributed systems. None of this behavior is specific to Elixir; it is all part of the Erlang runtime. But don’t let that fool you. Network communication brings a whole new set of trade-offs to consider.

For instance, in order to exchange messages between processes in these examples, you named the IEx process running on chip@macbook. To uniquely name a process, you need to use a process registry. The process registry used here is a local process registry. We’ll talk more about them in ​Finding Processes​ so we’ll just give you a quick working definition now.

A local process registry is straightforward to implement but it’s limited in capabilities. For example, to check if a process exists on a given node, you’ll always need to use the network to ask that node if the process is alive. Furthermore, a local process registry only guarantees uniqueness locally, but a distributed process registry must guarantee that a name is unique across the whole cluster.

Different process registries will choose different trade-offs and those choices will impact the design of your applications. To understand how this affects your systems, you’ll need to understand state, persistence, and replication.