A collection of tightly interacting modules in Erlang is called an application. Erlang systems consist of a set of loosely coupled applications. It is good practice to design an application to provide a single point of entry for calls originating from other applications. Collecting all externally exported functions into one module provides flexibility in maintaining the code base, since modifications to all “internal” modules are not visible to external users. The documentation for this module gives a complete description of the interface to the application. This single point of entry also facilitates tracing and debugging of the call flow into the application. In large systems, it is good practice to prefix the modules in a particular application with a short acronym. This ensures that the choices of module names in different applications will never overlap, which would cause problems if both applications were used together in any larger system.

Modules are your basic building blocks. When designing your modules, you should try to export as few functions as possible. As far as the user of a module is concerned, the complexity of a module is proportional to the number of exported functions, since the user of a module needs to understand only the exported functions. Having as small an interface as possible also gives the maintainer of the application greater flexibility, as it makes it much easier to refactor the internal code.

You should try to reduce intermodule dependencies as much as you can. It is harder to maintain a module with calls to many modules instead of calls to just a few modules. Reducing interdependencies not only facilitates the maintenance of the modules being called, but also makes it easier to refactor them as a smaller number of external calls have to be maintained.

Intermodule dependencies should form an acyclic graph, as shown in Figure 20-1; that is, there should be no module X, say, that depends on another module that (through a chain of dependencies) depends on module X.

Figure 20-1. Intermodule dependencies

Within a particular module, place related functions close to each other.^[48] For example, you should place functions such as start and stop and init and terminate next to each other, and place all of the message-handling functions in the same part of a module. Keeping related functions close together makes your code easier to follow and inspect, especially for cases where you do or open something in one function and undo or close it in another.

compile:file(foo, [compile_all,...]).

keysearch(Key, [{Key, Value}|_]) -> Value;
keysearch(Key, [_|Tail])         -> keysearch(Key, Tail);
keysearch(_key, [])              -> false.

keysearch(Key, [{Key, Value}|_]) -> {ok, Value};
keysearch(Key, [_|Tail])         -> keysearch(Key, Tail);
keysearch(_key, [])              -> false.

{ok, Status} = get_status(BladeId),
NewStatus = reset(BladeId, Status)

NewStatus = reset(BladeId, get_status(BladeId))

tax_to_death(Person) ->
  case is_swede(Person) of
    true ->
       {ok, raise_taxes(Person)};
    {error, Nationality} ->
       io:format("Person not Swedish:~p~n",[Nationality]),
       error
   end.

tax_to_death(Person) ->
  case is_swede(Person) of
    true ->
      {ok, raise_taxes(Person)};
    {error, Nationality} ->
      {error, Nationality}
  end.

-module(q).
-export([add/2, fetch/1]).

add(Item, Q) -> lists:append(Q, [Item]).

fetch([H|T]) -> {ok, H, T};
fetch([])    -> {error, empty}.

NewQ = [],
Queue1 = q:add(joe, NewQ),
Queue2 = q:add(klacke, Queue1).

Newq = q:empty(),
Queue1 = q:add(joe, Newq), ...

Processes are the basic structuring elements of a system in Erlang. A fundamental principle of design in Erlang is to create a one-to-one mapping between the parallel processes in your Erlang program and the set of parallel activities in the system you are modeling. This could add up to quite a few simultaneous processes, so where possible, avoid unnecessary process interaction and unnecessary concurrency. Ensure that you have a process for every concurrent activity, not for only two or three! If you are coming from an object-oriented background, this will not equate to a process for every object or every method applied to an object. Or, if you are dealing with users logged in to a system, this will probably not equate to a process for every session. Instead, you will have a process for every event entering the system. This will equate to massive numbers of transient processes and a very limited number of persistent ones.

You should always implement a process loop and its peripheral functions in one module. Avoid placing code for more than one process loop in the same module, as it becomes very confusing when you try to understand who is executing what. You might have many process instances executing the same code base, but ensure that the loop in the module is unique.

Hide all message passing in a functional interface for greater flexibility and information hiding. Instead of having the following expression in your client code:

resource_server ! {free, Resource}

replace it with this function call:

resource_server:free(Resource)

Place the client functions in the same module as the process. Client functions are functions called by other processes that result in a message being sent to the process defined in the module. This makes it easy to follow the message flow without having to jump between modules, let alone having to find out in which module the clause receiving the messages is located. From your end, it reduces and simplifies the documentation you have to produce, as you need to describe only the functional API, and not the message flow. You will also get more flexible code, as you are hiding the following:

If you use a functional interface, you have the flexibility to change all of this without affecting the client code.

Registered processes should, where possible, have the same name as the module in which they are implemented. This facilitates the troubleshooting of live systems and improves code readability and maintainability. Registered processes should have a long life span; you should never toggle between registering and deregistering them. As the space used by atoms is not garbage-collected, avoid dynamically creating atoms to register processes, as this might result in a memory leak.

Processes should have well-defined behaviors and roles in the system. You should seriously consider using the OTP behaviors described in Chapter 12, including servers, event handlers, finite state machines, supervisors, and applications.

When working with message passing, all messages should be tagged. It makes the order of clauses in the receive statement unimportant, and as a result, it facilitates the addition of new messages without changing the existing behavior, thus reducing the risk of bugs.

Avoid pattern matching only on unbound variables in receive clauses. In the following example, what if you want your process to also handle the message {get, Pid, Variable}?

loop(State) ->
  receive
    {Mod, Fun, Args} ->
      NewState = apply(Mod, Func, Args),
      loop(NewState)
  end.

It is easy to see that you would have to match the message above the {Mod, Fun, Args} pattern, but what if your receive statement was matching on many more messages? It would not be that obvious. Or, what would happen if you want to apply a function in the module get? By tagging messages, you get full flexibility in rearranging the order of your messages and avoid incorrectly pattern matching messages in the wrong clause:

loop(State) ->
  receive
    {apply, Mod, Fun, Args} ->
      NewState = apply(Mod, Func, Args),
      loop(NewState);
    {get, Pid, Variable} ->
      Pid ! get_variable(Variable),
      loop(State)
  end.

When using receive clauses, you should not be receiving unknown messages. If you do, they should be treated as bugs and you should allow your system to crash, just as an OTP application will do. If you don’t let the system crash, make sure you log the unexpected messages. This allows you to ascertain where they are coming from, and to diagnose how to deal with them. You will probably find that these messages originate from ports or sockets or from bugs that should be picked up during a unit test.

If you do not handle unknown messages, you will notice that the CPU usage of your system will start to increase and that the response time will decrease, until the Erlang runtime system runs out of memory and crashes. The increase in CPU usage and response time is explained by the fact that whenever a process receives a message, it has to traverse the potentially hundreds or thousands of messages in the mailbox before reaching one that matches. The more messages that are not matched, the more memory is leaked.

When you are sending and receiving messages, you should hide the internal message protocol from the client. Always tag the messages using the client function name, as it facilitates the hop from the client function to the location in your process loop where you handle the message:

free(Resource) ->
   resource_server ! {free, Resource}.

Use references. In complex systems where similar responses and requests might originate from different processes, use references to uniquely identify responses from specific requests:

call(Message) ->
  Ref = make_ref(),
  resource ! {request, {Ref, self()}, Message},
  receive
    {reply, Ref, Reply} -> Reply
  end.

reply({Ref, Pid}, Message) ->
  Pid ! {reply, Ref, Message}.

Be careful with timeouts. If you use them, always flush messages that arrive late. If you don’t, your next request will result in the response that previously timed out. You can avoid this problem by using references and assuming that no other processes will send messages of the format {reply, Ref, Reply}. Just flush messages you do not recognize:

call(Message) ->
  Ref = make_ref(),
  resource ! {request, {Ref, self()}, Message},
  wait_reply(Ref).

wait_reply(Ref) ->
  receive
    {reply, Ref, Reply} -> Reply;
    {reply, _,   Reply} -> wait_reply(Ref)
  end.

If you are using timeouts to deal with the case when a process might have terminated, it is much better to use a link or a monitor.

Be very restrictive in trapping exits, and when doing so, do not toggle. Make sure you remember to flush exit signals, and when linking and unlinking processes, be aware of race conditions. When you link to a process, it might have already crashed. When you unlink, it might have terminated, and its EXIT signal will be waiting to be retrieved from the process mailbox.

Separate error recovery and normal code, as combining them will increase the complexity of your code. It will also ensure that crashes and recovery strategies are handled consistently. Never try to fix an error that should not have occurred and then continue. If an unexpected error occurs, make your process crash and let a supervisor process deal with it. It is more likely that in trying to handle errors, you will generate more bugs than you think you are solving. In the following example, what would you do if the variable List is not a list?

bump(List) when is_list(List) ->
  lists:map(fun(X) -> X+1 end, List);
bump(_) ->
  {error, no_list}.

Everywhere you call bump/1, you would have to cater for two return values in a case clause, but still not make the person reading the code any wiser as to why List got corrupted. Don’t be defensive, and instead write:

bump(List) ->
  lists:map(fun(X) -> X+1 end, List).

If List does not contain a list, a runtime error will occur in lists:map/2 and will result in your process terminating. Let your supervisor handle the exit signal and let it decide the recovery strategy consistently with all of the other processes it is supervising. Make sure the crash is recorded and can be used for post-mortem debugging; log these errors and crashes in a separate error logger process.

As we should be hiding processes and message passing behind functional interfaces, ensuring that the dependencies form an acyclic graph also ensures that there are no deadlocks between these processes. Deadlocks are extremely rare in Erlang, but they do occur if the concurrency is not well thought out and properly designed. Ensuring that there are no cycles in your module dependencies is a step in the right direction.

Programs are written not just to be executed by a computer, but also to be read and understood by their authors and other programmers. Writing your programs in a way that makes them easier to read and understand will help you remember what your program does when you come back to it six months down the road, or will help another programmer who has to use or modify your program. It will also make it easier for someone to spot errors or other problems, or to interpret debugging information. So, being consistent in the way that you write programs will help everyone. In this section, we give a set of commonly used conventions for style in writing Erlang programs.

First, avoid writing deeply nested code.^[50] In your case, if, receive, and fun clauses, you should never have more than two levels of nesting in your code. Here is how not to do it:

reset(BladeId, AdminState, OperState) ->
  case AdminState of
    enabled ->
      case OperState of
        disabled ->
          enable(BladeId);
        enabled ->
          disable(BladeId),
          enable(BladeId)
      end;
    disabled ->
      {error, admin_disabled}
  end.

A common trick to reduce indentation is to create temporary composite data types. If you have nested if and case statements, join them together in a tuple and pattern-match them in one clause:

reset(BladeId, AdminState, OperState) ->
  case {AdminState, OperState} of
    {enabled, disabled} ->
      enable(BladeId);
    {enabled, enabled} ->
      disable(BladeId),
      enable(BladeId);
    {disabled, _OperState} ->
      {error, admin_disabled}
  end.

You can reduce indentation by introducing pattern matching in your function heads. By pattern matching on the AdminState and the OperState in the function clause, not only do we make the code more readable, but we also reduce the level of nested clauses to zero and reduce the overall code size:

reset(BladeId, enabled, disabled) ->
  enable(BladeId);
reset(BladeId, enabled, enabled) ->
  disable(BladeId),
  enable(BladeId);
reset(_BladeId, disabled, _OperState) ->
  {error, admin_disabled}.

Avoid using if clauses when case clauses are a better fit. This is especially common with programmers coming from an imperative background who still do not feel at ease with pattern matching. Always ask yourself whether you can rewrite your if clause into a case clause using pattern matching and guards that will avoid pattern matching on the atoms true and false. If so, is it more readable and compact?

get_status(A,B,C) ->
  if
    A == enabled ->
      if
        B == enabled ->
          if
            C == enabled ->
              enabled;
            true ->
              disabled
          end;
        true ->
          disabled
      end;
    true ->
      disabled
  end.

The preceding example is an extreme occurrence of the misuse of the if statement. By creating a composite data type with the variables A, B, and C, and replacing the if with a case statement, we reduce the level of indentation and make the code more readable and compact:

get_status(A,B,C) ->
  case {A,B,C} of
    {enabled,  enabled,   enabled } -> enabled;
    {_status1, _status2, _status}   -> disabled
  end.

Aim to keep your modules to a manageable size. Short modules facilitate maintenance and debugging as well as the understanding of your code. A manageable module should have no more than 400 lines of code, comments excluded. Split long modules in a logically coherent way and remember that long lines will not solve your problem.

Lines of code should not be too long, or drift across the page. A line of code should never be more than 80 characters long. Too many times developers have tried to convince us that their long lines were readable by widening their editor window to cover the whole screen. That will not solve your problem.

If your code is drifting across the page, spending a few minutes on reformatting will save the next person from spending hours trying to maintain and debug it. The following code style is a typical example of what happens when someone writes code, tests it, but never goes back to review it:

name(First, Second) ->
  case person_exists(First, Second) of
    true ->
      Y = atom_to_list(First) ++ [$ |atom_to_list(Second)] ++
                                          [$ |get_nickname(First,
                                             Second)],
      io:format("true person:~s~n",[Y]);
    false ->
           ok
  end.

You can easily rewrite this to the following:

name(First, Second) ->
  case person_exists(First, Second) of
    true ->
      Y = atom_to_list(First) ++
          [$ |atom_to_list(Second)] ++
          [$ |get_nickname(First, Second)],
      io:format("true person:~s~n",[Y]);
    false ->
      ok
  end.

After a second iteration and a bit of thinking, the preceding code would convert to the following:

name(First, Second) ->
  case person_exists(First, Second) of
    true ->
      NickName = get_nickname(First, Second),
      io:format("true person:~w ~w ~s~n",[First, Second, NickName]);
    false ->
      ok
  end.

If your code drifts across the page, solve the problem by:

Choose meaningful function and variable names. If the names consist of several words, separate them with either capital letters or an underscore. Some people will pick one style for variables and the other for functions, without mixing them together. Whatever the case, when you pick a style, stick to it throughout your code; this is true of all the conventions discussed here: consistency makes code readable, whereas chopping and changing between styles will distract a reader away from understanding what is going on.

Avoid long names, as they are a prime reason for code drifting across a page. A long name might look all right in the function head, but imagine calling it in the second level of a clause together with long variable names. If you are using longer names, it makes more sense to use them in functions—which will potentially be used across your code base—rather than for variables whose scope will be limited to a particular function.

Use abbreviations and acronyms to shorten your names, but ensure that they make sense and are easy to understand. A common pitfall is to use names that are very similar, and so can easily be confused: Name, Names, and Named all look pretty similar! If you use prefixes, pick ones that will provide hints regarding the variable types or function return values. Finding meaningful function and variable names takes practice and patience, so don’t underestimate the task.

Avoid the underscore on its own, when using “don’t care” variables. Even if you are not using the variable, its contents might be of interest to whoever is reading the code. Not using an underscore in front of unused variables will result in compiler warnings. But be warned that variables of the format _name are regular single-assignment variables, so once they have been bound, they cannot be reused as “don’t care” variables.

In the following example, if the variable AdminState is bound to the atom disabled, _State will also be bound to disabled. If OperState is bound to enabled, the second case clause will fail with a case clause error, as none of the patterns match:

restart(BladeId, AdminState, OperState) ->
  case AdminState of
    enabled ->
      disable(BladeId);
    _State ->
      ok
  end,
  case OperState of
    disabled ->
      ok;
    _State ->
      stop(BladeId)
  end.

Use records as the principal data structure to store compound data types. If records are used by one module only and are not exported, the definition should be placed at the beginning of the module to stop others from using it in their modules. Records used by many modules should be placed in an include file and properly documented.

Use record selectors to access a field and pattern matching to access values in more than one field:

Cat = #cat{name = "tobby", owner ="lelle"},
#cat{name = Name, owner = Owner} = Cat,
Name2 = Cat#cat.name.

Never, ever use the internal tuple representation of records, as this defies the flexibility that records bring to the table:

Cat = #cat{name = "tobby", owner = "lelle"},
{cat, Name, _owner} = Cat

If you write code such as this and then add a field to your record, you will have to update the code that uses the tuple representation even if the function is not affected by the field.

Use variables only when you need them. There are two reasons for using a variable. The first is when you need to use the value they are bound to more than once. The second is to improve clarity in your code and shorten the length of a line. You might be tempted to write the following:

Sin = sin(X),
Cos = cos(X),
Tan = Sin / Cos

But passing the return values of the function directly to another function will make your code much clearer and more compact:

Tan = sin(X) / cos(X)

Always ask yourself whether the variable really makes the code clearer; the more you program in Erlang, the less you will find yourself using variables.

Be careful when using catch and throw. It is nearly always preferable to use the try ... catch construct than to use catch and throw. Try to minimize their usage, and ensure that any throw is caught in the same module as a catch. As you saw in Chapter 3, it makes sense to use catch only when you can safely ignore the return value of a function.

If you are using catch and throw, always make sure you also handle runtime errors that might be caught. For instance, the following code appeared in code that was about to be sent into production:

Value = (catch get_value(Key)),
ets:insert(myTable, {Key, Value})

What would have happened if a runtime error occurred in the get_value/1 call? You would have caught the runtime error {'EXIT', Reason}, bound it to the variable Value, and stored it in the ETS table.

Be very careful with the process dictionary; in fact, avoid it like the plague. The BIFs put and get are destructive operations. They will make your functions nondeterministic and hard to debug, as determining the contents of the process dictionary after the process has crashed will be very difficult, if at all possible, to do. Instead of the process dictionary, introduce new arguments in your functions.

If you do not know what the process dictionary is,^[51] move along; there is nothing to see here, move along.

Use the -import(mod, [fun/arity...]) directive with care. It can make the code hard to follow and will initially confuse the most experienced of Erlang programmers. A temptation might be to use it when reducing the length of lines, but this comes at the expense of comprehension. There are some cases where it makes sense:

Developing your own Erlang programming style might take years. Make sure you are consistent with yourself in any single system. This includes your use of indentation and spaces as well as your choice of variable, function, and module names. In large projects, style guidelines are often given to you.

The user should be able to predict what will happen when using the system. The foundation of this determinism originates in predictable and consistent results being returned by your functions. A consistent system in which modules do similar things is easier to understand and maintain than a system in which modules do similar things in quite different ways.

Minimize the number of functions with side effects, collecting them in specific modules. These modules could handle files or encapsulate database operations. Functions with side effects will cause permanent changes in system state. Knowledge of these states is imperative for these functions to be used and debugged, making reusability and maintainability of the code difficult. Try to reduce the number of side effects by maximizing the number of pure functions, separating side effects into atomic functions rather than combining them with functional transformations.

Some of the most challenging bugs are those caused by race conditions, especially in the advent of multicore systems in which code is truly running in parallel. Here’s a common scenario: you run a test and are able to re-create the same bug every time. As soon as you turn on trace printouts, the overhead causes the process to run more slowly, changing the order of execution. All of a sudden, your bug is not reproducible anymore. What can you do to avoid this? Make your code as deterministic as possible.

A deterministic program is one that will always run in the same manner and provide the same result or manifest the same bug, regardless of the order of execution. What makes a solution deterministic? Assume a supervisor has to start five workers in parallel and that the start order of these processes does not matter. A nondeterministic solution would spawn all of them and check that they have started correctly. A deterministic solution would spawn the processes one at a time, ensuring that each one has started correctly before proceeding to the next one. Although both might provide the same result, the nondeterministic solution may make start errors hard to reproduce, providing different results based on the order in which the processes have been spawned. Although determinism is not guaranteed to eliminate race conditions, it will certainly reduce the number of them and will make your system more predictable, as well as making debugging much more straightforward.

Abstract out common design patterns. If you have the same code in two places, isolate it in a common function.^[52] If you have similar patterns of code, try to define the difference in terms of a variable or a separate function, combining the pattern in a common call. Where appropriate, replace your recursive functions iterating on lists with a call to one of the higher-order functions in the lists module. You can encapsulate your functionality on the list elements in a fun and choose a recursive pattern using one of the higher-order functions provided in the lists module. To see what is happening to the elements in the list, all you need to do is inspect the fun. To see what recursive pattern is being applied, examine which function in the lists module is being called.

When you are designing a library of your own, you will find that many of the functions follow similar patterns, such as fold or map on lists. You can then write your own higher-order functions to encapsulate these patterns, allowing you and other users of the module to abstract their functions. With higher-order functions encapsulating common computation patterns, many recursive functions with multiple clauses and base cases become much more compact, readable, and maintainable.

Avoid defensive programming. Trust your input data if it originates from within the system, testing it only if it enters through an external interface. Once the data has entered the runtime system, it is the responsibility of the calling function to ensure that the input data is correct, and not of the function that was called. If your function is called with erroneous input data, the advice is to make it crash.

An example of defensive programming would be a function converting atoms denoting months to their respective numbers:

month('January') -> 1;
month('February') -> 2;
...
month('December') -> 12;
month(_other) -> {error, badmonth}.

It might be tempting to add the last clause acting as a catchall. If month/1 is called with an incorrect atom, it would return the tuple {error, badmonth}. This return value would either force the user of the function to test for and handle this error value, or would cause a runtime error somewhere in the system, where a function expecting an integer receives the tuple. Removing the last defensive clause would instead cause a function clause error, providing the call chain and the misspelled atom in the crash report.

Do not future-proof your code. Don’t try to write code that will be able to deal with every possible eventuality as the system evolves. It will make your code harder to understand and maintain, adding unnecessary complexity. And best of all, you will probably not end up using what you have added anyhow. Be kind to Erlang consultants supporting and maintaining your code in the years to come. Do not try to predict what will happen to your code after it has gone into production. Just implement what is needed.

Here are two final pieces of advice that are not necessarily related to Erlang, but to programming in general: avoid cut-and-paste programming and do not comment out dead code, just delete it, as you can always get it back from your repository if you need to sometime in the future.^[53]

The Erlang virtual machine is constantly being optimized and improved. What might have been inefficient constructs or necessary workarounds in earlier releases are not necessarily a problem in the current version. So, beware when reading about efficiency, workarounds, and optimizations in old performance guides, blog entries, and especially old posts in newsgroups and mailing list archives. If in doubt, always refer to the release notes and documentation of the runtime system you are using. And most importantly, benchmark, stress test, and profile your systems accordingly.

month('January') -> 1;
month('February') -> 2;
month(String) when is_list(String) -> {error, badmonth};
month('March') -> 3;
...
month('December') -> 12;
month(_other) -> {error, badmonth}.

month(String) when is_list(String) -> {error, badmonth};
month('January') -> 1;
month('February') -> 2;
...

1> Str = [$h,[$e,[$l,$l],$o]].
[104,[101,"ll",111]]
2> io:format("~s~n",[Str]).
hello
ok
3> lists:append([[1,2,3],[4,5,6]]).
[1,2,3,4,5,6]

lines(Str) ->
  "Hello " ++ Str ++ " World".

lines(Str) ->
  ["Hello ", Str, " World"].

double([X|T], Buffer) ->
  double(T, Buffer ++ [X*2]);
double([], Buffer) ->
  Buffer.

double([X|T], Buffer) ->
  double(T, [X*2|Buffer]);
double([], Buffer) ->
  lists:reverse(Buffer).

List1 ++ List2 ++ List3

lists:append([List1, List2, List3])

even_multiple(List, Multiple) ->
  Even = lists:filter(fun(X) -> (X rem 2) == 0 end, List),
  Multiple = lists:map(fun(X) -> X * Multiple end, Even),
  lists:sum(Multiple).

even_multiple(List, Multiple) ->
  Fun = fun (X, Sum) when (X rem 2) == 0 ->
              X + Sum;
            (X, Sum) ->
              Sum
        end,
  Multiple * lists:foldl(Fun, 0, List).

spawn_opt(Module, Function, Args, OptionList)

Let others review your code, as they will provide you with feedback and comments on style and optimizations. Always try to write clean and understandable code; type-specify your interfaces and model your data structures. Put effort in choosing algorithms and constructs that scale effectively. You have to think about real-time efficiency from the start, as it is not something that you can easily resolve later. And finally, never optimize your code in your first stage of development; instead, always program with maintainability and readability in mind. When you have completed your application, profile it and optimize your code only where necessary.

Common mistakes often made by beginners include:

A programmer will be able to understand a problem in full only when he has solved it at least once. And when he has found a solution, he will have thought of much better ways he could have solved the problem. Always try to go back and rewrite your code, keeping the aforementioned beginner errors in mind. You will quickly discover that your refactored program will consist of an elegant and efficient code base which, as a result, becomes easier to test, debug, and maintain.^[54] Expect code reductions of up to 50% when rewriting your first major Erlang programs. As you become more experienced and develop your programming style, this reduction will decrease as you start getting things optimal the first time around.

The golden rules when working with Erlang should always be as follows:

You will quickly discover that in the majority of cases, your code will be fast enough. Happy Erlang programming!