6
Designing a predictable API

This chapter covers

• Being consistent to create intuitive APIs
• Adding features to simplify use and adapt to users
• Adding metadata and metagoals to guide users

In the previous chapter, we started our journey to learn how to build usable APIs and discovered fundamental principles we can use to create straightforward APIs that are easy to understand and easy to use. This is good—we now know how to design a decent API. But we can do better. What about designing an awesome API? What about designing an API that users will be able to use instinctively without thinking about it, even if it is the very first time they’re using it? How can we do that?

Have you ever felt tremendous pleasure when using an unfamiliar object or application for the first time? You know, when everything is so intuitive and easy that you feel outrageously smart as you discover on your own all of its possibilities? This is possible not only because you are actually outrageously smart, but also because the thing you are using has been designed to make it totally predictable. Of course, not everything is capable of providing such a tremendous feeling, but every day you might encounter situations where predictability gives you a hand without you realizing it. Why do you know how to open a door in a building you’ve never been to? Because it looks like the doors you have encountered before. How can you use an ATM in a country using a language you don’t understand? Because it adapts its interface to you. How can you find your way in a huge and labyrinthine subway station? Because there are signs telling you where to go.

Can we really design such intuitive APIs? Yes, we can! Just like any object, an API can be predictable because it shares similarities that other users have encountered before, because it can adapt to the users’ will, or because it provides information to guide them.

6.1 Being consistent

If you encounter a washing machine like the one shown in figure 6.1, with a button showing a triangle icon oriented from left to right, you can guess this button’s purpose easily. Why? Because you’ve already seen this icon on various media players.

Figure 6.1 A washing machine and cassette player sharing the same icon

Since the mid-1960s, all media players have used such an icon. From obscure DCC (Digital Compact Cassette) players to compact disc players and software media players, each one of these devices uses the same triangle icon for the Start Play button. Therefore, you can guess that this button starts the washing machine.

I’m sure you know what the standard Pause button looks like too. What if a media player does not use this standard icon for its Pause button? Users will be puzzled and will have to make an effort to understand how to pause the audio or video that’s playing.

Again, what is true for real-world human interfaces is also true for APIs. It is essential to keep the design of an API consistent—to make it predictable. This can be done with a little bit of discipline by ensuring consistency of data and goals inside and across all APIs, using and meeting prescribed standards, and by shamelessly copying others. But if consistency can lead to an awesome API design, it must never be used at the expense of usability.

6.1.1 Designing consistent data

Data is at the core of APIs—resources, parameters, responses, and their properties shape an API. And all of their meanings, names, types, formats, and organization must be consistent in order to help consumers understand these easily. So, designing consistent APIs starts with choosing consistent names, as shown in figure 6.2.

Figure 6.2 Inconsistent and consistent naming

In a poorly designed Banking API, an account number could be represented as an accountNumber property for the get accounts goal’s result, as a number property for get account, and as a source property for the money transfer goal’s input. Here, the same piece of information in three different contexts is represented with totally different names. Users will not make the connection between them easily.

Once users have seen an account number represented as accountNumber, they expect to see that piece of information always represented by accountNumber. People are used to uniformity in design. So, whether this property is part of a fully detailed account listing, a summary of the account list, or used as a path parameter, an account number must be called an accountNumber.

When choosing names for various representations of the same concept, take care to use similar ones. When used for a money transfer to identify the source account, the original name should remain recognizable but can be altered in order to provide more information about the nature of the property; we might call it sourceAccountNumber, for example. Now consumers are able to make a connection between these properties and guess that they represent the same concept.

That also works for unrelated data of a similar type or for representing similar concepts. For example, balanceDate, dateOfCreation, and executionDay all represent a date, but the first one uses the suffix Date; the second one, the prefix dateOf; and the third, the suffix Day. Using a generic suffix or prefix in a name to provide additional information about the nature of what is named is a good practice, as long as it is done consistently. Here (figure 6.2), the good design uses the same Date suffix for all dates, but you can choose another solution as long as you remain consistent. But even correctly named, a property, for example, can still be subject to inconsistency, as shown in figure 6.3.

Figure 6.3 Inconsistent and consistent data types and formats

An account number could be represented as the string "0001234567" in the get accounts result, as the string "01234567" for get account, and as the number 1234567 for the money transfer goal’s input. Such variations will inevitably cause bugs on the consumer side. To fix them, consumers must standardize these different representations and know when to convert one type or format to another to use it in a given context.

People and software don’t like to be surprised with such inconsistencies. Once consumers have seen the first accountNumber property as a string with a specific format, they expect all other account number representations to be strings with the same format. Even if they have different names, different representations of the same concept should use the same type and format.

Choosing a data type or format can also have an overall impact on the API. How do you think consumers will react if they see the balanceDate of a bank account as an ISO 8601 string (such as 2018-03-23), the creationDate of an account represented by a UNIX timestamp (such as 1423267200), and the executionDate of a transfer as YYYY-DD-MM date (such as 2018-23-03)? They won’t like it because it’s inconsistent.

People seek global uniformity in design. Once consumers have seen one date and time property represented by an ISO 8601 string, they expect all date and time properties be ISO 8601 strings. Once a data format has been chosen for a type of data, it should be used for all representations of the same data type.

Consumers seek global uniformity in all aspects of the API, however, not just data types and formats. What is the problem with a URL’s /accounts/{accountNumber}, which represents an account, or /transfer/{transferId}, which represents a money transfer? It’s /accounts versus /transfer—plural versus singular. Once consumers are familiar with the use of plural names for collections, they expect to see all collections with plural names. You can use a singular for collections if you want, but whatever your choice, stick to it! And this doesn’t only apply to URLs: it concerns every single name and value you choose.

Now, what’s the problem with the two URLs and the two data structures shown in figure 6.4? Their data organizations are inconsistent.

Figure 6.4 Inconsistent organization

The /accounts/{accountNumber} and /transfers/delayed/{transferId} URLs don’t have the same organization. The /transfers/delayed/{transferId} URL introduces an unexpected level between the collection name and resource ID, making the URL harder to understand. We could use /delayed-transfers/{transferId} instead, for example.

Each level of a URL should always have the same meaning. Once consumers are used to a data organizational pattern, they expect to see it used everywhere. Again, this doesn’t only apply to URLs; data organization in inputs and outputs can present patterns too. In the bottom part of figure 6.4, the elements of two collections are represented in two different ways. If every collection resource you have designed so far is represented by an object containing a property called items, which is an array, do not dare to design one as a simple array. Why? Because consumers will be surprised by this variation.

Once data organization conventions have been chosen, follow those strictly. Basically, every bit of an API’s data must be consistent. But APIs are not only made of static data; they are made to do things, and all of an API’s behaviors must be consistent too.

6.1.2 Designing consistent goals

An API behavior is determined by its goals: these await inputs and return success or error feedbacks, and all these goals can be used to form various flows. Obviously, all of this must be consistent.

What’s the problem with the read account and get user information goals? These two goal names are inconsistent—they represent the same type of action but use different verbs. It would be wiser to name them read account and read user information, especially if they have to be represented as functions in code, like readAccount() and readUserInformation(). Hopefully, for REST APIs, the programmatic representation of these goals will magically be consistent, thanks to the use of the HTTP protocol. Indeed, both of these goals will be represented by a GET /resource-path request using the same HTTP method (on different paths).

As you saw in section 6.1.1, data must be consistent, and so must a goal’s inputs. For example, it could be helpful when listing an account’s transactions with a GET /accounts/{accountId}/transactions request to be able to retrieve only the ones that occurred between two dates. Query parameters such as fromDate=1423267200 and untilDay=2015-03-17 can do the job but are obviously inconsistent. It would be better to use fromDate=2015-02-07 and toDate=2015-03-17. You must use consistent names, data types, formats, and organization when designing a goal’s inputs.

The same goes for the feedback returned in case of success or error. If all goals leading to the creation of something return a 200 OK status code without taking advantage of codes such as 201 Created or 202 Accepted, it would be wise to avoid introducing new goals returning different success HTTP status codes instead of the usual 200 OK. Indeed, you are free to use only a small subset of all existing HTTP status codes; that can make sense in some contexts. But even if consumers are supposed to treat any unexpected 2XX status as a 200 OK, such inconsistency might surprise some of them.

Consistency matters for error messages too. You must obviously return consistent HTTP status codes to signify errors, but the informative data returned must also be consistent. You learned to design such data in section 5.2, and if you have defined generic codes like MISSING_MANDATORY_PROPERTY to signify that a mandatory property is missing, always use this code across your API.

Finally, consistency concerns not only how the API looks, but also how it behaves. If all previously designed sensitive actions (like a money transfer) consist of a control <action> and a do <action> goal—the first one doing all possible verifications without executing the action, and the second one actually executing the action—any new sensitive actions must be represented with goals having the same behavior. When designing APIs, you must also take care to create consistent goal flows.

So every single aspect of the interface contract, every behavior of an API, must be consistent. But we only talked about being consistent within an API; indeed, this is only the first level of consistency in API design.

6.1.3 The four levels of consistency

When you look at the buttons of a TV remote, you can see that these are consistent; the number buttons, for example, all have the same shape. If you look at a TV remote and a Blu-Ray or a DVD player from the same manufacturer, these may have little differences, but they are mostly consistent as they usually share common features (the same number buttons, for example). If you look at any media device or media player, these too are consistent, presenting the same controls, especially the play button. There may be some little differences again, but you still feel comfortable switching from one to the other. And finally, if you encounter a play button on a washing machine, you know what its purpose is because you probably have seen it before, perhaps on different media players. These examples show four levels of consistency that can be applied to the design of APIs:

• Level 1 —Consistency within an API
• Level 2 —Consistency across an organization/company/team’s APIs
• Level 3 —Consistency with the domain(s) of an API
• Level 4 —Consistency with the rest of the world

We have just seen the first level, how an API must be consistent with itself—proposing consistent data, goals, and behaviors. Every time you make a design choice, you must ensure that it will not introduce a variation in the API, or worse, a contradiction. When looking at the API as a whole, consumers must see a regular interface. When they jump from one part to another, they must have the feeling that this new part is familiar. They must be able to guess how this new part works even if they have never used it before.

Just as consistency is important within an API, it is also important across the APIs that an organization provides. This is the second level of consistency. An organization might have one team with a single API designer or multiple teams with many designers, and everything in between. The consumers of an organization’s APIs do not care if these APIs were designed by one or many designers. What they care about is that the APIs share common features so they can understand and use any part of the API easily, once they have learned to work with one.

Just as different goals within an API must share common features, different APIs within an organization must also share common features. Sharing common features (such as data organization, data types, or formats) enhance interoperability between APIs. It’s easier to take data from an API and feed it to another if the features are consistent.

The third level is about being consistent with the domain(s) of or used by an API. For example, representing an address is not done in the same way if you just want to get some customers’ addresses compared to if you want to provide formatted addresses to be printed on envelopes. If you have to calculate distances in an API for marine navigation, you will use nautical miles and not miles or kilometers. There usually are standard or at least common practices that you must follow when working on a specific domain.

And the fourth and last level: APIs have to be consistent with the rest of the world. There are common practices—standards, if you will—that you can use. Not only does following these make your APIs predictable for people who have never used any of your APIs before, thereby enhancing your APIs interoperability with the rest of the world, but it also makes your API designer’s job easier. Let’s see how this is possible.

6.1.4 Copying others: Following common practices and meeting standards

Why reinvent the wheel when someone has already done that? There are thousands of standards that you can use in your API; there are thousands of APIs whose designers design using common practices, and there are some reference APIs that you can shamelessly copy. You know the meaning of the play and pause symbols shown in figure 6.5 because you have seen them on various devices.

Figure 6.5 Play and pause symbols defined by the ISO 7000 standard

You might have learned their meaning by reading the user manual of the first device using them that you encountered, but after that, every time you saw these symbols, you were able to guess what they meant. On each device using these symbols, their purpose is the same.

The look and meaning of the play and pause symbols are defined by the ISO 7000 standard (https://www.iso.org/obp/ui/#iso:pub:PUB400008:en). Once users have encountered these symbols on one device, they are able to guess their purpose on any other device. They are able to use a new device without prior experience with it because they have experience with other devices using the same standards. Any designer willing to create a start/pause-something button will probably use these symbols instead of reinventing new ones. Like any real-world device, an API can take advantage of standards (in a broad sense) to be easier to understand.

Our Banking API might have to provide information about amounts in various currencies. Creating our own currency classification and always using it in all of our banking-related APIs is a good thing. That way, we are at least consistent within our organization. But it would be better to use the ISO 4217 international standard (https://www.iso.org/iso-4217-currency-codes.html), which lets you represent currencies by a three-letter code (USD, EUR) or a three-digit code (840, 978). Using such a standard, we can be consistent with the entire world! Anyone who has ever used the ISO 4217 standard elsewhere will understand the meaning of the ISO 4217 currency codes without having to learn a nonstandard classification. Similarly, the ISO 8601 standard we saw earlier to represent date and time values is not only a human-friendly format, but is also widely adopted in the software industry.

There are standards for data formats, data naming and organization, and even processes. And not all of them are defined by the ISO; there are many other organizations defining standards and recommendations that you could use in your API design. Use your favorite search engine and look for something like “<some data> standard” or “<some data> format,” and you will probably find a format that you can use to represent “<some data>.” Try to find out how to represent phone numbers, for example.¹ 

¹ You should find the E.164 format, which is recommended by the ITU-T (the ITU Telecommunication Standardization Sector).

But being standard does not always mean following ISO’s or another organization’s specifications. If our Banking API represents a delayed transfer with /delayed-transfers/{transferId} URL, you might guess that using the HTTP method DELETE would cancel a delayed transfer. If you get a 410 Gone response, you might guess that the delayed transfer was executed or canceled before you tried to delete it. How can you guess that? Because you expect the Banking API, which claims to be a REST API, to strictly follow the HTTP protocol, which is defined by RFC 7231 ([https://tools.ietf.org/html/rfc7231).

The DELETE HTTP method can be used on a resource to delete, undo, or cancel the concept represented by a URL. The 410 Gone response is quite explicit; according to the standard, it “… indicates that the resource requested is no longer available and will not be available again.” And further, it “… should be used when a resource has been intentionally removed and the resource should be purged.”

So, REST APIs can be consistent by simply applying the HTTP protocol’s rules to the letter. That way, anyone can quickly get started using any REST API.

In the API design world, there are common practices that can be followed, such as the one shown in figure 6.6.

Figure 6.6 A common URL pattern

As you saw in section 3.2.3, while there are no standard rules for the URL structure of REST APIs, many of these use the /resources/{resourceId} pattern. Here, resources is a collection identified by a plural noun. It contains elements of type resource.

Even if not everything is standardized in the API design world, there are common practices that are very close to being standards. It is wise to follow them in order to ensure that your API will be easily understood by consumers based on their experience with other popular APIs.

And finally, in many cases, you can simply copy what others have done. Why bother rethinking pagination parameters from the ground up when this problem has been dealt with by so many other API designers? You can look at some well-known APIs and reuse the design you prefer. Doing this will simplify your life as an API designer, and if your users have used those APIs, they will feel at home when using yours for the first time. Everybody wins.

Here’s a scenario to practice this topic. Let’s say you have to design an API that processes images to create a matching color palette for web developers. Free-tier users can request up to 10 palettes per month; if they want more they have to purchase a subscription. Both free and paid users can send a maximum of one request per second. Sent images cannot be larger than 10 MB. Consider the following questions:

• How would you represent colors in a standard way that fits web developers’ needs?
• Which explicit HTTP status codes could you use to tell

• Free-tier users they have to pay to get more?
• Any users they have exceeded their requests-per-second quota?
• Any users that an image is too large?

• Which RFC (Request For Comments) could you use to provide straightforward error feedback?
• Bonus: fully design and describe this API using the OpenAPI Specification.

As this section has demonstrated, being consistent within and across APIs is a good thing. It lessens the need for practicing with our APIs to use them effectively—all we need to do is follow commonly used standards or practices and even (shamelessly or not) copy others. It also makes our APIs interoperable, especially when using conventional standards. And as icing on the cake, it makes our job as API designers easier so that we don’t lose time reinventing the wheel. Consistency seems to simplify everything; but unfortunately, that’s not always true.

6.1.5 Being consistent is hard and must be done wisely

You must be aware of two things about consistency: it’s hard to be consistent, and consistency must not be applied blindly. Being consistent across APIs simply requires following the same conventions when designing different APIs. It also requires knowing which APIs actually exist. That is surprisingly hard and requires discipline.

You must formally define your design with rules in a document called the “API Design Guidelines” or the “API Design Style Guide.” Even if you are the only API designer in your organization, such guidelines are important because over time we tend to forget what we have done previously (even in a single API). Defining such guidelines not only facilitates standardizing the overall organization’s API surface, it also facilitates the API designer’s job. You will learn in chapter 13 how to create such guidelines.

You’ll also need access to existing API designs in order to stay consistent with your organization’s APIs. Once you have your API design cheat sheet and your API directory, you can concentrate on solving real problems and not waste your time reinventing the wheel you created a few months ago.

Consistency is good but not at the cost of usability or common sense. Sometimes you will realize that if you push consistency too far it will ruin flexibility and make the process of design outrageously complicated, leading to consistent but totally unusable APIs. The important thing is to understand that sometimes you can be inconsistent because of a given context for the sake of usability (as we saw in section 3.4 when talking about design trade-offs). You will discover also in chapter 11 that there is not a single way of creating APIs: designing APIs requires us to adapt to the context.

So being consistent is a great way of being predictable, and doing so helps consumers intuitively use your API. But you can also cheat and let people choose what they want to get out of using your API, which makes it even more predictable.

6.2 Being adaptable

When you buy a book from an online retailer, you can often buy it in different versions. It might be sold as a printed book, an e-book, or an audio book. It can also be presented in various languages, like the French translation of this book, Le Design des APIs Web, that I hope to do one day. All these versions are different representations of the same book; it’s up to you to indicate which version you want when you add the book to your shopping cart.

And when you read the ordered book, you usually do not read it all at once. You read it page by page, and when you stop, you mark the last page you read. When you continue reading, you jump directly to the last page read without rereading the book from the beginning. When you read some kinds of books, especially technical ones, you might jump directly to a specific chapter or section and, hence, a specific page, so you might not read the chapters in their natural order. You might also only read the parts concerning a specific topic.

Managing different representations of the same concept and providing a partial, selected, or adapted representation of some content is not reserved for books; we can do that with APIs too. We can create such an adaptable API design that helps to make the API predictable and also satisfy different types of users. If consumers can specify what they want, they can predict what they will get.

Here we discuss three common ways of making an API design adaptable: providing and accepting different formats; internationalizing and localizing; and providing filtering, pagination, and sorting features. This is not an exhaustive list of options—you can find others and even create your own when needed.

6.2.1 Providing and accepting different formats

JSON is the obvious way of representing an account’s transactions list in our Banking API. As shown on the left side of figure 6.7, a list of transactions could be an array of JSON objects, each composed of three properties: a date, a label, and an amount.

Figure 6.7 A list of transactions as JSON and CSV

But JSON is not the only option. The right side of the figure shows the same data represented in a comma-separated values (CSV) format. In this case, a list of transactions is represented by lines of text. Each line representing a transaction is composed of three values separated by commas (,): the first one is the date, the second one the label, and the third one the amount. This transactions list could also be represented by a PDF file. You can also allow consumers to choose among different formats, depending on their needs. The only limit is your imagination.

But if the get account’s transactions goal can return a list in various formats, how can consumers tell which format is needed? Figure 6.8 shows two different ways to do so.

Figure 6.8 Two options to request the transactions list as a CSV document

As shown on the left in figure 6.8, we could add a format parameter to this goal in order to let consumers specify if they want the transactions list in JSON, CSV, or PDF format. To get the list as a CSV document, for example, consumers could send a GET request that specifies format=CSV like so:

GET /accounts/{accountId}/transactions?format=CSV

That’s a possibility, but because the Banking API is a REST API, we could also take advantage of the HTTP protocol and use content negotiation. When sending the GET /accounts/{accountId}/transactions request to the API server, consumers can add an Accept: text/csv HTTP header after the HTTP method and URL to indicate that they want this transactions list as CSV data. (This approach is shown on the right in figure 6.8.) If everything is OK, the API server responds with a 200 OK HTTP status code followed by a Content-type: text/csv header and the list of transactions as a CSV document.

Consumers can also send Accept: application/json or Accept: application/pdf to get JSON data or a PDF file, respectively, with the server returning a response with a Content-type: application/json or Content-type: application/pdf header followed by the document in the appropriate format. This example has introduced two new features of the HTTP protocol: HTTP headers and content negotiation. Let’s take a closer look at these.

HTTP headers are colon-separated name/value pairs. They can be used in both requests and responses to provide some additional information. In a request, these headers are located after the request line containing the HTTP method and URL. In a response, they are located after the status line containing the HTTP status code and reason phrase. There are around 200 different standard HTTP headers, and you can even create your own if needed. They are used for various purposes, one of which is content negotiation.

Content negotiation is an HTTP mechanism that allows the exchange of different representations of a single resource. When an HTTP server (hence a REST API server) responds to a request, it must indicate the media type of the returned document. This is done in the Content-type response header. Most REST APIs use the application/json media type because the documents these return are JSON documents. But consumers can provide an Accept request header containing the media type they want to get. As shown in figure 6.9, in the Banking API, the three possible media types for the account’s transactions list are application/json, application/pdf, and text/csv.

Figure 6.9 Requesting three different representations of an account’s transactions list

If the consumer requests a media type like audio/mp3 that the provider does not handle, the server will respond with a 406 Not Acceptable error. Note that a request without an Accept header implies that the consumer will accept any media type. In that case, the server will return a default representation—JSON data, for example.

That also works when the consumer has to provide data in the body of the request. In section 5.2.1 in the last chapter, you saw that to create a transfer, consumers have to send a POST /transfers request, whose body contains a source account, a destination account, and an amount. This body was expected to be a JSON document, but it could also be of another media type. For example, a consumer might send an XML document containing the information needed to create a transfer.²  To do that, they must provide the Content-type: application/xml header. If the API server is unable to understand XML, it returns a 415 Unsupported Media Type error. If the consumers also want to get the result as an XML document instead of a JSON one, they must provide the Accept: application/xml header along with the Content-type header. This tells the server, “I am sending you XML and I would like you to respond using XML too.”

² XML (eXtensible Markup Language) is a markup language that is supposed to be both human- and machine-readable. This was the de facto standard for API and web services before JSON. In XML, a property such as the amount would be represented as <amount>123.4</amount>.

That’s great—content negotiation, whether provided by the protocol used or handled manually, lets consumers choose the format they want to use when communicating with an API, as long as it is supported. But we can do more than that.

6.2.2 Internationalizing and localizing

Even translated into French, the Le Design des APIs Web e-book is only another representation of the same book. How can we apply this concept to the Banking API example?

Back in section 5.2.3, you learned to design straightforward error feedback. When the customer attempts a money transfer, for example, the API can return an error with the message Amount exceeds safe to spend. Such a message could be shown to all end users—but what if they do not understand English? The developers building the application or website using the Banking API will have to manage translation of this message on their side. From a technical point of view, this is possible because the error is identified with a clearly identifiable AMOUNT_OVER_SAFE type. But maybe we, the API designers, can give a hand to developers using our API and propose a way to get error messages in languages other than English.

We could add a language parameter to all the Banking API goals, with its value being an ISO 639 language code (http://www.loc.gov/standards/iso639-2/php/code_list.php). For example, the ISO 639 code fr stands for French, and en stands for English. In section 6.1.4, you learned that using standards is good in order to ensure that a value will be easily understandable and interoperable. But wait—en simply means English. UK English and US English can be considered different languages, just like French and French Canadian. So, ISO 639 isn’t a good idea; it would be better to use a more accurate standard to identify the language.

RFC 5646 (https://tools.ietf.org/html/rfc5646), which defines language tags, is the standard we’re looking for. This format uses ISO 639 language codes and ISO 3166 country codes: US English is en-US, UK English is en-UK, French is fr-FR, and French Canadian is fr-CA.

As you can see, choosing a standard might not be straightforward. You have to be careful when choosing one and be sure that it really fulfills your needs.

Now that we’ve found the right standard, we can create translations of all our error messages in all the languages we want to support. For example, when using the transfer money goal with the language parameter set to fr-FR, the AMOUNT_OVER_SAFE human-readable error message might be Le montant dépasse le seuil autorisé. Note that any text returned by the API, not only error messages, can be returned in the language indicated in the language parameter. It can also be represented as a query parameter, but because the Banking API is a REST API, we can take advantage of the HTTP protocol to provide it instead.

Content negotiation not only applies to data formats, but also to languages. Just as consumers can use the Accept and Content-type HTTP headers to specify a media type, they can also use the Accept-Language and Content-Language headers to indicate which language they are speaking, as shown in figure 6.10.

Figure 6.10 Negotiating content language with an API

When using POST /transfers to transfer money, if consumers provide no headers, the API server can return a response with a Content-Language: en-US header to indicate that any textual content is in US English. If, however, consumers provide an Accept-Language: fr-FR HTTP header with their requests to indicate that they want to get textual content in French, the API server responds with a Content-Language: fr-FR header, and any textual data will be translated into French. If the requested language—Italian (it-IT), for example—is not supported, the server returns a 406 Not Acceptable HTTP status code. Because this status code can also be returned when the consumer requests a media type that is not supported, it’s also a good idea to provide straightforward error feedback with a clear error code like UNSUPPORTED_LANGUAGE and a message like Requested language not supported.

Adapting data values to developers, their applications, and their end users isn’t only about language translation, though. In the US, for example, people use the imperial system of measurement, while in France they use the metric system. People in the US and France do not use the same units, the same date and number formats, or the same paper sizes. Being able to adapt to all these variations is possible if your API supports internationalization and localization (often called i18n and l10n, with the numbers indicating the number of characters between the first and last letter of the word).

For our REST Banking API, internationalization means being able to understand that an Accept-Language: fr-FR header means that the consumer wants a localized response using French language and conventions. On the server side, it means that if the requested localization is supported, the content will be returned localized along with a Content-Language: fr-FR header. If it’s not supported, the server returns a 406 Not Acceptable status code.

For the Banking API, localization means being actually able to handle the fr-FR locale. The data returned should be in French, using the metric system, and a PDF should be generated using the A4 size and not the US letter size, for example. This topic is not specific to APIs; these issues apply to all areas of software development.

But as API designers and providers, should we really care about internationalization and localization? This is a totally legitimate question that must be answered sooner or later when designing an API. It depends on the nature of your API and the targeted consumers and/or their end users. If you’re lucky, the data exchanged through your API might not be impacted at all by localization concerns, so you might be able to bypass it. If you do not target people in different locales, you might not need to handle internationalization. Be cautious, though, because sometimes people can use different locales in the same country (the en-US or es-US locales in the US, for example).

If you don’t think you need them, you can start without internationalization features and update your API later if needed. But be aware that while adding internationalization features to an existing API can be done easily and transparently from the consumer’s point of view, modifying an implementation that was not built with internationalization in mind might be trickier.

Note that there are other aspects of content negotiation, like priorities when requesting multiple variants of a resource and content encoding, that we will not explore in this book. You can read more about this in RFC 7231 (https://tools.ietf.org/html/rfc7231).

We’ve seen that consumers can specify not only the data format they want to use but also the language and units, for example. Is it possible to provide an even more customizable API in order to be even more predictable? Yes, it is!

6.2.3 Filtering, paginating, and sorting

A bank account that has been open for many years can have thousands of transactions in its history. A customer who wants to get an account’s transactions using the Banking API probably does not want to see all of these transactions at once and would prefer to get a subset. Maybe they want to see the 10 most recent ones and then possibly go deeper into the list. As shown in figure 6.11, this could be done by adding some optional parameters to this goal, such as pageSize and page.

Figure 6.11 Simple pagination

On the server, the transactions list is split virtually into pages, with each page containing pageSize transactions. If pageSize is not provided, the server uses a default value, and if page is not provided, the server returns the first page by default. To get the first page of 10 transactions, consumers would have to provide pageSize=10 and page=1. To get the second page of 10 transactions, they would provide pageSize=10 and page=2.

In our REST Banking API, these pagination parameters could be passed as query parameters, as in GET /accounts/1234567/transactions?pageSize=10&page=1. But we could also take advantage of the HTTP protocol and use the Range HTTP header. To get the first page of 10 transactions, this header would be Range: items=0-9. To get the next page, the header is Range: items=10-19.

The Range header was created to allow a web browser to retrieve a portion of a binary file. The value of a request’s Range header is <unit>=<first>-<last>. A standard unit is bytes, so bytes=0-500 would return the first 500 bytes of a binary file.

We can use a custom unit like items. Sending a Range header with the items=10-19 value tells the server, “I want the collection’s items from indexes 10 to 19.” I could have chosen another unit name, such as transactions, but that would mean that if we wanted to paginate the /accounts collection resource, the unit would be accounts. The unit name used to paginate can be guessed from the collection name, but I prefer to favor the generic name items. That way, there’s no need to guess the unit for paginating collections: it is always the same.

If consumers want a subset of transactions, however, they might like to have more control over this subset. Maybe they only want to see transactions that have been categorized as restaurant transactions. To get these specific transactions, the consumer might send a GET /accounts/1234567/transactions?category=restaurant request. The category query parameter is used here to filter the transactions and only return the ones categorized as restaurant.

This filtering example is a really basic one. If you want to practice, here’s a problem you will have to solve sooner or later as an API designer: filtering a collection on numerical values. Let’s say you’re designing an API dealing with secondhand cars. Users should be able to list available cars having a mileage between two values using a GET /cars request and one or more query parameters. Using a natural language, such a query would be something like, “List cars with mileage between 15,000 and 30,000 miles.” Try the following exercises:

• Find a way of designing such a filter.
• Find at least two other ways of doing the same thing by searching through existing APIs (or API design guidelines).
• Decide which one you prefer.
• Bonus: for all these different ways, describe the request and its parameter(s) using the OpenAPI Specification.

By default, transactions are ordered from latest to oldest; when consumers request transactions, they get the latest first. Consumers might also want to get the transactions sorted by descending amounts (higher amounts first) and in chronological order (from oldest to latest). To get such a list, they might send a request like this:

GET /accounts/1234567/transactions?sort=-amount,+date

The sort query parameter defines how the transactions list should be sorted. It contains a list of direction and property couples. The direction + is for ascending and - is for descending. The -amount and +date values tell the server to sort the transactions by amount in descending order and by date in ascending order. Note that this is only one way of providing sorting parameters; it can be done in other ways too.

These pagination, filtering, and sorting features can be used together. Using the category=restaurant&sort=-amount,+date&page=3 query parameters in a GET /accounts/1234567/transactions request returns the third page of the restaurant transactions, ordered by descending amount and ascending date.

As this section has demonstrated, besides making our API look familiar, a good way of making it predictable is to let consumers say what they want and give it to them. A third way of making an API predictable is by giving consumers some clues about what they can do with it.

6.3 Being discoverable

In most books, you know which page you are reading because its number is printed on it. Sometimes, the current chapter or section is also indicated at the top or bottom of the page. Earlier, we saw that when you read some books, you can jump directly to a specific chapter or section. This is possible because the book comes with a handy table of contents (TOC) listing the chapters and sections and on which pages they start. When you read a book, therefore, you read its content, but you also have access to additional information about the content itself. You could read the book without using this extra information; if all of it were removed, the content would be unaffected. But reading the book would be far less convenient.

If the book is a novel, not having a TOC (or even page numbers) isn’t really a problem. A novel is more interesting read page after page, without being spoiled by a too-explicit preview of the contents (like “Chapter XI: The character you have become so attached to dies”). But if the book is a practical one, like the one you are reading now, you might want to scan the TOC before you begin reading to get a better idea of what the book is about in order to be sure it is relevant for you. You might also want to jump to a specific section because you have a specific problem to solve. Without a TOC and page numbers, it would not be easy to find what you are looking for. This additional information makes a book discoverable. It’s not mandatory, but it greatly improves the reading experience.

Like books, APIs can be designed in order to be discoverable. This is done by providing additional data in various ways, but discoverability can also be improved by taking advantage of the protocol used. REST APIs have the discoverable feature in their genes because they use URLs and the HTTP protocol.

6.3.1 Providing metadata

In section 6.2.3, you discovered the pagination feature. When accessing an account’s transactions list, consumers of the Banking API can indicate which page of transactions they want. But how do they know that there are multiple pages available?

For now, the server’s response when consumers request an account’s transactions list consists only of an object containing an items property, which is an array of transactions. Sounds like a book without page numbers and a TOC. This response could be improved by adding some data about pagination, as shown in figure 6.12.

Figure 6.12 Providing metadata to explain “Where am I and what can I do”

If a first call to the get account’s transactions goal is made without pagination parameters, the server could return the items array along with the current page’s number (page, whose value is 1) and the total number of pages (totalPages, whose value could be 9). This will tell consumers that there are eight more pages of transactions ahead.

Thanks to the additional data, the transactions list is now discoverable. In computer science such data is called metadata; it’s data about data. Metadata can be used to tell consumers where they are and what they can do.

Let’s look at another example, just to show that metadata is not limited to pagination. Using the Banking API, consumers can transfer money from one account to another immediately or on a predefined date. When listing past transfer requests, the server can return both executed and postponed requests. An already executed request cannot be canceled, but a postponed one that has not yet been executed can be. As shown in figure 6.12, we could add some metadata describing the possible actions on each transfer request. For an already executed money transfer, the actions list would be empty. For a postponed one, it could contain a cancel element. This will tell consumers which ones they can use the cancel money transfer goal on.

As you can see, an API can return metadata along with data in order to help consumers discover where they are and what they can do. The API can be used without this extra information, but metadata greatly facilitates its use. By adding metadata, we are basically applying what we learned in section 5.1—we are providing ready-to-use data. This can be done with any type of API. Depending on the API in question, you can rely on other mechanisms to provide such information, especially by taking advantage of some of your chosen protocol’s features.

6.3.2 Creating hypermedia APIs

Using the REST Banking API, consumers can retrieve a list of accounts by calling GET /accounts. Each account comes with a unique id that can be used to build its URL (/accounts/{accountId}) and to retrieve detailed information about it using the GET HTTP method. This id can also be used to retrieve an account’s transactions with GET /accounts/{accountId}/transactions. Thanks to the pagination metadata we have just added, consumers will know if there are more transactions than the ones returned by their first call. In that case, they can use GET /accounts/{accountId}/transactions?page=2 to get the next page of transactions. They can even jump directly to the last page of transactions. They just have to take the lastPage value and use it to GET /accounts/{accountId}/transactions?page={lastPage value}.

Sounds like a well-designed API with crystal-clear URLs and even metadata that helps consumers, right? Now, let’s imagine a situation where you’re browsing a bank’s website and all the hypermedia links have been removed. Would you be happy as a customer to have to read a user’s manual to learn what all the available URLs are? If you wanted to see detailed information about one of your accounts, would you be happy about having to construct the page’s URL yourself by copying and pasting the account number? The World Wide Web without its hypermedia links would be quite terrible to use.

Fortunately, this isn’t how it works. Once on a website, you can discover its content simply by clicking links and going from one page to another. REST APIs rely on World Wide Web principles, so why not take advantage of these? As shown in figure 6.13, a hypermedia Banking API would provide an href property for each account returned by GET /accounts.

Figure 6.13 Hypermedia Banking API

For the 1234567 account, its value would be /accounts/1234567. Consumers wanting to get access to this account’s detailed information would then just have to GET this ready-to-use relative URL without needing to construct it themselves. And the response to this request would have a transactions property, whose value could be an object containing an href property with a value of /accounts/1234567/transactions.

Again, consumers would just have to GET this href value to get the account’s transactions list. And of course, the pagination metadata would provide URLs like next and last using properties whose values could be /accounts/1234567/transactions?page=2 and /accounts/1234567/transactions?page=9, respectively. Consumers would then be able to browse the API without the need to know its available URLs and their structures.

REST APIs provide links just like web pages. This facilitates API discovery and, as you will see later in this book, API updating. There is no standard way to provide this hypermedia metadata, but there are common practices, mostly based on how links are represented in HTML pages and the HTTP protocol.

Hypermedia metadata usually uses names such as href, links, or _links. Although there is no standard, several hypermedia formats have been defined. The best-known ones are HAL, Collection+JSON, JSON API, JSON-LD, Hydra, and Siren. These formats come with differing constraints regarding the data structure.

HAL (http://stateless.co/hal_specification.html) is relatively simple. A basic HAL document has a links property containing the available links. Each link is an object identified by its relationship (or _rel) with the current resource. The self relationship is used for the resource’s link. The link object contains at least an href property with the full URL or relative URL. For a bank account resource, the link to its transactions would be located there as transactions as the following listing shows.

Listing 6.1 A bank account as a HAL document

{
  "_links" : {
    "self": {
      "href": "/accounts/1234567"    ①  
    },
    "transactions": {
      "href":"/accounts/1234567/transactions"    ②  
    }
  }
  "id": "1234567",
  "type": "CURRENT",
  "balance": 10345.4
  "balanceDate": "2018-07-01"
}

①   Link to the bank account resource itself

②   Link to the bank account’s transactions

The concept of link relationship is not specific to HAL; it is defined by RFC 5988 (https://tools.ietf.org/html/rfc5988). Hypermedia APIs do not only provide available URLs; they can also provide available HTTP methods. For example, with the Siren hypermedia format (https://github.com/kevinswiber/siren), we can describe the cancel action on a postponed money transfer. Siren also comes with constraints regarding the data structure: the properties are grouped in properties, links to other resources are located in links, and actions are in actions. The next listing shows an example of a Siren document.

Listing 6.2 A money transfer as a Siren document

{
  "properties" : {    ①  
    "id": "000001",
    "date": "2135-07-01",
    "source": "1234567",
    "destination": "7654321",
    "amount": "1045.2"
  },
  links: [    ②  
    { "rel": ["self"],
      "href": "/transfers/000001" }
  ],
  actions: [    ③  
    { "name": "cancel",
      "href": "/transfers/000001",
      "method": "DELETE" }
  ],
}

①   Groups a resource’s properties under properties

②   Equivalent to HAL’s _links

③   Describes an action with a name, URL, and HTTP method

Providing hypermedia metadata is the most common way of taking advantage of the web roots of REST APIs to create predictable APIs, but the HTTP protocol provides features that can be used to make REST APIs even more predictable.

6.3.3 Taking advantage of the HTTP protocol

So far, we have used GET, POST, PUT, DELETE, and PATCH HTTP methods. The following listing shows that an OPTIONS /transfers/000001 request can be used to identify the available HTTP methods on a resource.

Listing 6.3 Using the OPTIONS HTTP method

OPTIONS /transfers/000001
 
200 OK
Allow: GET, DELETE

If the API server supports this HTTP method and the resource exists, it can return a 200 OK response along with an Allow: GET, DELETE header. The response clearly states that GET and DELETE HTTP methods can be used on /transfers/000001. Like metadata that can provide information about the data (section 6.3.1), such metagoals can provide information about API goals.

Earlier in this chapter, you saw that an account’s transactions list could be returned as a JSON, CSV, or PDF document. The following listing shows that when responding to a GET /accounts/1234567/transactions request, an API server can indicate other available formats with a Link header.

Listing 6.4 A response indicating other available formats with the Link header

200 OK
Allow: GET
Content-type: application/json    ①  
Link: </accounts/1234567/transactions>;    ②  
           type=application/pdf,    ②  
      </accounts/1234567/transactions>;    ②  
           type=text/csv    ②  
 
{
 "items" : [
   ...
 ]
}

①   The transactions list returns as a JSON document.

②   It is also available in PDF and CSV formats. (Note that it is actually a single line.)

Note that such use of the HTTP protocol by REST APIs cannot be widespread. Like choosing a standard, you should check if such features are really useful to consumers. If so, you might have to explain them in detail in your documentation for people who are not HTTP protocol experts.

If you want to practice the lessons from this section, you can try updating the Shopping API we were working with in chapters 3 and 4, using the OpenAPI Specification as follows:

• Add hypermedia features using HAL (or Siren) to represent links between resources.
• Add pagination, filtering, and sorting features with relevant metadata and hypermedia controls.
• Add a content negotiation feature to support the CSV format.
• Add the OPTIONS HTTP method where necessary.

In the next and final chapter about usability, you will learn to organize and size your APIs in order to keep them usable.

Summary

• To create APIs whose operations can be guessed, consistently define conventions and follow common practices and standards.
• Being consistent in your design not only makes your API easier to use, but also makes its design simpler.
• Always check if your API needs to provide different representation and/or localization and internationalization features.
• For each goal dealing with lists, consider whether paging, filtering, and sorting features will facilitate its use.
• In order to guide consumers, provide as much as metadata as possible (like hypermedia links, for example).
• Always check the underlying protocol and use its available features to make your API predictable, while taking care not to confuse users with complex or totally unused features.