7
Designing a concise and well-organized API
This chapter covers
- Organizing an API’s data, feedback, and goals
- Managing the granularity of data, goals, and the API
Now that you know how to design straightforward and predictable APIs, we have one last thing to cover in order to be sure we’re designing usable APIs. TV remote controls sometimes look intimidating with their numerous and not always well-organized buttons. Some microwave ovens or washing machines offer far too many functions for mere mortals. Overwhelming, disorganized, indistinct, or motley everyday interfaces, at best, puzzle their users and, at worst, frighten them.
“Less is more” and “a place for everything and everything in its place” are two adages that every API designer should apply. Organizing and sizing an API’s data, feedback, and goals is important in order to provide an API that can be easily understood and will not overwhelm users. If this is not done, all that we have learned about creating straightforward and predictable APIs is worth nothing.
7.1 Organizing an API
If you have ever used a TV remote control, you should be able to understand the meaning of any button on the four examples shown in figure 7.1. All of them propose exactly the same functions, using 15 buttons; but, depending on the buttons' organization, the usability goes from terrible to perfect.
Figure 7.1 The organization of a TV remote control’s buttons affects its usability.
On the first remote control (on the left), the buttons are randomly placed, making it difficult to find any given one. On the second remote control, buttons are grouped by type, making those easier to find. If users are looking for the 7 button, for example, they know they can find it in the group of number buttons. The channel (CH) + and - buttons and volume (VOL) + and - buttons are also grouped together.
But just grouping buttons is not sufficient. On the third remote control, the buttons in each group have been sorted. The 7 button is now easier to find thanks to the ascending sorting in the numbers group. Putting the volume + button on top of its group is also better for two reasons:
- +/- are usually placed that way to mimic what they represent: up/down.
- The channel + button is on top of its group, so this change means the two groups are consistent.
Finally, the fourth remote control’s button groups are rearranged to make it even easier to use. The power button is placed at the top to prevent it from being pressed inadvertently while the user is holding the remote control. The channel and volume button groups are also switched to match common practice on remote controls. Now that the buttons are placed intuitively, you can easily use the remote control—maybe even in the dark while watching a movie.
As this example shows, it’s easier to find a specific element in a set and understand its purpose if elements are logically grouped and sorted. It’s the same for APIs—organization matters. Just like an everyday object, an API can be either unusable or perfectly intuitive, depending on the organization of its data, feedback, and goals.
7.1.1 Organizing data
Figure 7.2 Grouping data in the bank account’s representation
Designing a well-organized API starts with the data, so let’s look at a concrete example. The initial bank account representation shown in figure 7.2 (a) contains two highlighted properties: overdraftProtection and limit. The overdraftProtection property indicates whether overdraft protection is active on the account, and limit tells how much overdraft protection is available. Here its value is 100, meaning that any transaction that causes the account to go more than $100 over its balance will be blocked. These two properties are related, but nothing explicitly tells us that in the design.
We can make some changes to make this relationship more obvious. A first idea would be to move the two properties closer to each other (b), but it’s still not clear that they’re related. Renaming limit to overdraftLimit gives a better result (c), but combining these two techniques is better still (d). This way, we create a virtual boundary around those. We could also create a more solid boundary by putting these two properties into an overdraftProtection substructure (e).
Grouping data is the first step, but it’s not enough. Having to constantly scroll up and down through the API’s response to find the most important data can be terribly annoying. Sorting data can enhance readability for human beings (programs do not care at all about this).
As shown in figure 7.3, two other groups can be created by moving type and typeName and also safeToSpend and balance closer together. In each group, properties are sorted from more important (on top) to less important (on the bottom). All the groups are also sorted by importance.
Figure 7.3 Sorting data in the bank account’s representation
This organization will also be visible in the documentation or code generated from the specification of your API. Grouping properties in a dedicated structure can also help to provide a better vision of what is required or not, as shown in figure 7.4.
Figure 7.4 Grouping data to manage an optional group of required properties
In this example, according to the JSON Schema on the left, both overdraftProtection and overdraftLimit are optional. But from a functional point of view, if overdraftProtection is true, then overdraftLimit is mandatory. Grouping these two properties in an optional overdraftProtection object containing two mandatory active and limit properties solves this problem.
Not that such a strategy is basically exposing the provider’s perspective—here, a JSON Schema limitation that does not allow one to describe the required/optional properties' combinations. But it is always good to know this trick; sometimes it can be of great help in order to provide a highly accurate JSON Schema.
To design usable data, you must organize it by creating data groups—moving related properties closer together, using common prefixes, or creating substructures—and sorting the data in those groups and the groups themselves from more important to less important.
7.1.2 Organizing feedback
A well-organized API provides well-organized feedback. In section 5.2, you learned how we can use HTTP status codes to provide informative feedback. As a reminder, table 7.1 shows some of the use cases you’ve seen.
Table 7.1 HTTP status code examples
HTTP status codes are grouped into classes. A response in the 2XX class means that everything went OK, while a 4XX response means that there’s a problem with the request and the consumer should fix it. Grouping HTTP status codes this way makes these easier to understand.
If an API returns a 413 status code to your request, you know that the problem is on your side, even if you’ve never seen this status code before.1 Why? Because it’s a 4XX-class code. This unknown 4XX should be treated the same way as a 400 status code (see section 5.2.3).2
1 This code actually means that your request is larger than the server is willing or able to process (https://tools.ietf.org/html/rfc7231#section-6.5.11).
2 RFC7231 states that, “… a client MUST understand the class of any status code, as indicated by the first digit, and treat an unrecognized status code as being equivalent to the x00 status code of that class …” (https://tools.ietf.org/html/rfc7231#section-6).
Organizing feedback isn’t just about status codes, though. You can also organize more specific or custom feedback. In section 5.2.4, you saw that when returning multiple errors it can be helpful to categorize them. Figure 7.5 shows a response to a delayed money transfer creation request that specifies an amount exceeding the safe-to-spend limit, that’s missing the destination account, and that provides the execution date as a UNIX timestamp.
Figure 7.5 Grouped and sorted errors
As you can see, each error has a type, which gives us a clue about the source of the problem. The first error from the BUSINESS_RULE group has obviously triggered a business control. The MISSING_MANDATORY_PARAMETER error obviously concerns a missing mandatory parameter. And the obvious BAD_FORMAT_OR_TYPE error tells us that the property’s value does not conform to the expected type or format. The errors are also sorted from most- to least-critical: the Amount exceeds safe to spend error is the most serious one, followed by Destination is mandatory, and the less-critical Date must use ISO 8601 YYY-MM-DD format.
When designing an API, you must organize feedback to facilitate its interpretation by taking advantage of the organization of the underlying protocol’s feedback, creating your own feedback organization, and sorting multiple errors from most-to-least critical.
7.1.3 Organizing goals
Last but not least, goals also deserve to be well-organized. If you are familiar with object-oriented programming, you can compare this to organizing methods in classes. An API’s goals can be organized both virtually and physically. As shown in figure 7.6, the OpenAPI Specification you discovered in chapter 4 can be used to organize an API’s goals virtually.
On the left in figure 7.6 is a totally disorganized Banking API definition. On the right, the goals have been grouped into two categories, Account and Transfer, by adding a tags property on each operation.
Figure 7.6 Grouping goals with tags in an OpenAPI Specification document
But how did we choose each goal’s group? There’s no magic recipe, but the idea is to group together goals that are related from a functional point of view. If you’re designing a REST API, you must not be fooled by the paths when doing that; you must focus on the functionality of the goals and not their representations, as shown in figure 7.7.
Figure 7.7 Grouping goals with tags based on functionality versus URL representation
As you can see on the left of this figure, if we were to focus on paths, we might end up with three categories: Beneficiary (for the /beneficiaries paths), Transfer (for the /transfers paths), and Account (for the /accounts paths). But it doesn’t make sense to separate the goals represented by the /transfers and /beneficiaries paths because they cannot exist without each other.
On the right, we’ve organized the goals into two categories. This is better, but having the Transfer category before Account does not really reflect how people will use the API. Users are likely to first be interested in operations related to the account domain before trying to use the API to transfer money. As shown in figure 7.8, we can sort the categories by adding a tags definition on the root level. The tags property is a list in which each item contains a tag’s name and its description.
Figure 7.8 Sorting goal groups by adding sorted tags definitions
All we need to do to sort our Account and Transfer tags as we want is to sort the tag definitions in this tags list. Then we can add a description for each tag (or each category). Now that the groups are sorted, we should also sort the operations within the groups, as shown in figure 7.9.
Figure 7.9 Sorting goals in the OpenAPI Specification document
This can only be done by sorting the goals by order of importance in the specification document itself (this is how it should have been done from the beginning, by the way). Here GET /accounts is more important than GET /accounts/{id} because consumers will usually list accounts before accessing an account’s detailed information. Note also that the order of the HTTP methods is the same for all resources: GET, POST, DELETE; POST and GET in /beneficiaries have been swapped. Remember what you learned about consistency in section 6.1? Choose one way to sort the HTTP methods and stick to it for all resources! But this is only a virtual organization that is not exposed in the API design itself. We could add /account and /transfer root paths to the resource paths to actually group those as shown in figure 7.10.
Figure 7.10 Grouping resources by paths
Users would then be able to make connections between resources by just looking at their paths. Be warned, however, that this could make the paths less simple to guess in some cases.
Now it’s your turn. Suppose the goals shown in figure 7.11 belong to the API of a famous image-sharing social network called Imagebook. How would you organize those using what you’ve learned so far?
Figure 7.11 How would you organize these goals?
Organizing an API’s goals virtually or physically facilitates understanding. This can be done by sorting goals in the definition document and taking advantage of the API specification format. And you can possibly add an organization level when designing the programming interface (add a level in the path for the REST HTTP method, for example).
We now know how to design a well-organized API. But as API designers, we must ensure that our APIs are concise too.
7.2 Sizing an API
In Joe Dante’s 1984 movie, Gremlins, Randall Peltzer, the main protagonist’s father, tries to sell the “invention of the century”: the Bathroom Buddy (shown in figure 7.12).
Figure 7.12 The not-so-handy (and kind of gross) Bathroom Buddy, which does too many things
It’s an all-in-one device for travelers. Imagine a huge Swiss Army knife-like thing including a razor, a shaving cream dispenser, a shaving mirror, a toothbrush, a toothpaste dispenser, a toothpick, a dental mirror, a comb, a nail clipper, and probably some other more or less useful features.
The problem with the Bathroom Buddy is not simply that each demonstration fails miserably, ending with the inventor covered with toothpaste or shaving cream. The real problem is that it wants to do too many things, and it doesn’t really look that handy. It’s too big to fit well in hand, and using each function seems to be quite a challenge. Finding where the comb is hidden might not be easy the first few times, and the idea of using the same device to brush my teeth and clip my toenails is quite disgusting, to say the least. A separate toothbrush and nail clipper are far more convenient (and appealing) to use!
And sizing doesn’t only matter for everyday objects. What is the right size for a database table? A class? A method? A function? An application? These are questions that come up constantly when you’re working with software—and APIs are no exception. Each aspect of an API, including its data and its goals, should be sized wisely. Sometimes you’ll find that what you’d considered as a single API can be worth splitting into different ones, just like the the Bathroom Buddy’s toothbrush and nail clipper.
7.2.1 Choosing data granularity
The bank account’s JSON representation in figure 7.13 contains 32 properties and has a maximum depth of 4.
Figure 7.13 Number of properties and maximum depth of a bank account representation
Holding 32 properties seems reasonable; but, depending on the context, it can be too much. What if this representation is used in a list? In that case, it might not be relevant to provide all of an account’s information when users might need only a summary. Also, if we take a closer look, we can see at least one potential problem: this bank account contains a transactions list. This representation might be trying to do too many things at once, and it might not be easy to manipulate the transactions list from within the bank account. These representations should be separated.
And regarding the maximum depth of 4, it is also quite reasonable, if slightly above the recommended level. This depth is a direct result of grouping using substructures to keep the data readable. As shown in figure 7.14, data granularity has two dimensions: the number of properties and the depth.
Figure 7.14 Choosing the number of properties and maximum depth
The number of properties that’s reasonable for an API to return in a data structure is a matter of functional relevance; the provided properties must all be functionally appropriate in the context in which they are used. The more data an API returns, however, the more the designer must be careful about its organization (remember section 7.1.1) and relevance because, even though these are all pertinent, having a high number of properties does not facilitate usage.
In my experience, I would say that above 20 properties, you should definitely think about organizing those and possibly challenge each one. But this is not a silver bullet; there are fields where it could make sense to have so many properties. Try to define your own rules based on your domain. Recall also that, as we saw in section 5.2.1, we must request the minimum data possible to ensure usability. The number of properties is quite critical in this context.
Regarding the depth, we also have the same input/output duality; but for both contexts, it is recommended to try not to go beyond three levels of depth. Having more than three levels of depth makes manipulating the raw data, coding, and reading the documentation more complicated. Again, this rule might need to be adapted to your context.
Organizing data also helps to make your API easier to understand, so you will have to find a balance. Keep an eye on data granularity, but remember that it is mainly a matter of functional context and not a matter of numbers. But granularity doesn’t only matter for data—it also matters for goals.
7.2.2 Choosing goal granularity
Take another look at figure 7.15. Is it a good idea to have the transactions included in the bank account representation? Probably not.
Figure 7.15 Different goal granularity when reading and modifying data
If the get bank account goal returns the account and its transactions list, we’ll have to deal with managing a potentially large number of transactions. Always returning all the transactions can be cumbersome, and managing transaction pagination can be complex (as you learned in chapter 6). A GET /accounts/{id}?page=2 or GET /accounts/{id}?transactionPage=2 REST API request seems quite awkward. It is better to provide a separate goal to get a bank account’s transactions (GET /accounts/{id}/transactions).
Choosing the right granularity for your goals is about ensuring that a goal is not doing two (or more) quite different things. Note also that the granularity of goals is not always consistent. As shown in figure 7.15, it can differ when reading or modifying data, for example.
The Banking API currently allows us to modify an account holder’s addresses by updating the bank account resource. While it might be useful for this resource to provide information about the account holder, including their addresses, updating an address by updating the bank account resource is not really natural. Doing the update that way hides the update address goal within another one.
There are two issues here. First, it might not be obvious at first sight that you can update an address by updating a bank account. Second, the account holder’s data is independent of a specific bank account. The same account holder might own multiple accounts, so updating the address through an account seems quite awkward. And what if there are other properties of the bank account that can be updated through this resource, such as the overdraft feature?
Requesting minimal inputs and managing errors for the update bank account goal could become quite complex for both designers and consumers because this goal would encompass several subgoals. It would be wiser to provide an independent update address goal as seen at the bottom right in figure 7.15, but it is totally acceptable to give access to the address information through the bank account if it makes sense from a functional point of view.
Remember that a goal’s granularity should be determined by context and usability. We’ll talk more about goal granularity in chapter 8, but first, if granularity matters for data and goals, it matters, of course, for APIs as well.
7.2.3 Choosing API granularity
When we organized the Banking API goals in section 7.1.3, borders appeared around the goals. As shown in figure 7.16, we have grouped the goals into Account and Transfer categories.
Figure 7.16 From one Banking API to separate Bank Account and Money Transfer APIs
But these are more than just simple categories. Each group of goals could be totally independent. Therefore, why don’t we split the Banking API into two smaller but functionally useful APIs? These smaller Money Transfer Bank Account APIs will be easier to manage and can be reused independently in different contexts. Note that in the Money Transfer API, goals have been grouped in the initial Transfer and Beneficiary categories. It now makes sense to organize them into smaller groups.
We have been working with the programming interface representation, but organizing and splitting an API’s goals can be done during the first design steps when identifying goals. Try to apply what you have learned here to the Shopping API we designed in chapters 3 and 4. How would you organize and split the goals list shown in table 7.2 into independent, smaller APIs? It’s up to you to fill in the Category and API columns; try to come up with at least two different versions. (Hint: thinking about who the users are can help you to find one version.)
Table 7.2 How would you organize and split this Shopping API goals list?
| Goal | Category | API |
| Create user | ||
| Search for products | ||
| Get product’s information | ||
| Add product to shopping cart | ||
| Remove product from cart | ||
| Check out cart | ||
| Get cart detail | ||
| List orders | ||
| Add product to catalog | ||
| Update a product | ||
| Replace a product | ||
| Delete a product | ||
| Get an order’s status | ||
| Update user | ||
| Delete user |
Remember that once an API is organized into groups of goals while identifying goals or designing the programming interface, it can be split into smaller but functionally significant APIs that can be used independently.
This concludes part 2 of this book; you now know how to design usable APIs. That is already great, but we won’t stop here: there is still a lot of ground to cover. In the third part of this book, you will learn to design APIs while taking care of the whole context around them.
Summary
- Organize data properties by sorting them, naming them using patterns, or grouping them in data structures.
- Categorize feedback and sort it by its importance.
- Group goals by focusing on functionality and not representations; you can use API description format features or naming patterns (OpenAPI tags and URL prefixes for REST APIs).
- Keep the number of properties and depth levels as low as possible in data structures.
- Avoid creating does-it-all goals.
- Split data structures, goals, and even APIs into smaller but functionally significant elements when possible.