8
Designing a secure API
This chapter covers
- The intersection between API security and API design
- Defining user-friendly scopes for access control
- Adapting API design to meet access control needs
- Adapting API design to handle sensitive material
Designing APIs that make sense for their users and are usable is definitely important, but this must not be done without considering security. API security is not an afterthought that you can assume will be handled later (whenever that is) by the security people (whoever they are). Indeed, design and security are inextricably linked when creating an API or anything else.
Regularly, there is some news about a company having been “hacked” through their APIs, especially private ones used for mobile applications. I put quotation marks around hacked because sometimes such hacking is at a kindergarten level. Indeed, in some cases, hackers simply inspect the API responses and discover sensitive data that should have never left the depth of the provider’s systems. There is also the classic “What happens if I change the user ID in a request?” … “I get other users’ data!”
This is not because an API is private or for partners and only used by trusted consumers, so it can expose anything without us giving thought to security. Public API security is usually treated more seriously as long as the people involved actually know what API security means. Security matters for all types of APIs; and, as an API designer, you have a part to play in API security.
You must at least have a basic understanding of API security in order to design secure APIs and communicate efficiently with the people involved in their creation. An entire book, if not several, could be written about this topic; the aim of this chapter is only to give an overview without providing details about the actual implementation of API security. Books such as OAuth 2 in Action, written by Justin Richer and Antonio Sanso, or API Security in Action, written by Neil Madden, both published by Manning (https://www.manning.com/books/oauth-2-in-action and https://www.manning.com/books/api-security-in-action, respectively), provide more detailed information about this topic. While this discussion will obviously be far from complete, it should be sufficient to help you understand how API designers can create APIs that are secure by design. Figure 8.1 zooms in on an API call from a security perspective to illustrate where API security and API design collide.
Figure 8.1 Zooming in on an API call from the security perspective
When the Banking API’s provider receives API calls, it must do some high-level access control in order to be sure that the consumers are known and allowed to use the requested API goal. Then the provider proceeds with some lower-level access control measures to ensure, for example, that the end users only see their own accounts.
In certain cases, the request and response can contain sensitive personal data that should be handled carefully. The goal itself could also be a sensitive one, such as transferring money, and require extra care. Access control and sensitive material are the primary areas where API security and API design intersect. We will dive into both of these, but first, we need to dig a little deeper into API security.
8.1 An overview of API security
In order to design secure APIs, you must know some basic API security principles. It is important to also understand what these principles mean, not only for the developers of both API consumers and providers, but also for the end users. We will explore all of this by walking through the three steps of a first API call: registering a consumer, getting credentials allowing consumption of the API, and making the API call. Besides providing a glimpse of an API ecosystem, what follows gives a very simplified and partial vision of the OAuth 2.0 authorization framework defined by RFC 6749 (https://tools.ietf.org/html/rfc6749). Although it is not the only option, it is commonly used to secure web APIs.
8.1.1 Registering a consumer
Secure APIs only allow known consumers to use them. When developers want to use an API in their consumer applications, whether they are mobile applications or backend ones, they must first register them. To do so, they can use the API provider’s developer portal.
An ideal developer portal is a website offering documentation, tutorials, FAQs, forums, support, and any other useful resources and tools that developers can take advantage of to understand what the API does and how to use it. It also provides information about API consumption; developers can check how their consumers use the API and can also have access to logs of the API calls. Developers have to register themselves (and perhaps provide payment information if the API is not a free one). Once that is done, they can register their consumers. Figure 8.2 shows a basic consumer application registration process for the Banking API.
Figure 8.2 Registering a consumer to select which part of the API will be used and to get credentials
Each consumer is given a name. Here we have an Awesome Banking Application and a Boring Financial Dashboard. Then developers have to select which scopes of the API their consumers will use. A scope corresponds to one or more goals of the API. The Boring Financial Dashboard uses only the read accounts and transactions scope, which corresponds to the list accounts, read account, and list transactions goals. Therefore, this consumer will only be allowed to use these three goals. The Awesome Banking Application uses all scopes and will, therefore, be allowed to use all of the API’s goals. Once this configuration is done, developers can retrieve a client ID for their applications, which will be used in the next step.
Not all APIs, especially private ones, come with a developer portal. The same configuration can be done by storing (securely, of course) the application name, scopes, and credentials in a database, for example. Whatever the means, once the consumers are registered, the developers can proceed to the next step: getting the credentials that allow consumers to use the API. Let’s see what this means for the Awesome Banking Application.
8.1.2 Getting credentials to consume the API
If you’ve ever used a Sign Up with Google/Twitter/GitHub/Facebook button, what you’ll see now should look familiar. The Awesome Banking Application is a third-party application created by the Awesome Company. Its end users are customers of the Banking Company that provides the Banking API. The Awesome Banking Application and its end users must be authenticated in order to get a token, which contains the credentials allowing the application to consume the Banking API.
Figure 8.3 shows the roles played by the different parties involved. (This is a simplified and partial view of the OAuth 2.0 implicit flow, which is one way, among others, of getting credentials to allow consumption of an API. Its intent is only to be a support example; it is not a prescription. Security measures must be chosen by security experts. Do not use this specific method without knowing what you are doing.)
Figure 8.3 Roles involved in the OAuth 2.0 implicit flow
The resource server is the application providing the Banking API. The client is the third-party Awesome Banking Application that consumes the Banking API. The resource owner is the end user using the Awesome Banking Application. And last but not least, the authorization server is the application that verifies the identity of the consumer and the end user. Figure 8.4 shows how these parties work together.
Figure 8.4 Authenticating the Awesome Banking Application and its end users, which provides an access token allowing them to consume the Banking API
To retrieve the credentials, the Awesome Banking Application first contacts the Banking Company’s authorization server to request the end user’s authentication. This is usually done by opening a web browser window pointing to the authorization server’s URL and including the client ID in a query parameter (https://auth.banking.com?client_id=XTF129DHDJDNQLK, for example).
When the authorization server receives this request, it checks that the provided client ID corresponds to a known consumer. If so, it retrieves the consumer’s name (Awesome Banking Application) and the scopes it requests (read accounts and transactions, transfer money, and manage beneficiaries), according to the configuration in figure 8.2. Then it returns a login web page including this information. End users have to check that what the application wants to do (the requested scopes) is appropriate. If so, they provide their login and password and then click the Authorize button.
Clicking this button sends the user’s login and password, along with the list of scopes, to the authorization server. It checks that the requested scopes match the ones defined in the client’s (the Awesome Banking Application) configuration. And it checks that the end user’s login and password are valid. If everything is in order, it generates an access token, the credential that allows the Awesome Banking Application to consume the Banking API. The authorization server stores this token along with the end user’s ID, the client ID, and the granted scopes; and, finally, it returns the access token to the application.
Note that any communication between consumer and provider takes place over a secured channel, ensuring that nobody can intercept the exchanged data. When using the HTTP protocol, this is done using Transport Layer Security (TLS) encryption (formerly known as the Secure Sockets Layer, or SSL). If you’ve ever used a URL of the form https://example.com, you have used TLS.
Now that the Awesome Banking Application has its access token, it can start to consume the Banking API. We look at that in the next section.
8.1.3 Making an API call
As a first call to the Banking API, the Awesome Banking Application can request to list the user’s accounts, as shown in figure 8.5.
Figure 8.5 The Awesome Banking Application requests to list accounts.
To do so, it sends a list accounts request to the Banking API (over a secured channel, of course). This request also contains the access token retrieved previously. When the Banking API receives this request, it contacts the authorization server to check if the access token is valid. If it is a valid token, the authorization server can return the data attached to it, such as the end user’s ID, the client ID, and the granted scopes.
The Banking API’s implementation first checks that the list accounts goal belongs to one of the scopes granted to the Awesome Banking Application. Because the read accounts and transactions scope was granted to the end user, as described in the previous section, the implementation proceeds to the next step.
The consumer requests the list accounts goal without further explanation. Should the implementation return all the available accounts? Certainly not! The consumer’s request is made in the context defined by the data attached to the access token. The end user’s ID is attached to the access token, and the implementation uses it to filter the accounts. That way, it returns only the accounts that should be returned to the user making the request.
We’ve now been through all the steps of our first API call. We understand the mechanics, but let’s see what all this means from the API designer’s perspective.
8.1.4 Envisioning API design from the perspective of security
Figure 8.6 sums up the basic security principles you have seen so far and which of these have an impact on API design.
Figure 8.6 How API security and API collide
You have seen that all communication between consumer and provider should take place over a secured channel, ensuring that no data can be intercepted. This has nothing to do with API design, but it’s always good to know that the communications are secure. You have also seen that only registered consumers should be allowed to consume an API. These consumers should only be allowed to use the parts of the API they really need and to which end users have requested access.
On the other side, the implementation needs to know which consumer is requesting a goal in order to verify that it is actually allowed to use this goal. Defining scopes (a), partitioning the API in order to grant access to only selected goals, cannot be done without input from the API designers because these groups of goals must make sense for developers as well as end users. Besides knowing if a consumer is allowed to use a goal, the implementation (b) also needs to know on whose behalf a request is being made in order to adapt its behavior to the specific end user’s rights. If end users are actually involved, that is not always the case. That matter is the implementation’s job, but API designers should keep this in mind because it can impact the design of the API.
Is that all? Do API designers just have to think about access control to design secure APIs? No. There is another, far less obvious aspect of API security that matters for API designers.
The Awesome Banking Application can list accounts over a secure connection only if it provides a valid access token, and the scopes attached to that token encompass this goal. The returned accounts are only the ones owned by the end user attached to the access token. That sounds totally secure, doesn’t it? But what if for each account, the data returned contains a list of all the debit cards attached to it? And what if for each card, the provided data includes its number, security code, expiration date, and the account holder’s full name? That would be a huge problem because this data is highly sensitive.
Should the Banking API provide such data? Probably not. Thinking about API design from the perspective of security is not only a matter of access control; it also involves asking the question, “Should we really expose this data (c) through our API?” And this question matters not only for data but also for goals (d).
So, to create APIs that are secure by design, API designers must take care of application access control, end user access control, and sensitive material. Let’s see how we can do that.
8.2 Partitioning an API to facilitate access control
Let’s start with one of the most obvious aspects of secure API design: how to partition an API to facilitate access control. But first, let’s talk about why we must group goals in scopes in order to grant consumers access to only selected goals, and why we must take care with the design of these groups.
In the real world, a hotel is often seen as a whole entity (especially by its guests). Besides rooms, hotels can provide services such as a swimming pool, fitness center, sauna, or spa. These services are usually offered to all guests; they are included in the room price. But sometimes, some of these services are optional. Guests have to tell the hotel staff they want to use them and pay for them in addition to their room price. Some of the services can even be accessible to people who are not staying at the hotel. These customers pay to use only these services without using the rest of the hotel’s features. This means that, from the hotel guest and non-guest customer’s perspectives, the hotel is but a business providing multiple services that can be accessed independently. At the hotel’s reception desk, customers can request access to all or some of these services. The customers then will only be able to use the services they have been granted access to.
This can, even must, happen in the API world too. As we saw in section 8.1.1, two different consumers might not use the same goals. Figure 8.7 shows the different goals used by the Awesome Banking Application and the Boring Financial Dashboard.
Figure 8.7 Different consumers with different needs do not use the same goals.
The Awesome Banking Application uses all the available goals, while the Boring Financial Dashboard only uses the list accounts, read account, and list transactions goals without ever needing to use the others. But does that mean that this consumer should not be allowed to use those other goals? Why would we implement such access controls on an API? The Open Web Application Security Project (OWASP; https://www.owasp.org), a worldwide not-for-profit charitable organization focused on improving the security of software, states that
“Every feature that is added to an application adds a certain amount of risk to the overall application. The aim for secure development is to reduce the overall risk by reducing the attack surface area.”
OWASP
By limiting consumers' access to only the goals they really need, you reduce the likelihood of an attack. This is the principle of least privilege, which can also be applied to data. Because web APIs can be exposed on the internet, the fewer open doors there are, the better. And as with hotels, some of your API’s features can require different paid subscriptions, so you would not want to see consumers who haven’t paid the fee using those freely. Enabling access controls eases the process of granting access to different areas of your API to selected consumers only. That’s two good reasons for designing scopes. But we must also take care in how we design these in order to provide the best developer and end user experiences possible.
When developers register their consumers to use an API, they have to select the appropriate scopes. The same goes for end users: when they allow a third-party application to access an API on their behalf, they have to check the requested scopes and possibly select these by themselves. So an API’s goals should be partitioned into various groups, called scopes, in order to enable access control mechanisms, and these scopes should be carefully designed. Let’s see how we can do that.
8.2.1 Defining flexible but complex fine-grained scopes
A first, a simple way of defining scopes is to bluntly define a scope for each goal. While configuring their consumers via the Banking API’s developer portal, developers might see configuration screens such as those shown in figure 8.8, allowing them to select the scopes (or goals, in this case) they need for each consumer.
Figure 8.8 Controlling access with fine-grained, goal-based scopes
Configuring the Awesome Banking Application, which provides full access to all of the Banking API’s services, requires selecting all of the scopes. Only the list accounts, read account, and list transactions scopes are selected for the Boring Financial Dashboard, which focuses on account activity analysis. Configuring the PayFriend App, which proposes to allow its users to easily send money to friends, is trickier. The money transfer-related goals have to be selected carefully. The validate an uncommon transfer scope is unselected for this application because it is meant to be used only by bank advisors to validate certain money transfers that are judged to be unusual, based on their destination or amount. Once the configuration is done, each of these consumers will only be able to use the goals corresponding to the scopes selected for them. For example, the PayFriend App cannot list accounts but can trigger a money transfer.
With 12 scopes corresponding to the 12 goals of the Banking API, the access control configuration is quite flexible, but it can be considered complex. Each scope has to be carefully selected. What if there are more goals and, therefore, more scopes to deal with? The complexity increases. And what if we allow third-party applications to use the API on the behalf of their end users? Those end users will have to deal with this complex configuration too. Both developers and end users can feel overwhelmed or annoyed by the process of carefully selecting a few scopes from a lengthy list.
Maybe we can define less fine-grained and more user-friendly scopes by basing these on concepts and actions. If you have been designing your API using the method described in section 3.2, this should be quite simple because you’ve already done something like that. Figure 8.9 shows how to define such scopes for our Banking API.
Figure 8.9 Defining fine-grained scopes based on concepts and actions
The first step consists of identifying the main concept (or resource) for each goal. You begin by identifying the main noun in the goal. For example, both the list accounts and read account goals deal with the concept of an account. Then you identify the CRUD (Create, Read, Update, Delete) action that best represents the goal’s main verb. For these two goals, it is Read; therefore, these fall under the account:read scope. Note that the scope-naming convention {concept}:{action} is quite common but might not be too user-friendly. Such scope names are usually accompanied by a helpful description, such as
"account:read": list accounts and access detailed information about those
Unfortunately, this technique does not always reduce the number of scopes. For the beneficiary-related goals, we still end up with three scopes matching the list beneficiaries, create a beneficiary, and delete a beneficiary goals. In some cases, this can even cause problems.
The update a transfer and validate an uncommon transfer goals both update a money transfer and, therefore, could be grouped under the transfer:update scope. But that would not be very secure! By allowing a consumer to update a money transfer, we would also allow them to use the far more critical validate an uncommon transfer goal. In this case, it would be wiser to keep this goal under a specific transfer:validate scope that uses a custom action instead of a CRUD one.
Partitioning based on concepts and actions can produce scopes that are still flexible, but a little less fine-grained and complex. This must be done carefully, however, to avoid inadvertently granting undue access to critical goals, and the improvement is fairly minimal. Let’s think back to what we learned in chapter 7 about designing a concise and well-organized API. Can we use those concepts to try to organize the goals into coarser scopes and provide a more usable solution?
8.2.2 Defining simple but less flexible coarse-grained scopes
In section 7.1.3, you learned how to organize goals into categories. Why not use them as scopes? Figure 8.10 shows what these category-based scopes would look like for the Banking API.
Figure 8.10 Defining category-based scopes is usually not a good idea.
The account scope seems acceptable, but the transfer one is too broad; it might grant undue access to beneficiary-related goals and the critical validate an uncommon transfer. Categories, therefore, are usually not fit to be taken as a basis for scopes because they have different purposes:
- Categories —Organize goals from a functional perspective and help consumers understand how to use the API
- Scopes —Ensure that consumers are only granted access to the goals they really need
If categories do not work well, how are we supposed to organize our goals into meaningful but secure groups? We can try to base them on what the users can achieve using the API. Such scopes correspond more or less to the whats we identified when filling in the API goals canvas (see section 2.3). Figure 8.11 shows a partial API goals canvas for the Banking API focusing on transfers.
Figure 8.11 Defining role- and functionality-based scopes
This canvas tells us that to access account information, account owners list accounts, read an account, and list its transactions. Therefore, we could create an accessing account information scope matching this what and comprising these three goals. The same goes for bank advisors, who are responsible for validating uncommon transfers. The corresponding goals can be grouped under a validating transfers scope. We could proceed the same way for the transfer money and manage transfers whats, but it might not really make sense to separate these. For example, after creating recurring transfers, consumers might want to be able to delete those once they aren’t needed anymore. Therefore, maybe we should create a transferring money scope comprising these two what’s.
To practice, I’ll let you complete the API goals canvas with the other goals and define the scopes accordingly. As you can see, this way we end up with fewer, less flexible, but more user-friendly scopes. Such coarse-grained scopes can be seen as a kind of shortcut to grant consumers access to several goals at once.
You can also create totally arbitrary scopes. In some use cases, this might make sense. For example, you could define administrator-level scopes on each resource: a beneficiary:admin scope granting access to all goals related to beneficiaries, and so on. You’ll have to be careful when defining such scopes, however, because they could allow undue access if granted to applications without the appropriate care.
You’ve now seen a few different ways of defining scopes. Which one should you use?
8.2.3 Choosing scope strategies
Which strategy is the best when it comes to defining scopes to enable access control on an API? The flexible but complex fine-grained strategy or the less flexible but more user-friendly coarse-grained one? And within each strategy, which approach should you adopt? Unfortunately, there is no one right answer. Depending on the API’s consumers, developers, and end users, the most suitable approach can vary. But there’s good news: you might not have to choose a single one!
Beyond creating coarse-grained and user-friendly scopes, the whats-based strategy shows something interesting about scopes. In figure 8.12, you can see the goals covered by two of the scopes we just defined.
Figure 8.12 The same goals can belong to different scopes.
The list transfers goal belongs to two scopes! This is the good news: it means that there can be some overlap between scopes. This allows us to define different levels of scopes.
We could, for example, use a concepts- and actions-based approach, having scopes such as beneficiary:read and beneficiary:delete along with an arbitrary one, where all goals covered by these scopes would also be covered by a beneficiary:admin scope. We could even add a more user-friendly third level of scope using the whats-based strategy for third-party integrations involving end users. We are under no obligation to show or allow all of our API scopes to all of our consumers; we can provide different views of the available scopes when necessary. And, as we’ve done before, we can take advantage of the API description format to define these scopes.
8.2.4 Defining scopes with the API description format
If the API you’re designing is associated with an API description format, be sure to check if it allows you to describe scopes. As illustrated in the following listing, the OpenAPI Specification 3.0 allows this.
Listing 8.1 Describing scopes
components:
securitySchemes:
BankingAPIScopes: ①
type: oauth2
flows:
implicit:
authorizationUrl: ②
"https://auth.bankingcompany.com/authorize"
scopes: ③
"beneficiary:create": Create beneficiaries
"beneficiary:read": List beneficiaries
"beneficiary:delete": Delete beneficiaries
"beneficiary:manage": Create, list, and delete beneficiaries
① Reusable scopes are defined in components.securitySchemes.
② You can put in a dummy URL during the design phase if needed. (Note: URL should be on the same line.)
③ A security scheme can contain more than one scope.
The scope definition is done in the components.securitySchemes section of the OpenAPI document. The BankingAPIScopes scheme defines all the flows and scopes that can be used in the Banking API when using the oauth2 security type. For now, only the implicit flow is available, and when this flow is used, only the scopes related to the beneficiaries are allowed.
There are four scopes defined in this listing. Each scope definition comes with a description. The first three are goal-based ones: beneficiary:create, beneficiary:read, and beneficiary:delete. The fourth one, beneficiary:manage, is an arbitrary scope encompassing the three beneficiary-related goals. Defining these scopes is not enough, though. They must be linked to goals, as shown in the next listing.
Listing 8.2 Linking a goal to scopes
paths:
/beneficiaries:
get:
tags:
- Transfer
description: Gets beneficiaries list
security: ①
- BankingAPIScopes: ②
- "beneficiary:read"
- "beneficiary:manage"
responses:
"200":
description: The beneficiaries list
① Lists the security schemes used
② References a security scheme from components.securityScheme and lists scopes needed to use this goal
The list beneficiaries goal is represented by GET /beneficiaries. It is covered by the beneficiary:read and beneficiary:manage scopes of the BankingAPIScopes security scheme we defined in listing 8.1. This means that only consumers that have been granted access to at least one of these two scopes can use this goal.
If an API uses two types of scopes, it can be useful to make a distinction between them in the API description. This could be done simply to provide clearer documentation or to enable showing only specific types of scopes, depending on who is using the API, for example. The following listing shows how to organize scopes in different groups.
Listing 8.3 Grouping scopes
components:
securitySchemes:
ConceptActionBasedSecurity: ①
type: oauth2
flows:
implicit:
authorizationUrl: "https://auth.bankingcompany.com/authorize"
scopes:
"beneficiary:create": Create beneficiaries
"beneficiary:read": List beneficiaries
"beneficiary:delete": Delete beneficiaries
ArbitraryBasedSecurity: ②
type: oauth2
flows:
implicit:
authorizationUrl: "https://auth.bankingcompany.com/authorize"
scopes:
"beneficiary:manage": Create, list, and delete beneficiaries
① Contains only goal-based scopes
② Contains the higher-level scope
Again, we apply what you discovered in chapter 7: in designing a concise and well-organized API, we organize the API components to ease understanding and use. The BankingAPIScopes security scheme has been split into two new security schemes, each containing a given type of scope. The following listing shows how to use these new security schemes.
Listing 8.4 Linking a goal to scopes from different groups
paths:
/beneficiaries:
get:
tags:
- Transfer
description: Gets beneficiaries list
security: ①
- ConceptActionBasedSecurity:
- "beneficiary:read"
- ArbitraryBasedSecurity:
- "beneficiary:manage"
responses:
"200":
description: The beneficiaries list
① The security section can contain references to different security schemes.
The security section now contains references to the ConceptActionBasedSecurity and ArbitraryBasedSecurity security schemes, each of which lists the scope that’s used, just like in listing 8.2.
To practice, you can expand this example by adding the other Banking API goals and three levels of scopes (goal-based, arbitrary, and action-based). You could also add another type of OAuth flow, such as the client credential one (hint: look for the “OAuth Flows Object” description in the OpenAPI Specification documentation at https://github.com/OAI/OpenAPI-Specification).
Now that you’ve seen how to define scopes, let’s look at how the API’s interface contract can be impacted by lower-level access control matters.
8.3 Designing with access control in mind
These days, hotel guests are usually given access cards instead of good old keys. They can use these cards in elevators to gain access to the floors where their rooms are located; and, of course, they can use them to open the doors to their rooms. And, obviously, they cannot use their cards to open the doors of other guests' rooms.
The hotel staff members also have cards, which they can use to access staff-reserved floors or guests' rooms. Different staff members can have access to all of the hotel’s rooms, or only the ones located on a given floor. Both guests and staff members have access to the hotel’s floors and rooms, but they don’t all have the same level of access.
In the API world, low-level, fine-grained access control of this sort exists too. It is mostly handled by the implementation, but API designers also have a role to play here. To ensure that everything goes smoothly, API designers must know what data is needed to actually implement the access controls and adapt their designs if necessary.
8.3.1 Knowing what data is needed to control access
Just because the Awesome Banking Application is allowed to list accounts doesn’t mean it should be allowed to list all accounts. It can only retrieve the ones belonging to its end users, the Banking Company’s customers. Now, what if the Banking Company creates a Bank Advisors App using the Banking API for its bank advisors managing the customers' accounts? What should the list accounts goal return for this consumer? Should it return all the customers' accounts or only the ones related to customers managed by the bank advisors? Whatever the answer, list accounts will not have the same behavior when it is triggered for a customer or for an advisor. As shown in figure 8.13, each consumer has a different view of the bank accounts accessible via the list accounts goal.
Figure 8.13 Two seemingly identical requests giving different results
In this example, the design of the API does not seem to be affected. In both cases, the representation of the list accounts goal is the same: as a REST API, it would be something like GET /accounts—without any parameters. Its behavior will be modified according to the identity of the end user; but there is no endUserId parameter in the GET /accounts request, so how do we know who this end user is? Actually, this parameter does exist, but it is somewhat hidden.
As you saw in section 8.1.3, consumers must send an access token with their request to make an API call, just like hotel guests have to use their access card to open doors. When this token has been created, it is stored somewhere, along with some minimal security data. This data is a hidden part of the API interface contract. As an API designer, you must know about it in order to be sure that what you design actually works and is secure.
Such data is relatively standard; it usually consists of the consumer ID, the granted scopes, the end user’s ID, and possibly the end user’s role (the type of user) or permissions (what this specific user is allowed to do). When the implementation receives the request along with the token, it can retrieve the security data and do all necessary access control directly, based on the request’s data or based on some other retrieved data.
For example, if the Awesome Banking Application sends a GET /accounts/1234567 request, the implementation must check that the end user whose ID is attached to the access token sent along with the request is referenced as one of the 1234567 account’s holders. To be sure about this, however, you’ll need to check with those in charge of implementation or, more precisely, those in charge of the security layer. In our case, it’s quite simple, but be warned that the access control mechanism is not always so transparent. Also, a goal might need slightly different representations for different end users or roles.
8.3.2 Adapting the design when necessary
In section 5.2, we focused on the transfer money goal, which transfers a sum of money from a source account to a destination account. A money transfer is always initiated by a customer’s request. The source account belongs to this customer. The destination account is an account also belonging to this customer or to a beneficiary registered beforehand by the customer. The REST representation of this goal consists of a POST /transfers request, whose body contains the three properties amount, source, and destination.
If this goal is used by the Awesome Banking App, which is used by Banking Company customers, a customer’s ID is attached to the token sent along with the request. Therefore, the implementation can easily check that everything is in order. It checks that the source account belongs to the customer and also that the destination account belongs to the same customer or a beneficiary registered by this customer.
But what if this transfer money goal is used by the Bank Advisor App? This application is not used by customers, only by bank advisors. Customers, however, can call their bank advisors to request a money transfer. In this case, the user ID attached to the token is the advisor’s ID and not the customer’s.
The controls in the implementation are less simple but still feasible. The implementation checks that the source account belongs to a customer managed by the advisor. The destination account should belong to the customer owning the account or a beneficiary registered by the same customer. Everything seems to be OK, but there’s a problem. The system needs to trace which customer requested the money transfer.
If the bank account belongs to a single customer, it is easy to find the customer’s ID because it’s the one attached to the bank account, and only the owner of the account can request a transfer. But if the account is a joint one belonging to more than one customer, this doesn’t work. The goal needs to be modified in order to convey the correct customer ID to the implementation.
We could add an optional customerId property into the POST /transfers request’s body. We could also create a new representation of the transfer resource and use the POST method on it like this:
POST /customers/{customerId}/transfers
The second option seems to make more sense if we step back and look at the API from a higher level. A bank advisor might need to list a customer’s transfers, so a
GET /customers/{customerId}/transfers
using the same resource path could be useful. Whatever the chosen solution, the implementation will now be able to check that, when the transfer money goal is made on the behalf of advisors, it can only do that for customers for whom they have the rights to do so.
Actually, if you use the API goals canvas and the method described in section 2.3, which was to identify all users, what they do, how they do it, and especially what they need to do it, you should be able to deal easily and almost seamlessly with API design versus API security questions. As a reminder, this method is shown again in figure 8.14.
Figure 8.14 The API goals canvas
Working with scopes has shown that even though API designers should avoid the provider’s perspective in their designs, they sometimes need to know what’s happening in the implementation after all.
There is one last API security topic to work on, and this is probably the most important one from the API designer’s point of view. An API exposes data or capabilities. It’s therefore up to the API designer to check if these are sensitive, if they should really be exposed, and, if so, to choose the most secure way to include them in the API.
8.4 Handling sensitive material
When people stay in a hotel, they usually do not bring all of their belongings with them. It would not make sense to bring valuable or sensitive items like one’s income tax return, a great-great-great-grandmother’s jewelry, or a precious and rare Samba de Amigo Maracas set (needed to play the Sega Dreamcast video game of the same name). They probably won’t be needed, and they could be either lost or stolen. But there are other valuable items (ID cards, passports, phones, or cameras, for example) that people do need to bring. If these belongings have to be left in the hotel room, they can be secured in the room’s safety deposit box. Cash and credit cards can also be secured the same way.
In the API world, as in the software world in general, we must always check whether what can be requested, provided, or done through an API involves sensitive material. If so, we must ensure that it is really needed and then create the most secure design possible. This can be done either by tuning how sensitive material is represented or by choosing adapted access control mechanisms. This matter primarily concerns the API’s data, goals, and feedback, but we must also take care with the underlying protocol and architecture used by the API.
8.4.1 Handling sensitive data
When consumers use the read account goal, they get detailed information about an account. This can consist of the account’s number, its balance, and the list of debit cards linked to it. In the Banking Company’s systems, the basic information linked to a card consists of its holder’s name, its number (also called the Primary Account Number, or PAN), its expiration date, and its CVV (Card Verification Value, located on the card’s back). Cards provided by the Banking Company can be blocked (useful when you think your card might be lost or stolen). And if customers want to, they can also define a monthly ceiling (they receive an alert by SMS, email, or via a notification when their total monthly payments exceed this level). The data returned might look like what is shown on the left in figure 8.15. But even if this data is sent over a secured connection (see section 8.1.3), is it really wise to return all the available data?
Figure 8.15 Designing a secure representation of a banking card
The monthly ceiling, monthly payments to date, and blocked flag are non-sensitive data; no harm can be done with them. On the other hand, the card’s number, CVV, holder’s name, and expiration date are quite sensitive; this information can be used to make payments online or by phone. Maybe we should think twice before returning all this information.
The CVV is only used for online payments, and it is written on the card itself, so we can remove it. The card number is useful to identify a card, but we could keep only the last four digits; these are sufficient to identify a card in the context of a bank account. We can keep the expiration date and the card holder’s name because this is relevant information. Now that we have removed the CVV and chosen a secured representation of the card number, nobody can take advantage of this data to do harm. As you can see, by identifying sensitive data items and choosing to remove some and adapt the representation of others, we’re able to design a secure representation of a debit card that remains meaningful and relevant.
So the first step of secure API design is to identify the sensitive data that will be requested and provided via the API. The problem is that the term sensitive data covers a wide range of data and can be different, depending on your domain or industry. Identifying sensitive data is sometimes quite simple, as in the debit card example, because what should be considered sensitive is pretty obvious. Similarly, nobody would expose sensitive data such as customers' usernames and passwords. But sometimes, it’s not that obvious.
There are many country-specific, domain-specific, and international regulations, standards, or best practices that affect whether and how you can manipulate data. When any system deals with bank cards, it has to comply with the global Payment Card Industry Data Security Standard (PCI DSS). In the US, the Health Insurance Portability and Accountability Act (HIPAA) is a set of standards created to secure protected health information (PHI) by regulating healthcare providers. The General Data Protection Regulation (GDPR) is a European Union regulation impacting any company dealing with European citizens' data.
Whatever the reason why such regulations or standards exist, they can impact the data manipulated through your API. And this is not only a functional matter; there are best practices on the technical side as well, which API designers and implementers should follow. For example, usually it is not recommended to return sequential database keys that would give hints about critical data like how many customers your company has; such sequential data could be used to try to access data belonging to others (although, if your implementation handles access control perfectly, that shouldn’t be a problem).
Once you know which data is sensitive, the second step consists of choosing appropriate representations for that data. As a reminder, doing such adaptations is designing the API. Therefore, it requires us to focus on what the consumers can do using the API and not on the data itself.
How the representations will be adapted highly depends on how the data will be used. In the debit card use case, the goal was only to provide detailed information about an account and its linked cards, not to allow users to execute a payment. In that case, it makes sense to not provide the full card number and its CVV. Having this goal-oriented mindset will simplify the adaptation. Figure 8.16 shows four techniques that you can use to create secure representations.
Figure 8.16 Adapting representations of sensitive data to make it non-sensitive
The first technique is simple: you remove any sensitive data that is not required (1). This can have a nice side effect: some sensitive data related to the data that’s removed can become non-sensitive. In figure 8.16, for example, removing the sensitive number property (a card number) makes the expirationDate property non-sensitive.
If sensitive data cannot be removed, you might be able to replace it with a non-sensitive adaptation, as we did with the card number (2). Here, the value of number is truncated to make it non-sensitive. Also, the sensitive averageMonthlyBill is replaced by a non-sensitive vip flag, whose value is the result of someFunction(averageMonthlyBill). In this case, a goal-oriented design process would probably have solved this problem from the start.
Sometimes it can be useful to combine value and data replacement. For example, if the sensitive data is used as an identifier, it might be more practical to simply create a new non-meaningful identifier to identify a resource instead of using a meaningful but sensitive one. The truncated number, such as 5678, can be useful to show to end users but not to use as a resource identifier. A /cards/5678 resource path will probably be unique in the context of a customer, but perhaps not in the wider context of an advisor. Therefore, the card representation might benefit from a totally new id containing an opaque ID such as c4ca76a1. The resulting /cards/c4ca76a1 resource path would be unique in both contexts; and, depending on your needs, this new ID could be a replacement of number or an addition to the card representation. Note that like removing sensitive data, replacing sensitive data can make other sensitive data non-sensitive. Here, the sensitive expirationDate becomes non-sensitive once number is replaced.
Data replacement can be applied to multiple properties (3). Replacing a set of precise and sensitive values by a more fuzzy but still meaningful one can also do the trick. In the technique illustrated in figure 8.16, the sensitive merchant and address properties of a card transaction are combined in a new non-sensitive type property, whose value is the result of someFunction(merchant, address).
As a last resort, if sensitive data is really needed as is and if the communication channel’s encryption is not enough, the sensitive values can be encrypted (4). In the example in figure 8.16, the value of number, which is 1234123412345678, is encrypted as Ed3hZ9ylz1mYlq. But this technique has some downsides. Notably, consumers will have to decrypt the encrypted data to use it. When it comes down to this, it might be simpler and more effective to encrypt the entire messages exchanged over a secured connection established between the consumer and provider. The safest option would be to encrypt the data specifically for each consumer and provide each with the correct key to decrypt the data.
All of this is the job of the people managing the security layer and implementing the API. But as an API designer, you must ensure that all this security stuff is consumer-friendly. By talking to the security people and using the techniques you have learned about how to design APIs, you should be able to design secure representations. But the data exchanged via an API is not the API’s only sensitive material—some API goals can be sensitive too.
8.4.2 Handling sensitive goals
There are two kind of sensitive goals: those that manipulate sensitive data and those that trigger actions with sensitive consequences. As with sensitive data, whatever the reason a goal is considered sensitive, the first question you ask yourself must be, “Is this goal really needed?” If not, to keep the API design secure, not including it in the API is the simplest way to deal with it. But that is not always possible.
Let’s say that, for a very good reason, the Banking API must provide a card’s sensitive information (such as number, CVV, expiration date, and holder’s name) to some consumers or end users. It would be wise to ensure that the access to this data is tightly controlled. As shown in figure 8.17, this matter could be handled in four different ways.
Figure 8.17 Controlling access to goals exposing sensitive data
A first option (1) could be to create a dedicated read card goal providing access to the card’s sensitive data. This new goal could be protected by a specific read card scope granted only to the consumers that really need it. The read account goal could stay as it is and still provide the non-sensitive version of the card data.
A second option (2) could be to use a scope to trigger the return of sensitive data. The read account goal would return non-sensitive representations of the card data by default, except for consumers who have been granted the show sensitive card data scope. The implementation of these goals would return the raw sensitive card data instead of the adapted non-sensitive data.
In the first two options, if end users are involved, they will be able to see the sensitive data when using a consumer with the right scope. That might not fit some security requirements. Therefore, a third option (3) would be to do the same thing, but handle the choice of returning the sensitive or non-sensitive representation based on the permissions or roles of the end user attached to the access token.
But using end user permissions only would mean that any consumer that has been authorized to use the API on the users' behalf would have access to the sensitive data. If that is a problem, a fourth option (4) could be to combine consumer and end user access control to ensure that access to sensitive data is possible only when an end user with the adequate permissions uses a consumer with the required scope.
The choice of options and which is appropriate to use depends on security constraints and also developer experience. As with designing secure data representations, you will have to talk to the security people in your organization in order to know which options are possible. Also, from a consumer experience perspective, if there’s a wide gap between the sensitive and non-sensitive versions of the data, it would be better to provide dedicated goals to access sensitive data instead of modulating the version returned, based on scopes or permissions.
But goals can be sensitive even if they do not manipulate sensitive data. Let’s take it for granted that in our Banking API, we’re working on a secure representation of the card data and none of its properties are considered sensitive. Besides the basic card data, this representation contains the blocked flag and the monthlyCeiling amount. These two properties could be updated by end users or, more precisely, by consumers on their behalf. We could add an update card goal allowing this data to be updated. But even if updating the threshold of payments above which an alert is sent is not a sensitive action, blocking the card is quite a sensitive one. Therefore, it would be wise to ensure that blocking a card is done only by end users or consumers allowed to to so.
We can use exactly the same options for this that we used with the goal dealing with sensitive data. We could rely on the end users' permissions and check the implementation to see if the user is allowed to update the blocked flag when using the update card goal. We could also rely on a block card scope that would allow a consumer to do the same thing when using this update card goal. And obviously, we could mix these two options. All this would work, but mingling sensitive and non-sensitive data updates in the same goal can make the API complex, especially if the update of sensitive data requires additional security measures like multifactor authentication of end users. In that case, the monthlyCeiling property can be updated at once, but the blocked property only when the user validates it. It would probably be better to create a dedicated block card goal to handle this update. If we step back and think more about what the API can do and less about the data itself, such a goal makes sense.
So handling sensitive goals requires us to identify whether these manipulate sensitive data or trigger sensitive actions. Once that’s done, and we’re really sure we need these, we can rely on scopes or permissions to ensure that only authorized consumers and end users can access those goals. But using only scopes or permissions might not be sufficient.
It is sometimes better to adapt the interface contract to provide dedicated, clearly identified, and fine-grained goals handling the sensitive parts (data or actions). Such design adaptation facilitates access control and keeps the API easy to understand and use. But what happens when consumers try to do something they are not allowed to do? What kind of feedback do they get? And should API designers also be concerned about security when designing other kinds of feedback?
8.4.3 Designing secure error feedback
In section 5.2, you learned how to design exhaustive and informative error feedback that helps consumers to solve problems themselves. There we identified two types of errors: those due to a malformed request and those caused by contravening a business rule. Here we need a new security error type to clearly signify security-related errors. The feedback for these errors could use the same kind of representation—a message and a code. What might happen when a consumer triggers a security-related error when calling the Banking API, say, by sending a GET /cards/c4ca76a1 request?
If a consumer sends a request without providing a token or provides an invalid one, the Banking API server can return a 401 Unauthorized response. This response might contain a body with a message such as Missing or invalid token. If the same consumer retries its request with a valid token but that consumer has not been granted access to the read card scope, the server can return a 403 Forbidden response accompanied by a Consumer has not been granted the "read card" scope message and a SCOPE code. And if the consumer tries again after the developer has updated the configuration to add the missing scope, the server can return another 403 Forbidden response with an End user is not allowed to access this card message and a PERMISSIONS code. This last response obviously means that the end user does not have permission to read this specific card.
In some circumstances, this last error could be considered an information leak. Indeed, explicitly mentioning that the user is not allowed to access the c4ca76a1 card implicitly confirms that this card actually exists. To avoid such a leak, the Banking API server can instead say that the card does not exist in the context of the end user. It can do that by returning a 404 Not Found response with a Card does not exist message, just like it would do when a consumer requests a card that actually does not exist.
You should also keep the risk of information leaks in mind when designing malformed request or functional error feedback. For example, a consumer might try to update a card with a PATCH /cards/c4ca76a1 request that uses a secure card ID. If the consumer provides an invalid monthlyCeiling value to update this card, it would be a terrible idea to return a message containing sensitive information, such as the full card number, as in Impossible to update the 123412341234124 card.
And there is another type of error that we did not discuss yet and that is prone to a critical information leak: the unexpected ones; for example, a good old java.lang.NullPointerException (trying to something.do() when something is null in Java) or the quite annoying Unable to extend table in tablespace (when there’s not enough disk space to extend an on-premise database table to add some data). Obviously such errors never happen because everything is fully tested, monitored, and automated … until they happen, usually at the worst possible moment.
When using the HTTP protocol, such errors are represented by a 5XX class status code, usually the 500 Internal Server Error. While a 4XX status means it’s the consumer’s fault, a 5XX means it’s the provider’s. When such unexpected server errors occur, the implementation must never provide detailed information about the technical stack behind the API. You can provide an error ID for further investigation, but be careful not to provide any information that can give a hint about what is actually running behind the interface. So no stack trace and detailed error-describing software versions, server addresses, or the like can appear in such errors. Table 8.1 sums up the various error types you have seen so far.
Table 8.1 Error feedback use cases
As you can see, security error feedback resembles the feedback for other types of errors. When designing a REST API, you just have to use the appropriate HTTP status for these errors, such as 401 or 403. But don’t forget that you must be careful about what kind of information you provide with security or technical error feedback (and any other type of feedback) to avoid inadvertently providing access to sensitive information. We’re almost done, but there’s one last topic to be aware of to be sure that your API handles sensitive material appropriately.
8.4.4 Identifying architecture and protocol issues
How much should we trust the architecture and protocol that will support the APIs we’re designing? Not much if we don’t know much about them. Figure 8.18 shows a basic (and flawed) API architecture that might be used for the REST Banking API. This example highlights that the secure connection between consumer and provider might not always be as secure as we think.
Figure 8.18 A not-so-secure connection between consumer and provider
In this basic architecture, the consumer passes through a proxy to access the internet and make HTTP calls to the Banking API. The Banking API is not exposed directly on the internet; there is another proxy, an HTTP load balancer, that distributes the requests to various instances of the Banking API server application. The communication between nodes is secured using TLS.
Everything seems to be totally secure, but if we take a closer look, we can see that the load balancer logs some data (the HTTP method; URL, including query parameters; an HTTP status code; and response time). The content of these log files is sent to a monitoring tool that can be used to create shiny, useful dashboards showing how the Banking API is used and behaves. That means that anyone who has access to the load balancer’s logs or the monitoring tool can see the data contained in a resource path and its query parameters.
Of course, the Banking Company is very strict when it comes to security. All these logs are stripped of sensitive data. The Banking Company, however, does not control what happens on the proxy used by the consumer, which can also log the HTTP calls. Obviously, it is up to the consumer to secure that. But the Banking API can be designed to ensure that if URLs are logged, they do not contain any sensitive information.
For example, if the list of accounts can be filtered by customer name using a request like GET /accounts?customerLastName=Smith, that’s a problem. A name is sensitive information, and the customerLastName query parameter will be traced in every HTTP log on the wire between the consumer and the provider. To avoid this potential data leak, it would be safest to propose a POST /accounts/search request, whose body contains the search parameters.
If you are designing a REST HTTP API, take care not to put any sensitive information in path parameters or query parameters because they can be logged. And, for the protocol used to communicate and the architecture built for the API, always check if there are any possible data leaks. If there are potential leaks that could expose sensitive data, you must adapt the API’s design accordingly to prevent this.
In the next chapter, you will learn how designing an API’s evolution requires extra care to avoid provoking errors on the consumer side and how to minimize this risk from the ground up when designing an API.
Summary
- API designers contribute heavily to API security by minimizing the attack surface.
- An API should only expose and request what is really needed.
- Consumers should only be allowed to use what they really need.
- To ensure security, the design of an API must be done from the user’s perspective, keeping in mind what data is needed to control access.
- Sensitive data and goals cover a wide range; exactly what should be considered sensitive might not be obvious and should be identified with the help of technical, security, business, and legal experts.
- API designers must be aware of potential leaks due to the underlying protocol or architecture in order to fully secure the API design.