This chapter covers

• Connecting your serverless function to an external service using an HTTP API
• Dealing with common problems in async communication with Claudia API Builder

As you learned in the previous chapter, handling asynchronous operations in AWS Lambda is easy with Claudia API Builder. In that chapter, you also learned how to create a database for your pizza orders and created functions to store, retrieve, update, and delete them. But your application is capable of much more than that.

This chapter shows you how to connect your serverless application to an external HTTP service by enabling Aunt Maria’s pizzeria to use the Some Like It Hot Delivery Company’s API and offer more home delivery services. You will learn how to formulate an HTTP request from AWS Lambda, handle response errors, and set up a webhook with Claudia API Builder. You will also learn about the most common problems and pitfalls, how to solve them, and how to avoid encountering them in the first place.

4.1 Connecting to an external service

Ring, ring! Aunt Maria is on the phone again. She sounds pleased and thanks you for your current work, but you can sense that something is bothering her. It’s not long until she asks you for a favor.

It’s about the deliveries. Each time the pizzeria wants to deliver a pizza order, they need to phone the Some Like It Hot Delivery Company. That wasn’t a problem until the recent rise in pizza orders (thanks to you!). But now the process is starting to take up more and more time, so Aunt Maria wants you to find an alternative. Luckily for you, the Some Like It Hot Delivery Company has an API. How can you connect to it?

As we discussed earlier, your serverless application can connect to any of the following:

• A database (DynamoDB, Amazon RDS)
• Another Lambda function
• Another AWS service (SQS, S3, and many others)
• An external API

The Some Like It Hot Delivery API belongs to the last category.

4.2 Connecting to the delivery API

Let’s start with the createOrder handler, which is in the create-order.js file in your project’s handlers folder. After the createOrder handler saves the order to your database, you want to contact the Some Like It Hot Delivery Company’s API to schedule a delivery. The flow of your application should look like figure 4.1.

Before you start connecting the dots, take a quick look at the Some Like It Hot Delivery Company’s API, described in the following section.

4.2.1 The Some Like It Hot Delivery API

Aunt Maria is happy with the professionalism of Some Like It Hot Delivery Company. For a reasonable price, they pick up and deliver the pizzas while they’re still hot. Even their call center is good; the agents are polite and take orders quickly. But that’s still a bottleneck—they don’t have many agents, and despite the speed of their service, you need to wait on the line for a free agent, which is a problem if you need to deliver a lot of pizzas every day.

You decide to look at their website to see if there’s something that can simplify the workflow. Even a simple web form would be better than a phone call. Surprise, surprise—not only do they have a better solution, but they have a fully working API!

The API offers the following endpoints:

• POST /delivery creates a new delivery request and returns the delivery ID and a time estimation.
• GET /delivery returns all the scheduled deliveries for your restaurant.
• GET /delivery/{id} returns information on the status of a selected delivery.
• DELETE /delivery/{id} cancels the delivery, but only in the first 10 minutes following the creation of the delivery request.

It’s not the best API ever, but it’s good enough to allow you to automate the process.

We don’t dive deep into the Some Like It Hot Delivery API documentation right now. Instead, you’ll see the most important things about each API endpoint as you connect them.

4.2.2 Creating your first delivery request

As Aunt Maria described to you, when an order is placed she usually makes a phone call to create a delivery request. Instead, you’d like to create the delivery request automatically. Take a few seconds and, if you can, come up with a diagram of the flow.

When a customer orders a pizza, you need to

1. Validate the order.
2. Contact the Some Like It Hot Delivery API to see when the Some Like It Hot Delivery Company can deliver it.
3. Save the order to the database.

The flow is illustrated in figure 4.2.

Before implementing the flow from figure 4.2, you need to learn a bit more about creating a delivery request via the Some Like It Hot Delivery API. Let’s look at that now.

The most important feature of the Some Like It Hot Delivery API is its POST /delivery route, which creates a delivery request. This API endpoint accepts the following parameters:

• pickupAddress —The pickup address for the order. By default, it’ll use the address from your account.
• deliveryAddress —The delivery address for the order.
• pickupTime —The pickup time for the order. If the time isn’t provided, the order will be picked up as soon as possible.
• webhookUrl —The URL for a webhook that should be called to update the delivery status.

The Some Like It Hot Delivery API returns the delivery ID, the pickup time for the order, and the initial delivery status, which is “pending.” When the order is picked up, the Some Like It Hot Delivery API needs to make a POST request to your Pizza API webhook and send the new delivery status (“in-progress”) along with the delivery ID.

It’s time to update your create-order.js handler. It needs to send a POST request to the Some Like It Hot Delivery API, wait for its response, and then save the pizza order to the database. But you need to add a delivery ID to the database, so you can update the status of the order when your webhook receives the data.

The updated create-order.js with the delivery request should look something like the following listing.

Listing 4.1 create-order.js updated to create a delivery request before saving the delivery to the database

'use strict'
const AWS = require('aws-sdk')
const docClient = new AWS.DynamoDB.DocumentClient()
const rp = require('minimal-request-promise')
module.exports = function createOrder(request) {
  if (!request || !request.pizza || !request.address)
    throw new Error('To order pizza please provide pizza type and address where pizza should be delivered')
  return rp.post('https://some-like-it-hot.effortless-serverless.com/delivery', {    ①  
    headers: {    ②  
      "Authorization": "aunt-marias-pizzeria-1234567890",    ②  
      "Content-type": "application/json"
    },
    body: JSON.stringify({    ④  
      pickupTime: '15.34pm',
      pickupAddress: 'Aunt Maria Pizzeria',
      deliveryAddress: request.address,    ⑤  
      webhookUrl: 'https://whpcvzntil.execute-api.eu-central-1.amazonaws.com/chapter4_1/delivery',    ⑥  
    })
  })
    .then(rawResponse => JSON.parse(rawResponse.body))    ⑦  
    .then(response => {
      return docClient.put({    ⑧  
        TableName: 'pizza-orders',
        Item: {
          orderId: response.deliveryId,    ⑨  
          pizza: request.pizza,
          address: request.address,
          orderStatus: 'pending'
        }
      }).promise()
    })
    .then(res => {
      console.log('Order is saved!', res)
      return res
    })
    .catch(saveError => {
      console.log(`Oops, order is not saved :(`, saveError)
      throw saveError
    })
}

Note a few new things here:

• minimal-request-promise —As its name states, this is a minimal promise-based API for HTTP requests. You can pick the module you like the most. We recommend minimal-request-promise because of its minimal required implementation. For more details, you can take a look at its source code on GitHub: https://github.com/gojko/minimal-request-promise.
• Authorization —Making a request to an external service usually requires some kind of authorization, but because the Some Like It Hot Delivery API is not a real API, anything you pass in the Authorization header will work.
• webhookURL —The Some Like It Hot Delivery API needs an endpoint where it will send its delivery status updates.

As previously mentioned, a webhook is a simple API endpoint that accepts POST requests. There are two things you need to do:

1. Create a route handler for the webhook
2. Create a route called /delivery that accepts POST requests

Let’s start with the first one. Go to the handlers directory in the root of your Pizza API project, and create a new file named update-delivery-status.js.

The webhook route handler flow should be as follows:

1. Your webhook should receive a POST request with its delivery ID and the delivery status in the request body.
2. Find the order in the table using the delivery ID you received from the Some Like It Hot Delivery API.
3. Update that order with a new delivery status.

But there’s a tricky part here. DynamoDB has two actions: get and scan. The get command allows you to query the database only by key columns, whereas scan can query on any column. Another important difference is that scan loads up the whole table and then applies a filter on its collection; the get command directly queries the table.

These differences seem to be limiting, but in reality, you just need to do a bit more planning. Besides a single primary key, DynamoDB supports a composite key, too—it consists of a primary or hash key and a sort or range key, and requires the combination of those two to be unique. Another way to handle similar problems is to add a secondary index. To learn more about both approaches, see the official documentation: http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Introduction.html.

In your case, there’s an even easier solution—the delivery ID is unique, and you’ll get it before you store the order to the pizza-orders table, so you can use the delivery ID as an order ID. Doing so allows you to query the database by both order and delivery ID, because they are the same, and also to remove the uuid module, because you don’t need it anymore.

Let’s try to implement that. The following listing shows the code.

Listing 4.2 The update delivery status handler receives the data from the Some Like It Hot Delivery API and updates the order in the table.

'use strict'
const AWS = require('aws-sdk')
const docClient = new AWS.DynamoDB.DocumentClient()
module.exports = function updateDeliveryStatus(request) {
  if (!request.deliveryId || !request.status)    ①  
    throw new Error('Status and delivery ID are required')

  return docClient.update({    ②  
    TableName: 'pizza-orders',
    Key: {
      orderId: request.deliveryId    ③  
    },
    AttributeUpdates: {
      deliveryStatus: {
        Action: 'PUT',
        Value: request.status
      }
    }    ④  
  }).promise()
    .then(() => {
      return {}    ⑤  
    })
}

Before you can test your webhook, you need to add a route to the api.js file in the root of your project. To do that you need to require your new handler at the top of the file by adding the const updateDeliveryStatus = require('./handlers/update-delivery-status') line. Then you need to add another POST route, the same way you did it in chapter 2. The following listing shows the last few lines of the updated api.js file.

Listing 4.3 Last few lines of updated api.js file, with the new route for the delivery webhook

// Rest of the file    ①  
api.delete('/orders/{id}', request => deleteOrder(request.pathParams.id), {
  success: 200,
  error: 400
})

api.post('/delivery', request => updateDeliveryStatus(request.body), {    ②  
  success: 200,    ③  
  error: 400    ④  
})

// Export a Claudia API Builder instance
module.exports = api

Awesome—you have the webhook, and all the ingredients are finally in place. Let’s taste the webhook—pardon, let’s test it. To do so, you need to deploy your API using the claudia update command. After updating the API, use the same curl command you used in chapters 2 and 3 to test creating an order:

curl -i 
  -H "Content-Type: application/json" 
  -X POST 
  -d '{"pizza":4,"address":"221b Baker Street"}' 
  https://whpcvzntil.execute-api.eu-central-1.amazonaws.com/chapter4_1/orders

The curl command should return {}, status 200, so everything is fine. But what is happening in the background?

As you can see in figure 4.3, your Pizza API contacts the Some Like It Hot Delivery API first, then it saves the order to the pizza-orders table. Then, a bit later, the Some Like It Hot Delivery API contacts your webhook and updates the delivery status to “in-progress.” And finally, it contacts your webhook again to set the status to “delivered.”

That’s it!

What else do you need to connect to the Some Like It Hot Delivery API?

Because you have a webhook, you don’t need to contact the Some Like It Hot Delivery API to get the delivery status. But you do need to contact the API if you want to cancel a delivery request. That would be a nice exercise, and you can try to do that in section 4.4. But before the exercise, let’s explore some of the common issues with async requests from AWS Lambda using Claudia.

4.3 Potential issues with async communication

As you’ve seen, handling asynchronous requests with AWS Lambda using Claudia is easy. But sometimes issues arise when you want to connect to an external service or do an async operation.

It’s hard to summarize all the potential issues, but here are the most common errors people make:

• Forgetting to return a promise
• Not passing the value out of .then or .catch statements
• Not wrapping the external service in a promise if it doesn’t support JavaScript promises out of the box
• Hitting the timeout before the async function finishes its execution

As you can see, most of the issues are promise-related. But let’s take a look at them one by one.

4.3.1 Forgetting to return a promise

The most common problem with integration of an external service or an async operation is caused by omitting the return keyword. An example of this error is shown in the following listing. This issue is hard to debug because the code will run without an exception, but execution will be stopped before the async operation is done.

Listing 4.4 Breaking the code by not returning a promise

module.exports = function(pizza, address) {
  docClient.put({    ①  
    TableName: 'pizza-orders',
    Item: {
      orderId: uuid(),
      pizza: pizza,
      address: address,
      status: 'pending'
    }
  }).promise()

Why is this a problem? As you can see in figure 4.4, if your async operation doesn’t return a promise, Claudia API Builder won’t know that the operation is asynchronous, and it will tell AWS Lambda that the function has finished its execution. It will also send undefined as the value of your function because you never returned anything meaningful.

The solution for this problem is easy: make sure you always return a promise, and if your code is not working, first check that all of your promises are returned.

4.3.2 Not passing the value from the promise

This problem is almost the same as the previous one. The following listing shows an example.

Listing 4.5 Breaking the code by not returning a value from the promise

module.exports = function(pizza, address) {
  return docClient.put({    ①  
    TableName: 'pizza-orders',
    Item: {
      orderId: uuid(),
      pizza: pizza,
      address: address,
      status: 'pending'
    }
  }).promise()
  .then(result => {
    console.log('Result', result)    ②  
  })

As you can see in figure 4.5, the main difference is that the async operation finishes its execution in this case, but the result is never passed back to your handler function, and your promise chain is broken. Again, undefined is returned as the result of your serverless function.

The solution for this problem is the same as for the previous one—make sure you always return the values.

4.3.3 Not wrapping the external service in a promise

Sometimes external or async services don’t have native support for promises. In that case, another common mistake is not wrapping the operations in a promise, as you can see in the next listing.

Listing 4.6 Breaking the code by not wrapping the non-promise async operation in a promise

module.exports = function(pizza, address) {
  return setTimeout(() => {    ①  
    return 'Are we there yet?'    ②  
  }, 500)
})

As you can see in figure 4.6, the problem is exactly the same as the first one.

But the solution is a bit different. As shown in the following listing, you need to return a new, empty promise. Then, execute an async operation inside it and finally resolve it when the async operation has finished its execution.

Listing 4.7 Fixing the broken code by wrapping the non-promise async operation in a promise

module.exports = function(pizza, address) {
  return new Promise((resolve, reject) => {    ①  
    setTimeout(() => {    ②  
      resolve('Are we there yet?')    ③  
    }, 500)
  })
})

4.3.4 Timeout issues with long async operations

This last common problem is one with AWS Lambda timeouts. As you may remember from chapter 1, the default execution time is three seconds. So what happens when your asynchronous operation takes more than three seconds, as in the following listing?

Listing 4.8 Breaking the code by executing a function that takes more time than an AWS Lambda timeout

module.exports = function(pizza, address) {
  return new Promise((resolve, reject) => {    ①  
    setTimeout(() => {
      resolve('Are we there yet?')    ①  
    }, 3500)    ②  
  })
})

Well, as you can see in figure 4.7, it just stops, and your Lambda function never returns any value. The main difference here is that even Claudia API Builder isn’t executed in this case. Imagine someone unplugging your computer during some operation—the effect is the same.

How do you fix this issue?

Unless you can optimize the speed of your async operation and be sure that your function executes in less than three seconds, the solution is to update the timeout for your function.

Claudia allows you to set the timeout only during the function’s creation. To do that, invoke the create command with the --timeout option, like this:

claudia create --region eu-central-1 --api-module api --timeout 10

The value for this option is given in seconds.

If you already have a function, the best way to update it is by running the following AWS CLI command:

claudia update --timeout 10

For more information about this command, see the official documentation at http://docs.aws.amazon.com/cli/latest/reference/lambda/update-function-configuration.html.

After running the command, your function should be updated with the 10-second timeout. If you run the example from listing 4.9 again, it should work without a problem.

This list of potential issues is not complete, but the four we mentioned should cover the clear majority of them.

Now go play a bit more with the options and try to break your serverless API in a more creative way!

4.4 Taste it!

As you’ve seen, connecting external services is not that hard—so now, try to do it on your own.

4.4.1 Exercise

Remember that you can integrate cancelation of a delivery request using the Some Like It Hot Delivery API?

Your exercise for this chapter is exactly that—update the delete-order.js handler to cancel the delivery request via the Some Like It Hot Delivery API before it deletes the order from the database.

Before you start, here’s some info about the DELETE method of the Some Like It Hot Delivery API:

• To delete the delivery request, you need to send a DELETE request to the /delivery/{deliveryId} route of the Some Like It Hot Delivery API.
• You need to provide the delivery ID as a path parameter in the URL.
• The full URL for the Some Like It Hot Delivery API is https://some-like-it-hot.effortless-serverless.com/delivery.
• An order can be deleted only if the status is “pending.”

If that’s enough information, go ahead and try it on your own.

In case you need additional tips, here are a few:

• You need to read the order from the pizza-orders table first in order to get its status.
• If the status is not “pending,” throw an error.
• If the status is “pending,” contact the Some Like It Hot Delivery API; only when you receive a positive answer should you delete an order from the pizza-orders table.

If you need more help, or you want to see the solution, check out the next section.

In case this exercise was too easy and you want an additional challenge, try to build the Some Like It Hot Delivery API you used in section 4.2.1. The solution for that exercise is not shown in the book, but feel free to compare your solution to the original source code at https://github.com/effortless-serverless/some-like-it-hot-delivery.

4.4.2 Solution

Let’s start with the flow. As we said, first you need to contact the pizza-orders database table to see if the order has the “pending” status. Cancel it using the DELETE method of the Some Like It Hot Delivery API, and finally delete it from the pizza-orders table. See figure 4.8 for a visualization of the flow.

How should you update your delete-order.js handler?

It’s easy. First, import the minimal-request-promise module, because you’ll want to use it to contact the Some Like It Hot Delivery API.

Then update your deleteOrder function to read an order from the pizza-orders DynamoDB table. If no order with the specified ID exists, the function automatically throws an error and status 400 is returned to the customer. If the order does exist, check if the status of that order is “pending”; if it’s not, you’ll need to throw an error manually.

If the order status is “pending,” use the minimal-request-promise module to send a DELETE request to the Some Like It Hot Delivery API. Remember that the order ID is the same as the delivery ID, so you can use that ID to delete the delivery request. An error from the API will automatically throw an error in your deleteOrder function, so the response status will be 400 as expected.

When the API successfully deletes the delivery request, you need to delete the order from the pizza-orders DynamoDB table—and that’s it!

See the following listing for the complete delete-order.js handler’s code after the update.

Listing 4.9 Deleting an order from the pizza-orders DynamoDB table

const AWS = require('aws-sdk')
const docClient = new AWS.DynamoDB.DocumentClient()
const rp = require('minimal-request-promise')    ①  

module.exports = function deleteOrder(orderId) {
  return docClient.get({    ②  
    TableName: 'pizza-orders',
    Key: {
      orderId: orderId
    }
  }).promise()
    .then(result => result.Item)
    .then(item => {
      if (item.orderStatus !== 'pending')    ③  
        throw new Error('Order status is not pending')

      return rp.delete(`https://some-like-it-hot.effortless-serverless.com/delivery/${orderId}`, {    ④  
        headers: {
          "Authorization": "aunt-marias-pizzeria-1234567890",
          "Content-type": "application/json"
        }
      })
    })
    .then(() => {
      return docClient.delete({    ⑤  
        TableName: 'pizza-orders',
        Key: {
          orderId: orderId
        }
      }).promise()
    })    ⑥  
}

Summary

• With AWS Lambda, you can connect to any external service the same way you would from any regular Node.js app, as long as your async operations are correct.
• If you are connecting to an external API, make sure that your HTTP library supports promises, or wrap the operations manually.
• There are some potential problems with connecting to external services; most of the time they are related to a broken promise chain.
• Another common problem is with a timeout—if your Lambda function takes more than three seconds to complete, increase the timeout of the function.