This chapter covers
In the previous chapter, you created a simple API for handling pizza information and orders. You also learned that unlike with a traditional Node.js server, AWS Lambda state is lost between subsequent invocations. Therefore, a database or an external service is required to store Aunt Maria’s pizza orders or any other data you want to keep.
As Node.js executes asynchronously, you will first learn how serverless affects asynchronous communication: how it works with Claudia, and, more importantly, the recommended way of developing your serverless applications. As you grasp these concepts, you will see how easy it is to connect AWS Lambda to an external service, and you will learn how to use it to store your pizza orders by using AWS DynamoDB.
Because our brains aren’t good at asynchronous reading, and books are written in a synchronous manner, let’s go step by step.
Ring, ring! You just had a short phone call with Aunt Maria. She is impressed by your speed, though she still can’t use your application, as you aren’t storing any of her pizza orders. She still needs to use the old pen-and-paper method. To complete the basic version of your Pizza API, you need to store your orders somewhere.
Before starting development, you should always have an idea of which details you want to store. In your case, the most elementary pizza order is defined by the selected pizza, the delivery address, and the order status. For clarity, this kind of information is usually drawn as a diagram. So as a small exercise, take a minute to try to draw it yourself.
Your diagram should be similar to figure 3.1.
Now that you have an idea of what to store, let’s see how you should structure it for the database. As you previously learned, you can’t rely on AWS Lambda to store state, which means that storing order information in your Lambda filesystem is off the table.
In a traditional Node.js application, you would use some popular database, such as MongoDB, MySQL, or PostgreSQL. In the serverless world, each of the serverless providers has a different combination of data storage systems. AWS doesn’t have an out-of-the-box solution for any of those databases.
As the easiest alternative, you can use Amazon DynamoDB, a popular NoSQL database that can be connected to AWS Lambda easily.
To put it simply, DynamoDB is just a database building block for serverless applications. DynamoDB is to NoSQL databases what AWS Lambda is to computing functions: a fully managed, autoscaled, and relatively cheap cloud database solution.
DynamoDB stores the data in its data tables. A data table represents a collection of data. Each table contains multiple items. An item represents a single concept described by a group of attributes. You can think of an item as a JSON object, because it has the following similar characteristics:
The table is just the storage representation of the model you previously defined, as shown in figure 3.1.
Now you need to transform your previously defined model to the structure your database understands: a database table. While you are doing that, keep in mind that DynamoDB is almost schemaless, which means that you need to define only your primary key and can add everything else later. As a first step, you’ll design a minimum viable table for your orders.
Ready?
As in any other database, you want to store each order as one item in the database table. For your pizza order storage, you’ll use a single DynamoDB table, which will be a collection of your orders. You want to receive your orders via an API and store them to the DynamoDB table. Each order can be described by a set of its characteristics:
You can use those characteristics as keys in your table. Your orders table should look like table 3.1.
Order ID | Order status | Pizza | Address |
1 | pending | Capricciosa | 221B Baker Street |
2 | pending | Napoletana | 29 Acacia Road |
The next step is to create your table—let’s name it pizza-orders
. As with most things in AWS, you can do this several ways; our preferred method is to use the AWS CLI. To create a table for the orders, you can use the aws dynamodb create-table
command, as shown in listing 3.1.
You need to supply a few required parameters when creating the table. First, you need to define your table name; in your case, it will be pizza-orders
. Then you need to define your attributes. As we mentioned, DynamoDB requires only primary key definition, so you can define only the orderId
attribute and tell DynamoDB that it will be of type string. You also need to tell DynamoDB that orderId
will be your primary key (or, in DynamoDB’s world, hash key).
After that, you need to define the provisioned throughput, which tells DynamoDB what read and write capacity it should reserve for your application. Because this is a development version of your application, setting both read and write capacity to 1 will work perfectly fine, and you can change that later through the AWS CLI. DynamoDB supports autoscaling, but it requires the definition of the minimum and maximum capacity. At this point, you won’t need to use autoscaling, but if you want to learn more about it, visit http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/AutoScaling.html.
Finally, you need to select the region where you want to create your table. Pick the same region as you did with your Lambda function to decrease latency in database communication. The following listing shows the complete command.
Listing 3.1 Create a DynamoDB table using the AWS CLI
aws dynamodb create-table --table-name pizza-orders ①
--attribute-definitions AttributeName=orderId,AttributeType=S ②
--key-schema AttributeName=orderId,KeyType=HASH ③
--provisioned-throughput ReadCapacityUnits=1,WriteCapacityUnits=1 ④
--region eu-central-1 ⑤
--query TableDescription.TableArn --output text ⑥
When you run the command in listing 3.1, it prints the ARN of your DynamoDB table and looks similar to this:
arn:aws:dynamodb:eu-central-1:123456789101:table/pizza-orders
That’s it! Now you have the pizza-orders
DynamoDB table. Let’s see how you can connect it to your API’s route handlers.
To be able to connect to your DynamoDB table from Node.js, you need to install the AWS SDK for Node.js. You can get the aws-sdk
from NPM, as you would any other module. In case you are unfamiliar with that process, see appendix A.
You now have all the ingredients, and it’s time for the most important step: combine all the pieces, just as you would prepare a pizza. (Fortunately for you, we have a pizza recipe in the last appendix.)
The easiest way to communicate with DynamoDB from your Node.js application is through the DocumentClient
class, which requires asynchronous communication. DocumentClient
, like any part of the AWS SDK, works perfectly with Claudia, and you will use it in the API route handlers you made in chapter 2.
Connecting your pizza order API to the newly created database is easy. Storing an order to your DynamoDB table takes just two steps:
DocumentClient
.POST
method to save an order.Because you split your code into separate files in chapter 2, let’s start with the create-order.js file in the handlers folder. The following listing shows how to update create-order.js to save a new order to the pizza-orders
DynamoDB table.
Listing 3.2 Saving an order to the DynamoDB table
const AWS = require('aws-sdk')
const docClient = new AWS.DynamoDB.DocumentClient() ①
function createOrder(request) {
if (!request || !request.pizza || !request.address)
throw new Error('To order pizza please provide pizza type and address where pizza should be delivered')
return docClient.put({ ②
TableName: 'pizza-orders',
Item: {
orderId: 'some-id', ③
pizza: request.pizza,
address: request.address,
orderStatus: 'pending'
}
}).promise() ④
.then((res) => { ⑤
console.log('Order is saved!', res)
return res
})
.catch((saveError) => { ⑥
console.log(`Oops, order is not saved :(`, saveError)
throw saveError
})
}
module.exports = createOrder ⑦
When you finish this step, the POST /orders
method of your Pizza API should look and work the way it is presented in figure 3.2.
Let’s explain what happens here. After importing the AWS SDK, you need to initialize the DynamoDB DocumentClient
. Then you can replace the empty object you are returning on line 7 of your create-order.js handler with the code that saves an order to your table, using the DocumentClient
you imported previously.
To save an order to DynamoDB, you use the DocumentClient.put
method that puts a new item in the database, either by creating a new one or replacing an existing item with the same ID. The put
method expects an object that describes your table by providing the TableName
attribute and the item by providing the following Item
attribute as an object. In your database table plan, you decided that your item should have four attributes—ID, pizza, address, and order status—and that’s exactly what you want to add to the Item
object you are passing to the DocumentClient.put
method.
As Claudia API Builder expects a promise for async operations, you should use the .promise
method of DocumentClient.put
. The .promise
method converts a reply to a JavaScript promise. Some of you are probably wondering if there are any differences in how promises work in serverless applications and how Claudia handles asynchronous communication. The following section gives a short explanation of promises and how they work with Claudia and Claudia API Builder. If you are already familiar with these concepts, jump to section 3.3.
The pizzeria processes include dough rising, baking, pizza ordering, and so on. These are asynchronous operations. If they were synchronous, Aunt Maria’s pizzeria would be blocked and stopped from working on anything else until the operation in progress finished. For example, you would wait until the dough had risen, and then do something else. And for such time-wasting, Aunt Maria would fire anyone, even you! Because most of the JavaScript runtimes are single-threaded, many longer operations, such as network requests, are executed asynchronously. Asynchronous code execution is handled by two known concepts: callbacks and promises. At the time of this writing, promises are the recommended way to go in all Node.js applications. We do not explain callbacks, as you are most likely already familiar with them.
A promise is like a real-world promise made to partners, friends, parents, and kids:
And a couple of hours later, guess who took out the garbage?
Promises are just pretty wrappers around callbacks. In real-world situations, you wrap a promise around a certain action or operation. A promise can have two possible outcomes: it can be resolved (fulfilled) or rejected (unfulfilled).
Promises can have conditions related to them, and this is where their asynchronous power comes into play:
This example displays how certain actions can occur only after fulfilling a certain asynchronous operation. In the same way, the execution of certain code blocks waits for the completion of a defined promise.
The following listing is a JavaScript promise representation of the example sentence.
Listing 3.3 Johnny’s play—the promise way
function tellJohhny(homework) {
return finish(homework) ①
.then(finishedHomework => {
return getOut(finishedHomework); ②
})
.then(result => { ③
return play();
})
.catch(error => {
console.log(error);
});
}
Promises have several features:
catch
block allows you to easily and properly manage errors and propagate them to the responsible error handler.Some customers order multiple pizzas in one order, but those pizzas are not delivered one by one. If they were, customers would be furious with such an inefficient process. Instead, the pizza chef usually bakes them all at the same time; then the delivery person waits until all of them are finished before delivery.
The following listing is a code representation of this process.
Listing 3.4 Pizza parallel baking
function preparePizza(pizzaName) {
return new Promise((resolve, reject) => {
// prepare pizza
resolve(bakedPizza);
});
}
function processOrder(pizzas) {
return Promise.all([
preparePizza('extra-cheese'),
preparePizza('anchovies')
]);
}
return processOrder(pizzas)
.then((readyPizzas) => {
console.log(readyPizzas[0]); // prints out the result from the extra-cheese pizza
console.log(readyPizzas[1]); // prints out the result from the anchovies pizza
return readyPizzas;
})
As you can see in listings 3.3 and 3.4, promises help a lot. They allow you to handle any situation Aunt Maria’s pizzeria could have and also help you properly describe all the processes. Claudia fully supports all promise features, so you can easily use them.
In the next listing, you can see a simple Claudia example of a handler replying after one second. Because setTimeout
is not returning a promise, you need to wrap it by using a new Promise
statement.
Listing 3.5 Wrapping an async operation that doesn’t support promises with a promise
const Api = require('claudia-api-builder')
const api = new Api()
api.get('/', request => {
return new Promise(resolve => { ①
setTimeout(() => { ②
resolve('Hello after 1 second') ③
}, 1000) ②
})
})
module.exports = api
As you see in listing 3.5, as opposed to some popular Node.js frameworks, Claudia API Builder only exposes the request in the route handler. In chapter 2, to reply to it you would return a value, but in the case of an asynchronous operation, you should return a JavaScript promise. Claudia API Builder receives it, waits for it to be resolved, and uses the value returned as a reply.
After the small detour into the world of promises, run claudia update
again from your pizza-api folder and deploy the code. In less than a minute, you’ll be able to test your API and see if it works.
To test your API, reuse the curl
command from chapter 2:
curl -i
-H "Content-Type: application/json"
-X POST
-d '{"pizza":4,"address":"221b Baker Street"}'
https://whpcvzntil.execute-api.eu-central-1.amazonaws.com/chapter3_1/orders
Oh! The curl
command returns this:
HTTP/1.1 400 Bad Request
Content-Type: application/json
Content-Length: 219
Date: Mon, 25 Sep 2017 06:53:36 GMT
{"errorMessage":"User: arn:aws:sts::012345678910:assumed-role/pizza-api-executor/book-pizza-api
is not authorized to perform: dynamodb:PutItem on resource:
arn:aws:dynamodb:eu-central-1:012345678910:table/pizza-orders"}
What’s wrong?
This error is telling you that the role your Lambda function is using (arn:aws:sts::012345678910:assumed-role/pizza-api-executor/book-pizza-api
) is not allowed to perform a dynamodb:PutItem
command on your DynamoDB database (arn:aws:dynamodb:eu-central-1:012345678910:table/pizza-orders
).
To fix the issue, you need to add an IAM policy that allows your Lambda function to communicate with your database. You can do that with claudia create
by providing a --policies
flag. Be careful, though; that flag doesn’t work with the claudia update
command, as Claudia never duplicates things that you can do with a single AWS CLI command.
First, define a role in a JSON file. Create a new folder in your project root, and call it roles. Then create a role file for DynamoDB. Call it dynamodb.json, and use the content from the following listing. You want to allow your Lambda function to get, delete, and put items in the table. Because you might have more tables in the future, apply this rule to all tables, not just the one you have right now.
Listing 3.6 JSON file that represents DynamoDB role
{
"Version": "2012-10-17", ①
"Statement": [ ②
{
"Action": [ ③
"dynamodb:Scan",
"dynamodb:DeleteItem",
"dynamodb:GetItem",
"dynamodb:PutItem",
"dynamodb:UpdateItem"
],
"Effect": "Allow", ④
"Resource": "*" ⑤
}
]
}
Now you can use the AWS CLI put-role-policy
command to add a policy to your role, as shown in the next listing. To do so, you’ll need to provide the role that your Lambda function is using, the name of your policy, and the absolute path to your dynamodb.json file. Where can you find the role? Remember the claudia.json file that Claudia created in the root folder of your project? Open that file, and you’ll see the role
attribute in the lambda
section.
Listing 3.7 Add a policy to the Lambda role to allow it to communicate with DynamoDB tables.
aws iam put-role-policy ①
--role-name pizza-api-executor ②
--policy-name PizzaApiDynamoDB ③
--policy-document file://./roles/dynamodb.json ④
When you run the command from listing 3.7, you won’t get any response. That’s OK, because an empty response means that everything went well.
Now, rerun the same curl
command and try to add an order:
curl -i
-H "Content-Type: application/json"
-X POST
-d '{"pizza":4,"address":"221b Baker Street"}'
https://whpcvzntil.execute-api.eu-central-1.amazonaws.com/chapter3_1/orders
The curl
command should return {}
with status 201. If that’s the case, congratulations! Your database connection is working! But how do you see whether the order was really saved to the table?
The AWS CLI has an answer to that question, too. To list all the items in your table, you can use the scan
command in the dynamodb
section of the AWS CLI. The scan
command returns all the items in the table unless you provide a filter. To list all the items in the table, run the command in the following listing from your terminal.
Listing 3.8 An AWS CLI command that lists all the items from the pizza-orders
table
aws dynamodb scan ①
--table-name pizza-orders ②
--region eu-central-1
--output json ③
This command “scans” your pizza-orders
table and returns the result as a JSON object. You can change the output value to text
, and you’ll get the result in text format. A few more formats are available, including XML.
The command should return something like the value in the following listing: a JSON response with the count and an array of all your table items.
Listing 3.9 Response from the scan
command for your pizza-orders
table
{
"Count": 1, ①
"Items": [ ②
{
"orderId": { ③
"S": "some-id" ④
},
"orderStatus": { ③
"S": "pending" ④
},
"pizza": { ③
"N": 4 ④
},
"address": { ③
"S": "221b Baker Street" ④
}
}
],
"ScannedCount": 1, ⑤
"ConsumedCapacity": null ⑤
}
Awesome—it seems that your API is working as expected!
Try to add another pizza order now with the same curl
command—for example, a Napoletana for 29 Acacia Road. If you then run the AWS CLI command from listing 3.8 again to scan the database, you’ll see only one item in your table; the previous one doesn’t exist anymore.
Why did that happen?
Remember that you hardcoded an orderId
in your create-order.js handler, as shown in listing 3.2?
Each of the orders should have a unique primary key, and you used the same one, so your new entry replaced the previous one.
You can fix that by installing the uuid
module from NPM and saving it as a dependency. uuid
is a simple module that generates universally unique identifiers.
After you download the module, update your create-order.js handler as shown in the next listing. You can simply import and invoke the uuid
function to get a unique ID for the order. Keep in mind that this listing shows only the part of the create-order.js file affected by this change; the rest of the file is the same as the one in listing 3.2.
Listing 3.10 Adding UUIDs for the orders while creating them
const AWS = require('aws-sdk')
const docClient = new AWS.DynamoDB.DocumentClient()
const uuid = require('uuid') ①
function createOrder(request) {
return docClient.put({
TableName: 'pizza-orders',
Item: {
orderId: uuid(), ②
pizza: request.pizza,
address: request.address,
status: 'pending'
}
}).promise() ③
// Rest of the file stays the same
After you redeploy the code by invoking the claudia update
function, use the same curl
command to test your API again and then scan the database with the AWS CLI command from listing 3.8. As you can see, the new orderId
for your new order is some unique string like this one: 8c499027-a2d7-4ad9-8360-a49355021adc
. If you add more orders, you’ll see that all of them are now saved in the database, as expected.
After storing an order in the database, retrieving one should be fairly easy. The DocumentClient
class has a scan
method, which you can use to retrieve the orders.
The scan
method works the same way as in the AWS CLI, with a small difference: You need to pass an object to it as a parameter, along with some options. In the options, the only required attribute is the name of your table.
Besides scanning the database, your get-orders.js handler can get a single item by an ID. You can do that with a scan by filtering the results, but that’s inefficient. A more efficient way is to use the get
method, which works almost the same way but requires a key for your item, too.
Let’s update your get-orders.js file in the handlers folder to scan the orders from your table, or to get a single item if an order ID is provided. When you update your code, it should look like the code in the following listing. Once you’ve made these changes, deploy the code using the claudia update
command.
Listing 3.11 get-orders.js handler reads the data from the pizza-orders
table
const AWS = require('aws-sdk')
const docClient = new AWS.DynamoDB.DocumentClient() ①
function getOrders(orderId) {
if (typeof orderId === 'undefined')
return docClient.scan({ ②
TableName: 'pizza-orders'
}).promise()
.then(result => result.Items) ③
return docClient.get({ ④
TableName: 'pizza-orders',
Key: { ⑤
orderId: orderId
}
}).promise()
.then(result => result.Item) ⑥
}
module.exports = getOrders
Let’s test it! First, scan all the orders with the following curl
command:
curl -i
-H "Content-Type: application/json"
https://whpcvzntil.execute-api.eu-central-1.amazonaws.com/chapter3_2/orders
When you run it, it should display something like this:
HTTP/1.1 200 OK
[{
"address": "29 Acacia Road",
"orderId": "629d4ab3-f25e-4110-8b76-aa6d458b1fce",
"pizza": 4,
"orderStatus":"pending"
}, {
"address": "29 Acacia Road",
"orderId": "some-id",
"pizza": 4,
"status": "pending"
}]
Don’t worry if the order ID is different from yours; it should be unique.
Now try using an ID from one of the returned orders to get a single order. You can do that by running the following curl
command from your terminal:
curl -i
-H "Content-Type: application/json"
https://whpcvzntil.execute-api.eu-central-1.amazonaws.com/chapter3_2/orders/629d4ab3-f25e-4110-8b76-aa6d458b1fce
The result should look something like this:
HTTP/1.1 200 OK
{
"address": "29 Acacia Road",
"orderId": "629d4ab3-f25e-4110-8b76-aa6d458b1fce",
"pizza": 4,
"status": "pending"
}
It works! Awesome and easy, right?
As you’ve seen, saving orders to the database and retrieving them is easy. But Aunt Maria has told you that sometimes customers make mistakes and order the wrong pizza, so she wants the capability to change or cancel a pizza order.
To fulfill Aunt Maria’s request, you need to connect two more API endpoints to the database:
pizza-orders
DynamoDB table.pizza-orders
DynamoDB table.When you finish both endpoints, your API should have the same structure as the one in figure 3.3.
The solution’s code is in the next section. Before looking at it, try to complete the exercise yourself, but if you’re struggling, peek a little.
A few hints:
DocumentClient
for both updates and deletions.DocumentClient.update
method. Besides TableName
, this method requires a few more items in the object you are providing, including Key
, UpdateExpression
, and others. See the official documentation for the full list: http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/DynamoDB/DocumentClient.html#update-property.update
method seems too complex for you, remember that DocumentClient.put
will replace an existing order with a new one, so you can try using that one.DocumentClient.delete
method. To delete an item, you need to provide an object that contains the TableName
and the Key
for that item. For more information, see the official documentation: http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/DynamoDB/DocumentClient.html#delete-property.In case this is too easy, here are a few additional things you can do:
The solutions to these additional tasks are available in the final application source code, along with code annotations.
Finished already or peeking a little? If you’re finished, that’s great, but even if you weren’t able to complete the exercise without any help, don’t worry. DynamoDB is a bit different from the other popular noSQL databases, and you may need more time and practice to understand it.
Let’s take a look at the solution. The following listing shows the updates for the update-order.js file in the handlers folder of your project.
Listing 3.12 Updating an order in the pizza-orders DynamoDB table
const AWS = require('aws-sdk')
const docClient = new AWS.DynamoDB.DocumentClient() ①
function updateOrder(orderId, options) {
if (!options || !options.pizza || !options.address)
throw new Error('Both pizza and address are required to update an order')
return docClient.update({ ②
TableName: 'pizza-orders',
Key: { ③
orderId: orderId
},
UpdateExpression: 'set pizza = :p, address=:a', ④
ExpressionAttributeValues: { ⑤
':p': options.pizza,
':a': options.address
},
ReturnValues: 'ALL_NEW' ⑥
}).promise()
.then((result) => { ⑦
console.log('Order is updated!', result)
return result.Attributes
})
.catch((updateError) => {
console.log(`Oops, order is not updated :(`, updateError)
throw updateError
})
}
module.exports = updateOrder ⑧
It’s not that different from create-order.js. The two major differences are
DocumentClient.update
method with a Key
, which is orderId
in your caseorderId
and new values to update (pizza
and address
)The following listing shows the updates for the delete-order.js file in your handlers folder. The required updates are similar to those in both the create-order.js and update-order.js files; the only difference is that you’re using the DocumentClient.delete
method here.
Listing 3.13 Deleting an order from the pizza-orders DynamoDB table
const AWS = require('aws-sdk')
const docClient = new AWS.DynamoDB.DocumentClient() ①
function deleteOrder(orderId) { ②
return docClient.delete({ ③
TableName: 'pizza-orders',
Key: { ④
orderId: orderId
}
}).promise() ⑤
.then((result) => {
console.log('Order is deleted!', result) ⑥
return result
})
.catch((deleteError) => {
console.log(`Oops, order is not deleted :(`, deleteError) ⑥
throw deleteError
})
}
module.exports = deleteOrder ⑦
Seems easy, right?
Now you need to run the claudia update
command from your pizza-api folder one more time to deploy your code. To test whether everything works, you can use the same curl
commands you were using in chapter 2. Copy them from listings 3.14 and 3.15, and paste them in your terminal. Don’t forget to update your orderId
value. Using the one provided in those listings won’t work because it’s just a placeholder.
Listing 3.14 curl
command for testing PUT /orders/{orderId}
route
curl -i
-H "Content-Type: application/json"
-X PUT
-d '{"pizza": 3, "address": "221b Baker Street"}'
https://whpcvzntil.execute-api.eu-central-1.amazonaws.com/chapter3_3/orders/some-id ①
This command should return the following:
HTTP/1.1 200 OK
{
"address": "221b Baker Street",
"orderId": "some-id",
"pizza": 3
"status": "pending"
}
Listing 3.15 curl
command for testing DELETE /orders/{orderId}
route
curl -i
-H "Content-Type: application/json"
-X DELETE
https://whpcvzntil.execute-api.eu-central-1.amazonaws.com/chapter3_3/orders/some-id ①
This command should return
HTTP/1.1 200 OK
{}
aws-sdk
Node module. Among other things, the AWS SDK also exposes the DynamoDB DocumentClient
class, which allows you to save, query, edit, and delete items in DynamoDB tables.