Working with a server

Most applications today rely on a specific server environment. An administrator creates a server environment, either physical or virtual, configures it, and then provides any required resources a developer may need. The developer then places an application created and tested on a test server of (you hope) precisely the same characteristics on the server. After some testing, the administrator comes back and performs additional configuration, such as setting up accounts for users. Other people may get involved as well. For example, a DBA may set up a database environment for the application, and a web designer may create a mobile interface for it. The point is that a lot of people get involved in the process of getting this application ready for use, and they remain involved as the application evolves. The time and money spent to maintain the application is relatively large. However, the application environment you create provides a number of important features you must consider before moving to a serverless environment:

• The server is completely under the organization’s control, so the organization chooses every feature about the server.
• The application environment tends to run faster than even the best cloud server can provide (much less a serverless environment, in which you have no control over the server configuration).
• Any data managed by the application remains with the organization, so the organization can reduce the potential for data breaches and can better adhere to any legal requirements for the data.
• Adding more features to the server tends to cost less after the organization pays for the initial outlay.
• A third party can’t limit the organization’s choice of support and other software to use with the application, nor can it suddenly choose to stop supporting certain software functionality (thereby forcing an unexpected application upgrade).
• Security tends to be less of a concern when using a localized server as long as the organization adheres to best practices.

Working in a serverless environment

Using a localized server does have some significant benefits, but building, developing, and maintaining servers is incredibly expensive because of the staffing requirements and the need to buy licenses for the various pieces of software. (You can mitigate software costs by using lower-cost open source products, but the open source products may not do everything you need or may provide the same services in a less efficient environment.) However, organizations have more than just cost concerns to consider when it comes to servers. Users want applications that are flexible and work anywhere today. With this in mind, here are some reasons that you may want to consider a serverless environment for your application:

• Lower hardware and administration cost: You don’t have hardware costs because Amazon provides the hardware, and the administration costs are theoretically zero as well. However, you do pay for the service and need to consider the trade-off between the cost of the hardware, administration, and services.
• Automatic scaling: You can bring on additional hardware immediately without any startup time or costs.
• Improved development team efficiency: Reducing the number of people in a process generally increases efficiency because you have fewer lines of communication and fewer human failure points as well. In addition, the developer needs to worry only about writing great code — not whether the server will function as expected.

• Low learning curve: Working with Lambda doesn’t require that you learn any new programming languages. In fact, you can continue to use the third-party libraries that you like, even if those libraries rely on native code. Lambda provides an execution environment, not an actual coding environment. You use a Lambda function (explained in the “Creating a Basic Lambda Application” section, later in this chapter) to define the specifics of how your application runs.

  Lambda does provide a number of prebuilt function templates for common tasks, and you may find that you can use one of these templates instead of building your own. It pays to become familiar with the prebuilt templates because using them can save you considerable time and effort. You just need to tell Lambda to use a particular template with your service resources.

• Increased reliability: In general, because Amazon can provide additional systems immediately, a failure at Amazon doesn’t spell a failure for your application. What you get is akin to having multiple sets of redundant failover systems.

Many of Amazon’s services come with hidden assumptions that could cause problems. For example, with Lambda, Amazon fully expects that you use other Amazon services as well. A Lambda app can react to an event such as a file being dropped into an S3 bucket by a user, but it can’t react to an event on your own system. The user may drop a file onto a folder on your server, but that event doesn’t create an event that Lambda can see. What you really get with Lambda is an incredible level of flexibility with significantly reduced costs as long as you want to use the services that Amazon provides with it. In the long run, you may actually find that Lambda locks you into using Amazon services that don’t really meet your needs, so be sure to think about the ramifications of the choices you make during the experimentation stage.

Selecting a Lambda blueprint

Lambda supports events from a number of Amazon-specific sources such as S3, DynamoDB, Kinesis, SNS, and CloudWatch. This chapter relies on S3 as an event source, but the techniques it demonstrates work with any Amazon service that produces events that Lambda can monitor.

When working with blueprints, you need to know in advance the requirements for using that blueprint. For example, Figure 6-2 shows the blueprint used in this chapter, s3-get-object-python. The blueprint name tells you a little about the blueprint, but the description adds to it. However, the important information appears at the bottom of the box. In this case, you see that the blueprint uses Python 2.7 and S3. Every blueprint includes these features, so you know what resources the blueprint requires before you use it.

Amazon provides a number of blueprints, and finding the one you need can be time consuming. Adding a condition to the Filter field or choosing a programming language from the Language field reduces the search time. For example, to locate all the S3-specific blueprints, type S3 in the Filter field. Likewise, to locate all the Python 2.7 blueprints, choose Python 2.7 in the Languages field.

Amazon licenses most of the blueprints under the Creative Commons Zero (CC0) rules (see https://creativecommons.org/publicdomain/zero/1.0/ for details). This means that Amazon has given up all copyright to the work, and you don’t need to worry about getting permission to use the blueprint as part of anything you do. However, the operative word in the Amazon wording on the blueprint page is “most,” which means that you need to verify the copyright for every blueprint you use to ensure that no hidden requirements exist that oblige you to get a license.

Configuring a function

Using the Lambda console and a blueprint means that the function-creation process is less about coding and more about configuration. You need to tell Lambda which blueprint to use, but the blueprint contains the code needed to perform the task. In addition, you tell Lambda which resources to use, but again, it’s a matter of configuration and not actual coding. The only time that you might need to perform any significant coding is when a blueprint comes close to doing what you want to do but doesn’t quite meet expectations.

The example that follows uses the S3 bucket that you see how to create in the “Performing a Few Simple Tasks” section of Chapter 2. However, you can use any bucket desired. The bucket simply holds objects that you want to process, so it’s a matter of choosing the right bucket to perform the required work. The blueprint used in this section, s3-get-object-python, simply reports the metadata from the objects dropped into the bucket.

1. Click s3-get-object-python.

   You see the Configure Event Sources page, shown in Figure 6-3.

   Even though the blueprint automatically chooses event-source information for you, you can still control the event source in detail. For example, you can change the Event Source Type field to choose a service other than S3, such as Kinesis, S3, CloudWatch, or DynamoDB.

2. Select an object source in the Bucket field.

   The example assumes that you want to use the bucket that Chapter 2 tells you how to create. However, any bucket you can access that receives objects regularly will work fine for this example. AWS simply chooses the first S3 bucket, so configuring this field is essential.

3. Choose the Object Created (All) option in the Event Type field.

   S3 supports three event types: Object Created (All); Object Removed (All); and Reduced Redundancy Lost Object. Even though Lambda receives all the events, you use the entries in the Prefix and Suffix fields to filter the events so that you react only to the important events. For example, you can choose to include a folder path or part of a filename as a prefix to control events based on location or name. Adding a file extension as the suffix means that Lambda will process only files of a specific type. The example provides simple processing in that it reacts to any item created in the bucket, so it doesn’t use either the Prefix or Suffix fields.

4. Click Next.

   You see the Configure Function page, shown in Figure 6-4. As with the Configure Event Sources page, it pays to check out the Runtime field. In this case, you can choose between Python 2.7 and Java 8. Even when the blueprint description tells you that it supports a specific language, you often have a choice of other languages you can use as well.

5. Type MyFunction in the Name field.

   Normally, you provide a more descriptive function name, but this name will do for the example and make it easier to locate the function later when you want to remove it.

6. Scroll down to the next section of the page.

   You see the function code. At this point, you can use the online editor to make small coding changes as needed. However, you can also upload a .zip file containing the source code you want to use, or you can upload existing source code from S3. This example relies on the example code that Amazon provides, so you don’t need to make any changes.

   Notice that the Python code contains a function (specified by the def keyword) named lambda_handler. This function handles (processes) the information that S3 passes to it. Every language you use has a particular method for defining functions; Python uses this method. As part of configuring the Lambda function, you need to know the name of the handler function.

7. Scroll down to the next section of the page.

   You see two sections: the Lambda Function Handler and Role section and the Advanced Settings section, as shown in Figure 6-5. The blueprint automatically defines the Handler field for you. Note that it contains the name lambda_handler as the handler name. When you use custom code, you must provide the name of the function that handles the code in this section.

   The first part of the entry, lambda_function, is the name of the file that contains the handler function. As with the function name, the blueprint automatically provides the appropriate name for you. However, if you upload a file containing the code, you must provide the filename (without extension) as the first entry. Consequently, lambda_function.lambda_handler provides the name of the file and associated handler function. The filename is separated from the handler function name by a period.

8. Choose S3 Execution Role in the Role field.

   You must tell AWS what rights to use when executing the lambda code. The environment provides several default roles, or you can create a custom role to use instead. AWS opens a new page containing the role definition, as shown in Figure 6-6. AWS fills in the details for you. However, you can click View Policy Document to see precisely what rights you’re granting to the Lambda function.

9. Click Allow.

   AWS returns you to the Lambda Function Handler and Role section of the page shown previously in Figure 6-5.

   The Advanced Settings section contains the settings you need to use for this example. The Memory field contains the amount of memory you want to set aside for the Lambda function (with 128MB being the smallest amount you can provide).

   The Timeout field determines how long the Lambda function can run before AWS stops it. Setting the value too high can make your application pause when it encounters an error (resulting in frustrated users); setting it too low may unnecessarily terminate a function that could complete given more time. In most cases, you must experiment to find the best setting to use for your particular function, but the default setting provided by the blueprint gives you a good starting point.

   You use the VPC field to define which VPC to use to access specific resources. This particular example doesn’t require the use of a VPC.

10. Click Next.

    The Review page, shown in Figure 6-7, shows the settings for this function.

    Pay particular attention to the Enable Event Source check box. If you select this option and create the function, the function becomes active immediately. Amazon recommends that you leave this check box blank so that you can test the function before you enable it.

11. Click Create Function.

    If you chose not to enable the function, the function exists, but it doesn’t do anything. You aren’t incurring any costs at this point. AWS displays the page shown in Figure 6-8. Note the Test button in the upper-left corner.

Using ensembles of functions

Sometimes you can accomplish some incredibly interesting tasks without performing any coding at all by creating ensembles of functions available as blueprints. For example, you can use the s3-get-object blueprint to retrieve objects from an S3 bucket of specific types and then pass the object onto DynamoDB, where another Lambda function, such as microservice-http-endpoint, passes it onto a microservice that your company owns for further processing.

You can even double up on blueprints. The same DynamoDB content can trigger another Lambda function, such as simple-mobile-backend, to send alerts to mobile users about new content. You can achieve all these tasks without any significant coding. All you really need to do is think a little outside the box as to how you can employ the blueprints that Amazon provides and combine them in interesting ways.

Most of the Amazon blueprints are used for other purposes online. For example, you can see how to modify the s3-get-object-python blueprint to accommodate a product named Logentries (https://logentries.com/) at https://logentries.com/doc/s3-ingestion-with-lambda/. Another product, Logsene (https://sematext.com/logsene/) also uses the blueprint as a starting point (see details at https://github.com/sematext/logsene-aws-lambda-s3). For an example of a combined-service use, check out the article at https://micropyramid.com/blog/using-aws-lambda-with-s3-and-dynamodb/, which is about using S3 with DynamoDB. These blueprints get real-world use by third-party companies that use the blueprint as a starting point to do something a lot more detailed and interesting.

Creating the test setup

Before you can test your new function, you need to upload a file to the bucket you created in Chapter 2. This means opening the S3 Management Console, selecting the bucket, and uploading a file to it just as you did in Chapter 2. The example assumes that you’ve named this file HappyFace.jpg.

Filenames are case sensitive. Consequently, if you name your file happyface.jpg and try to access it from the test code as HappyFace.jpg, the output will tell you that the code can’t access the file. At first, you may think that you have a permissions problem because of the error message you receive from Amazon. However, verifying the capitalization of the filenames you use during the testing process can save you a lot of time and frustration.

Testing the function

Every Lambda handler function receives two pieces of information: an event and its context. The event and context can include any sort of data, but to make things simple during testing, the test functionality relies on strings, that is, text that appears within double quotation marks ("). The text is bundled within a pair of curly brackets ({})so that the test function receives information just as it normally would. The following steps show how to test your new Lambda function:

1. Click Test.

   AWS displays the default testing template, which is Hello World. You need to select the testing template for S3 or the test will fail.

2. Choose S3 Put in the Sample Event Template field.

   You see the Input Test Event page, shown in Figure 6-9. To make this template function correctly, you make three small code changes, as the next steps explain.

3. Change the "key" entry as shown here:

   "key": "HappyFace.jpg"

4. Change the "arn" entry to match the arn for your bucket.

   The Amazon Resource Name (ARN) tells AWS where to find the bucket. The ARN for your S3 bucket will differ from the ARN I use here. However, your ARN entry will look something like this:

   "arn": "arn:aws:s3:::johnm.test-bucket",

   Make sure to create the ARN correctly. The number of colons between various elements is essential. The best way to avoid problems is to copy the ARN from a screen location that you know is correct and paste it into your code. Otherwise, you can spend hours trying to find the one missing or extra colon in your code.

5. Change the "name" field to match the name of your bucket.

   In this case, you provide only the bucket name. It should look something like this:

   "name": "johnm.test-bucket",

6. Click Save.

   AWS saves the test code for you and returns you to the test page shown in Figure 6-8.

At this point, you can click Test again. In most cases, Amazon won’t charge you anything because your S3 resource usage will remain within the limits of the free tier. However, by executing the test code, you can incur a small cost. Figure 6-10 shows an example of the output you see. The output correctly shows that the content type for HappyFace.jpg is "image/jpeg".

Checking the function metrics

In addition to various levels of testing, you can also view the metrics for your function by clicking the Monitoring tab. Figure 6-11 shows typical metrics for Lambda functions. In this case, you see how many times events have triggered the function, the duration of each call, and the number of errors that the functions have experienced.

Deleting the function

You don’t want to keep a function around any longer than necessary because each function invocation costs money. One way to keep costs down is to disable the function by clicking the Enabled link for the function in the Event Sources tab and choosing Disable when you see the configuration dialog box. The function will continue to perform its task as long as just one of the event sources remains enabled, so you must disable all event sources to stop the function from responding to events.

At some point, you simply won’t need the function, and keeping it around is a recipe for unexpected costs. To delete a function, choose Actions ⇒   Delete Function. AWS will ask whether you’re sure that you want to delete the function. Click Delete to remove the function and all its configuration information.