This chapter covers

• How CodePen uses serverless for its preprocessors
• How MindMup runs client APIs and file conversions

We’ve come to the end of this serverless journey. Through this book, you’ve learned what serverless is, how to use it in a new application, and how to migrate your existing applications to a serverless architecture. But we know that there is one important question you want to ask: Who is using serverless in production?

Hearing about companies using a new approach in production is always useful, along with the problems they’ve encountered and how it’s worked for them. In this case, another interesting question is why these companies felt serverless with Claudia.js was the right solution for them.

To answer that question, we chose two companies to ask about the challenges they’d faced, how and why they decided to go serverless, their architecture before and after going serverless, and the approximate costs.

Many companies are using serverless in production, and it was hard to pick the two best examples. But because you can read many success stories from enterprise companies using serverless on the AWS blog, we chose two companies that have successful products with small teams behind them. We think serverless allows you to move and scale quickly with a small team and a relatively low infrastructure cost, and we decided to show you how CodePen and MindMup are doing just that.

15.1 CodePen

CodePen (https://codepen.io) is a popular web application for creating, showcasing, and testing snippets of user-created HTML, CSS, and JavaScript. It’s an online code editor and open source learning environment where developers can create code snippets (creatively named “pens”), test and share them with other developers, and work collaboratively.

The CodePen developers had already done an episode on their podcast regarding their serverless applications with Node and Claudia.js. Alex Vazquez, one of the founders of CodePen, said that they extensively use AWS Lambda with Claudia for preprocessors, so we contacted Alex for an interview. The following is a description of what we learned.

15.1.1 Before serverless

CodePen allows developers to write and compile HTML, CSS, and JavaScript live in the editor. To do so, it needs to be able to display the developer’s code; if the developer is using preprocessors (such as SCSS, Sass, and LESS for CSS, and even Babel for JavaScript), it needs to preprocess it, too.

Therefore, the initial CodePen architecture was based on two monolithic Ruby on Rails applications—the main website and another application dedicated to preprocessors—and a single, relatively small database service.

You can see the CodePen architecture before moving to serverless in figure 15.1.

CodePen aims to allow its users to run their own code as frequently as they want. Another goal of CodePen is for people to get excited about code examples they find on the site and for pens to go viral—but there are moments that it gets “too awesome.” Naturally, CodePen gets huge spikes for different pens, and they can be impossible to predict. For these situations, CodePen needs to be able to scale quickly. Also, most CodePen users use CodePen free of charge, so it needs both a quick and an inexpensive way to make sure that it can serve all its users.

The team behind CodePen is small and distributed around the world, and they have one DevOps engineer. Because CodePen is running other people’s code, they’re always striving to keep security at a maximum. So, they started investigating how to separate the users' code execution for security purposes. That was the first time they heard about AWS Lambda: their DevOps engineer, Tim Sabat, suggested it as a possible solution. Initially, the CodePen developers rejected the idea because they didn’t really see the point, thinking it would be a hassle to set up and configure. They already had preconfigured servers, and it didn’t seem as though they needed individual Lambda services.

But one day they had a need to format code on demand using a new tool, and this “Lambda concept” seemed like a perfect solution for this task, especially with Claudia.js as its deployment tool. Alex said that they learned a lot about all the different things you can do with serverless functions, such as using API Gateway to set up a full HTTP API, connecting to S3, and setting up cron jobs. It suddenly dawned upon the team how powerful serverless applications are.

15.1.2 Serverless migration

CodePen runs a lot of preprocessors. Each of the preprocessors is dedicated to processing a specific code type, whether it’s HTML, CSS, or JavaScript. They also perform differently, have different CPU and memory requirements, and should run asynchronously. You can see how the preprocessors work in figure 15.2.

As you can see, each preprocessor does a different job, and one’s execution could affect another’s. Therefore, when the developers analyzed their current architecture, they realized that separating them was important for both performance and security reasons. Running all the individual preprocessors on the same server made it much harder to optimize each one and to determine where any bugs were. They decided that it made a lot of sense to split them up and run individual AWS Lambda functions, like their previous “code-prettifying-on-demand” service.

The team decided to completely refactor their monolithic Ruby on Rails preprocessor application, dividing it into a single serverless function called the router and many other individual preprocessor serverless functions. The router’s job is to invoke the needed preprocessor functions, each of which has the goal of preprocessing a certain code type.

Using Claudia, the team realized that even their front-end engineers could help with the migration, taking the strain off the lone DevOps engineer. CodePen’s lead front-end developer, Rachel Smith, was able to handle the whole refactor of the router to AWS Lambda, which has enabled CodePen to serve more than 200,000 concurrent requests during peak times. At the same time, Alex and the other developers refactored and migrated the individual preprocessors. You can see the resulting architecture in figure 15.3.

After migrating to serverless, CodePen still has the main monolithic Ruby on Rails application, but instead of a monolithic preprocessor application, it has the router serverless function and about a dozen serverless preprocessor functions. The router is available over API Gateway and uses the AWS SDK to directly invoke every serverless function it needs. It receives an array of tasks to accomplish in the payload. For example, one of the preprocessors is Babel; it has a default version, and the payload sent to the router states the version of Babel that it needs. The router knows which Babel versions are available, because each of the preprocessor Lambda functions comes with an associated version number. The design follows the Command pattern, where you provide the command to run and the router knows which Lambdas it needs to invoke.

In addition to the performance and security benefits, separating the monolithic preprocessor application into serverless functions means that as CodePen grows, it will be trivial to add more preprocessors as serverless functions and make them available to users. For the CodePen developers, that’s the biggest benefit: before, everything was stuck in the “mini-monolith,” as they called their previous Ruby on Rails preprocessor application, and changing or adding new features wasn’t easy.

15.1.3 Cost of the infrastructure

Costs have decreased significantly for CodePen after going serverless, because it had to reserve instances up front before; now it pays based on demand. The load fluctuates, so when there isn’t a lot of traffic, it’s great that CodePen isn’t charged for extra capacity. Additionally, with API Gateway caching, CodePen was able to save even more money. Compared with paying for reserved servers, AWS Lambda costs are cheap.

Additionally, one important consideration most people don’t factor in is the amount of time spent managing servers and the inherent complexity. To see what a difference it made, CodePen even tried changing all of their Lambdas to the maximum (3 GB) memory. Everything was even faster, because it was running on great hardware (better than was required, really). The monthly cost went up $1,000, which is a lot, but even at that level it was still affordable—and according to Alex Vasquez, even the additional cost for the maxed-out AWS Lambdas didn’t compare with the cost of the developer time required to set up and deploy a server.

The right Lambda size

CodePen doesn’t have a standard for defining the size of its serverless functions. The monoliths and the functions (if they get bigger) are separated using the logical unit approach. This approach specifies that, for example, if an image manipulation is needed, a serverless function will be created that includes everything related to that image manipulation.

Likewise, for an audio manipulation, a serverless function dedicated to that audio type will be created.

In CodePen’s case, the preprocessors are the logical units: Babel, Autoprefixer, Sass, LESS, and so on. For each one, an individual Node.js serverless function has been created. This allows for deployment of multiple versions of a specified preprocessor. Breaking up the functionality even further doesn’t make sense for CodePen, because the functions would become difficult to manage with all the directories and repositories.

Additionally, each CodePen serverless function must have a specific memory allocation. The general CodePen serverless function memory allocation is 512 MB. Babel is the only preprocessor that has a slow start time, so it is allocated 1,024 MB.

15.1.4 Testing and challenges

CodePen mostly has unit tests, written with Jest. Because both CodePen’s serverless functions are written in Node.js with Claudia, their front-end engineers are also able to write tests. There aren’t many integrated tests, but there are many defined test cases—mostly for debugging or manual testing of the entire system.

Additional scaling

AWS Lambda’s default limit for the number of concurrent requests is 1,000, and because of the unpredictable traffic patterns to its site, CodePen managed to hit this limit.

After sending a simple request to AWS, CodePen received an upgrade to 5,000 concurrent requests almost immediately.

Cold starts

The asynchronous nature of CodePen’s preprocessors softens the impact of their serverless function cold starts. It doesn’t affect their operations or the user experience. CodePen also relies heavily on API Gateway cache. They create a unique URL for every defined data set because API Gateway can cache only unique URLs.

Monitoring

For monitoring their serverless system, the CodePen team use only CloudWatch. For error reporting, they use HoneyBadger and its Node.js SDK. The entire system is monitored, and whenever a timeout or an error occurs, it is reported to HoneyBadger.

Security

CodePen uses JSON Web Token (JWT) for security. The token is generated by the monolith and shared to the client so that the client can easily authenticate the requests sent afterward to the router serverless function.

15.2 MindMup

MindMup (https://www.mindmup.com) is a popular mind mapping web application written primarily in JavaScript. According to Gojko Adzic, one of the company’s co-founders, MindMup serves almost half a million active users monthly with just a two-person team. To accomplish that, they started using serverless on AWS extensively, which significantly reduced their costs. More importantly, serverless pushed them to improve the architecture and is now allowing them to move quickly and experiment more.

15.2.1 Before serverless

From its inception in 2013, MindMup was optimized for a small team, relatively cheap infrastructure, and rapid development. At the beginning, the developers chose to combine Heroku with AWS to build an infrastructure that required minimal maintenance but was still scalable.

MindMup had one Heroku app that was the system core; it served a single-page web app and an API. The API was in charge of authentication, authorization, providing data, and subscriptions. MindMup began as a free service, but after almost two years, the creators added a paid option for collaboration. To support paid users, they needed a simple but scalable database, so they added AWS DynamoDB.

Along with the single-page application for mind map creation, one of the most important parts of MindMup is exporters—a feature that allows users to export mind maps they’ve created in different file formats, such as PDF, Word, or even as a PowerPoint presentation. They had lots of converters, and usage and memory consumption for each of them varied. For example, the PDF exporter was used all the time and required a lot of CPU and memory for file generation. The text exporter required only a small amount of resources, and the Markdown exporter was used much less frequently. Also, different exporters required different applications:

• For PDF, they needed Ghostscript, a software suite for PDF manipulation.
• For Word, they used Apache POI, a library that provides Java libraries for reading and writing files in Microsoft Office formats.
• For other exporters, they needed Ruby and Python.

To export mind maps in different formats, they used AWS S3 file storage. But instead of uploading files from the API, they uploaded files directly from the browser to an AWS S3 bucket, using signed URLs generated by the API. Notifications about newly uploaded conversion request files were then pushed to an AWS Simple Queue Service (SQS) queue. They had one SQS queue per conversion file type and a few applications to read from the queue and convert the file to the specified file type.

The converter applications were Heroku apps, but supporting lots of independent exporters required almost 30 dynos (Heroku’s lightweight Linux-based containers), which was too expensive. To reduce the cost, they bundled the exporters into a few Heroku apps; each app was a group of all the exporters that required the same programming language, because Heroku dynos require programming language selection on setup. The MindMup architecture at that point was similar to figure 15.4.

Bundling exporters reduced the cost of the infrastructure, but it also brought certain issues, such as the following:

• There was less isolation and more situations where different libraries could collide.
• Adding a new exporter required a lot of coordination because it required a new app and a new SQS queue. SQS queues were not too expensive, but they were an additional layer of complexity.
• Because it was all bundled, performing small experiments was difficult.
• Updating packages for different Heroku apps with different programming languages was difficult to manage.

Also, there was bit of a scandal with the way Heroku was doing routing—it routed requests to a random dyno instead of to a free one, even if the random dyno was already busy. To learn more about this incident, see https://genius.com/James-somers-herokus-ugly-secret-annotated.

At that time, exporters were the biggest pain point of the MindMup architecture.

15.2.2 Serverless migration

In January 2016, the developers were planning to add a new exporter, and they were looking at AWS Lambda because they had seen that files saved in S3 could invoke Lambda functions. As an experiment, Gojko created an exporter in Node.js with a bash script to automate the deployment of a new Lambda function. They liked how it worked, and in contrast with Heroku, which required them to reserve capacity, the AWS Lambda model was pay-per-use. That meant they could conceivably have a separate function for each exporter. At that moment, they decided to start gradually moving all the exporters to AWS Lambda, one by one.

Gojko saw that for the first exporter he had 30 lines of Node.js code and more than 200 lines of bash script. He realized that the risk had shifted from the code to the deployment and that he needed a tested tool to be able to fully migrate to serverless. The ecosystem of serverless tools wasn’t developed enough, however, so he decided to “roll his own” solution—which later was open-sourced and published as Claudia.js.

Because the MindMup website was a monolith that served an API and performed server-side rendering, the developers decided to start moving each of the API services to AWS Lambda functions. Migration of an API could be done through API Gateway, but they used this opportunity to rewrite and refactor some of the legacy code and to fix the three-year-old codebase that had grown “in many weird ways,” according to Gojko.

As the migration progressed, they ended up with a single application on Heroku that was responsible for rendering the index.html file. They replaced it with a static site on an AWS S3 bucket, with AWS CloudFront as a CDN and caching layer. The only exporter that was left on Heroku was the PowerPoint exporter, written in Java. It ran fine for some time, but they eventually moved that to AWS Lambda, too.

The entire migration took about a year of slow rewriting, and MindMup was a fully serverless application in February 2017. Everything was divided into small Lambda functions that are organized in logical units, and a single function is not necessarily doing just one small thing. In the current architecture, the MindMup website is loaded from AWS S3 and CloudFront. A browser communicates directly with the so-called Gold API, which is a Lambda function behind an API Gateway. The Gold API is connected to the DynamoDB database, and it is used by Stripe and PayPal webhooks.

The Gold API also generates presigned URLs that allow a browser to write files directly to an S3 bucket and to poll the bucket to check if the files were generated. When a file is uploaded to the S3 bucket, it triggers an SNS notification, which then triggers the converters.

The biggest difference is with the exporters, because each exporter—regardless of how many requests it handles—is in its own Lambda function.

When a file is converted, an exporter directly stores the result to an S3 bucket, and the browser gets the converted file.

Even more Lambda functions are used for analytics purposes, triggered by the different parts of the system (for example, by an API or by one of the exporters) via SNS notifications. Those Lambda functions process the data and store it to a few different services, such as S3 buckets and DynamoDB tables.

There’s also an authorization component in front of an API that communicates with the static website and the Gold API Lambda function. MindMup uses Cognito for Google sign-in, but they also have a custom authorizer for the rest of the app.

The current MindMup architecture is shown in figure 15.5.

Although MindMup is fully serverless now, the team is constantly working on improvements. For example, one of the next steps is to add scalable real-time collaboration support, using Kinesis streams and AWS Lambda functions.

15.2.3 Cost of the infrastructure

One of the most amazing things about MindMup’s serverless migration was the improvement in application infrastructure cost. A comparison of the infrastructure costs and the number of users for December 2016 and December 2015 showed that the user base had grown by about 50%, whereas the costs had decreased by about 50%. At that point, they decided to drop Heroku and SQS completely; they moved everything to AWS, reduced data transfers, and reduced latency. With Heroku, they had been paying for reserved capacity. That changed when they went serverless. The Gold API was the gateway for everything else, and it had been their main bottleneck. As a serverless function, it scales automatically, so it’s no longer a bottleneck.

In September 2017, MindMup had around 400,000 active users. Their total monthly AWS bill was $102.92 USD. See table 15.1 for cost breakdown by services.

They managed to reduce the cost because they let the front-end application talk to some services, such as S3, directly, without putting an API between the browser and S3. This affects the cost because different AWS services have different pricing models. For example:

• AWS charges for the number of requests, execution duration, and memory consumption.
• API Gateway charges for the number of requests and for data transfer.
• Amazon S3 charges only for data transfer.

In a traditional application, you would upload images through the API, but by doing this, you would incur costs for data transfer in S3 and API Gateway and for the number of requests, request duration, and memory consumption in AWS Lambda. In a serverless application, you can get the presigned URL for S3 from an API, which takes less than 100 ms and requires minimal memory. You can then upload the file directly to an S3 bucket from the front end and pay for only that data transfer.

15.2.4 Testing, logs, and challenges

Knowing that Claudia is well covered with automated tests, it was interesting to hear from Gojko how they test MindMup. As expected, MindMup has a lot of unit and integration tests. Everything is separated into libraries, and they use Hexagonal Architecture extensively.

Tests are written using the Jasmine framework. Each of the Lambda functions has many unit tests, and a few integration tests where needed. For most of the Lambda functions, they have a lambda.js file that does almost nothing, and is responsible just for wiring other components. This lambda.js file is rarely covered with tests because it contains only three to four lines of code. Then they have the main.js file, which is the main file for that Lambda function; it receives an event from lambda.js and processes it. This main.js file is connected to different libraries (for example, FileRepository), configured, and passed from the lambda.js file.

The biggest chunk of their testing is unit tests for the main.js file. They also have integration tests that connect main.js with the MemoryRepository. The FileRepository is tested with both unit and integration tests, but separately, because it’s a separate library.

The testing flow of the typical AWS Lambda function is depicted in figure 15.6.

For some of the exporters, they also conduct visual tests using Appraise, a tool for visual approval testing that we mentioned in chapter 11. To learn more about Appraise, see http://appraise.qa/.

For logs, MindMup uses CloudWatch. They also track errors, payment information, and access; for example, front-end exceptions are tracked through Google Analytics. They store some events in S3 so they can search them. They do not need real-time logging with extensive search capabilities.

It’s important to note that MindMup uses labels for the environments, which is a bit contrary to AWS best practices. MindMup labels functions for development; those functions then talk to a development S3 bucket, DynamoDB table, and so on. Moving from development to production requires only a label update. It’s safer to have different accounts for different environments, but the small size of the team is what makes this approach feasible and MindMup able to grow faster.

Summary

• Serverless architecture allows you to build a scalable product fast with a small team and an inexpensive infrastructure.
• The combination of Claudia and AWS Lambda allows front-end developers to develop and deploy production-ready services without extensive back-end knowledge.
• Going serverless can lead to a significant cost reduction if you optimize your application for the platform.
• With serverless, risk shifts from the code to deployment and integrations, and it’s important to cover those things with tests.
• Using Hexagonal Architecture reduces code complexity and allows you to test your serverless application more easily.
• Migration to serverless can be used to refactor a whole legacy application.

As well as learning a lot about the benefits of serverless, we hope you enjoyed Aunt Maria’s company throughout this book, and haven’t become too hungry while reading it. Bon appétit!