JavaScript
Article
By Kev Zettler

Create Serverless Microservices with Node.js and AWS Lambda

By Kev Zettler

Create Serverless Microservices with Node.js and AWS Lambda was peer reviewed by Camilo Reyes, Joan Yin, Yaphi Berhanu and Moritz Kröger. Thanks to all of SitePoint’s peer reviewers for making SitePoint content the best it can be!

Create Serverless Microservices with Node.js and AWS Lambda

If you’ve ever wanted to write a web app or API without messing around with the server, Amazon’s Lambda might be what you’re looking for. AWS is a collection of developer tools that Amazon develops and publicly offers.

In this article we will get up and running with Lambda, a new tool in the Amazon Web Services (AWS) suite. We’ll be using Lambda to create a HTTP GET endpoint that will make requests using the GitHub API to pull repository info from GitHub and return a JSON response.

To follow along with this article you will need an AWS account of your own. You can create a free AWS account at https://aws.amazon.com/

What is AWS Lambda?

The Lambda tag line is “Run Code without Thinking about Servers.” At first glance this may sound confusing. Where or how does the code actually run then?

Serverless and Functions as a Service

“Serverless” is a new software infrastructure term you may have heard. It is used for describing solutions for on-demand code execution. The term “serverless” can be misleading because, in fact, there are still servers in the equation. A better descriptor is FaaS, or “Functions as a Service.”

Both definitions are used to describe a new development and deployment experience. This new experience is considered “serverless” because you as a developer no longer have to manage, monitor, or scale any servers that are running your code. You just upload your code to a FaaS provider (AWS Lambda, in this case) and the FaaS provider executes it and manages any infrastructure for you behind the scenes.

The pros and cons of Serverless architecture

Given this expanded definition of the “Serverless” architecture, let’s look at some of the pros and cons when working with Lambda.

Pros

  • On-demand usage pricing.

    Traditional server hosting uses a recurring billing cycle. Your server is always up and running using resources and waiting for input. You pay a monthly or yearly fee to keep it running for the duration of your billing cycle.

    Lambda uses on-demand pricing, where billing is calculated per function use. This means that if your project utilizes Lambda functions that are not being used in high demand, you can save a significant amount of money over traditional hosting solutions.

    Lambda pricing is as follows:

    • $0.20 per 1 million requests
    • $0.00001667 for every GB-second of compute time, with every execution rounded up to the nearest 100ms

    Learn more at: https://aws.amazon.com/lambda/pricing/

  • Built-in auto scaling

    In a traditional hosted infrastructure, there comes a time where you may need to worry about performance and scaling. As the traffic and usage of your application increase, you may need to add more hosted servers to your infrastructure to keep up with demand. This can cause failures and bottlenecks for your users. Lambda takes care of scaling silently when needed, removing additional cognitive overhead.

Cons

  • Inconsistent local development work flow.

    You can write Lambda function code locally and test it in isolation, but you won’t be able to simulate a production environment locally without creating your own hacked-together version of Lambda.

Lambda Key Concepts

Function code and triggers

Lambda has two main concepts: code and triggers. Code is self-explanatory. In our case, it’s the JavaScript code that you write and upload to Lambda to produce your desired behaviors.

Once uploaded, code will not execute on its own. Lambda has an additional concept called “triggers.” Triggers are events fired by other AWS services that pass data to the Lambda function for execution.

Some example triggers are:

  • An HTTP request to AWS API Gateway fires lambda code
  • An event fired on an interval, like a cron job from CloudWatch Events
  • A DynamoDB table is updated and triggers Lambda code

Lambda code function signature

You define a Lambda function by exporting a regular function from JavaScript that matches the expected Lambda signature.

exports.myLambdaFunction = (event, context, callback) => {
   // Use callback() and return 
}

The function receives three arguments:

  1. event — A key-value pair dictionary of ‘trigger data’ that Lambda passes to the function.

  2. context — AWS internal information such as AWS request ID, Lambda expiration timeout, and Log info.
    For more info see: http://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-context.html

  3. callback — A standard async JavaScript callback handler.
    For more info see: http://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-handler.html#nodejs-prog-model-handler-callback

Building a Lambda Function

To get started creating a new Lambda function, visit the Lambda dashboard at:
https://console.aws.amazon.com/lambda/home?region=us-east-1#/functions?display=list

It should look similiar to this:
Screenshot of the AWS Lambda dashboard

Click the blue button Create a Lambda function to get started.

Select a blueprint

The next screen should prompt you to Select a blueprint and present a list of filter-able blueprints. Click the Blank Function option, it should be first in the list of blueprints. This page can used for future reference to check other utilities of Lambda.

Configure trigger

The following screen is Configure Triggers and it should look like:

Screenshot of the Configure triggers screen

Click Next to bypass this screen. We will assign a trigger later, after we’ve set up our function.

Configure function

Screenshot of the fields on the Configure function screen

In this section, we’ll give our Lambda function a name. I’ll use the name GithubGet. Optionally, you can also give a description to the function.

Specifying Lambda function code

By default, the Lambda UI is set to Edit code inline. You should be presented with an inline editor that has a sample function that looks like this:

Screenshot of the online function editor

The inline editor requires little overhead to get lambda code up and running, but for this tutorial we’re going to do something more advanced.

Creating complex functions with dependencies

In most real-world scenarios you’re going to want to create more complex functions, using 3rd-party libraries installed via npm.

Let’s create a custom function that uses an npm dependency and upload it to Lambda. You can follow the steps below, or feel free grab the code from the example repository.

Creating a new function

Let’s setup a folder for our new function and, from within, initialize npm with a default package.json file:

npm init -f

Next we’ll install the GitHub client:

npm install github

Create the file index.js with the following code:

var GitHubApi = require('github');
var github = new GitHubApi();

exports.handler = (event, context, callback) => {

  github.search.repos({
    q: 'sitepoint',
    sort: 'stars'
  }, function(err, res){
    if(err){
      callback(err);
    }

    var results = res.items.map((repo) => {
      return {
        url: repo.html_url,
        stars: repo.stargazers_count
      };
    });

    callback(null, {
      statusCode: 200,
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify(results)
    });
  });

};

Here is a breakdown of what this code is doing:

  1. Including and initializing the GitHub API.
  2. Defining a function handler that matches the Lambda signature.
  3. When handler is called, it makes a search request to GitHub for all repos that match ‘sitepoint’ as a query.
  4. With the response form Github, a map is created that includes url and star count for each repo.
  5. Finally, invoking the lambda callback with a HTTP response like object that, matches API Gateway’s expected integration.

Upload to AWS Lambda

Create a zip file of the function’s files using whatever zip utility you are familiar with. I used the command line zip utility on OSX, like this:

zip -r lambdaupload.zip ./index.js ./node_modules/

To upload code to Lambda, change the Code entry type option from Edit code inline to Upload a zip file, and upload your code using the form.

Animated GIF showing the upload to Lambda process

Configure a function handler and role

Under this section we want to set a few values. Handler is the reference to your Lambda function in the uploaded JavaScript file. By default this is set to index.handler. This maps to our uploaded code by looking for a file index (which in our case is index.js) and looks for an export statement for the handler function, which corresponds to our export.handler.

Fill in the ‘Role’ fields with the following info to create a basic role for our Lambda function:

  • Role: Create new role from template(s)
  • Role name: LambdaGetRole
  • Policy templates: Simple Microservice permissions

Screenshot of the settings mentioned above

Once that section is complete, click the blue “Next” button to continue. You should see a ‘Review’ screen summarizing the configuration:

Review screen summarizing the chosen options

If everything looks OK, click the Create function button to continue.

Assigning a trigger to our new function

Now that our function has been created and initialized, we need a way to invoke it. It’s time to assign a trigger to the function. For our trigger we will be using API Gateway.

API Gateway is another AWS service that automatically creates HTTP endpoints that you can configure responses from. We will be attaching our Lambda function as an API Gateway response.

  1. Click the ‘Triggers’
  2. Click ‘Add trigger’
  3. Click the empty square next to lambda
  4. Select “API Gateway”
  5. Under Security select “Open”
  6. Click Submit

These steps are shown in the following GIF:

Animated walk-through of the steps to add an AWS Lambda function trigger

After the trigger is successfully added, you should see it attached to your function under the Triggers tab.

Triggers tab, showing the newly created API Gateway trigger

There will be a URL listed under the API Gateway ID. You can visit this URL in your browser and you should see a JSON response similar to:

[{"url":"https://github.com/bodrovis/Sitepoint-source","stars":106},
{"url":"https://github.com/Azzurrio/moviestore","stars":80},
{"url":"https://github.com/bodrovis/SitepointMiniChat","stars":54},
{"url":"https://github.com/upchuk/d8-demo-modules","stars":34},

Congratulations! You’ve successfully deployed and triggered code on Lambda.

Next Steps and the Future of Lambda

Hopefully this project gave you a good foundation for working with AWS Lambda. Although we used a 3rd party client (GitHub) integration within our function code, this can be replaced by any other client API or a database client connection.

Serverless frameworks

The process of setting up Lambda that has been demonstrated in this article is very manual and somewhat ephemeral, but there are other ways to configure and initialize Lambda driven by the AWS API.

Currently, there are some frameworks being built on top of the AWS API that help streamline this process.

  • https://serverless.com/

    Serverless is the most robust serverless framework at the moment. However, at the writing of this article, the framework is undergoing a major version upgrade from 0.5 to an official 1.0 release. Unfortunately, this upgrade is not backwards compatible. The majority of popular plugins from 0.5 are currently broken awaiting completion of the 1.0 release.

    Serverless, with its working plugins, promises to deliver a very comprehensive Lambda experience. It provides local development environment setup for rapid iteration, automated Lambda code deployment, multiple deployment staging environments, and much more.

  • https://open-lambda.org/

    OpenLambda attempts to simulate a Lambda environment by providing a local development experience. It also offers tools to make it easier to deploy Lambda code, allowing for rapid iteration. This remedies one of the cons of Lambda listed above.

What do you think?

Will you consider FaaS for future projects? Perhaps you’ve already started using Lambda as part of your application architecture? Either way, I’d love to hear what you think. Please share your thoughts in the comments below!

  • TerryGauchat

    Is there any chance that AWS Lambda will support triggers from other subscribable sources; Firebase, for example? Or is there a Google Cloud computing equivalent to Lambda (Firebase is a Google company.)?

  • rKurtz

    Upon running the api gateway url I receive the message:

    {
    message: “Missing Authentication Token”
    }

Recommended
Sponsors
Get the latest in JavaScript, once a week, for free.