AWS Serverless: Caching

AWS Serverless: Caching

Simon Høiberg's photo
Simon Høiberg
·

8 min read

Why is caching important?

The reasons to implement a proper caching strategy in your applications are many, but two central aspects are

  • User Experience
  • Cost

Let's break them down.

User Experience

A 2017 study revealed that loaders and spinners on web applications induce stress proportional to the amount induced when watching horror movies.
Making sure that data is available fast is absolutely crucial for delivering a good user experience!

If we look at how AWS defines the term 'caching', they're stating that:
"a cache is a high-speed data storage layer which stores a subset of data, typically transient in nature so that future requests for that data are served up faster than is possible by accessing the data’s primary storage location".

So in essence, by using a proper caching strategy, we can spare full round trips to the primary data-source and have data served to the end-user faster, ultimately delivering better user experience.

Cost

Since we established that we can spare full round trips, we can also bring down the cost of our application.
Cost can be measured on various units: it can be computational resources, it can be with respect to rate limits of third party APIs, and finally, it can be the cost of using hardware in terms of economy.

Caching in Serverless Applications

Building applications on a serverless stack can offer great benefits in terms of simplicity, scalability, and freedom from managing complicated infrastructure.
However, since we have no 'server' running, it may be harder to identify and plan for good caching strategies.

In this article, we will dig deeper into the universe of Amazon Web Services (AWS), and see what they have to offer in terms of caching in serverless applications.

First of all, let's see which components are typically included in a serverless application on AWS

This give us the following options:

  • Client (User)
  • CloudFront
  • API Gateway
  • Lambda
  • DynamoDB

Let's go through them one by one, and compare the benefits.

Client

In general, we want to cache as 'close to the user' as possible.
So the first thing we want to consider is how to cache data on the client.

For web browsers, we have various options:

  • Cookies
  • localStorage
  • sesionsStorage
  • IndexedDB
  • Cache API
  • Service Workers

You can read more in-depth about these options on MDN's article about client-side storage.

Another option for caching in the client is saving data in-memory using memoization techniques.
Popular front-end frameworks such as React has great inbuilt options for memoizing data.

CloudFront

So, back to the back-end.
If you investigate the diagram above, we see, that when fetching static resources such as images and files, the first place we hit is CloudFront.

CloudFront is what we refer to as an 'edge', and has some built-in caching capabilities.
It is 'close to the user', it offers great flexibility, and we don't have to write any additional code to get it working.

By using AWS CLI or the AWS's web interface, we can easily customize the behavior of how CloudFront is caching.

For instance, we can cache based on query parameters, e.g.
http://cdn.my-domain.com/image.jpg?color=red;size=large

We can also cache content based on request headers and cookies.

You can read more in-depth about CloudFront's caching capabilities here on AWS official docs.

API Gateway

The next place we need to look into is API Gateway.
Some of the limitations of CloudFront is that it only allows caching on GET, HEAD and OPTIONS requests.

With API Gateway we are offered more extensive caching capabilities which also includes POST, PUT, and PATCH.
Typically, we use API Gateway in front of Lambdas (see diagram above).
That also means that, besides serving data faster to the user, we can also save execution time on our Lambdas, which can be a great way to reduce cost.

Additionally, API Gateway offers more control over what to cache.
For instance, if you use an endpoint which accepts multiple paths or query parameters, e.g. http://my-endpoint/v1/{profile}/{user}/?userId={userId}&profileId={profileId}

You can control which parts to include in the cache key.

An important note is, that API Gateway is using in-memory caching, which means that you will be paying for the uptime of the Memcached node that API Gateway is using behind the scenes.
It's important to compare this uptime-pay with Lambda's pay-per-use, in order to be efficient about cost.

Lambda

Using Lambda to cache is generally overlooked when we talk about well-architected serverless frameworks.
But here it goes: you can use Lambda's internal memory as a caching mechanism.

A Lambda container is kept 'alive' after an invocation has finished and will stay idle for some time, waiting for the next invocation.
The data that is loaded in the container's memory will remain there and be available for the next invocation.
This can be used as an in-memory caching mechanism.

In its simplicity, everything we declare outside of the handler is reused on the next invocation.

Let's take a look at an example:

// Let this be our "cache"
const userData = {};

const getUserDataFromID = async (id) => {
  // If user data already exist in cache, return it
  if (userData[id]) {
    return userData[id];
  }

  // Otherwise, reach into DynamoDB or similar expensive task
  const newData await new Promise((res) => {
    dbb.getItem({ id: { S: id } }, (err, data) => res(data.Item));
  });

  // Store for later
  userData[id] = newData;
  return newData;
}

module.exports.handler = async (event) => {
  return {
    statusCode: 200,
    body: await getUserDataFromID(event.id),
    headers: {
      'content-type': 'application/json'
    }
  }
}

This is, of course, a very basic and oversimplified example of how to use Lamdas internal memory as a caching mechanism.
The good thing about using this approach is that it's down to code.
We can implement any business-logic to invalidate/expire our cached data, as we see fit for our specific domain and use case.

There are of course some pitfalls to using this method as well.
First of all, we want to take care of memory management.
Our lambdas have a specified limit of memory to use, and we need to take into account that the primary business logic in itself needs memory to work.
What is left, is what we have left for caching (roughly).

Another aspect we want to consider is that there are no guarantees for this cache.
After a while of idling, the container of our Lambda will be closed down, and all memory will be wiped.
When multiple invocations of our Lambda happens concurrently, AWS will spin up a container for each, and memory will not be shared across these containers.

The cache miss can be pretty high. So caching in Lambdas can be a good and cheap solution to "take the top off" when making requests to DynamoDB or external APIs.
But it should be used when some degree of cache miss is accepted.

ElastiCache

To solve the issue with cache miss, we can use AWS's ElastiCache.
ElastiCache allows caching and sharing data across many Lamdas. It's also a great option for caching more complex data structures where searching and querying is a recurrent requirement.

The downside to ElastiCache is that we will need to pay for the uptime of an ElastiCache cluster.

DynamoDB

Finally, we can use DynamoDB to cache data. For this, we use DAX (DynamoDB Accelerator).

DAX is a really good option if we're already using DynamoDB.
It's a data-layer that lies on top of our DynamoDB, and it's fully managed by AWS, so it requires minimal changes to our implementation to get set up.

The downside here is the lack of control - querying through DAX can end up serving stale data (if items in DynamoDB has been updated recently), and it's not easy to manually 'invalidate on the fly'.
DAX is a good option if the data from DynamoDB is not changing very frequently.

Another downside is that we will be paying for the uptime of the cache node that lies on top of DynamoDB.

Conclussion

Building applications on a serverless stack can offer great benefits in terms of simplicity, scalability, and freedom from managing complicated infrastructure.
However, caching is still an important part of a well-architected application, and must be carefully considered in order to optimize performance, user experience, and cost.

We have looked into 5 different ways of introducing caching in AWS.
Of course, the options are not limited to only these five, nor do you have to pick one over the other. As part of a well-designed architecture, a combination of these is prone to be used.