Distributed Cache On Steroids: Amazon ElastiCache

Web applications can often be made to perform better and run faster by caching critical pieces of data in memory.  Frequently accessed data, results of time-consuming/expensive database queries, search results, session data and results of complex calculations are usually very good candidates for cache storage. We will understand basics of caching post which we will deep dive in Amazon ElastiCache

Types of Caching Architectures

Local In-Memory Caching

The Cache objects are stored locally in-memory (example Java Heap) of the application servers. Open source frameworks like EHCache, OSCache can be used inside Java Application Servers for serving cached items from the heap. Since the cache items are accessed from the same process heap of the application programs, this model offers very fast “Set/Get” operations for cache items. On the other hand because of the limited nature of RAM only few GB can be cached in the Local In-Memory Caching model.

Network Attached Caching

The Cache objects are stored in a separate Cache Server accessible over TCP network. Cache Clients or Cache Drivers need to be embedded on the Application Servers which enable the GET/PUT/Sync operations. Cache behaves like a network attached RAM. Terracotta Cache is an example of network attached RAM caching. Since the cache operations need to travel over the network, few milliseconds of latency can be felt on “Set/Get” operations. Since the cache is centralized and consolidated it is easy manage, extend and maintain this tier.

Distributed Caching

Distributed caching may run on multiple cache servers so that it can grow in size and in transactional capacity. It is mainly used to store application data residing in database and web session data. The web applications may access the distributed cache deployed locally or remotely using a client library. The most popular distributed caching software is Memcached. Deploying large farms of Memcached has become possible because memory is affordable and cheap now and networks have become very fast.  Distributed cache works well on low cost commodity machines and cloud providers. One such popular distributed cache on AWS we are going to discuss in this article is Amazon ElastiCache.

Introducing Amazon ElastiCache:

Amazon ElastiCache is a managed distributed caching service provided by Amazon Web services.  Amazon ElastiCache currently uses memcached as the caching engine, so memcached compatible programs can be easily ported to Amazon ElastiCache usually without code change. With Amazon ElastiCache operating as a separate tier, AWS team offloads typical cache tier management tasks like the ones listed below from the application and infrastructure teams of the customer:

• Managing the work involved in setting up a distributed in-memory cache tier
• Provisioning the server resources you request to installing the caching software
• Common administrative tasks such as failure detection, recovery and software patching
• Adding / removing Cache nodes from the cluster

Now lets us explore in detail the major components of Amazon ElastiCache.

Components of Amazon ElastiCache

Amazon ElastiCache node:

Amazon ElastiCache node is a memcached instance with a unique endpoint URL and port. You can configure the memcached configuration file used by the popular memcached clients with this endpoint URL(s) provided. Memcached clients are available for all popular programming languages and the latest list can be found here. Amazon ElastiCache nodes are protocol compliant with memcached server and usually the same memcached client mentioned above can be switched to Amazon ElastiCache nodes without requiring code changes. You can use standard Memcached operations like get, set, incr and decr in exactly the same way as you would in your existing Memcached deployments and as well perform requests using both Binary and Text messages over TCP protocol. A sample pseudo code illustrating SET operation in Text TCP protocol using Java Spymemcached library and Amazon ElastiCache Tier:

import java.io.IOException;
import java.util.concurrent.ExecutionException;
import net.spy.memcached.AddrUtil;
import net.spy.memcached.DefaultConnectionFactory;
import net.spy.memcached.MemcachedClient;
import net.spy.memcached.*;
import java.util.*;
public class ElastiCacheTextTCPSet {
 public void setTextValue() throws IOException {
 MemcachedClient memcachedClient = new MemcachedClient( new DefaultConnectionFactory(),AddrUtil.getAddresses("ecache1a.sqjbuo.0001.use1.cache.amazonaws.com:11211"));
 String key = "1000002";
 String value = "1000002-Test value in Text";
 Integer expires = Integer.parseInt("1000");
 try {
 Future<Boolean> result = memcachedClient.set(key, expires, value);
…………
…………
 } catch (InterruptedException e) {
 e.printStackTrace();
 } catch (ExecutionException e) {
 e.printStackTrace();
 }
 memcachedClient.shutdown();
 }

}

A sample program illustrating GET operation in Text TCP protocol using Java Spymemcached library and Amazon ElastiCache Tier:

import java.io.IOException;
import net.spy.memcached.AddrUtil;
import net.spy.memcached.ConnectionFactory;
import net.spy.memcached.DefaultConnectionFactory;
import net.spy.memcached.HashAlgorithm;
import net.spy.memcached.MemcachedClient;
import java.util.*;

public class ElastiCacheTextTCPGet {
 public void getTextValue() throws Exception {
 MemcachedClient memcachedClient = new MemcachedClient( new DefaultConnectionFactory(),AddrUtil.getAddresses("ecache1a.sqjbuo.0001.use1.cache.amazonaws.com:11211"));
 String key = "1000002";
 Object obj=null;
 try {
 obj = memcachedClient.get(key);
 System.out.println("Value = :"+obj.toString());
 } catch (Exception e) {
 e.printStackTrace();
 }
 memcachedClient.shutdown();
 }

}

You can also communicate with Amazon ElastiCache using Binary protocol of Memcached. A sample program illustrating SET operation using Java Spymemcached-TCP Binary Protocol and Amazon ElastiCache Tier:

import java.io.IOException;
import java.util.concurrent.ExecutionException;
import net.spy.memcached.AddrUtil;
import net.spy.memcached.DefaultConnectionFactory;
import net.spy.memcached.BinaryConnectionFactory;
import net.spy.memcached.MemcachedClient;
import net.spy.memcached.*;
import java.util.*;
public class ElastiCacheBinaryTCPSet {
 public void setBinaryValue() throws Exception {
 MemcachedClient memcachedClient = new MemcachedClient( new BinaryConnectionFactory(),AddrUtil.getAddresses("ecache1a.sqjbuo.0001.use1.cache.amazonaws.com:11211"));
 String key = "1000004";
 String value = "1000004-Test value in Binary-TCP protocol";
 Integer expires = Integer.parseInt("1000");
 try {
 Future<Boolean> result = memcachedClient.set(key, expires, value);
 } catch (InterruptedException e) {
 e.printStackTrace();
 } catch (ExecutionException e) {
 e.printStackTrace();
 }
 memcachedClient.shutdown();
 }
}

Memcached software version 1.4.5 is used as the cache engine in the Amazon ElastiCache nodes. Cache nodes accept connections on port 11211 and only for Amazon EC2 network. In order to allow network requests to your Cache Nodes from web/application EC2 instances, you will need to authorize access in AWS security groups. Amazon ElastiCache provides variety of cache node types (capacities) and pricing models to suit the customer requirements. The latest cache node type information can be found here

Amazon ElastiCache Cluster:

An Amazon ElastiCache Cluster is a collection of one or more Amazon ElastiCache Nodes. All Cache Nodes within a Cache Cluster will be of the same Node Type (capacity). A node type is an associated memory size of the cache nodes to be deployed in your cache cluster. Currently cache nodes come in 11 different capacities in AWS to suit a variety of your needs, new capacity types will be added by AWS in future. If you need cache nodes of different capacities in your application systems you need to create multiple cache clusters in your account. The Cache Cluster should be supplied with an Identifier name by the customer during its creation process. For example: if you give a name “ecache1a” for the cache cluster, the endpoint URL of a particular cache node may look like this ecache1a.sqjbuo.0001.use1.cache.amazonaws.com:11211.This name identifies a particular Cache Cluster when interacting with the Amazon ElastiCache API and commands. The Cache Cluster Identifier must be unique for that customer in an AWS region. While configuring a cluster you need to mention the availability zone in in which you would prefer to deploy your Cache Cluster. Currently a single Amazon ElastiCache Cluster cannot span multiple Availability zones inside an Amazon EC2 region. It is advised to keep the cache cluster and related web/application ec2 instances in the same Availability zone for better latency. Refer here for common Amazon ElastiCache deployment architectures in AWS infrastructure.  Parameters and Security group settings of all the nodes inside the Amazon ElastiCache cluster can be grouped and controlled at cluster level. This clustering/grouping of cache nodes helps in easy management and maintenance of huge farms of cache nodes.  Amazon ElastiCache Clusters are can be created, using the AWS Management Console, Amazon ElastiCache APIs, or Command Line Tools. In addition to the above you can specify whether you would like to receive SNS notifications related to your cache nodes/cluster using Amazon Simple Notification Service (SNS) system. You can either select an existing SNS topic or disable notifications completely as well for the entire Amazon ElastiCache Cluster.

Amazon ElastiCache Parameter Group:

Amazon ElastiCache Parameter Groups allows you to control the runtime parameters and cache engine configuration values of your Nodes. The values configured will get passed to memcached nodes during startup and gets applied to all the nodes associated in the Amazon ElastiCache Cluster. If you do not specify a Cache Parameter Group for your Cache Cluster, then a default Cache Parameter Group (default.memcached1.4) will be used. Usually the default parameter group contains engine defaults and Amazon ElastiCache system defaults optimized for the Cache Cluster nodes you are running. Since Amazon ElastiCache by default chooses the optimal configuration parameters for your Cache Cluster taking into account the Node Type’s memory/compute resource capacity, it does not require change for most use cases. However, if you want your Cache Cluster to run with your custom configuration values, you can simply create a new Cache Parameter Group, modify the desired parameters and use it for new or existing clusters. In event the custom parameter group is applied to running cache cluster the changes will not be applied to the Cache Nodes until the Cache Cluster is rebooted.

Amazon ElastiCache Security Group:

An Amazon ElastiCache Security Group acts like a firewall, controlling network access to your Cache Cluster. Since IP-range based access control is currently not enabled for Amazon ElastiCache Clusters, All clients to a Cache Cluster must be within the EC2 network and authorized via security groups.  If you want your Web application EC2 instances to access your Cache Cluster, you must explicitly enable access from hosts in their specific EC2 security groups. For example: To allow network access to your Cache Cluster, you first need to create a Cache Security Group and link the desired EC2 security groups (which in turn specify the EC2 instances allowed) to it.

What features Amazon ElastiCache offers over MemCached on EC2

Operation 1: Modification of Cache Cluster

An Amazon ElastiCache cluster can be modified anytime.  Properties like Cache Security group, Cache Parameter group, Maintenance window period, SNS notifications and Auto Minor version upgrade can be modified at cluster level applying to all nodes. Security group changes may take a few minutes to be applied to the cluster. Parameter Group changes will take effect only after you reboot your Cache Nodes.  Instance capacity type of a Cache Node cannot be modified at runtime.

Operation 2: Reboot

Using this option you can reboot a single cache node or the entire cache cluster. The cache nodes will not be available during the cluster reboot operation. In case a single cache node alone is rebooted, other cache nodes are available for processing requests. Reboot operation can be done using the AWS console or API. Reboot is usually performed:

• When a cache node(s) is not responsive
• Parameter Group changes applied need to take effect on all the nodes of the cache cluster

In our tests, we found reboot of 5 X m1.large cache node type in Amazon EC2 US-EAST region takes around ~140 seconds. This may vary time to time depending upon the cache node type and Amazon EC2 regions in consideration. Because of the ephemeral nature of the Cache, the cache node begin empty (also called “cold”) after reboot operation, and depending on your workload pattern, it may take some time to be re-populated with data (also called “warming up” phase).

Operation 3: Delete and Removal

You can remove a one or more nodes from a cache cluster or delete the entire cluster. In case only one node is present in the cache cluster, you need to delete the cache cluster itself. Deleting one or more nodes takes ~200 seconds. This may vary time to time depending upon the cache node type and Amazon EC2 regions in consideration. In case a cluster is deleted, you can recreate a cache cluster in the same name at later point of time in the same Amazon EC2 region.

Operation 4: Addition of Cache Nodes

Amazon ElastiCache as the name suggests you can automatically/manually add cache nodes to the existing ElastiCache cluster making the whole tier elastic. One or more cache nodes can be added to the existing cache cluster at a time. In our tests, we found adding 5 X m1.large cache node type in Amazon EC2 US-EAST region takes around ~320 seconds. This may vary time to time depending upon the cache node type and Amazon EC2 regions in consideration. While adding new cache nodes in existing cluster, the cache node type should be the same size as the existing node type. You cannot change that size, in case you need different cache node capacity a new cache cluster needs to be created. With a normal hashing algorithm, increasing the number of memcached cache nodes can cause many keys to be remapped to different cache node resulting in huge set of cache misses. Imagine you have 10 Amazon ElastiCache Nodes in your cache Cluster, adding an eleventh server may cause ~40%+ of your keys to suddenly point to different servers than normal. This activity may cause cache misses and swamp your backend Database with requests. To minimize this remapping process it is recommended to follow consistent Hashing in your cache clients. Consistent Hashing allows for more stable distribution of keys and minimal remapping when new cache nodes are added. Using this algorithm, adding an eleventh server should cause less than 10% of your keys to be reassigned. This % may vary in production but it is far more efficient in such elastic scenarios compared to normal hash algorithms.  It is also advised to keep cache node ordering and number of cache node entries same in all the client configurations while using consistent Hashing algorithm. Java Applications can use “Ketama library” through Spymemcached library to integrate this algorithm into their applications.  Refer URL to know more about consistent Hashing in memcached. A sample Pseudo code illustrating SET operation in TEXT-TCP protocol using Java Spymemcached –Ketama Hashing library and Amazon ElastiCache Tier:

import java.io.IOException;
import java.util.concurrent.ExecutionException;
import net.spy.memcached.AddrUtil;
import net.spy.memcached.DefaultConnectionFactory;
import net.spy.memcached.MemcachedClient;
import net.spy.memcached.*;
import java.util.*;
public class ElastiCacheKetamaTCPSet {
 public void setValue() throws IOException {
 ConnectionFactory connFactory = new DefaultConnectionFactory(
 DefaultConnectionFactory.DEFAULT_OP_QUEUE_LEN,
 DefaultConnectionFactory.DEFAULT_READ_BUFFER_SIZE,
 HashAlgorithm.KETAMA_HASH);
 MemcachedClient memcachedClient = new MemcachedClient(connFactory,AddrUtil.getAddresses("ecache1a.sqjbuo.0001.use1.cache.amazonaws.com:11211"));
 String key = "1000001";
 String value = "1000001-Test value in Consistent Hashing";
 Integer expires = Integer.parseInt("1000");
 try {
 Future<Boolean> result = memcachedClient.set(key, expires, value);
………………..
………………..
 } catch (InterruptedException e) {
 e.printStackTrace();
 } catch (ExecutionException e) {
 e.printStackTrace();
 }
 memcachedClient.shutdown();
 }
}

Frequently Asked Questions about Amazon ElastiCache