Amazon ElastiCache is a fully managed, highly available, scalable, in-memory data store and cache and provides support for two popular open source engines, Redis and Memcached, and is protocol compliant with both. ElastiCache provides an easy migration path for those desiring a fully managed alternative to self-managed in-memory store/cache, often as easy as simply changing an application configuration setting. ElastiCache is a member of the AWS database family of services which also includes RDS, Redshift and DynamoDB.
(A hi-res, PDF version of this diagram is located here)
ElastiCache provides support for both VPC and non-VPC (ala EC2 classic) clusters, For this summary, I’ve only included notes for ElastiCache clusters in VPCs. For details about non-VPC installations, see the documentation.
Both ElastiCache Memcached and Redis are commonly used as fast, in-memory data stores for application session information and as caches for application objects and pages. As a cache, ElastiCaches serves to reduce latencies associated with common or frequently accesses data by keeping the data close to the end users that use it.
As a session store, ElastiCache provides a fast, in-memory repository for shared application state, allowing systems to scale in and out seemlessly without regard for the session affinity of particular systems. This provides additional benefits by also increasing the fault tolerance and availablity of well-architected systems.
In addition to serving as a data store and cache, ElastiCache Redis can also act as a data structure store (hashes, sets, etc) and as a message broker (pub/sub functionality), for advanced application specific functionality. Some use Redis as a primary data store.
Amazon ElastiCache pricing includes both compute and storage costs.
ElastiCace node compute costs are based on hourly pricing, with partial hours are billed as an hour. Unlike EC2 instances, ElastiCache nodes are not billed by the minute or second. Currently, the smallest Elasticache node, the cache.t2.micro with 0.555 GB of memory, costs $0.017 per hour ($12.44/mo using 732 hour month). The largest node type, the cache.r4.16xlarge with 407 GB RAM is $7.280 per hour ($5328.96/mo). All examples are from US East (N.Virginia).
Similar to EC2 instances, ElastiCache nodes may also be purchased as Reserved nodes. In exchange for a 1 or 3 year commitment and an upfront payment, ElastiCache Reserved nodes offer a significant discount. For example, the 1 year Reserved node cost for the cache.t2.micro includes a $51 up-front payment and $0.006 hourly fee ($8.64/mo). For a 3 year commitment, costs include a $109 up-front payment and $0.004/hr ($5.96/mo).
ElastiCache includes storage one snapshot for free. Additional snapshots may be stored at $0.085/GB/month
There are no data transfer changes associated with ElastiCache directly. However, if an EC2 instance access an ElastiCache node in a different Availability Zone, theree will be data transfer costs for the EC2 instance.
ElastiCache clusters may be scaled vertically by changing instance types. This may be to adjust the amount of RAM available per node or/and to scale network performance. The behavior of the cluster depends on the type.
Scaling a memcached cluster requires cluster replacement and the applicaiton must be updated to use the new cluter endpoint. No data is copied.
Scaling a redus cluster vertically up is automatically handled by ElastiCache. ElastiCache will block all reads & writes, create a new cluster, copy/replicate data, change DNS, enable reads & writes, delete the old cluster. This should (obviously) be done during a maintenance window. Scaling down is entirly manual, you are responsible for all aspects of the scaling effort, including data retention.
Memcached can be scaled horizontally by adding or removing nodes to adjust capacity and/or read/write performance. Up to 20 nodes are allowed per cluster, up 100 nodes are allowed per region, although these may modified upon request to AWS. Client applications must be aware of changes to nodes and must repartition key space. Consistent hashing is the preferred method and supported by many client libraries. AWS provides an Auto Discover client for some languages and platforms.
Write capacity and performance may be modified by adding or removing shards from a Redis cluster. 1 to 5 read-replicas may be added to a shard to improved read capacity and performance (and fault tolerance).
Online cluster resizing ElastiCache Redis (cluster-enabled) clusters may be resized dynamically while still handling traffic (albeit, with some performance degradation). During a cluster resize operation, key slots are remapped equally among cluster nodes (by count) and keys transferred among nodes as required.
ElastiCache will automatically delect and recover failed nodes. Nodes will be repaired or replaced, DNS and IP address will remain the same. SNS notifications can be sent to indicate node recovery has occured. ElastiCache clusters can span multiple Availability Zones (AZ) to reduce the impact of a failed AZ.
ElastiCache Redis supports sharding and read replicas to increase availability. Shards may be added to reduce the impact of a failed shard. In the event a primary nodes fails and is unable to accept writes, a read replica will be promoted to take it’s place (the one with shortest repliciation lag).
You can manually trigger a failover event for a Redis cluster to test the failure of a primary node.
VPC support is provided for ElastiCache, including ability to place clusters in specific subnets and use VPC security groups. Security groups are used to control access and act as firewalls. Access to ElastiCache clusters requires explicit ingress rules.
ElastiCache Redis includes support for encryption in flight (AWS manages all certificates), and encryption at rest. Encryption at rest is used during backup and restore, and when storing backups and snapshots in S3.
- Can’t run in VPC configured for dedicated tenancy
- 15 shards per cluster
- 5 replicas per shard
- 100 nodes per region
- Redis 2.8.22 reserve 50% memory for non-data, otherwise 25%, to account for increased memory usage during backup operations.
Tooling, metrics, automation, migration. Can backup Redis RDB to S3 from on-premise, then restore to RDS
See the ElastiCache product page for additional information.
- Nov 21, 2017: AWS CodeBuild Adds New Capabilities to Support Amazon Virtual Private Cloud Resources and Dependency Caching
- Nov 20, 2017: Amazon ElastiCache Now Supports the R4 Node Family
- Nov 16, 2017: Utilization Alerts for Amazon Redshift, Amazon RDS, and Amazon ElastiCache Reservations
- Nov 15, 2017: Amazon ElastiCache for Redis In-Transit and At-Rest Encryption Now Available in Additional Regions
- Nov 10, 2017: Monitor your Amazon Redshift, Amazon RDS, and Amazon ElastiCache reservations using AWS Cost Explorer’s RI Utilization report
- Nov 9, 2017: Amazon ElastiCache for Redis introduces dynamic addition and removal of shards while continuing to serve workloads