Scaling up containers with Dynamic Local Cache
Problem Statement: There is a local cache in service and it's dynamic, it changes whenever there is an API request from Platform. We need to scale this service up to 6 containers without moving to a centralized cache (e.g. Redis).
The challenge in scaling was because when there are multiple containers and there is an API request for the service, the request will be routed to any of those 6 containers, so only that particular container refreshes the cache while all others are left with stale data.
As depicted in the diagram when a request from refresh was made only one of the six containers was able to re-build the local cache, all others are still having an out-dated cache.
We can solve this problem by using a messaging queue like Kafka, AWS Kinesis, etc. In our case, we used AWS Kinesis to build a simple solution. Idea was to create a Kinesis Stream with a single shard where all the 6 containers can produce to and consume from.
Each time there is an API request to refresh the cache doesn't matter which one of the containers receives the request but a message is produced to Stream asking for the container to refresh the local cache.
Write a Kinesis client for producing and consuming the refresh message. The above snippet shows the simple code for producing a message as a flag to refresh the cache for the containers. In a similar fashion, we can write the code for consumers.
Now, whenever the request comes and we produce the message to the Kinesis Stream that message will be consumed by all the container (including the one which produced the message), and soon as the message is received the individual containers will refresh their local cache.
That’s a good solution but is it scalable ? Say we have to scale the Service to 60 or 600 Containers, does this solution still hold ? Yes. it does, but, what about the cost ? Any Engineer who has used AWS will know how costly services like Kinesis gets. So, lets look at alternate solution for this problem.
A good solution will be using Redis Keyspace Notifications. Keyspace notifications allow clients to subscribe to Pub/Sub channels in order to receive events affecting the Redis data set in some way. Events are delivered using the normal Pub/Sub layer of Redis, so clients implementing Pub/Sub are able to use this feature without modifications.
Have a simple entry onto Redis dictionary, e.g. “cache-refreshed”. As per current use case whenever there is an event received for refreshing the request is made on the Redis cache to delete the value. As soon the record is deleted from the cache all the containers subscribed to the event receive a Notification. One sample implementation for similar use case can be found here.
You have a better approach ? Feel free to drop a note in comments.
