Caching design

Similar Posts:

Name Summary
Locality Of Reference Principle Recently requested data is likely to be requested again
Cache invalidation Declare cached content as invalid, then take actions(purge/refresh/ban/etc)

Q: What are common types of database caches?
Link: DB caching with redis

  1. Database-integrated caches. e.g Amazon Aurora offers an integrated cache that is managed within database engine and has built-in write-through capabilities. link
  2. Local caches: cache within your application
  3. Remote caches: separate instance(s) dedicated for storing the cached data in-memory. e.g, redis, memcached.
  4. Client side cache
  5. CDN

Q: Cache Invalidation: when to update the cache
Link: cache

If the data is modified in the database, it should be invalidated in the cache; if not, this can cause inconsistent application behavior.

Num Name Summary
1 Write-through Two sync writes: db and cache
2 Write-back Sync write to cache, and async write to db
3 Cache-aside*/*Write-around Lazy loading. If cache hit miss, load from db
4 Refresh-ahead When cache TTL hits, load from DB actively, instead of triggered by visits

Q: Cache validation

Name Summary
Control freshness of cached data by TTL Once the set time has passed, the key is deleted from the cache
Add some time jitter to your TTLs Avoid lots of hot data expire at the same time.
Set lower TTLs for dynamic data; longer for static ones  

Q: How cache evictions are supported in cache service?

General Cache eviction policies

Eviction Policy Description
LRU (Least recently used)  
LFU (Least frequently used)  
RR (Random Replacement)  
FIFO (First in first out)  
LIFO (Last in first out)  
MRU (Most recently used)  

Link: Using Redis as an LRU cache:

Eviction Policy Description
allkeys-lru The cache evicts the least recently used (LRU) keys regardless of TTL set.
allkeys-lfu The cache evicts the least frequently used (LFU) keys regardless of TTL set.
volatile-lru The cache evicts the least recently used (LRU) keys from those that have a TTL set.
volatile-lfu The cache evicts the least frequently used (LFU) keys from those that have a TTL set.
volatile-ttl The cache evicts the keys with the shortest TTL set.
volatile-random The cache randomly evicts keys with a TTL set.
allkeys-random The cache randomly evicts keys regardless of TTL set.
no-eviction The cache doesn’t evict keys at all. This blocks future writes until memory frees up

In general as a rule of thumb:

  • Use the allkeys-lru policy when you expect a power-law distribution in the popularity of your requests, that is, you expect that a subset of elements will be accessed far more often than the rest. This is a good pick if you are unsure.
  • Use the allkeys-random if you have a cyclic access where all the keys are scanned continuously, or when you expect the distribution to be uniform (all elements likely accessed with the same probability).
  • Use the volatile-ttl if you want to be able to provide hints to Redis about what are good candidate for expiration by using different TTL values when you create your cache objects.

Generally, least recently used (LRU)-based policies are more common for basic caching use cases. Also, if you are experiencing evictions with your cluster, it is usually a sign that you should scale up or scale out to accommodate the additional data.

Q: Pros and cons of each cache eviction policy. And describe the typical use case for each.


Q: How to support distributed caching service?


  • Redis Sentinel provides high availability for Redis. (link)