Caching

Akka HTTP’s caching support provides a lightweight and fast in-memory caching functionality based on futures. The primary use-case is the “wrapping” of an expensive operation with a caching layer that, based on a certain key of type K, runs the wrapped operation only once and returns the the cached value for all future accesses for the same key (as long as the respective entry has not expired).

Akka HTTP comes with one implementations of the Cache API built on Caffeine featuring frequency-biased cache eviction semantics with support for time-based entry expiration.

To enable caching support add a library dependency onto:

sbt
libraryDependencies += "com.typesafe.akka" %% "akka-http-caching" % "10.0.11"
Gradle
dependencies {
  compile group: 'com.typesafe.akka', name: 'akka-http-caching_2.12', version: '10.0.11'
}
Maven
<dependency>
  <groupId>com.typesafe.akka</groupId>
  <artifactId>akka-http-caching_2.12</artifactId>
  <version>10.0.11</version>
</dependency>

Basic design

The central idea of the cache API is to not store the actual values of type T themselves in the cache but rather the corresponding futures, i.e. instances of type CompletableFuture<T>Future[T]. This approach has the advantage of taking care of the thundering herds problem where many requests to a particular cache key (e.g. a resource URI) arrive before the first one could be completed. Normally (without special guarding techniques, like so-called “cowboy” entries) this can cause many requests to compete for system resources while trying to compute the same result thereby greatly reducing overall system performance. When you use an Akka HTTP cache the very first request that arrives for a certain cache key causes a future to be put into the cache which all later requests then “hook into”. As soon as the first request completes all other ones complete as well. This minimizes processing time and server load for all requests.

All Akka HTTP cache implementations adheres to the Cache interfaceCache class, which allows you to interact with the cache.

Along with the cache API, the routing DSL provides several caching directives to use caching in your routes.

Frequency-biased LFU cache

The frequency-biased LFU cache implementation has a defined maximum number of entries it can store. After the maximum capacity is reached the cache will evicts entries that are less likely to be used again. For example, the cache may evict an entry because it hasn’t been used recently or very often.

Time-based entry expiration is enabled when a time-to-live and/or time-to-idle expirations are set to a finite duration. The former provides an upper limit to the time period an entry is allowed to remain in the cache while the latter limits the maximum time an entry is kept without having been accessed, ie. either read or updated. If both values are finite the time-to-live has to be strictly greater than the time-to-idle.

Note

Expired entries are only evicted upon next access (or by being thrown out by the capacity constraint), so they might prevent garbage collection of their values for longer than expected.

For simple cases, configure the capacity and expiration settings in your application.conf file via the settings under akka.http.caching and use LfuCache.create()LfuCache.apply() to create the cache. For more advanced usage you can create an LfuCacheLfuCache with settings specialized for your use case:

Java
import akka.http.caching.javadsl.Cache;
import akka.http.caching.javadsl.CachingSettings;
import akka.http.caching.javadsl.LfuCacheSettings;
import akka.http.caching.LfuCache;

final CachingSettings defaultCachingSettings = CachingSettings.create(system());
final LfuCacheSettings lfuCacheSettings = defaultCachingSettings.lfuCacheSettings()
  .withInitialCapacity(25)
  .withMaxCapacity(50)
  .withTimeToLive(Duration.create(20, TimeUnit.SECONDS))
  .withTimeToIdle(Duration.create(10, TimeUnit.SECONDS));
final CachingSettings cachingSettings = defaultCachingSettings.withLfuCacheSettings(lfuCacheSettings);
final Cache<Uri, RouteResult> lfuCache = LfuCache.create(cachingSettings);
final Route route = cache(lfuCache, keyerFunction, () -> innerRoute);
Scala
import akka.http.caching.scaladsl.Cache
import akka.http.caching.scaladsl.CachingSettings
import akka.http.caching.scaladsl.LfuCacheSettings
import akka.http.caching.LfuCache

val defaultCachingSettings = CachingSettings(system)
val lfuCacheSettings =
  defaultCachingSettings.lfuCacheSettings
    .withInitialCapacity(25)
    .withMaxCapacity(50)
    .withTimeToLive(20.seconds)
    .withTimeToIdle(10.seconds)
val cachingSettings = defaultCachingSettings.withLfuCacheSettings(lfuCacheSettings)
val lfuCache: Cache[Uri, RouteResult] = LfuCache(cachingSettings)
val route = cache(lfuCache, keyerFunction)(innerRoute)
The source code for this page can be found here.