erl-cache

A generic in-node caching library with memoization support.

This is a fork of spilgames/erl-cache for Alert Logic.

Purpose

This application is meant to facilitate the process of caching function calls within an erlang node. Do not expect a complex distributed application here. There are far more complex products out there intended to act that way. Instead, erl_cache intends to be a simple solution that unifies common caching patterns within an erlang node which you probably have implemented a thousand times in one thousand slightly different ways. It's also a nice library if you want to do memoization in Erlang.

Functional Description

The erl_cache module acts as the main application interface, allowing you to start/stop independent cache servers and interact with them.

Each entry in a cache can be either valid, overdue or evict. The difference between overdue and evict is that, when a overdue entry is requested, and in case a refresh callback was indicated when first setting it, it will be refreshed and so back into valid state. Evict entries will be removed from cache when hit and never returned to the client.

From a user point of view, those independent cache servers provide independent namespacing. Each cache server uses its own set of default options and can crash without affecting any of the others.

From a system point of view, erl_cache acts as a server of caches, holding the cache names and their associated defaults in a protected ets table. erl_cache is also responsible for option validation in every call.

erl_cache_server holds the actual cache in its own protected ets table and implements the logic behind the refreshing overdue entries when hit and the eviction of old entries

Status of an entry

Every cached entry can be in one of the following three states:

• Valid: any request of that key will result in a hit and the stored value will be returned. An entry is in the valid state as long as indicated by the validity option.
• Overdue: the entry validity period has passed but validity + evict period hasn't. Any request for a key in such state will be considered a overdue hit and never a miss. In case a refresh_callback was specified for the entry when set, the value will be refreshed and set back to valid state. Depending on whether the wait_for_refresh, the client will be served the old value or the refreshed one.
• Evict: the entry is no longer valid and won't be refreshed. An entry in evict state will never be returned to the user. Eventually, entries in evict state are removed from the cache.

Caching Error Values

When caching a function call without checking the return value, or when using the ?CACHE macro, the user might transparently be caching an error result instead of a valid one. When dealing with long validity times, this could lead to errors in your cached call to consistently propagate to the upper layer for a long time.

Not caching error values at all is also a questionable practice. Think of an overload situation when your cached call can't handle the load and returns an error. By not caching that error at all, the already overloaded application is being hit by more traffic and the overload situation is getting worse.

Since there is no silver bullet, but seeming clear that error values are to be dealt with using special care, erl_cache provides is_error_callback and error_validity. By default this application considers the atom error and tuples {error, _} to be errors. The default validity for such values is a fifth of the default validity for a normal value.

Since refreshing an error will produce a rather unpredictable result, error entries never enter the overdue state: either they are valid or to be evicted.

Also, when refreshing a valid overdue entry produces an error, that error will not be used to refresh the entry. Instead, the error will be logged and the automatic refresh will be disabled for that entry. The cached value will remain in overdue state without being refreshed until it expires normally.

Eviction of expired entries

It's important to know that this application works with a lazy eviction strategy. This means entries are marked to be evicted but still kept in the cache. Once an entry is marked to be evicted, from a user perspective it's as if the entry does not exist, since it's impossible to retrieve it or refresh it without explicitly setting it again.

Once every evict_interval, the cache is scanned for entries to be evicted and those are erased from the cache. Only at that point the evict stats are updated and the memory used by such entries is freed.

Configuration

This application accepts only one configuration option: cache_servers. This is a list of 2-tuples indicating the name and the default options for each of the caches the application should bring up at startup. The format is the same used in erl_cache:start_cache/2. i.e.

[{erl_cache, [
    {cache_servers, [{my_cache_server, [{wait_until_done, true}, {validity, 5000}, {evict, 3000}]}]}
]}].

The default config options unless otherwise specified are:

• wait_for_refresh: true
• wait_until_done: false
• validity: 300000
• error_validity: 60000
• evict: 60000
• evict_interval: ServerLevelEvict + ServerLevelValidity
• max_cache_size: undefined
• mem_check_interval: 10000
• refresh_callback: undefined
• is_error_callback:

  fun (error) -> true; ({error, _}) -> true; (_) -> false end

The evict_interval option controls how often entries to be evicted will be deleted from the cache. There is only one global evict_interval per cache_server and specific validity and evict values passed to set operations or to the ?CACHE macro will not affect it. The evict_interval, max_cache_size and mem_check_interval options will be ignored in all function calls except for erl_cache:start_server/2 and erl_cache:set_cache_defaults/2.

Configuration options for a cache server can be overwriten at runtime by using erl_cache:set_cache_defaults/2.

The max_cache_size (in MB) and max_check_interval options provide a very simple mechanism to inform and react on caches consuming too much memory. In short, the mechanism scans periodically the amount of memory consumed by the cache ets and if it goes over a limit prints a warning and forces an eviction cycle. Keep in mind this is far from accurate and can work particularly badly when caching large binaries, since the memory consumed by these is not reported by ets.

The key_generation option allows the specification of a controlled key format when using the macro. The {key_generation, Module} can be used in the macro usage or in the defaults for the server. Module must implement the erl_cache_key_generator behaviour.

The ?CACHE macro

For ease of use, this application provides the ?CACHE macro. This macro can be placed on top of any public function. Every time the function is invoked, erl_cache will try to retrieve the associated return value from cache and, in case it's not there, perform the regular function call and cache the result. Here you can see an example of how to use the macro to avoid sums being performed everytime sum/2 is called:

?CACHE(my_cache_namespace, [{validity, 10000}, {evict, 2000}}]).
sum(A, B) ->
    A + B.

Important Notes

This application has been designed for in node caching of small datasets. Keep in mind the memory limiting mechanism provided by max_cache_size is not strict in any way; it just acts as a helper to aliviate the pain if possible by forcing an extra purge of the cached entries when running out of the configured memory for the cache ets.

The memory control mechanism is not accurate and doesn't deal properly with cached large binaries, since the cache ets will only store references to them and it's not possible to figure the exact size of the referenced binaries.

Benchmarking

This application has a ready benchmarking suite. It is based on basho bench. To run the benchmark:

$ make benchmark

In order to render the results (gnuplot is needed):

$ ./deps/basho_bench/priv/gp_throughput.sh

For full usage guide please consult basho bench documentation.