Reference

AbstractCache{K, V} <: AbstractDict{K, V}

Supertype of the containers of GlobalCache and TaskLocalCache. A subtype C implements C{K, V}(; maxsize, by), get!(default, c, key), empty!(c), resize!(c; maxsize) and MemoizationKit.cache_stats(c); see Implementing a cache. Other AbstractDict methods are optional.

LRU and ClockCache are thread-safe dictionaries that convert keys to K.

LRU{K, V}(; maxsize::Integer = 10_000, by = nothing)

Least-recently-used cache, with the recency list stored as integer prev/next vectors instead of heap-allocated nodes. Evicts exactly the least recently used entry.

maxsize limits the number of entries, or, when by is given, the sum of by(value) over all entries.

ClockCache{K, V}(; maxsize::Integer = 10_000, by = nothing)

Cache with CLOCK (second-chance) eviction, an approximation of LRU. A hit only sets the entry's reference bit; on eviction a hand sweeps the slots, clearing set bits and evicting the first entry whose bit is already clear. New entries start with a clear bit, so entries that are never hit again are evicted first.

maxsize limits the number of entries, or, when by is given, the sum of by(value) over all entries.

resize!(c::AbstractCache; maxsize::Integer) -> c

Set the size limit of c, evicting entries until they fit.

cache_stats(c::AbstractCache) -> NamedTuple

(; hits, misses, length, currentsize, maxsize, by) of c, where currentsize is the total size of the entries and by the size measure (nothing when counting entries).

@cached function f(args...; kwargs...) ... end
@cached f(args...; kwargs...) = ...

Memoize a method. The body becomes a method of MemoizationKit.implementation, and f keeps its signature (including default values and keyword arguments) but looks up the result in a cache selected by CacheStyle(f, args...).

The key is chosen by MemoizationKit.cachekey, which defaults to the tuple of positional arguments, followed by a NamedTuple of the keyword arguments if there are any. Custom keys can merge calls with different argument types. Shared caches hold all key and value types of a function together (one cache per container type). The return type annotation, if present, may depend on where parameters; otherwise the value type is inferred, or Any if inference does not give a concrete type. Keys of different types are distinct entries.

Use stable keys and treat cached mutable results as shared objects. Cache entries are not automatically invalidated when a method or external state changes.

Works for any method definition: qualified names (function Base.f(...)), operators and callable objects ((x::Foo)(args...)). Use uncached to bypass the cache.

uncached(f, args...; kwargs...)

Call the @cached function f without consulting or filling any cache. Supply all positional and keyword arguments, including those with defaults in f.

MemoizationKit.implementation(f, args...; kwargs...)

The uncached body of a @cached function f. The macro moves the body of each cached method here, dispatching on typeof(f); call it through uncached.

MemoizationKit.cachekey(f, args...; kwargs...)

Return the key used to memoize a call of a @cached function. By default it is the positional argument tuple, followed by the keyword NamedTuple when nonempty. Specialize this function to map equivalent inputs to a shared key:

MemoizationKit.cachekey(::typeof(f), x; scale = 1) = (length(x), scale)
MemoizationKit.cachekey(::typeof(g), x) = (Hashed(x, customhash, customequal),)

The implementation, return-type inference, and cache strategies still receive the original arguments. Custom keys can merge calls to different methods or argument types; all merged calls must admit the same cached result, including its return type. MemoizationKit's RAM containers distinguish key types, so keys intended to share an entry must also have the same type. The key must remain stable while cached. uncached bypasses this hook.

RAM caches use hashing and equality; disk caches compare serialized bytes, so a canonical representation is needed to share disk entries between different inputs. After changing the mapping, empty existing RAM caches and update MemoizationKit.diskversion if using disk.

Hashed(value, hashf = Base.hash, eqf = Base.isequal)

Wrap value with custom hashing and equality for use as a dictionary or cache key. hashf(value, seed::UInt) computes its hash; eqf(a, b) compares the wrapped values. Retrieve the value with parent. Both functions may also be callable structs.

Two wrappers compare equal only when their hash and equality callables are identical (===) and eqf considers their values equal. The equality must be an equivalence relation, and equivalent values must have equal hashes for every seed. Values and callable state that affect hashing or equality must remain unchanged while stored.

Use MemoizationKit.cachekey to wrap inputs without changing a cached function's signature. Custom equality affects RAM caching; disk caching compares serialized key bytes.

CacheStyle

Supertype of the caching strategies used by @cached functions.

The strategy for a call f(args...) is chosen by CacheStyle(f, args...), which defaults to GlobalCache(). Specialize it to change the strategy per function or argument type:

MemoizationKit.CacheStyle(::typeof(f), x::SmallKey) = NoCache()
MemoizationKit.CacheStyle(::typeof(f), x::ThreadedKey) = TaskLocalCache{LRU}()

See also NoCache, GlobalCache, TaskLocalCache.

NoCache()

Strategy that calls the implementation every time.

GlobalCache{C}()
GlobalCache()

Strategy that stores results in one process-wide cache per function, of container type C <: AbstractCache. Without C, the container is the one set by the container preference (ClockCache by default), so that a CacheStyle method can return the default strategy.

GlobalLRUCache()

Alias for GlobalCache{LRU}().

TaskLocalCache{C}()
TaskLocalCache()

Strategy that stores results in caches local to the current task, of container type C. C is either an AbstractCache or an AbstractDict type such as Dict; if it is not already concrete, it is completed to C{K, V}. Without C, the container is the one set by the container preference. Task-local caches need no locking but are not shared, not bounded unless C is, and are not visible to cache_info.

cache_info() -> Vector{Pair{Any, AbstractCache}}
cache_info(f) -> Vector{Pair{Any, AbstractCache}}
cache_info(m::Module) -> Vector{Pair{Any, AbstractCache}}

The global caches, those of f, or those of the functions owned by the module m or its submodules, as f => cache pairs. A function has one cache, holding all its signatures (or one per container type, if its CacheStyle selects several), and is owned by the module that defines it, parentmodule(typeof(f)), wherever its @cached methods are written. The caches are live: they show their size and hit statistics, and can be inspected, emptied or resized directly. Task-local caches are not included.

empty_caches!()
empty_caches!(f)
empty_caches!(m::Module)

Empty every global cache, those of f, or those of the functions owned by the module m or its submodules (see cache_info). Statistics are kept.

set_cache_size!(f, maxsize::Integer; by = nothing)

Set the size limit of the global cache of f, overriding the preferences (see the configuration docs). The limit applies to the function as a whole, all signatures together. Without by, maxsize counts entries; otherwise it bounds the sum of by(value) over the entries, e.g. with by = MemoizationKit.cachesize for bytes. Changing by discards the cache of f.

set_cache_preferences!(; settings...)
set_cache_preferences!(package::Module; settings...)
set_cache_preferences!(f; settings...)

Store default cache settings in LocalPreferences.toml: globally (section [MemoizationKit]), for the functions of package ([<package>.MemoizationKit]), or for the function f ([<owner>.MemoizationKit.<f>], where <owner> is the package that defines f). Settings are maxsize, measure ("count" or "bytes"), disk and disk_path (see the disk caching docs), and, globally only, container ("ClockCache" or "LRU"). A value of nothing removes the setting.

Settings apply to functions whose first cache is created afterwards, so in practice after a restart; changing container recompiles MemoizationKit. See the configuration docs for how settings combine.

set_cache_preferences!(; maxsize = 100_000)
set_cache_preferences!(MyPackage; measure = "bytes", maxsize = 2^30)
set_cache_preferences!(MyPackage.expensive; maxsize = 50_000)

MemoizationKit.cachesize(x) -> Integer

Size in bytes of a cached value x, used by caches whose measure is "bytes". Defaults to Base.summarysize(x). Overload it for your own types when that is slow (it traverses the whole object) or inaccurate (memory shared between values is counted for each of them):

MemoizationKit.cachesize(t::MyTensor) = sizeof(t.data)

cache_dashboard(; interval = 1.0, filter = "")

Open an interactive terminal dashboard of the global caches: one row per function, with live hit rates and sizes, from which caches can be emptied or resized. A second tab lists the disk caches in use (see the disk caching docs), with their entries, size and hit rate. The statistics are re-read every interval seconds, and only functions whose name contains filter are shown.

The dashboard is a package extension: load Tachikoma.jl first, with using Tachikoma. See Dashboard for the keybindings.

DiskCacheStyle(f, args...)

The disk caching strategy for the call f(args...) of a @cached function: NoCache() (the default) or a DiskCache. It is independent of CacheStyle: the disk is consulted when the RAM cache misses, or on every call when CacheStyle is NoCache().

MemoizationKit.DiskCacheStyle(::typeof(f), args...) = DiskCache()

DiskCache{S}()
DiskCache(; serializer = Serialization.Serializer)

Disk caching strategy, returned by DiskCacheStyle: results are stored in an SQLite database per function and node, written with the serializer type S <: AbstractSerializer. Needs SQLite.jl (using SQLite).

MemoizationKit.diskversion(f) -> String

Version of the disk cache of f, part of its file name; "1" by default. Change it when the stored results of f are no longer valid, or no longer readable (e.g. after a Julia update).

MemoizationKit.disk_artifact(f) -> Union{Nothing, String}

A directory with a read-only disk cache of f, made by export_disk_cache, that is consulted before the cache of the node, e.g. artifact"results". Defaults to nothing.

disk_cache_info(f) -> Vector{Pair{Any, NamedTuple}}
disk_cache_info(m::Module) -> Vector{Pair{Any, NamedTuple}}

The disk cache of f on this node, or those of the functions owned by m or its submodules that have been used in this process, as f => (; path, entries, bytes) pairs.

MemoizationKit.disk_cache_stats() -> Vector{Pair{Any, NamedTuple}}

The disk lookups in this process of each function whose disk cache is open, as f => (; hits, misses) pairs, where a hit is a result read from disk (from the artifact or the database of the node) and a miss a result computed and written. Unlike disk_cache_info, it reads no files.

empty_disk_caches!(f)
empty_disk_caches!(m::Module)

Remove the entries of the disk cache of f on this node, or of the caches listed by disk_cache_info(m).

export_disk_cache(f, dir) -> String

Write the disk cache of f on this node into dir, as a compact read-only database named as MemoizationKit.disk_artifact expects it, and return its path.

disable_disk_caches!()

Turn all disk caches off in this process, until enable_disk_caches!.

enable_disk_caches!()

Turn disk caches back on, after disable_disk_caches!.

MemoizationKit.instrument_label(f, ::Val{phase}) -> String

Label of the phase of f in the sections recorded by enable_cache_timers!. Defaults to "lookup f" and "compute f", with the name of f; overload it to choose your own.

enable_cache_timers!(M::Module, timer::TimerOutput = TimerOutputs.get_defaulttimer())

Record the time spent in the cached functions owned by the package M, or any of its submodules, in timer: a section per function and phase, the lookup (hit or miss) with the computation of a miss nested in it, labelled by MemoizationKit.instrument_label. This includes methods of these functions cached in other packages. Modules outside packages, such as those defined in the REPL, count separately. Enabling M again replaces its timer; disable_cache_timers! stops.

Enabling and disabling define and delete a method of an internal hook, so they recompile the callers of the functions of M, and only take effect for code that starts afterwards (from the next top-level statement on, or through invokelatest). They cannot be used during precompilation.

This is a package extension: load TimerOutputs.jl first, with using TimerOutputs. See Timing for the details.

disable_cache_timers!(M::Module)

Stop timing the cached functions owned by M, started by enable_cache_timers!. Their calls recompile without the timers, at no cost again.