NFileCache <: FileCache

Represents a number-of-files-constrained file cache. Configurable with maximum number of files in the cache and how to choose which elements to discard.

NFileCache(root, nfiles, discard_func; predicate=x->true)

Construct a new file cache. Arguments:

• root: the root directory of the cache.
• nfiles: number of files to retain.
• discard_func: function that return the eviction order, see e.g. DiscardLRU and DiscardLFU.
• predicate: function that determines whether a file should be tracked/included in the cache. Currently only used for existing files when setting up the file cache. The default is to track every existing file.

Example usage:

# Create cache that keeps maximum 10 files and releases LRU objects
fc = NFileCache(root, 10, DiscardLRU())

# Add a new file:
path = add!(fc, key)
open(path, "w") do io
    write(path, filedata)
end

# Check to see if that file is there (and also increment its usage counters)
hit!(fc, key)

# Delete that file
delete!(fc, key)

SizeConstrainedFileCache <: FileCache

Represents a size-constrained file cache. Configurable with different policies for when to discard elements from the cache and how to choose which elements to discard.

SizeConstrainedFileCache(root, target_func, discard_func; predicate=x->true)

Construct a new file cache. Arguments:

• root: the root directory of the cache.
• target_func: function that return the (maximum) number of bytes that files in the cache may occupy, see e.g. TargetSizeKeepFree and TargetSizeConstant.
• discard_func: function that return the eviction order, see e.g. DiscardLRU and DiscardLFU.
• predicate: function that determines whether a file should be tracked/included in the cache. Currently only used for existing files when setting up the file cache. The default is to track every existing file.

Example usage:

# Create cache that keeps 10GB free and releases LRU objects
scfc = SizeConstrainedFileCache(root, TargetSizeKeepFree(10*1024^3), DiscardLRU())

# Add a new file:
path = add!(scfc, key, filesize)
open(path, "w") do io
    write(path, filedata)
end

# Check to see if that file is there (and also increment its usage counters)
hit!(scfc, key)

# Delete that file
delete!(scfc, key)

delete!(fc::FileCache, key)

Removes a key from the file cache, clearing all metadata about that key within the cache. Also removes the backing file on disk if it still exists. Returns true if the key was actually deleted.

add!(fc::NFileCache, key::AbstractString, new_size=0)

Reserves space for a new file within the file cache. If the number of files exceed the maximum number of entries allowed in the cache the cache delete a file as governed by the discard ordering.

add!(scfc::SizeConstrainedFileCache, key::AbstractString, new_size)

Reserves space for a new file within the file cache of the specified size. If the addition would result in a new total size that is greater than the target specified by the target_size function passed to the SCFC constructor, shrink!() the file cache until this new object can fit nicely within the cache without overstepping any file size bounds. If the new object cannot fit wihtin the cache at all, throws an ArgumentError.

add!()'ing an entry that already exists within the file cache overwrites it and clears all access counters, treating it as a semantically different object than the object that was replaced.

filepath(fc::FileCache, key::AbstractString)

Given a key, returns the path on disk for that key.

get_disk_freespace(path)

Returns the amount of available disk space within the filesystem that contains the given path. Internally uses statfs() which is only implemented on Linux and macOS; throws an error on all other systems.

hit!(fc::FileCache, key)

Returns true if the key exists within the cache, false otherwise. Increments access counters for the given key, recording last access time, number of times accessed, etc...

rebuild!(fc::FileCache; predicate::Function = x -> true)

Rebuilds the file cache datastructures within memory from disk. This is not a lossless operation; last access times may be inaccurate and number of times accessed will be set to 1 for any key that was not previously tracked. This is done automatically when creating a new cache; the file cache will scan its root directory and populate itself with the values gathered using this function.

The predicate keyword can be used to pass a predicate function (input: key, output: Bool) to filter which files in the root directory that should be tracked/included in the cache. The default is to include every file.

shrink!(fc::NFileCache, files_to_remove)

Shrinks a file cache by the amount given by discarding files within the file cache, stopping only when the target has been met or the list of keys to discard is exhausted. Returns the new total size of the file cache.

shrink!(scfc, size_to_shrink)

Shrinks a file cache by the amount given by discarding files within the file cache, stopping only when the target has been met or the list of keys to discard is exhausted. Returns the new total size of the file cache.