Stroika::Foundation::Cache::TimedCache< KEY, VALUE, TRAITS > Class Template Reference

Keep track of a bunch of objects, each with an associated time used to allow data to 'expire'. More...

#include <TimedCache.h>

Classes
struct	CacheElement
	everything here is optional ;-) But typically, its fKey, fValue, fLastRefreshedAt More...

Public Member Functions
	TimedCache ()
nonvirtual TimeStampDifferenceType	GetMaxAge () const
nonvirtual void	SetMaxAge (TimeStampDifferenceType maxAge)
nonvirtual Traversal::Iterable< CacheElement >	Elements () const
template<typename K = KEY> requires (IKeyedCache<K>)
nonvirtual Traversal::Iterable< K >	Keys () const
template<typename K = KEY> requires (IKeyedCache<K>)
nonvirtual optional< VALUE >	Lookup (typename Common::ArgByValueType< K > key) const
	Returns the value associated with argument 'key', or nullopt, if its missing (missing same as expired). Can be used to retrieve lastRefreshedAt.
template<typename K = KEY> requires (kTrackExpiration and IKeyedCache<K>)
nonvirtual optional< tuple< VALUE, TimeStampType > >	LookupDetails (typename Common::ArgByValueType< K > key) const
	Returns the value associated with argument 'key' (if IKeyedCache), or nullopt, if its missing (missing same as expired). Like Lookup but also returns expiration information.
template<typename K = KEY> requires (not IKeyedCache<K>)
nonvirtual optional< TimeStampType >	GetExpiration () const
	Get the Expiration of object or nullopt of item expired/not in cache.
template<typename K = KEY, Common::invocable_r< VALUE, KEY > CACHE_FILLTER_T> requires (IKeyedCache<K>)
nonvirtual VALUE	LookupValue (typename Common::ArgByValueType< K > key, CACHE_FILLTER_T &&cacheFiller)
	Lookup value, and if missing, fetch it with argument cacheFiller (and add/return its value).
template<typename K = KEY> requires (IKeyedCache<K> and IValuelessCache<VALUE>)
nonvirtual void	Add (typename Common::ArgByValueType< K > key)
template<Traversal::IIterableOfTo< KEY > ITERABLE_OF_KEY_TYPE>
nonvirtual void	RetainAll (const ITERABLE_OF_KEY_TYPE &items2Keep)
	like Mapping<>::RetainAll () - throw away all elements not in items2Keep
nonvirtual void	clear ()
nonvirtual void	ClearExpiredData ()
template<typename K = KEY>
nonvirtual optional< VALUE >	Lookup (typename Common::ArgByValueType< K > key, LookupMarksDataAsRefreshed successfulLookupRefreshesAcceesFlag)

Static Public Attributes
static constexpr bool	kTrackExpiration = TRAITS::kTrackExpiration
	Track on a per-item when it expires. If not tracked, we use expiresAt as whenAdded + maxAge This is false by default.
static constexpr bool	kTrackFreshness = TRAITS::kTrackFreshness
	freshness means when last added/updated (or if kAutomaticallyMarkDataAsRefreshedEachTimeAccessed) then last accessed too) This is true by default

Detailed Description

template<IKey KEY, IValue VALUE, TimedCacheSupport::ITraits< KEY, VALUE > TRAITS = TimedCacheSupport::DefaultTraits<KEY, VALUE>>
class Stroika::Foundation::Cache::TimedCache< KEY, VALUE, TRAITS >

Keep track of a bunch of objects, each with an associated time used to allow data to 'expire'.

This expiration time is handled PRINCIPALLY, in one of two ways: kTrackExpiration: In this case, at the time the data is ADDED, a time of expiration is captured and associated with the datum.

kTrackFreshness: In this case, the time the data is Added (last-refreshed) is associated with the datum, and expiration computed later. this is the default

More about kTrackFreshness: We define 'fresheness' somewhat arbitrarily, but by default, this means since the item was added. However, the TimedCache also provides other apis to update the 'freshness' of a stored object, depending on application needs.

Keeps track of all items - indexed by Key - but throws away items which are any more stale than given by the staleness limit.

When/Why to use?: (

For example, consider a system where memory is stored across a slow bus, and several components need to read data from across that bus. But the different components have different tolerance for staleness (e.g. PID loop needs fresh temperature sensor data but GUI can use more stale data).

This CallerStalenessCache will store when the value is updated, and let the caller either return the value from cache, or fetch it and update the cache if needed.

This differs from other forms of caches in that: o It records the timestamp when a value is last-updated o It doesn't EXPIRE the data ever (except by explicit Clear or ClearOlderThan call) o The lookup caller specifies its tolerance for data staleness, and refreshes the data as needed.

Note

Comparison with LRUCache The main difference beweeen an LRUCache and TimedCache has to do with when an element is evicted from the Cache. With a TimedCache, its evicted only when its overly aged. With an LRUCache, its more random, and depends a bit on luck (when using hashing) and how recently an item was last accessed.

TimedCache (since Stroika v3.0d23) fully supports the caching model from the Stroika v2.1 CallerStalenessCache (which is now obsolete). If the cache uses 'Freshness' instead of 'Expired' (the default) then the various Lookup APIs fully support specifying a (non-defaulted, caller specified) 'staleness'.

Example Usage

Use TimedCache to avoid needlessly redundant lookups

optional<String> ReverseDNSLookup_ (const InternetAddress& inetAddr)
{
    const Time::Duration                                        kCacheTTL_{5min};
    static Cache::TimedCache<InternetAddress, optional<String>> sCache_{kCacheTTL_};    // not threadsafe (not internally synchronized) by default - but checked with Debug::AssertExternallySyncrhonizedMutex
    return sCache_.LookupValue (inetAddr, [] (const InternetAddress& inetAddr) {
        return DNS::kThe.ReverseLookup (inetAddr);
    });
}

Example Usage

Assume 'LookupDiskStats_' returns DiskSpaceUsageType, but its expensive, and the results change only slowly...

struct DiskSpaceUsageType {
    int size;
};
// do the actual lookup part which maybe slow
auto LookupDiskStats_ ([[maybe_unused]] const String& filename) -> DiskSpaceUsageType { return DiskSpaceUsageType{33}; };
 
Cache::TimedCache<String, DiskSpaceUsageType> sDiskUsageCache_{5.0_duration};
OR
    struct CACHE_TRAITS_ : Cache::TimedCacheSupport::DefaultTraits<String, DiskSpaceUsageType> {
        static constexpr inline bool kAutomaticallyMarkDataAsRefreshedEachTimeAccessed = true; // to treat lookups as 'refreshing' the cache like LRU
    };
Cache::TimedCache<String, DiskSpaceUsageType, CACHE_TRAITS_> sDiskUsageCache_{5.0_duration};
 
// explicitly caller maintaining the cache
optional<DiskSpaceUsageType> LookupDiskStats_Try1 (String diskName)
{
    optional<DiskSpaceUsageType> o = sDiskUsageCache_.Lookup (diskName);
    if (not o.has_value ()) {
        o = LookupDiskStats_ (diskName);
        if (o) {
            sDiskUsageCache_.Add (diskName, *o);
        }
    }
    return o;
}
 
// more automatic maintainance of that update pattern
DiskSpaceUsageType LookupDiskStats_Try2 (String diskName)
{
    return sDiskUsageCache_.LookupValue (diskName,
                                    [](String diskName) -> DiskSpaceUsageType {
                                        return LookupDiskStats_ (diskName);
                                    });
}
 
// or still simpler
DiskSpaceUsageType LookupDiskStats_Try3 (String diskName)
{
    // maybe use eTreatFoundThroughLookupAsRefreshed depending on your application
    return sDiskUsageCache_.LookupValue (diskName, LookupDiskStats_);
}
void DoIt ()
{
    // example usage
    EXPECT_TRUE (Memory::NullCoalesce (LookupDiskStats_Try1 ("xx")).size == 33);
    EXPECT_TRUE (LookupDiskStats_Try2 ("xx").size == 33);
    EXPECT_TRUE (LookupDiskStats_Try3 ("xx").size == 33);
}

Example Usage

Same as above, but adding internal syncrhonization (automatic thread safety)

optional<String> ReverseDNSLookup_ThreadSafe_ (const InternetAddress& inetAddr)
{
    const Time::Duration                                        kCacheTTL_{5min};
    struct CACHE_TRAITS_ : Cache::TimedCacheSupport::DefaultTraits<InternetAddress, optional<String>> {
        static constexpr inline InternallySynchronized kInternallySynchronized{InternallySynchronized::eInternallySynchronized};
    };
    static Cache::TimedCache<InternetAddress, optional<String>, CACHE_TRAITS_> sCache_{kCacheTTL_};    // NOW the cache is threadsafe
    return sCache_.LookupValue (inetAddr, [] (const InternetAddress& inetAddr) {
        return DNS::kThe.ReverseLookup (inetAddr);
    });
}

Example Usage (caller specifies staleness in each context where lookup happens)

// keyed cache
optional<int> MapValue_ (int value, optional<Time::DurationSeconds> allowedStaleness = {})
{
    static CallerStalenessCache<int, optional<int>> sCache_;
    try {
        return sCache_.LookupValue (value, allowedStaleness.value_or (30), [=](int v) -> optional<int> {
            return v;   // typically more expensive computation
        });
    }
    catch (...) {
        // NOTE - to NEGATIVELY CACHE failure, you could call sCache_.Add (value, nullopt);
        // return null here, or Execution::ReThrow ()
    }
}
EXPECT_EQ (MapValue_ (1), 1);  // skips 'more expensive computation' if in cache
EXPECT_EQ (MapValue_ (2), 2);  // ''

Example Usage (no key, and 'caller staleness' style)

optional<InternetAddress> LookupExternalInternetAddress_ (optional<Time::DurationSeconds> allowedStaleness = {})
{
    static TimedCache<NonKeyedKeySentinalType, optional<InternetAddress>> sCache_;
    return sCache_.Lookup (allowedStaleness.value_or (30), []() -> optional<InternetAddress> {
        ...
        return IO::Network::InternetAddress{connection.GET ().GetDataTextInputStream ().ReadAll ().Trim ()};
    });
}
optional<InternetAddress> iaddr = LookupExternalInternetAddress_ ();    // only invoke connection logic if timed out

Example Usage

 To use TimedCache<> to 'own' a set of objects (say a set caches where we are the only
 possible updater) - you can make the 'VALUE' type a shared_ptr<X>, and  use Lookup (...,eTreatFoundThroughLookupAsRefreshed) instead 
 of Lookup ().

 In this example, there is a set of files on disk in a folder, which is complex to analyze
 but once analyzed, lots of calls come in at once to read (and maybe update) the set of files
 and once nobody has asked for a while, we throw that cache away, and rebuild it as needed.

 This example ALSO shows how to wrap a cache object in 'Synchronized' for thread safety.

 \code
     using ScanFolderKey_ = String;
     static constexpr Time::DurationSeconds kAgeForScanPersistenceCache_{5 * 60.0s};
     struct FolderDetails_ {
         int size; // ...info to cache about a folder
     };
     Synchronized<Cache::TimedCache<
         ScanFolderKey_,
         shared_ptr<FolderDetails_>,
         shared_ptr<FolderDetails_>>>
         sCachedScanFoldersDetails_{kAgeForScanPersistenceCache_};

     shared_ptr<FolderDetails_> AccessFolder_ (const ScanFolderKey_& folder)
     {
         auto lockedCache = sCachedScanFoldersDetails_.rwget ();
         if (optional<shared_ptr<FolderDetails_>> o = lockedCache->Lookup (folder, eTreatFoundThroughLookupAsRefreshed)) {
             return *o;
         }
         else {
             shared_ptr<FolderDetails_> fd = MakeSharedPtr<FolderDetails_> (); // and fill in default values looking at disk
             lockedCache->Add (folder, fd);
             return fd;
         }
     }

     void DoIt ()
     {
         auto f1 = AccessFolder_ ("folder1"_k);
         auto f2 = AccessFolder_ ("folder2"_k);
         auto f1again = AccessFolder_ ("folder1");  // if you trace through the debug code you'll see this is a cache hit
     }
 \endcode

Example Usage

Use 'void' as second type argument - just storing the first KEY value (like keyedCollection or just presence/absence test - like accessKeys)

Cache::TimedCache<Characters::String, void> validAccessKeyCache{30s};
EXPECT_TRUE (validAccessKeyCache.GetExpiration ("fred") == nullopt);
validAccessKeyCache.Add ("fred");
EXPECT_TRUE (validAccessKeyCache.GetExpiration ("fred"));

Note

This cache will keep using more and more memory until the cached items become out of date. For a cache that limits the max number of entries, use the

See also

LRUCache.

Note

This cache assumes one timeout for all items. To have timeouts vary by item,

See also

CallerStalenessCache.

Note

Satisfies Concepts: o ICache<TimedCache<KEY,VALUE>,KEY,VALUE> o moveable<TimedCache<KEY,VALUE>> o copyable<TimedCache<KEY,VALUE>>

Thread-Safety if (TRAITS::kInternallySynchronized == eInternallySynchronized) Internally-Synchronized-Thread-Safety

Thread-Safety if (TRAITS::kInternallySynchronized == eNotKnownInternallySynchronized) C++-Standard-Thread-Safety

we REQUIRE (without a way to enforce) - that the STATS object be internally synchronized, so that we can maintain statistics, without requiring the lookup method be non-const; this is only for tuning/debugging, anyhow...

Implementation Note: '_' private routines assume called with locks in place, and just do the minimal function (except when otherwise noted).

See also

LRUCache

Definition at line 572 of file TimedCache.h.

Constructor & Destructor Documentation

TimedCache()

template<IKey KEY, IValue VALUE, TimedCacheSupport::ITraits< KEY, VALUE > TRAITS>
requires (not is_empty_v<decltype (TRAITS::kDefaultMaxAge)>)

Stroika::Foundation::Cache::TimedCache< KEY, VALUE, TRAITS >::TimedCache

(

)

Note that TimedCache is copyable and moveable by value.

It MAYBE default constructible (if there is a kDefaultMaxAge - defaults to true).

Definition at line 44 of file TimedCache.inl.

Member Function Documentation

GetMaxAge()

template<IKey KEY, IValue VALUE, TimedCacheSupport::ITraits< KEY, VALUE > TRAITS>

auto Stroika::Foundation::Cache::TimedCache< KEY, VALUE, TRAITS >::GetMaxAge

(

)

const

When items are added to the timed cache, there is a universal (for the entire cache) minimum allowed freshness (how old item allowed to be before thrown away).

\alias Note - 'allowed freshness' == 'time to live' == 'TTL'. GetMinimumFreshness, GetTimeout, TTL, GetDefaultMaxAge ().

So an item added 30 seconds ago (freshness = 30s), would be thrown away/not returned as part of the cache if the minimum allowed freshness was 5 seconds.

Note

if kTrackFreshness, this is used in CHECKS of freshness (provides a default 'maxage' at the point of Lookup call). if kTrackExpiration, this is applied to the added item, as its default expiration. In both cases, these values can be overridden (either in Add or Lookup depending on which you are tracking).

Definition at line 123 of file TimedCache.inl.

SetMaxAge()

template<IKey KEY, IValue VALUE, TimedCacheSupport::ITraits< KEY, VALUE > TRAITS>
requires (TRAITS::kPerCacheMaxAge)

void Stroika::Foundation::Cache::TimedCache< KEY, VALUE, TRAITS >::SetMaxAge

(

TimeStampDifferenceType

maxAge

)

See also

GetMaxAge ()

Definition at line 134 of file TimedCache.inl.

Elements()

template<IKey KEY, IValue VALUE, TimedCacheSupport::ITraits< KEY, VALUE > TRAITS>

auto Stroika::Foundation::Cache::TimedCache< KEY, VALUE, TRAITS >::Elements

(

)

const

Note

This returns the non-expired elements of the current cache object.

@todo could use overload taking TTL argumnet (if track freshness)

Definition at line 145 of file TimedCache.inl.

Keys()

template<IKey KEY, IValue VALUE, TimedCacheSupport::ITraits< KEY, VALUE > TRAITS = TimedCacheSupport::DefaultTraits<KEY, VALUE>>

template<typename K = KEY>
requires (IKeyedCache<K>)

nonvirtual Traversal::Iterable< K > Stroika::Foundation::Cache::TimedCache< KEY, VALUE, TRAITS >::Keys

(

)

const

Note

This returns the non-expired keys of the current cache object.

@todo could use overload taking TTL argumnet (if track freshness)

Lookup() [1/2]

template<IKey KEY, IValue VALUE, TimedCacheSupport::ITraits< KEY, VALUE > TRAITS = TimedCacheSupport::DefaultTraits<KEY, VALUE>>

template<typename K = KEY>
requires (IKeyedCache<K>)

nonvirtual optional< VALUE > Stroika::Foundation::Cache::TimedCache< KEY, VALUE, TRAITS >::Lookup

(

typename Common::ArgByValueType< K >

key

)

const

Returns the value associated with argument 'key', or nullopt, if its missing (missing same as expired). Can be used to retrieve lastRefreshedAt.

Note

that the non-const overload of Lookup respects TRAITS::kAutomaticallyMarkDataAsRefreshedEachTimeAccessed, and will auto-refresh the item (similar to LRUCache) if found.

Occasionally, a caller might want to ASSURE it gets data, and just use the cached value if fresh enuf, and specify a lookup lambda to fetch the actual data if its not fresh, in which case call LookupValue ().

Note

difference between const and non-const overloads is just that some extra bookkeeping can be done and kAutomaticallyMarkDataAsRefreshedEachTimeAccessed respected in non-const overload.

for Lookup (key) whether we track freshness or expiration, the API looks the same (though the impl differs).

for Lookup (key,maxAge) only possible if you track freshness no non-const version of this, cuz no bookkeeping. If you are using this, the caller decides what gets cleaned up, explicitly.

LookupDetails()

template<IKey KEY, IValue VALUE, TimedCacheSupport::ITraits< KEY, VALUE > TRAITS = TimedCacheSupport::DefaultTraits<KEY, VALUE>>

template<typename K = KEY>
requires (kTrackExpiration and IKeyedCache<K>)

nonvirtual optional< tuple< VALUE, TimeStampType > > Stroika::Foundation::Cache::TimedCache< KEY, VALUE, TRAITS >::LookupDetails

(

typename Common::ArgByValueType< K >

key

)

const

Returns the value associated with argument 'key' (if IKeyedCache), or nullopt, if its missing (missing same as expired). Like Lookup but also returns expiration information.

Note

that the non-const overload of Lookup respects TRAITS::kAutomaticallyMarkDataAsRefreshedEachTimeAccessed, and will auto-refresh the item (similar to LRUCache) if found.

Occasionally, a caller might want to ASSURE it gets data, and just use the cached value if fresh enuf, and specify a lookup lambda to fetch the actual data if its not fresh, in which case call LookupValue ().

Note

difference between const and non-const overloads is just that some extra bookkeeping can be done and kAutomaticallyMarkDataAsRefreshedEachTimeAccessed respected in non-const overload.

GetExpiration()

template<IKey KEY, IValue VALUE, TimedCacheSupport::ITraits< KEY, VALUE > TRAITS = TimedCacheSupport::DefaultTraits<KEY, VALUE>>

template<typename K = KEY>
requires (not IKeyedCache<K>)

nonvirtual optional< TimeStampType > Stroika::Foundation::Cache::TimedCache< KEY, VALUE, TRAITS >::GetExpiration

(

)

const

Get the Expiration of object or nullopt of item expired/not in cache.

in the case of IKeyedCache, it returns the expiration of the given key (or nullopt if not present or expired). in the case of NOT IKeyedCache, it refers to the single cached item, but otherwise acts the same.

LookupValue()

template<IKey KEY, IValue VALUE, TimedCacheSupport::ITraits< KEY, VALUE > TRAITS = TimedCacheSupport::DefaultTraits<KEY, VALUE>>

template<typename K = KEY, Common::invocable_r< VALUE, KEY > CACHE_FILLTER_T>
requires (IKeyedCache<K>)

nonvirtual VALUE Stroika::Foundation::Cache::TimedCache< KEY, VALUE, TRAITS >::LookupValue	(	typename Common::ArgByValueType< K >	key,
		CACHE_FILLTER_T &&	cacheFiller
	)

Lookup value, and if missing, fetch it with argument cacheFiller (and add/return its value).

This operates as if: if (auto ov = Lookup (key)) { return *ov; } else { VALUE r = cacheFiller(); Add (r); return r; }

Note

this is the PREFERRED API for adding/looking up in TimedCache

Usually one will use this as:

VALUE v = cache.LookupValue (key, ts, [this] () -> VALUE {return this->realLookup(key); });

Note

This function may update the TimedCache (which is why it is non-const).

Any time arguments given constrain the lookup. They are not used for the Add ().

Add()

template<IKey KEY, IValue VALUE, TimedCacheSupport::ITraits< KEY, VALUE > TRAITS = TimedCacheSupport::DefaultTraits<KEY, VALUE>>

template<typename K = KEY>
requires (IKeyedCache<K> and IValuelessCache<VALUE>)

nonvirtual void Stroika::Foundation::Cache::TimedCache< KEY, VALUE, TRAITS >::Add

(

typename Common::ArgByValueType< K >

key

)

Adds/Updates the given value associated with key. if kTrackFreshness (the default) o The new items freshness is TRAITS::GetCurrentTimestamp (), or the value given as argument if kTrackExpiration o The new item's expiration is either given by expiresAt or now+ttl, or defaults to GetMaxAge()

Note

this API supports overloads of Add () where either the KEY or the VALUE is missing, depending on how the TimedCache TRAITS were declared.

clear()

template<IKey KEY, IValue VALUE, TimedCacheSupport::ITraits< KEY, VALUE > TRAITS>

void Stroika::Foundation::Cache::TimedCache< KEY, VALUE, TRAITS >::clear

(

)

Remove everything from the cache

Definition at line 769 of file TimedCache.inl.

ClearExpiredData()

template<IKey KEY, IValue VALUE, TimedCacheSupport::ITraits< KEY, VALUE > TRAITS>

void Stroika::Foundation::Cache::TimedCache< KEY, VALUE, TRAITS >::ClearExpiredData

(

)

May be called occasionally to free resources used by cached items that are out of date. Not necessary to call - but can save memory.

Can be triggered automatically - see TRAITS::kAutomaticPurgeFrequency

Definition at line 775 of file TimedCache.inl.

Lookup() [2/2]

template<IKey KEY, IValue VALUE, TimedCacheSupport::ITraits< KEY, VALUE > TRAITS = TimedCacheSupport::DefaultTraits<KEY, VALUE>>

template<typename K = KEY>

nonvirtual optional< VALUE > Stroika::Foundation::Cache::TimedCache< KEY, VALUE, TRAITS >::Lookup	(	typename Common::ArgByValueType< K >	key,
		LookupMarksDataAsRefreshed	successfulLookupRefreshesAcceesFlag
	)

Before Stroika 3.0d1, we used to remove the entry from the list (an optimization). But that required Lookup to be non-const (with synchronization in mind probably a pessimization). So instead, count on PurgeUnusedData being called automatically on future adds, explicit user calls to purge unused data.

 i = fData_.erase (i);

Definition at line 974 of file TimedCache.h.

Member Data Documentation

kTrackExpiration

template<IKey KEY, IValue VALUE, TimedCacheSupport::ITraits< KEY, VALUE > TRAITS = TimedCacheSupport::DefaultTraits<KEY, VALUE>>

constexpr bool Stroika::Foundation::Cache::TimedCache< KEY, VALUE, TRAITS >::kTrackExpiration = TRAITS::kTrackExpiration

staticconstexpr

Track on a per-item when it expires. If not tracked, we use expiresAt as whenAdded + maxAge This is false by default.

Note

kTrackFreshness and kTrackExpiration are mutually exclusive

Definition at line 593 of file TimedCache.h.

kTrackFreshness

template<IKey KEY, IValue VALUE, TimedCacheSupport::ITraits< KEY, VALUE > TRAITS = TimedCacheSupport::DefaultTraits<KEY, VALUE>>

constexpr bool Stroika::Foundation::Cache::TimedCache< KEY, VALUE, TRAITS >::kTrackFreshness = TRAITS::kTrackFreshness

staticconstexpr

freshness means when last added/updated (or if kAutomaticallyMarkDataAsRefreshedEachTimeAccessed) then last accessed too) This is true by default

Note

kTrackFreshness and kTrackExpiration are mutually exclusive

Definition at line 602 of file TimedCache.h.

---

The documentation for this class was generated from the following files:

• TimedCache.h
• TimedCache.inl