Loading...
Oct 2024 | 8 min time
Introduction
One of the most common needs for an application developer is to optimize the performance of various operations involving data access to the database. In some scenarios, such data doesn’t change for hours, days, or even months/years. So, the question to ask is: if we need to retrieve this data, does it always make sense to perform a query on the database, with a cost that could be more or less substantial and could affect the overall performance of the application?
Let’s take the example of a “Nation” table, containing the list of countries in the world. Every time I need to get this list, I have to run a query on the database, burdening it and with query response times that might not be very fast, depending also on the database load at that moment. The example of countries probably never generates a real performance issue (we are talking about about 200 records in the table), but the topic is general, and the question to ask is: can we avoid always going through the database to get static data? The answer is: Yes! With Cache!
We will address this topic in the Microsoft .Net world and present the various mechanisms that exist today to retrieve some data from a “container” different from the database.
Local Cache – MemoryCache
MemoryCache is the standard in-memory caching implementation in .Net, designed to store data directly in the memory of the server hosting the application. This allows you to query the database only the first time, while subsequent requests retrieve the data directly from the server’s memory. The cache remains valid for a predefined period, set by its configuration, unless it is manually or forcibly removed.
Practical Example
Let’s go back to the example of the “Nation” table: how can we effectively cache this table using .Net’s MemoryCache? Here’s the code:
public IMemoryCache Cache { get; }
private const string NATIONS_CACHE_KEY = "NATIONS";
private async Task<List<Nation>> DbGetNations()
{
return await dbContext.Nation.ToListAsync();
}
public async Task<List<Nation>> GetNations()
{
if (!Cache.TryGetValue(NATIONS_CACHE_KEY, out _))
{
var nations = await DbGetNations();
var cacheEntryOptions = new MemoryCacheEntryOptions()
.SetAbsoluteExpiration(TimeSpan.FromHours(24));
Cache.Set(NATIONS_CACHE_KEY, nations, cacheEntryOptions);
}
return Cache.Get<List<Nation>>(NATIONS_CACHE_KEY).ToList();
}
How the “GetNations” method works:
- It checks if the key NATIONS_CACHE_KEY exists in the cache.
1 - If it exists, we fetch the data from the cache.
2 - If it doesn’t exist, a database query is executed, and the data is set in the cache with the key NATIONS_CACHE_KEY.
- Subsequent calls use the data stored in memory.
In our case, the cache expires after 24 hours, so the key and its content will automatically be removed from the cache after 24 hours, with the effect that the next call will “enter the if” and re-execute the database query. Of course, this is a simple use of MemoryCache in .Net. In a real scenario, we could keep the data in the cache for more than 24 hours or without an expiration. In that case, if we add a new nation, we can explicitly reset the cache (e.g., with Cache.Remove(NATIONS_CACHE_KEY)) or simply restart the server.
Alternatives to Official Libraries: Z.EntityFramework.Plus
To simplify the use of MemoryCache with Entity Framework, there are third-party libraries, such as Z.EntityFramework.Plus, that offer methods to manage caching more easily.
Returning to the previous example:
private async Task<List<Nation>> GetNations()
{
return (await dbContext.Nation.FromCacheAsync(new MemoryCacheEntryOptions() { AbsoluteExpirationRelativeToNow = TimeSpan.FromHours(24) }, NATIONS_CACHE_KEY)).ToList();
}
As you can see, instead of directly executing ToList which would perform the query on the database, the FromCacheAsync method of Z.EntityFramework.Plus is used first, with its configuration. The library’s method will determine whether to query the database (and save the result for future queries) or retrieve the result from MemoryCache.
In reality, FromCacheAsync is much more advanced than a simple use of MemoryCache. For example, if we are in a scenario where the result to be saved is not just a single table, but filters are needed to obtain the query. Let’s take the example of a “Province” table containing the provinces of various countries in the world. This table obviously will also contain a reference to the nation they belong to. If we want to cache the list of provinces given a “CountryCode” input each time, we should create a key that considers the CountryCode filter used:
For example, PROVINCES_IT_CACHE_KEY, PROVINCES_ES_CACHE_KEY, etc., complicating the implementation.
With the FromCache/FromCacheAsync method of Z.EntityFramework.Plus, the system is able to read the different filter parameters of the input and build specific keys to save the specific result, all transparently to the developer.
In this case, the “key” in the FromCacheAsync method (like NATIONS_CACHE_KEY or PROVINCE_CACHE_KEY) serves a different function than in the custom cache implementation. Here, the key is used to manage the cache’s expiration: the query results associated with that key will expire after the set time. If needed, an explicit reset on that key will expire all stored results, even those with different parameters.
Effectiveness of MemoryCache
Does what we’ve described so far solve the cache problem? Unfortunately, not entirely. Let’s assume our application is load-balanced across multiple servers (2,4,6,8, etc.). Suppose a user makes a call to GetCountries. The load balancer directs the request to Server 1, queries the database, and saves the result in the server’s memory. A subsequent call to GetCountries also lands on Server 1, the data is in memory, and the results are returned without making another database query. But let’s say another call to GetCountries is made. This time, the load balancer directs the request to Server 2, but here the cache was never populated, so the system makes another database query and saves the result in Server 2’s memory. In essence, as the number of balanced servers grows, the caching mechanism explained so far becomes less effective. What’s the solution to this problem? The answer is: Distributed Cache!
Distributed Cache – IDistributedCache
Distributed cache has a great advantage, which is that queries and their results are not stored in the memory of the single server running the app, but rather an “external shared memory” is used, from which the app will draw regardless of the server it’s running on. Additionally, being external to the app, a restart of the app will not reset the cache. In this way, using the previous example, having saved the query/result in shared memory the first time, a GetCountries request executed on Server 2 will continue to access the “shared” cache without needing another database query to get the list of nations, as was the case in the MemoryCache example. The IDistributedCache interface in .Net provides methods to manipulate items in the distributed cache implementation. A common implementation of IDistributedCache uses Redis, an open-source in-memory data storage system, frequently employed as a distributed cache.
IDistributedCache: Is It Really the Solution?
So, the question we must ask is: has the distributed cache definitively solved the various problems related to implementing data caching mechanisms? Again, not entirely. While having a shared cache allows you to centralize data storage in a single “container,” making caching effective regarding application scalability, on the downside, access times to this shared structure will never be as performant as those on server memory. The risk, therefore, is to lose some of the time gained by not performing a database query.
Based on the previous example, the second request to GetCountries on Server 1, although it wouldn’t run a database query, would access a shared cache, which will certainly not be as fast as the second request on Server 1, which fetches the data from the server’s memory, as explained in the MemoryCache example. Another aspect to consider in implementing distributed cache is the need to serialize and deserialize data. Unlike local cache, such as MemoryCache, where objects can be managed directly in memory without these steps, in distributed cache, the data must be converted to a format that can be shared among the various nodes, thus introducing additional overhead in reading and writing operations.
Is There a Way to Combine the Benefits of Local and Distributed Cache?
The answer is yes: Hybrid Cache!
Hybrid Cache – FusionCache/HybridCache
The hybrid cache mechanism combines MemoryCache and IDistributedCache, following these steps:
- On a request, the system first checks if the data is present in the local MemoryCache.
- If the data is found, it is returned quickly.
- If it is not in MemoryCache, the system searches in the distributed cache (if configured).
- If the data is present in the distributed cache, it is returned and copied into the local MemoryCache for future requests.
- If the data is in neither cache, a query is executed on the database, and the data is saved both in the distributed cache and in the local MemoryCache.
This design, with a bit of overhead caused by data copying, allows obtaining the data using the most performant access mode at that moment and guarantees even faster access on subsequent requests (if, for example, the data is cloned from the distributed cache to the local one).
Following the previous example:
The first request to GetCountries is made on Server 1:
- The system finds nothing in either the local cache or the distributed one, so it queries the database to retrieve the data.
- The data is saved both in the distributed/shared cache and in the local MemoryCache.
The second request to GetCountries is also made on Server 1:
- The system finds the data in the local cache and returns it, achieving the best performance.
The third request to GetCountries is made this time on Server 2:
- The system finds nothing in the local cache and checks the distributed one.
- The data is found (because it was written by the first request that hit Server 1) and is returned to the caller.
- At the same time, the data is cloned into the local MemoryCache on Server 2.
A fourth request to GetCountries hits Server 2 again:
- The system finds the data in the local cache and returns it, achieving the best performance.
Distributed Cache Implementation
For the implementation of the hybrid cache, we have two methods available:
- A third-party library: FusionCache
- From version 9 of .NET, HybridCache will be available, which mainly aims to standardize/replace IMemoryCache + IDistributedCache.
Again, we won’t show implementation details here, as they are easily accessible on the web. The purpose of this article is to explain the various issues related to implementing and using caching, with a specific focus on the .NET world, and to describe the different approaches that gradually improve the solution.
Conclusions – “All That Glitters Is Not Gold”
In conclusion, there is no guarantee that one solution is absolutely better than another. Everything should be considered in context: if it’s true that the local cache is limited in some contexts, the distributed cache solves certain problems of the former but introduces others, and the hybrid cache tries to combine the positive aspects of both mechanisms. However, it’s also true that a more “complete” mechanism is also more “complicated” to manage in terms of use/implementation and additional structures to handle (e.g., the shared server part for the distributed cache).
Therefore, in specific contexts (e.g., small, unbalanced applications), the simple and pure use of MemoryCache might be sufficient and optimal.