Introducing FusionCache

Imagine you’ve just launched the next big thing on the web. Your site is innovative, user-friendly, and now, swamped with visitors. But then, during those peak traffic hours, you notice a dreaded slowdown. Parts of your website begin to lag, tarnishing the otherwise seamless experience you worked hard to create. You think, “Much of this data barely changes. Why not cache it?”

Caching is a secret weapon in the software developer’s arsenal, adept at tackling just these kinds of issues. It’s about being smart with your resources: storing frequently accessed data in a readily available spot to cut down on those tedious wait times. This not only slashes latency but also eases the load on your databases and servers, preventing them from becoming overwhelmed during those traffic surges. In fact, caching is so fundamental that many developers could script cache retrieval in their sleep—though, let’s be honest, this code itself could send them there too.

var data = cache.Get(“key”);

if (data == null)
{
    data = await _repo.GetMyData(id);
    cache.Set("key", data, TimeSpan.FromHours(1));
}

return datal

But as you’ll soon discover, not all caching libraries are created equal. Enter FusionCache—a game-changer in the world of caching, designed to make your life easier and keep your applications running smoothly, no matter what. Ready to dive into a world where caching headaches are a thing of the past? Let’s get started.

Introducing FusionCache

FusionCache abstracts away the tedious get/set code and adds some convenient resiliency features. It also supports any cache that implements IdistributedCache, which is almost every major cache and database (in memory, Redis, Memcached, DynamoDb, MongoDb, etc).

One of the resiliency issues it handles out of the box is the factory coordination problem. Say you have an API resource that gets a product by its ID. In a typical cache usage scenario, your factory would go to the data store once you find your product is not in the cache. So if you have 100 concurrent requests for that specific product, you would hit the database 100 times to get the data. In theory, it would be sufficient to query the database once and to satisfy the other 99 requests with the retrieved data. FusionCache does just that.

One other problem it solves, that I’m really keen on, is factory timeouts. Let’s say instead of a database, your data source is third-party API. This API is sometimes very slow and occasionally unavailable. FusionCache allows you to set a timeout on your request to the data source. If you set a timeout of 100 milliseconds and your API call takes more than 100 milliseconds, FusionCache will return the cached value it has – even if it’s stale. When your API call returns, it will update the cached value with the new value so future requests will have fresh data. This will smooth out the response times of your API as you will never wait more than 100 milliseconds to retrieve this data.

Let’s dive into some code!

Code

Consider the following (crummy) code to get weather from a weather API:

internal class WeatherService
{
    const string weatherUrl = "https://api.open-meteo.com/v1/forecast?latitude=40.440624&longitude=-79.995888&current=temperature_2m,wind_speed_10m&hourly=temperature_2m,relative_humidity_2m,wind_speed_10m";

    public WeatherService()
    {
    }

    public async Task<List<Weather>> GetWeather()
    {
        return await _getWeatherFromSource();    
    }

    private async Task<List<Weather>> _getWeatherFromSource()
    {
        var results = new List<Weather>();
        using (var client = new HttpClient())
        {
            var response = await client.GetAsync(weatherUrl);
            if (response.IsSuccessStatusCode)
            {
                var content = await response.Content.ReadAsStringAsync();
                Console.WriteLine(content);
                var weather = JsonConvert.DeserializeObject<Weather>(content);
                results.Add(weather);
            }
        }
        return results;
    }
}

First, we install the FusionCache NuGet package. Next, we register FusionCache in our DI and inject an IFusionCache in our constructor:

services.AddFusionCache()
      .WithDefaultEntryOptions(new FusionCacheEntryOptions
       {
           Duration = TimeSpan.FromHours(1)
       });

Then, we wrap our call to _ getWeatherFromSource() in a FusionCache call.

    public async Task<List<Weather>> GetWeather()
    {
        var weather = await _cache.GetOrSetAsync(
            "my-cache-key",
            _ => _getWeatherFromSource());

        return weather;
    }

That’s it! FusionCache will take care of checking the cache and invoking our weather function if the cache was empty. You can pass a FusionCacheEntryOptions object to .GetOrSetAsync(), or use the default like I did above.

In my example, the first request to GetWeather() took 1300ms, while the second, cached, invocation took 1 ms.

Next, let’s take at how FusionCache can address factory timeouts, which I think is an amazing feature. In this scenario, let’s say our weather API is occasionally slow or is down. We can set a timeout value so that our application never takes more than 100ms to get weather:

services.AddFusionCache()
      .WithDefaultEntryOptions(new FusionCacheEntryOptions
       {
           Duration = TimeSpan.FromHours(1),
           // FACTORY TIMEOUT
           FactorySoftTimeout = TimeSpan.FromMilliseconds(100),
           FactoryHardTimeout = TimeSpan.FromMilliseconds(1500)
       });

The FactorySoftTimeout returns the stale cached value if our factory doesn’t respond in 100ms. By default, the factory will continue to run in the background and update the cached value once it completes.

We set a FactoryHardTimeout value so that even without a fallback value, the factory cannot take more than 1.5 seconds to execute. When a hard timeout kicks, a SyntheticTimeoutException is thrown. We will have to handle this in our code and take whatever action we deem necessary.

Next, let’s take a look at how to handle factory errors.

In order to solve this problem, we need to set a few properties on our FusionCacheEntryOptions.

services.AddFusionCache()
      .WithDefaultEntryOptions(new FusionCacheEntryOptions
       {
           Duration = TimeSpan.FromHours(1),
           // FACTORY TIMEOUT
           FactorySoftTimeout = TimeSpan.FromMilliseconds(100),
           FactoryHardTimeout = TimeSpan.FromMilliseconds(1500),

           // FAILSAFE
           IsFailSafeEnabled = true,
           FailSafeMaxDuration = TimeSpan.FromHours(2),
           FailSafeThrottleDuration = TimeSpan.FromSeconds(30)
       });

IsFailSafeEnabled should be pretty self-explanatory. It tells FusionCache we want to set a fail-safe on our value. FailSafeMaxDuration sets the maximum duration a stale value can be used even after its logical expiration. FailSafeThrottleDuration sets how long an expired value can be used because of a fail-safe condition. For example, if your factory throws an exception, FusionCache will wait 30 seconds before trying again. This will prevent your application from hammering a down API or database. There are options to add Jitter to this, too.

So far, we’ve only used an in-memory cache. Next, let’s take a look at caching in Redis

Redis

In order for our code to use Redis as our distributed cache, we have to add an extension for Redis and a FusionCache serializer. Distributed caches do not work with object instances like memory caches; they work with binary data where the value you get/set is of type byte[] so FusionCache needs a serializer to be able to integrate with them.

You first need to install these two packages:

PM> Install-Package Microsoft.Extensions.Caching.StackExchangeRedis
PM> Install-Package ZiggyCreatures.FusionCache.Serialization.NewtonsoftJson

This is what our DI with Redis looks like:

services.AddFusionCache()
        .WithDefaultEntryOptions(new FusionCacheEntryOptions
        {
            Duration = TimeSpan.FromHours(1),

            // FACTORY TIMEOUT
            FactorySoftTimeout = TimeSpan.FromMilliseconds(100),
            FactoryHardTimeout = TimeSpan.FromMilliseconds(1500),

            // FAILSAFE
            IsFailSafeEnabled = true,
            FailSafeMaxDuration = TimeSpan.FromHours(2),
            FailSafeThrottleDuration = TimeSpan.FromSeconds(30)
        })
        .WithSerializer(
            new FusionCacheNewtonsoftJsonSerializer()
        )
        .WithDistributedCache(
            new RedisCache(new RedisCacheOptions()
            {
                Configuration = "CACHE CONNECTION STRING"
            })
        );

And that’s it! Now our wonderful weather service uses Redis as its distributed cache. If we ever want to switch our cache provider, we just have to change the WithDistributedCache() DI and the rest of the code will take care of itself.

Summary

With FusionCache seamlessly integrated into your website’s backend, launching the updated version has transformed your user experience. Pages now load in the blink of an eye, delighting your customers and, yes, positively impacting your bottom line. But the benefits don’t stop at just speed. You’ve bolstered your site’s reliability, safeguarding it against the unpredictability of slow or unresponsive external services. Plus, the flexibility to switch cache providers without a major overhaul means you’re future-proofed and ready for whatever comes next.

FusionCache has proven itself to be more than just a tool; it’s a strategic partner in your website’s success story. Ready to see the difference for yourself? Dive into the FusionCache documentation and join the community of developers redefining performance and reliability. Your next version could be your best yet—all thanks to FusionCache.

Complete code example on github