Episerver Commerce performance optimization – part 2

Or lock or no lock – that’s the question.

This is the second part of the series on how can you improve the performance of Episerver Commerce site – or more precisely, to avoid the deadlocks and 100% CPU usage. This is not Commerce specific actually, and you can apply the knowledge and techniques here for a normal CMS site as well.

It’s a common and well-known best practice to store the slow-to-retrieve data in cache. These days memory is cheap – not free – but cheap. Yet it is still much faster than the fastest PCIe SSD in the market (if your site is running on traditional HDD, it’s not even close). And having objects in cache means you won’t have to open the connection to SQL Server, wait for it to read the data and send back to you – which all cost time. And if the object you need is a complex one, for example a Catalog content, you will also save the time needed to construct the object. Even if it’s fast, it is still not instantaneous, and it will cost you both memory and CPU cycles. All in all – caching is the right way to go. But how to get it right?

One common mistake for to have no lock when you load the data for the first time and insert it into cache.

This code is quite easy to write and understand, but it contains a serious flaw:

public MyData LoadData(int id)
{
   var dataFromCache = CacheManager.Get(CreateCacheKey(id)) as MyData;
   if (dataFromCache != null)
   {
        return dataFromCache;
   }
   var data = LoadFromDb(int id);
   _cache.Insert(CreateCacheKey(id), data, new CacheEvictionPolicy(...));
   return data;
}

The problem is there might be multiple threads which hit the code at the same time (it’s not required to be exactly the same time, it just needs to hit during the time the first thread is loading data from database and constructing the object and inserting that to the cache – which can be from a few hundred of milliseconds to a few seconds). If you have things that appear on almost every page, say, the menu, then they are more vulnerable to this mistake. For some reasons, if the cache is cleared (perhaps one of the item was updated, or just because the site is starting up and the cache is empty), then all of the threads will be hitting the database, and probably competing each other on the same data. This will easily make the CPU usage/Database disk usage spike, and the site will stall for a while until things settle down (if you are lucky enough to not end up in deadlocks or auto site restart).

The obvious solution is to add a lock here, but beware! You might be introducing another issue:

private static object _lockObject = new object();

public MyData LoadData(int id)
{
   var dataFromCache = CacheManager.Get(CreateCacheKey(id)) as MyData;
   if (dataFromCache != null)
   {
        return dataFromCache;
   }
   lock (object)
   {
      var data = LoadFromDb(int id);
      _cache.Insert(CreateCacheKey(id), data, new CacheEvictionPolicy(...));
      return data;
   }
}

Can you spot the problem?

Lacking of lock is definitely an issue. But locking too much is also a problem. So here if we are getting the object with id = 1, we effectively block the getting data for all other id. Every call to LoadData must wait until MyData with id = 1 is loaded and stored to cache. This can also be a huge bottleneck.

Locking, either too little (read, none), or, too much, can both have performance implications on your site.

So what is the correct solution?

If you are using CMS 10+ (And you should!), and if you are using ISynchronizedObjectInstanceCache you’re in luck. There are new extensions with allow you to read from/insert to cache with confident, as it has lock builtin.

                _cache.ReadThrough(
                    CreateCacheKey(id),
                    () =>
                    {
                        return LoadFromDb(id);
                    },
                    x => new CacheEvictionPolicy(...),
                    ReadStrategy.Wait);

ReadStrategy.Wait is the flag you need. It allows only on thread to read the CreateCacheKey(id) (and therefore the action to load from database as well), while other threads are put to wait. When the thread is done, other threads will be signaled and they will happily load the data from cache.

If you are not using CMS 10, or if you are not using ISynchronizedObjectInstanceCache, you are not all of out luck, but it’ll require more work, of course. The idea here is to have a ConcurrentDictionary<int, object>. For each id, you can have a lock object for it. That way, you can still have the required lock, but also not block the loading of other id.

I’ll leave the implementation detail for you 😉