Can Episerver Commerce support our catalog size?

One of the questions customers usually raise during evaluation of Episerver Commerce is : “Can it support our catalog size? We have (a very big number of ) SKUs. Will it work?”

Definitely, maybe.

I’ve seen “big” catalogs. Some very big. Million and more products.

The correct term would be “Entries”, as in a standard implementation your catalog can contain products, variants (of those products, or standalone ones), bundle and package. However for the sake of simplicity, we will refer to them as “products”. So when we talk about 2 million products, it is the 2 million entries. (you can have 100k products, 1.800k variants and 100k packages)

Theoretically, Episerver Commerce can support up to 512 millions 1 billion of entries (!), so you can have pretty much anything in your catalog until you reach a hard technical limit. Just for comparison, Amazon.com, which is arguably the biggest eCommerce site on the world, has about 500 millions SKU(s) in 2015. But the number of entries is not everything. There are several factors which determine your catalog “size”. the number of entries is an important factor, but there are several other factors as well.

• The number of languages in your catalog. As you already know (if you haven’t known, the first chapter of my book is available for free), each language requires additional data on CatalogContentProperty and ecfVersionProperty table. If you are having a 100k entries catalog, but with 10 languages, then your data read/write will be almost as much a 500k entries language, but with only 2 languages.
• The number of metafields in your entries. Each metafield will need a row in above tables. This also applies to the density of metafields – if you have 300 metafields per entry (trust me, it’s a thing), but 250 of them are null, then it does not take much data processing than if you have only 50 metafields but all of them are not null! Of course, having too many metafields expose another (performance) problem: each content will have to carry the information of that metafield, even if it’s null. We will discuss on how to improve that later.
• The number of versions you want to have. By default, Commerce stores 20 versions of each content, but you can change that via UIMaxVersion setting. General speaking, too many versions can affect performance of Catalog UI (it will have to load more data), so unless you have to keep every version, it’s a good practice to have an upper limit. If you don’t care about versioning at all (For example if you use external PIM to manage product information), you can even disable version for catalog content, and in most of the cases, it improves performance.
• What is the the product update flow? Will it be updated solely by the merchandisers via Catalog UI, or by importing export from an external PIM system, or a combination of both
• What is the rate/frequency of product update? Is it monthly, weekly, daily, hourly or continuous?  How many products are updated each time, on average?

[quads id=2]

Now you have an idea of how to look at “Catalog size”, it’s time to consider more pragmatic question: how to have adequate (or even better, good) performance with my catalog?

• Use latest version – I have been advocating for this, and I don’t think I can emphasize this enough: for 99% of the cases, the newer version is also faster. The catalog performance improvements we made in Commerce 9 were big, but we didn’t stop there. When we spot a chance to make things run faster, we do it, and since Commerce 9, we have added dozens of optimizations and tuning, and the results, in some areas, are quite astonishing. Even if you don’t need new features, you’ll get better performance, for free.
• The structure of your catalog can affect performance – this has been discussed in depth in my book (in chapter 1, which is free).
• There are several best practices to improve catalog performance (and they apply to other parts as well): don’t load when you don’t really need, and cache smartly, etc. I wrote about those guidelines here and here (and will add more in the future).
• Profilers are your friends and a few hours with profiles can reveal a lot about your code (and how you can do better).
• When you have a performance problem, tell us. Our developer support services will be happy to help you to collect information needed, and we – as development team – will be happy to work on that.
• The general rule is: a bigger catalog needs a more powerful server, especially for SQL Server instance. It apparently needs to do more work, read more from disk(s) and cache more in the memory. The application instance also needs to have more cache. So plan accordingly. You can’t expect to have a great performance when you run a very big catalog on a “slow” server – if performance is your priority – throw hardware at it will certainly help (of course if you’ve done all above points)