Can Episerver Commerce support our catalog size?

One of the questions customers usually raise during evaluation of Episerver Commerce is : “Can it support our catalog size? We have (a very big number of ) entries. Will it work?”

The answer is (of course, as always): It depends!

I’ve seen “big” catalogs. Some big enough in number – 1 million entries catalogs are not very uncommon, and some are even (much) bigger. Theoretically, Episerver Commerce can support up to 512 millions 1 billion of entries (!), so you can have pretty much anything in your catalog until you reach a hard technical limit. Just for comparison, Amazon.com, which is arguably the biggest eCommerce site on the world, has about 500 millions SKU(s) in 2015. But the number of entries is not everything. There are several factors which determine your catalog “size”. the number of entries is an important factor, but there are several other factors as well.

Continue reading “Can Episerver Commerce support our catalog size?”

The Catalog UI trade-off: performance or better UI

I supposed this is a well known feature, but I was asked more than once about it, so it’s better to write something here to clarify the confusions.

If you have some very, very big catalogs, you probably have seen this “notification” in Catalog UI

By default, the Catalog UI groups a product and its variations in a parent-children view (they are not exactly parent-children, by the way). However, to do that, it needs to know about all the entries in that specific category. If it’s a small category, it should be no problem, but if it’s big one, then it’s inevitable slow. The lazy loading which the catalog content list only loads the contents when you scroll to them is not helping in this matter. Moreover, the grouping introduces an overhead for the UI, and having too many groups can severely affect the performance. Trust me, you won’t like a sluggish UI.

This improvement was introduced way back – 7.11 if I remember correctly – thanks to my colleague Magnus Rahl. To this day it’s still valuable – the performance was improved – but not that much to remove the threshold completely (And the improvement to the catalog versioning in Commerce 9 should have nothing to do with this).

When you see this notification, and if you’re unhappy with it, you have two (primary) options: Either to sub-categorize your category – i.e. introduce sub categories so each will have a smaller number of entries. Or increase the value of threshold.

Each approach has its own disadvantages. Sub-categorizing might break your SEO, while the second approach will undoubtedly effect the UI performance. Your call!

Now – the tricky part – which number to configure in SimplifiedCatalogListingThreshold setting. Obviously, it must be greater than the biggest number of entries in a category. But how to obtain that number? I’ve seen the confusion to raise that value to 3000, 5000, or even 10000 and it’s still not working. No, you can’t guess, you have to know for sure.

One simple option is to look at Commerce Manager Catalog Management. There is a small text in right corner of the list which shows the number of entries in that category (No, it’s not available in the Catalog UI, but I assume it would be helpful?)

 

The nuke option is to look at the database. Usually we recommend to avoid manipulate the database directly, as it can be dangerous – but here is a little code which only queries data (so practical harmless)

Now you know the biggest number of the entries in a category – just change the threshold in setting. Try it and see if the UI Performance is acceptable to you.

Find indexing job + HierarchicalCatalogPartialRouter: A note

I ran into this problem recently and while in the end it’s quite simple issue (Everything is simple if we understand it, right?), it costed me quite many hairs in the process – as it involved debugging with 3 solutions – Find.Commerce (where the problem appears), Commerce (where the router does the work), CMS Core (where the routers are handled). It was both fun, and confusing.

The problem as a customer has this code in an initialization module:

Continue reading “Find indexing job + HierarchicalCatalogPartialRouter: A note”

Episerver Commerce performance optimization – part 1

This is a first part of a long series (which have no planned number of parts) as the lessons I learned during trouble shouting customers’ performance problems. I’m quite of addicted to the support cases reported by customers, especially the ones with performance problems. Every time I jump into such support case, I’ll be with less hairs, but I also learn some new things:  Implementations are different from cases to cases, but there are some common mistakes which will hurt your website performance. This series will try to point out those mistakes so you get your performance gain, for (almost) free:

Mistake 1: Loading to much content

It’s easy to load contents, especially with the new content APIs. Given an universal ContentReference, you can load a content with a simple line of code. By default, the loaded content is cached, so you might think it’s cheap, or even free to load a content. Think again.

Continue reading “Episerver Commerce performance optimization – part 1”

Speed up your catalog entries indexing

Even with the raising popularity of FindCommerce, I suspect that many Commerce customers are still using the search provider system, as it comes with an undeniable benefit: It can be used within Commerce Manager. And while I suspect a majority of you have the eventual indexing turned on (aka the entry will be indexed as soon as it’s changed), many still index the entries on demand. Aka manually, or periodically via scheduled job.

We received a report from a customer recently as the indexing timeout, as he was indexing a large amount of entries (about 250.000 entries on 8 catalogs). When we looked into the problem, we discover a possible improvement which is almost free for you.

Continue reading “Speed up your catalog entries indexing”

EntryContentBase, MetaObject, CatalogEntryDto, Entry: which should you use?

It can be pretty confusing for new Commerce developers to understand how to work effectively with entries in Commerce. There are many things which represent the same concepts, however they are different and their APIs are not compatible. So which is which and what should you use?

Which is which

  • CatalogEntryDto is the DataSet to represent one or more entries (CatalogEntryDto can of course be empty). Beside the basic information like Name, Code, or MetaClassId, depends on how did you load it, a CatalogEntryDto can contain information about the assets, the associations or the variations (you can specify what to load by using CatalogEntryResponseGroup parameter. CatalogEntryDto, however, does not contain information of the metadata system of an entry – for example, if you add a metafield named “Description” to your entry metaclass – that is not available in a CatalogEntryDto.
    CatalogEntryDto can be loaded or saved by ICatalogSystem methods.

Continue reading “EntryContentBase, MetaObject, CatalogEntryDto, Entry: which should you use?”

Catalog Search APIs are for editing only!

If you are using Catalog Search APIs for any customer-facing features, you are doing it wrong!

I have seen this problem a couple of times – the search feature on the site is “dead” – it is very slow, and the log file is usually filled with dead lock or timeout error. As it turns out, the search feature was implemented by Catalog Search APIs, which is a big no-no.

To be clear, there are two builtin APIs related to searching in Episerver Commerce: the “fast” one, which can be done via SearchManager, ISearchCriteria and ISearchResults, is the SearchProvider APIs. It’s the indexed search (strictly speaking, you can make it not “indexed”, but that’s beside the point), and the actual search functions will be provided by providers, like LuceneSearchProvider, Solr35SearchProvider, or FindSearchProvider.

Continue reading “Catalog Search APIs are for editing only!”

The hidden danger of dot (Or why should your metafield not contain . in the name)

A dot (.) – it is harmless. What harm can it do, it looks pretty innocent.

And yet it can break your Catalog UI.

Psyduck, from Pokemon Go
A dot can look pretty harmless and innocent, just like a Psyduck. Frankly, its eyes are also two dots.

Catalog UI relies on the Shell UI from CMS to render properties and such. Shell UI, in its hands, needs to know about the metadata of the properties. When you have dot in the metafield names, the MetaDataPropertyMapper will create an Property with that name on site start up. And then when you open All properties mode, Shell UI will request your content type models, and CMS Core will happily return those properties.

Continue reading “The hidden danger of dot (Or why should your metafield not contain . in the name)”

Episerver Commerce MetaDictionary internals

This is an excerpt from my book – Pro Episerver Commerce – which is now already 2/3 complete.

Dictionary types.

Previously we discussed on how properties work with catalog content. However – if you have dictionary types in your MetaClasses, they will work differently. In this section we will examine these special data types – this applies to Order system metaclasses as well.

As we all know – there are three types of dictionary in Episerver Commerce:

  • Single value dictionary: editor can select a value from defined ones.
In Commerce Manager, you can create new metafield with type of Dictionary, but without "Multiline" option
In Commerce Manager, you can create new metafield with type of Dictionary, but without “Multiline” option

Single value dictionary type is supported in the strongly typed content types – you’ll need to define a property of type string, with backing type of typeof(PropertyDictionarySingle)

  • Multi value Dictionary: editor can select multiple values from defined ones. The only different from Single value dictionary is it has the “Multiline” option enabled.

Continue reading “Episerver Commerce MetaDictionary internals”

Import catalog: beware of warning messages

Recently I worked with a support case from a customer. They complained that the catalog import is too slow for them, it took them 6-7 minutes to import 200 entries. Their catalog is about 50k entries, which is not particularly big, but not small neither, and with that rate, it does mean they will have to wait for days for the catalog imported. This is of course not good and I jumped in when I became available. (In the end, I like performance tuning so much that I find it hard to refuse a case like this)

I did try to import their catalog on my development machine – a relatively powerful desktop with Intel Core i7 4790 CPU, 16GB of RAM and 256GB SSD on Windows 10/.NET 4.5.2. Everything went quite well – I achieved a rate of 200 entries per 9-10 seconds, which is, IMO, more than enough. (And this is on Commerce 9, on Commerce 8 then I would suspect it’s 3-4 times slower).

After some conversations with Episerver’s developer support team, it seems that the slowdown only happens with the re-import. I tried it again with re-import. Hmm, something is wrong here. The import is much slower and it eventually stops at some points. The debugger shows that it starts quite well, even faster than a new import (200 entries per 3-4 seconds, a very recent version of Commerce made an improvement of skipping the entries which are not changed to speed up the import), but it slows down by time. Profiling does not show anything obvious, most of the time was spent with loading the MetaObject:s – which is kind of expected.

I was starring at the import tab, and wondering what can be wrong, and suddenly I realized the answer is what I was looking at:

Continue reading “Import catalog: beware of warning messages”