Speed up catalog routing if you have multiple children under catalog

A normal catalog structure is like this: you have a few high level categories under the catalog, then each high level category has a few lower level categories under it, then each lower level category has their children, so on and so forth until you reach the leaves – catalog entries.

However it is not uncommon that you have multiple children (categories and entries) directly under catalog. Even though that is not something you should do, it happens. 

But that is not without drawbacks. You might notice it is slow to route to a product. It might not be visible to naked eyes, but if you use some decent profilers (which I personally recommend dotTrace), it can be fairly obvious that your site is not routing optimally.

Why?

To route to a specific catalog content, for example http://commerceref/en/fashion/mens/mens-shirts/p-39101253/, the default router have to figure out which content is mapped to an url segment. So with default registration where the catalog root is the default routing root, we will start with the catalog which maps to the first part of route (fashion ). How do it figure out which content to route for the next part (mens ) ? 

Until recently, what it does it to call GetChildren on the catalog ContentReference . Now you can see the problem. Even with a cached result, that is still too much – GetChildren with a big number of children is definitely expensive.

We noticed this behavior, thanks to Erik Norberg. An improvement have been made in Commerce 12.10 to make sure even with a number of children directly under Catalog, the router should perform adequately efficient.

If you can’t upgrade to 12.10 or later (you should!), then you might have a workaround that improve the performance. By adding your own implementation of HierarchicalCatalogPartialRouter, you can override how you would get the children content – by using a more lightweight method (GetBySegment)

    public class CustomHierarchicalCatalogPartialRouter : HierarchicalCatalogPartialRouter
    {
        private readonly IContentLoader _contentLoader;

        public CustomHierarchicalCatalogPartialRouter(Func<ContentReference> routeStartingPoint, CatalogContentBase commerceRoot, bool enableOutgoingSeoUri) : base(routeStartingPoint, commerceRoot, enableOutgoingSeoUri)
        {
        }

        public CustomHierarchicalCatalogPartialRouter(Func<ContentReference> routeStartingPoint, CatalogContentBase commerceRoot, bool supportSeoUri, IContentLoader contentLoader, IRoutingSegmentLoader routingSegmentLoader, IContentVersionRepository contentVersionRepository, IUrlSegmentRouter urlSegmentRouter, IContentLanguageSettingsHandler contentLanguageSettingsHandler, ServiceAccessor<HttpContextBase> httpContextAccessor) : base(routeStartingPoint, commerceRoot, supportSeoUri, contentLoader, routingSegmentLoader, contentVersionRepository, urlSegmentRouter, contentLanguageSettingsHandler, httpContextAccessor)
        {
            _contentLoader = contentLoader;
        }

        protected override CatalogContentBase FindNextContentInSegmentPair(CatalogContentBase catalogContent, SegmentPair segmentPair, SegmentContext segmentContext, CultureInfo cultureInfo)
        {
            return _contentLoader.GetBySegment(catalogContent.ContentLink, segmentPair.Next, cultureInfo) as CatalogContentBase;
        }
    }

And then instead of using CatalogRouteHelper.MapDefaultHierarchialRouter , you register your router directly

 var referenceConverter = ServiceLocator.Current.GetInstance<ReferenceConverter>();
            var contentLoader = ServiceLocator.Current.GetInstance<IContentLoader>();
            var commerceRootContent = contentLoader.Get<CatalogContentBase>(referenceConverter.GetRootLink());
            routes.RegisterPartialRouter(new HierarchicalCatalogPartialRouter(startingPoint, commerceRootContent, enableOutgoingSeoUri));

(ServiceLocator is just to make it easier to understand the code. You should do this in an IInitializationModule, so use context.Locate.Advanced instead.

This is applicable from 9.2.0 and newer versions. 

Moral of the story:

  • Catalog structure can play a big role when it comes to performance.
  • You should do profiling whenever you can
  • We do that too, and we make sure to include improvements in later versions, so keeping your website up to date is a good way to tune performance.

Refactoring Commerce catalog code, a story

It is not a secret that I am a fan of refactoring. Clean. shorter, simpler code is always better. It’s always a pleasure to delete some code while keeping all functionalities: less code means less possible bugs, and less places to change when you have to change.

However, while refactoring can bring a lot of enjoying to the one who actually does it, it’s very hard to share the experience: most of the cases it’s very specific and the problem itself is not that interesting to the outside world. This story is an exception because it might be helpful/useful for other Commerce developer.

Continue reading “Refactoring Commerce catalog code, a story”

Commerce batching performance – part 2: Loading prices and inventories

UPDATE: When looked into it, I realize that I have a lazy loading collection of entry codes, so each test had to spent time to resolve the entry code(s) from the content links. That actually costs quite a lot of time, and therefore causing the performance tests to return incorrect results. That was corrected and the results are now updated.

In previous post we talked about how loading orders in batch can actually improve your website performance, and we came to a conclusion that 1000-3000 orders per batch probably yields the best performance result.

But orders are not the only thing you would need to load on your website. A more common scenario is to load prices and inventories for product. So If you are displaying a product listing page, it’s quite common to load prices and inventories for all products in that page. How should it be loaded?

Continue reading “Commerce batching performance – part 2: Loading prices and inventories”

Commerce batching performance – part 1: Loading orders

One of best practices for better performance – not just with Commerce or Episerver Commerce, is to batch your calls to load data. In theory, if you want to load a lot of data, loading by both end will be problematic: if you load each record one by one, the overhead for opening the connection and retrieve data will be too much. But if you load all of them, then it is likely that you will end up with either time out exception in database end, or out of memory exception in your application. The better way is to of course, loading them by smaller batch: either 10, 20, or 50 records at one and repeat until the end.

That is the theory, but is it really better in practice? And if it is, which size of batch works best? As they usually say, reality is the golden test for theory, so let’s do it.

Continue reading “Commerce batching performance – part 1: Loading orders”

Speed up your catalog indexing performance – part 2

Almost two years ago I wrote part 1 here: https://vimvq1987.com/speed-catalog-entries-indexing/ on how to speed up your catalog indexing performance. If you have a fairly big catalog with frequent changes, it might take longer time than necessary to build the index incrementally. (Rebuild index, in other hands, just delete everything and rebuild from scratch, so it is not affected by the long queue in ApplicationLog). I have seen some cases where rebuilding the entire index, is actually faster than waiting for it to build incrementally.

The tip in previous blog post should work very well if you are using anything lower than Commerce 11.6, but that is no longer the case!

Continue reading “Speed up your catalog indexing performance – part 2”

A curious case of memory dump diagnostic: How Stackify can cause troubles to your site

It’s been a while since my last blog post, and this time it will be fun time with windbg. One of DXC customers has been having problem with instances of their site hang and restart. We at Episerver takes the availability and performance of DXC sites very seriously, therefore I was asked to help diagnosing the problem. It’s something I’d like to do and to learn anyway, so game is on.

The golden standard for looking at the kind of those problems is still the tried and trusted Windbg. With the help from the Managed Services Engineer, I was able to access several memory dumps from the site. Most of them were taken when the site stalled, which is usually the most useful moment because it should contain the most critical piece of information why the stall happened.

Continue reading “A curious case of memory dump diagnostic: How Stackify can cause troubles to your site”

Episerver Commerce: A problem-solution approach is now draft complete

6 months ago I announced that I was working on a second book on Episerver Commerce. Unlike the first one where I tried to provide a systematic approach about the framework, this book focuses on the bite-size recipes, each one is a solution to solve a real world problem.

It’s one problem solved, at a time.

The world is big and the possibilities are endless, there might be more than one solution to a problem, but I tried to give the best one to my knowledge, experience. I hope that by talking about it, I also give information about how the API:s work, what to use, what should be avoided. what can be better, what are the best practices … , so you can develop solutions to your own problems, or even, come up with better solutions than mine.

If you manage to do that, don’t forget to share your solutions as a blog post on World forums, to help the ones come after you!

The cover I spent a day one. At least it's colorful, right?
People asked me: why foods? Well, because it’s a recipe book.

And now I’m proud to announce that the book is draft complete. Which means I’m happy with the content of the book. I will not stop there, new recipes and/or improvements will be brought from time to time, but if you want to buy the book, this is the time.

As always, writing a book is not an easy task – even with a smaller book like this. I would not have done it without the help and supports from the community. I would like to thank buyers who have bought the book, even from the very early days. I also want to thank people on World who gave me ideas and suggestions for recipes.

Not every news is good news, however. Consider the time and effort I have put into the book, and the new pricing policies of Leanpub (they increased their cut recently from 10% + 50c to 20%), I will increase the asking price of the book to $24.99, from $19.99. If you bought the book, you own the book and there is no extra cost for you. But if you are buying the book now, you would have to spend more money…

But …

… a Commerce book announcement is incomplete without some discount.

Now it’s something special: The book is dedicated to my daughter, who was born today! In the honor of me becoming a father – and to welcome my baby girl to the world, I’m offering 50% off for today: https://leanpub.com/epicommercerecipes/c/HelloWorld which is valid until the end of July 31st, GMT time.

I hope you find something useful from my book, and I welcome all new recipe suggestions and improvements!

Getting all non published variations

I got a question from a colleague today: A customer has multiple languages (8 of them). They need to make sure all variants are published in all languages. That is of course a reasonable request, but there is no feature builtin for such requirement. But good news is that can be done with ease. If you want to try this as practice, go ahead – I think it’s a good exercise for your Episerver Commerce-fu skills.

To do this task, we need the snippet to traverse the catalog from here https://leanpub.com/epicommercerecipes/read_sample

Continue reading “Getting all non published variations”

Watch out for Singletons

If you are a seasoned Episerver developer, you should (and probably, already) know about the foundation of the framework: dependency injection. With the Inversion of control framework (most common, Structuremap, but recent versions of Framework allow much more flexible options), you can easily register your implementations, without having to manually create each and every instance by new operator. Sounds great, right? Yes it is.

And Episerver Framework allows you to make it even easier by this nice ServiceConfiguration attribute:

[ServiceConfiguration]
public class MyClass 
{
}

so your class will be automatically registered, and whenever you need an instance of MyClass, IoC framework will get the best instance for you, automatically, without breaking a sweat. Isn’t it nice? Yes it is.

But I guess you also saw this from place to place

[ServiceConfiguration(LifeCycle = ServiceInstanceScope.Singleton)]
public class MyClass 
{
}

So instead of creating a new instance every time you ask it to, IoC framework only creates the instance once and reuses it every time. You must think to yourself: even nicer, that would save you a lot of time and memory.

But is it (nicer)?

You might want to think again.

Singleton means one thing: shared state (or even worse, Global state). When a class is marked with Singleton, the instance of that class is supposed to be shared across the site. The upside, is, well, if your constructor is expensive to create, you can avoid just that. The downside, of course, shared state can be a real b*tch and it might come back to bite you. What if MyClass holds the customer address of current user. If I set my address to that, and because you get the same instance, you’ll gonna see mine. In Sweden it’s not a real problem as you can easily know where I live (even my birthday if you want to send a gift, or flowers), but I guess in the bigger parts of the world that is a serious privacy problem. And what if it’s not just address?

And it’s not just that, Singleton also make things complicated with “inherited singleton”. Let’s take a look at previous example. Now we see Singleton is bad, so let’s remove that on our class. But what if other class depends on our little MyClass:

[ServiceConfiguration(LifeCycle = ServiceInstanceScope.Singleton)]
public class MyOtherClass 
{
   private MyClass _myClass;
   public MyOtherClass(MyClass myClass)
   {
        _myClass = myClass;
   }
}

Now I hope you see where the problem is. One instance of MyOtherClass is shared inside the side. And it comes with an attached MyClass instance. Even if you don’t intend to, that MyClass instance will also be shared. Same problem after all.

Singleton was there to solve one problem (or two), but it can also introduce other problems if you don’t really think about if your instance should be shared or not. And not just your class, the classes which have dependency on your class as well.

And it’s not just Singleton , HttpContext and Hybrid might also subject to same problem, but to a lesser extend. Any lifecycle that shares state should be considered: if you really need it and what you are sharing.

Lifecycle is hard, but it can also work wonder, so please take your time to make it right. It’s worth it.

Multiple catalogs: Without catalog name(s)

My previous blog posts about multiple catalogs in multiple sites setting here and here has helped one customer (hopefully more) to address their problem. They tried the approach and it works for them. Now they came back and ask if they can remove the catalog name from the URL entirely. Is that something that can be done. Fortunately, yes.

Before moving on, let’s be clear: this is obviously custom development and is not officially supported by Episerver. I.e. it’s neither tested and documented. Normally you would have to go to Expert Services for such requirement. For this blog, I provide those samples for free (you don’t even have to buy my book 😉 ), but remember I can only do that much. You have to test to see if it works for you (I of course do the basic testing), and if you run into problems later, I might not be able/available to help.

Now get back to the problem. Previously the router looks like this

    
protected override CatalogContentBase FindNextContentInSegmentPair(CatalogContentBase catalogContent, SegmentPair segmentPair,
        SegmentContext segmentContext, CultureInfo cultureInfo)
    {
        if (catalogContent.ContentType == CatalogContentType.Root)
        {
            CatalogContent definedCatalogContent;
            var definedCatalogLink = _contentLoader.Get(RouteStartingPoint).CatalogLink;
            if (_contentLoader.TryGet(definedCatalogLink, cultureInfo, out definedCatalogContent))
            {
                return definedCatalogContent;
            }
        }
        return base.FindNextContentInSegmentPair(catalogContent, segmentPair, segmentContext, cultureInfo);
    }

And it works great with  http://commerceref/en/fashion/mens/mens-shoes/p-36127195/ or http://commerceref/en/products/mens/mens-shoes/p-36127195/. But now they want to completely remove the catalog name from the url. They can’t make it works with http://commerceref/en/mens/mens-shoes/p-36127195/.

Why? Because the router is returning the catalog for /mens/ segment. In next segment (mens-shoes), we fallback to the default implementation, which can’t find any matching content as children of that definedCatalogContent .

The solution: Because we want the first-level category to appear as direct child of the catalog root, we would have to return it ourselves. Instead of returning the catalog, we return the first node that matches /mens/ segment.

        protected override CatalogContentBase FindNextContentInSegmentPair(CatalogContentBase catalogContent, SegmentPair segmentPair,
            SegmentContext segmentContext, CultureInfo cultureInfo)
        {
            if (catalogContent.ContentType == CatalogContentType.Root)
            {
                CatalogContent definedCatalogContent;
                var definedCatalogLink = _contentLoader.Get<StartPage>(RouteStartingPoint).CatalogLink;
                if (_contentLoader.TryGet<CatalogContent>(definedCatalogLink, cultureInfo, out definedCatalogContent))
                {
                    var nodes = _contentLoader.GetChildren<NodeContent>(definedCatalogLink);
                    return nodes.FirstOrDefault(n => n.RouteSegment.Equals(segmentPair.Next, StringComparison.OrdinalIgnoreCase));
                }

                return null;
            }
            return base.FindNextContentInSegmentPair(catalogContent, segmentPair, segmentContext, cultureInfo);
        }

And it just works:

Now that’s just one part of the solution. We still need to make sure outgoing URLs respect that. I’ll leave that to you, as practice.