Category: Commerce

Get contact by email address

If you are using Episerver Commerce (or should I say, Optimizely B2C Commerce), you will, at some point, need to get the contact by an email address. That sounds like an easy enough task, until you realize that the class to manage customer contacts –CustomerContext has no such method. You will need to find another way, and this is one way you can do it

CustomerContact contact = customerContext.GetContacts().Where(m => m.Email == email)?.FirstOrDefault();

Of course this is not the optimal – avoid it if you can. First of all it loads a lot of contact just to find one. Also while it looks like you are getting all contacts (which is of course something to avoid), you are only get the first 1000 contacts by default, so the code above would return inaccurate result.

Is there a better way?

Yes of course.

Contact was built on “Business Foundation” – think of it as an ORM with extensions. Business Foundation allows great flexibility, with a few caveats. This is how you can find contact by email address:

            try
            {
                var filterEl = new FilterElement("Email", FilterElementType.Equal, email);
                return
                    BusinessManager.List(ContactEntity.ClassName, new[] { filterEl })
                        .OfType<CustomerContact>()
                        .FirstOrDefault();
            }
            catch (ObjectNotFoundException)
            {
                //Safe guard
                return null;
            }

BusinessManager does not have a Get method, so we are use List instead. Note that Email is supposed to be unique, so this should not have any down side with performance (see note below).

This is not limited to email, or to contact. You can find contacts by other properties, or get an Organization with same technique.

It is very important to note, however, this need a proper index on Contact tables, to make sure you are not killing your database.

Get exported Personalization catalog feeds

There are cases that you want to get your Personalization catalog feeds in zip format, maybe to make sure the customizations you have done are there (and are exactly what you want to), or you need to send them to developer support service for further assistance (like why your catalog feeds are not properly imported). Theoretically you can log in as an admin, and go to https://<yoursite>/episerverapi/downloadcatalogfeed to download the feed. But there are few problems with that.

First of all you might have more than one catalog, and the link above only allows you to download the latest one. Secondly it might fail to give you any catalog feed at all (With “There is no product feed available. Try run the scheduled job first.” error, even if you already ran the Export Catalog Feed job. There is currently no known fix for that). Is there a way to simply get the data?

Yes, there is. It’s not fancy, but it works. All your catalog feeds will be put in appdata\blobs\d4a76096689649908bce5881979b7c1a folder, so just go there and grab the latest ones. appdata is the path defined in your <episerver.framework>\<appData> section.

What if you are running on Azure? It is the same “folder”. Use Azure Storage Explorer to locate the blob. If you are running on DXP, get in touch with developer support service and they’d be happy to help.

I originally planned to write a small tool to easily download the catalog feeds, but it turned out the Episerver Blob APIs have no way to list content of a container, so a manual, simple way is better this time.

Don’t let order search kill your site

Episerver Commerce order search is a powerful feature. My colleague Shannon Gray wrote about is long ago https://world.episerver.com/blogs/Shannon-Gray/Dates/2012/12/EPiServer-Commerce-Order-Search-Made-Easy/ , and I myself as well https://world.episerver.com/blogs/Quan-Mai/Dates/2014/10/Order-searchmade-easy/

But because of its power and flexibility, it can be complicated to get right. People usually stop at making the query works. Performance is usually an after thought, as it is only visible on production environment when there are enough requests to bring your database to its knees.

Let me be very clear about it: during my years helping customers with performance issues (and you can guess, that is a lot of customers), order search is one of the most, if not the most common cause of database spikes.

Trust me, you never want to your database looks like this

As your commerce database is brought to its knees, your entire website performance suffers. Your response time suffers. Your visitors are unhappy and that makes your business suffer.

But what is so bad about order search?

Order search allows you to find orders by almost any criteria. And to do that, you often join with different tables in the database. Search for orders with specific line items? Join with LineItem table on a match of CatalogEntryId column. Search for orders with a specific shipping method? Join with Shipment table on a match of ShippingMethodId etc. etc. SqlWhereClause and SqlMetaWhereClause of OrderSearchParameters are extremely flexible, and that is both a cure, and a curse.

Let’s examine the first example in closer details. The query is easy to write. But don’t you know that there is no index on the CatalogEntryId column? That means every request to search order, end up in a full table scan of LineItem.

There are two bad news into that: your LineItem table usually have many rows already, which makes that scan slow, and resource intensive. And as it’s an ever growing table, the situation only gets worse over time.

That is only a start, and a simple one, because that can be resolved by adding an index on CatalogEntryId , but there are more complicated cases when adding an index simply can’t solve the problem – because there is no good one. For example if you search for orders with custom fields, but only of type bit . Bit is essentially the worst type when it comes to index-ability, so your indexes will be much less effective than you want it to be. A full table scan will likely be used.

In short:

Order search is flexible, and powerful. But, “With great power come great responsibility”. Think about what you join on your SqlWhereClause and SqlMetaWhereClause statements, and if your query is covered by an index, or if adding an index will make senses in this case (I have a few guidelines here for a good index https://vimvq1987.com/index-or-no-index-thats-the-question/). Or if you can limit the number of the orders you search for.

Your database will thank you, later.

Iterate through all carts/orders

While it’s not a common task to do, you might want to iterate through all carts, or all carts with a specific criteria. For example, you might want to load all carts that have been last modified for more than 1 week, but less than 2 weeks, so you can send a reminder to the buyer (Ideas on the implementation of that feature is discussed in my book – Episerver Commerce A problem solution approach). Or you simply want do delete all the carts, as asked here https://world.episerver.com/forum/developer-forum/Episerver-Commerce/Thread-Container/2021/1/removing-all-active-carts/ . How?

In previous versions of Episerver Commerce, what you can do is to use OrderContext to find orders and carts using the Order search API. However that does not work with non default implementations, such as the serializable carts. A better way would be to use the new abstraction – IOrderSearchService. It takes a OrderSearchFilter which allows things like paging to be set, and returns an OrderSearchResults<T> which contains the matching collection of carts or orders, and the total count. When you have a lot of carts or orders to process, it’s nice (even important) to let the end users know the progress. However, it’s also important to know that, counting the matching carts/orders can be very expensive, so I’d suggest to avoid doing it every time.

The pattern that you can use is to do a first round (which do not load many carts, except one), to load total count. For subsequent calls you only load the carts, but set ReturnTotalCount to false to skip loading the total count. If you want to delete all the carts (for fun and profit, obviously do not try this on production, unless if this is exactly what you want), the code can be written like this, with _orderSearchService is an instance of IOrderSearchService, and _orderRepository is an instance of IOrderRepository

            var deletedCartsTotalCount = 0;
            var cartFilter = new CartFilter
            {
                RecordsToRetrieve = 1,
                ExcludedCartNames = excludedCartNames,
                ReturnTotalCount = true
            };

            //Get the total carts for status update.
            var orderSearchResults = _orderSearchService.FindCarts(cartFilter);
            var totalCount = orderSearchResults.TotalRecords;
            cartFilter.ReturnTotalCount = false;
            cartFilter.RecordsToRetrieve = 100;

            var cartLoaded = 0;
            do
            {
                var searchResults = _orderSearchService.FindCarts(cartFilter);
                foreach (var cart in searchResults.Orders)
                {
                    _orderRespository.Delete(cart.OrderLink);
                    deletedCartsTotalCount++;
                }

                OnStatusChanged($"Deleted {deletedCartsTotalCount} in {totalCount} carts.");
                cartLoaded = searchResults.Orders.Count();
            }
            while (cartLoaded > 0);

A few notes:

  • You might or might not exclude carts based on name
  • CartFilter has a few filters that you can play with, not just names.

Don’t use AspNetIdentity FindByEmailAsync/FindByIdAsync

Or any of its equivalent – FindByEmail/FindById etc.

Why?

Reason? It’s slow. Slow enough to effectively kill your database, and therefore, your website.

If you want dig into the default implementation (which is using EntityFramework), this is what you end up with, either if you are using FindByEmailAsync, or its synchronous equivalent FindByEmail

    public virtual Task<TUser> FindByEmailAsync(string email)
    {
      this.ThrowIfDisposed();
      return this.GetUserAggregateAsync((Expression<Func<TUser, bool>>) (u => u.Email.ToUpper() == email.ToUpper()));
    }

It finds user by matching the email, but not before it uses ToUpper in both sides of the equation. This is to ensure correctness because an user can register with “[email protected]” but then try to login with “[email protected]”. If the database is set to be CS – case sensitive collation, that is not a match.

That is fine for C#/.NET, but it is bad for SQL Server. When it reaches database, this query is generated

(@p__linq__0 nvarchar(4000))SELECT TOP (1) 
    [Extent1].[Id] AS [Id], 
    [Extent1].[NewsLetter] AS [NewsLetter], 
    [Extent1].[IsApproved] AS [IsApproved], 
    [Extent1].[IsLockedOut] AS [IsLockedOut], 
    [Extent1].[Comment] AS [Comment], 
    [Extent1].[CreationDate] AS [CreationDate], 
    [Extent1].[LastLoginDate] AS [LastLoginDate], 
    [Extent1].[LastLockoutDate] AS [LastLockoutDate], 
    [Extent1].[Email] AS [Email], 
    [Extent1].[EmailConfirmed] AS [EmailConfirmed], 
    [Extent1].[PasswordHash] AS [PasswordHash], 
    [Extent1].[SecurityStamp] AS [SecurityStamp], 
    [Extent1].[PhoneNumber] AS [PhoneNumber], 
    [Extent1].[PhoneNumberConfirmed] AS [PhoneNumberConfirmed], 
    [Extent1].[TwoFactorEnabled] AS [TwoFactorEnabled], 
    [Extent1].[LockoutEndDateUtc] AS [LockoutEndDateUtc], 
    [Extent1].[LockoutEnabled] AS [LockoutEnabled], 
    [Extent1].[AccessFailedCount] AS [AccessFailedCount], 
    [Extent1].[UserName] AS [UserName]
    FROM [dbo].[AspNetUsers] AS [Extent1]
    WHERE ((UPPER([Extent1].[Email])) = (UPPER(@p__linq__0))) OR ((UPPER([Extent1].[Email]) IS NULL) AND (UPPER(@p__linq__0) IS NULL))

If you can’t spot the problem – don’t worry because I have seen experienced developers made the same mistake. By using the TOUPPER function on the column you are effectively remove any performance benefit of the index that might be on Email column. That means this query will do an index scan every time it is called. We have the TOP(1) statement which somewhat reduces the impact (it can stop as soon as it finds a match), but if there is no match – e.g. no registered email, it will be a full index scan.

If you have a lot of registered customers, frequent calls to that query can effectively kill your database.

And how to fix it

Fixing this issue will be a bit cumbersome, because the code is well hidden inside the implementation of AspNetIdentity EntityFramework. But it’s not impossible. First we need an UserStore which does not use the Upper for comparison:

public class FoundationUserStore<TUser> : UserStore<TUser> where TUser : IdentityUser, IUIUser, new()
{
    public FoundationUserStore(DbContext context)
        : base(context)
    { }

    public override Task<TUser> FindByEmailAsync(string email)
    {
        return GetUserAggregateAsync(x => x.Email == email);
    }

    public override Task<TUser> FindByNameAsync(string name)
    {
        return GetUserAggregateAsync(x => x.UserName == name);
    }
}

And then a new UserManager to use that new UserStore

    public class CustomApplicationUserManager<TUser> : ApplicationUserManager<TUser> where TUser : IdentityUser, IUIUser, new()
    {
        public CustomApplicationUserManager(IUserStore<TUser> store)
            : base(store)
        {
        }

        public static new ApplicationUserManager<TUser> Create(IdentityFactoryOptions<ApplicationUserManager<TUser>> options, IOwinContext context)
        {
            var manager = new ApplicationUserManager<TUser>(new FoundationUserStore<TUser>(context.Get<ApplicationDbContext<TUser>>()));

            // Configure validation logic for usernames
            manager.UserValidator = new UserValidator<TUser>(manager)
            {
                AllowOnlyAlphanumericUserNames = false,
                RequireUniqueEmail = true
            };

            // Configure validation logic for passwords
            manager.PasswordValidator = new PasswordValidator
            {
#if DEBUG
                RequiredLength = 2,
                RequireNonLetterOrDigit = false,
                RequireDigit = false,
                RequireLowercase = false,
                RequireUppercase = false
#else
                RequiredLength = 6,
                RequireNonLetterOrDigit = true,
                RequireDigit = true,
                RequireLowercase = true,
                RequireUppercase = true

#endif
            };

            // Configure user lockout defaults
            manager.UserLockoutEnabledByDefault = true;
            manager.DefaultAccountLockoutTimeSpan = TimeSpan.FromMinutes(5);
            manager.MaxFailedAccessAttemptsBeforeLockout = 5;

            var provider = context.Get<ApplicationOptions>().DataProtectionProvider.Create("EPiServerAspNetIdentity");
            manager.UserTokenProvider = new DataProtectorTokenProvider<TUser>(provider);

            return manager;
        }
    }

And then a way to register our UserManager

    public static IAppBuilder AddCustomAspNetIdentity<TUser>(this IAppBuilder app, ApplicationOptions applicationOptions) where TUser : IdentityUser, IUIUser, new()
    {
        applicationOptions.DataProtectionProvider = app.GetDataProtectionProvider();

        // Configure the db context, user manager and signin manager to use a single instance per request
        app.CreatePerOwinContext<ApplicationOptions>(() => applicationOptions);
        app.CreatePerOwinContext<ApplicationDbContext<TUser>>(ApplicationDbContext<TUser>.Create);
        app.CreatePerOwinContext<ApplicationRoleManager<TUser>>(ApplicationRoleManager<TUser>.Create);
        app.CreatePerOwinContext<ApplicationUserManager<TUser>>(CustomApplicationUserManager<TUser>.Create);
        app.CreatePerOwinContext<ApplicationSignInManager<TUser>>(ApplicationSignInManager<TUser>.Create);

        // Configure the application
        app.CreatePerOwinContext<UIUserProvider>(ApplicationUserProvider<TUser>.Create);
        app.CreatePerOwinContext<UIRoleProvider>(ApplicationRoleProvider<TUser>.Create);
        app.CreatePerOwinContext<UIUserManager>(ApplicationUIUserManager<TUser>.Create);
        app.CreatePerOwinContext<UISignInManager>(ApplicationUISignInManager<TUser>.Create);

        // Saving the connection string in the case dbcontext be requested from none web context
        ConnectionStringNameResolver.ConnectionStringNameFromOptions = applicationOptions.ConnectionStringName;

        return app;
    }

Finally, replace the normal app.AddAspNetIdentity with this:

        app.AddCustomAspNetIdentity<SiteUser>(new ApplicationOptions
        {
            ConnectionStringName = commerceConectionStringName
        });

As I mentioned, this is cumbersome to do. If you know a better way to do, I’m all ear ;).

We are also skipping the case sensitivity part. In most of the cases, it’ll be fine as you are most likely using CI collation instead. But it’s better to be sure than leave it to chance. We will address that in the second part of this blog post.

Register your custom implementation, the sure way

The point of Episerver dependency injection is that you can plug in your custom implementation for, well almost, everything. But it can be tricky at times how to properly register your custom implementation.

The default DI framework (and possibly any other popular DI frameworks) works in the way that implementation registered later wins, i.e. it overrides any other implementation registered before it. To make Episerver uses your implementation, you have to make sure yours is registered last.

  • Never register your customer implementation using ServiceConfiguration. Implementation with that attributes will be registered first in the initialization pipeline. You will run into either
    • The default implementation was registered in an IConfigurableModule.ConfigureContainer. As those will be registered later than any implementation using ServiceConfiguration , yours will be overridden by the default ones.
    • The default implementation was also registered using ServiceConfiguration. Now you run into indeterministic situation – the order will be randomized every time your website starts. Sometimes it’s yours, sometimes it’s the default one, and that might cause some nasty bug (Heisenbug, if you know the reference 😉 )
  • That leaves you with registering your implementation by IConfigurableModule.ConfigureContainer . In many cases, registering your implementations here will just work, because the default implementations are registered by ServiceConfiguration attribute. However, that is not always the case. There is a possibility that the default one was registered using IConfigurableModule.ConfigureContainer, and things will be tricky. First of all, unlike IInitializationModule when you can make your module depends on a specific module, the order in which IConfigurationModule.ConfigureContainer is executed is not determined. Even if you were allowed to make the dependency, it’s not clear which module you should depend on, and in many cases, that module is internal, so you can’t specify it

That is the point of this post then. To make sure your implementation is registered regardless of how the default one is registered, you can always fallback to use the ConfigurationComplete event of ServiceConfigurationContext. This is called once all ConfigureContainer have been called, so you can be sure that the default implementation is registered – time to override it then!

        public void ConfigureContainer(ServiceConfigurationContext context)
        {
            context.ConfigurationComplete += Context_ConfigurationComplete;
        }

        private void Context_ConfigurationComplete(object sender, ServiceConfigurationEventArgs e)
        {
            e.Services.AddSingleton<IOrderRepository, CustomOrderRepository>();
        }

Simple as that!

Note that this only applies to cases when you want to override the default implementation. If you register an implementation of your own interfaces/abstract classes, or you will be adding your implementation (not overriding the default one, an example is if you have an implementation of IShippingPlugin), you can register it in any way.

Don’t insert an IEnumerable to cache

You have been told, cache is great. If used correctly, it can greatly improve your website performance, sometimes, it can even make the difference between life and death.

While that’s true, cache can be tricky to get right. The #1 issue with cache is cache invalidation, which we will get into detail in another blog post. The topic of today is a hidden, easy to make mistake, but can wreck havok in production.

Can you spot the problem in this snippet?

var someData = GetDataFromDatabase();                
var dataToCache = someData.Concat(someOtherData);
InsertToCache(cacheKey, dataToCache);

If you can’t, don’t worry – it is more common than you’d imagine. It’s easy to think that you are inserting your data correctly to cache.

Except you are not.

dataToCache is actually just an enumerator. It’s not until you get your data back and actually access the elements, the enumerator is actually called to fetch the data. If GetDataFromDatabase does not return a List<T>, but a lazy loading collection, that is when unpredictable things happen.

Who like to have unpredictability on a production website?

A simple, but effective advice is to always make sure you have the actual data in the object you are inserting to cache. Calling either .ToList() or ToArray() before inserting the data to cache would solve the problem nicely.

And that’s applied to any other lazy loading type of data as well.

Name or Display name in Catalog UI: you can choose

Since the beginning of Catalog UI, it had always shown Name, in both Catalog Tree and the Catalog content list.

That, however, was changed to DisplayName since 13.14 due to a popular feature request here https://world.episerver.com/forum/developer-forum/Feature-requests/Thread-Container/2019/12/use-localized-catalog-in-commerce-catalog-ui/#214650

All is good and the change was positively received. However not every is happy with it – some want it the old way, i.e. `Name` to be displayed. From a framework perspective, it might be complex to let partners configure which field to display. But if you are willing to do some extra work, then it’s all easy.

Catalog content is transformed using CatalogContentModelTransform, this is where DisplayName is added to the data returned to the client. If you override that, you can set DisplayName to whatever you want, for example, Name.

Here is what the implementation would look like

using EPiServer.Cms.Shell.UI.Rest.Models.Transforms;
using EPiServer.Commerce;
using EPiServer.Commerce.Catalog;
using EPiServer.Commerce.Catalog.ContentTypes;
using EPiServer.Commerce.Catalog.Linking;
using EPiServer.Commerce.Shell.Rest;
using EPiServer.Framework.Localization;
using EPiServer.ServiceLocation;
using Mediachase.Commerce.Catalog;
using Mediachase.Commerce.Customers;
using Mediachase.Commerce.InventoryService;
using Mediachase.Commerce.Markets;
using Mediachase.Commerce.Pricing;

namespace EPiServer.Reference.Commerce.Site.Infrastructure
{
    [ServiceConfiguration(typeof(IModelTransform))]
    public class BlahBlahBlah : CatalogContentModelTransform
    {
        public BlahBlahBlah(ExpressionHelper expressionHelper, IPriceService priceService, IMarketService marketService, IInventoryService inventoryService, LocalizationService localizationService, ICatalogSystem catalogContext, IRelationRepository relationRepository, ThumbnailUrlResolver thumbnailUrlResolver, CustomerContext customerContext) : base(expressionHelper, priceService, marketService, inventoryService, localizationService, catalogContext, relationRepository, thumbnailUrlResolver, customerContext)
        {
        }

        public override TransformOrder Order
        {
            ///Yes, this is very important to make it work
            get { return base.Order + 1; }
        }

        protected override void TransformInstance(IModelTransformContext context)
        {
            var catalogContent = context.Source as CatalogContentBase;
            var properties = context.Target.Properties;

            if (catalogContent is NodeContent nodeContent)
            {
                properties["DisplayName"] = nodeContent.Name;
            }
            if (catalogContent is EntryContentBase entryContent)
            {
                properties["DisplayName"] = entryContent.Name;
            }
        }
    }
}

And here is how it looks




A few notes:

  • CatalogContentModelTransform, and other APIs in Commerce.Shell, are not considered public APIs, so they might change without notice. There is a risk for adding this, however, it’s quite low.
  • This (or the bug fix) does not affect breadcrumb, it has been, and still is, showing Name.

Export catalog, with linked assets

If you are already using a PIM system, you can stop reading!

If you have been using Commerce for a while, you probably have seen this screen – yes, in Commerce Manager

This allow you to export a catalog, but without a caveat: the exported catalog, most likely, does not contains any linked assets. The reason for that was the asset content types need to be present at the context of the site. In Commerce Manager, the general advice is to not deploy the content types there for simpler management.

Import/Export are also missing features in Catalog UI compared to Commerce Manager. I wish I could have added it, but given my Dojo skill, it’s better to write something UI-less, and here you go: a controller to let you download a catalog with everything attached. Well, here is the entire code that you can drop into your project and build:

using EPiServer.Data;
using EPiServer.Framework.Blobs;
using EPiServer.Logging;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Web.Http;
using EPiServer.Commerce.Catalog.ContentTypes;
using Mediachase.Commerce.Catalog;
using Mediachase.Commerce.Catalog.ImportExport;
using System.IO;
using System.IO.Compression;
using System.Text;
using System.Xml;

namespace EPiServer.Personalization.Commerce.CatalogFeed.Internal
{
    /// <summary>
    /// Download a catalog.
    /// </summary>
    public class CatalogExportController : ApiController
    {
        private readonly CatalogImportExport _importExport;
        private readonly IBlobFactory _blobFactory;
        private readonly IContentLoader _contentLoader;
        private readonly ReferenceConverter _referenceConverter;
        internal const string DownloadRoute = "episerverapi/catalogs/";
        private static readonly Guid _blobContainerIdentifier = Guid.Parse("119AD01E-ECD1-4781-898B-6DEC356FC8D8");

        private static readonly ILogger _logger = LogManager.GetLogger(typeof(CatalogExportController));

        /// <summary>
        /// Initializes a new instance of the <see cref="CatalogExportController"/> class.
        /// </summary>
        /// <param name="importExport">Catalog import export</param>
        /// <param name="blobFactory">The blob factory.</param>
        /// <param name="contentLoader">The content loader.</param>
        /// <param name="referenceConverter"></param>
        public CatalogExportController(CatalogImportExport importExport,
            IBlobFactory blobFactory,
            IContentLoader contentLoader,
            ReferenceConverter referenceConverter)
        {
            _importExport = importExport;
            _blobFactory = blobFactory;
            _contentLoader = contentLoader;
            _referenceConverter = referenceConverter;

            _importExport.IsModelsAvailable = true;
        }

        /// <summary>
        /// Direct download catalog export for admins.
        /// </summary>
        /// <param name="catalogName">Name of catalog to be exported.</param>
        /// <returns>
        /// Catalog.zip if successful else HttpResponseMessage containing error.
        /// </returns>
        [HttpGet]
        [Authorize(Roles = "CommerceAdmins")]
        [Route(DownloadRoute)]
        public HttpResponseMessage Index(string catalogName)
        {
            var catalogs = _contentLoader.GetChildren<CatalogContent>(_referenceConverter.GetRootLink());
            var catalog = catalogs.First(x => x.Name.Equals(catalogName, StringComparison.OrdinalIgnoreCase));
            if (catalog != null)
            {
                return GetFile(catalog.Name);
            }

            return new HttpResponseMessage
            { Content = new StringContent($"There is no catalog with name {catalogName}.") };
        }

        private HttpResponseMessage GetFile(string catalogName)
        {
            var container = Blob.GetContainerIdentifier(_blobContainerIdentifier);
            var blob = _blobFactory.CreateBlob(container, ".zip");
            using (var stream = blob.OpenWrite())
            {
                using (var zipArchive = new ZipArchive(stream, ZipArchiveMode.Create, false))
                {
                    var entry = zipArchive.CreateEntry("catalog.xml");

                    using (var entryStream = entry.Open())
                    {
                        _importExport.Export(catalogName, entryStream, Path.GetTempPath());
                    }
                }
            }

            var response = new HttpResponseMessage
            {
                Content = new PushStreamContent(async (outputStream, content, context) =>
                {
                    var fileStream = blob.OpenRead();

                    await fileStream.CopyToAsync(outputStream)
                        .ContinueWith(task =>
                        {
                            fileStream.Close();
                            outputStream.Close();

                            if (task.IsFaulted)
                            {
                                _logger.Error($"Catalog download failed", task.Exception);
                                return;
                            }

                            _logger.Information($"Feed download completed.");

                        });
                }, new MediaTypeHeaderValue("application/zip"))
            };
            return response;
        }
    }
}

And now you can access to this path http://yoursite.com/episerverapi/catalogs?catalogName=fashion to download the catalog named “Fashion”.

A few notes:

  • This requires Admin access, for obvious reasons. You will need to log in to your website first before accessing the path above
  • It can take some time for big catalogs, so be patient if that’s the case ;). Yes another approach is to have this as a scheduled job when you can export the catalog in background, but that make the selection of catalog to export much more complicated. If you have only one catalog, then go ahead!

Disabling Catalog Dto cache: maybe, don’t?

Recently (as recent as this morning) I was asked to look into a case when the Find indexing performance was subpar. Upon investigation – looking at a properly captured trace from dotTrace – it was clear that at least 30% percent of time was spending in loading the CatalogDto

This is something that should not happen, as the CatalogDto should have been cached. Also, a normal site should have very few catalogs, so the cache should be very effective. However, data does not lie – it has been hitting database a lot, and a quick check on the site settings revealed that the entire DTO cache has been indeed, disabled

 <Cache enabled="false" collectionTimeout="0:0:0" entryTimeout="0:0:0" nodeTimeout="0:0:0" schemaTimeout="0:0:0" />

By setting these timeout settings to 0, the cache is immediately invalidated, rendering them useless. The CatalogDto, therefore, is loaded everytime from database, causing the bottleneck.

The reason for setting those timeout to 0 was probably – I guess – to reduce the memory footprint of the site. However, Catalog DTOs are fairly small in size, and since Commerce 11, it has been smart enough to skip caching the DTOs if there is cache on a higher (content ) level, thanks to my colleague Magnus Rahl. So DTOs should not be of any concerns, if you are not actively using them (and in most of the case, you should not). By re-enabling the cache, the indexing time can be cut, at least 30%, according to the aforementioned trace.

As you might wonder, Catalog content provider still uses the DTOs internally, therefore it would load those for data.

Moral of the story:

  • The cache settings are there, but because you can, does not mean you should. I personally think cache settings should be as hidden as possible from accidental changes. Disabling cache, and in a lesser extend, changing default cache timeout, can have unforeseeable consequences. Only do so if you have strong reasons to do so. Or better, let us know why you need to do that, and we can figure out a compromise.