Optimizing T-SQL COUNT

This is a continuation of my previous post about paging in SQL Server. When it comes to paging, you would naturally want to know the total number of rows satisfying, so you can display some nice, useful information to your end-users.

You would think, well, it’s just a count, and a simple query like this would be enough:

SELECT COUNT(Id) FROM MySecretTable

There should be nothing to worry about, right? Actually, there is.

Let’s get back to the example in previous post – we have to count the total number of orders in that big table.

SELECT COUNT(ObjectId) FROM OrderGroup_PurchaseOrder

Because ObjectId is the clustered index of OrderGroup_PurchaseOrder, I did expect it to be use that index and be pretty fast. But does it? To my surprises, no.

Continue reading “Optimizing T-SQL COUNT” →

The art of paging

No this is not really “art” – I’m just trying to have a more clickbait title. It’s more about understanding what you have at your disposal and use them for your benefits – in this case – how new SQL statement can drastically improve your performance.

In this blogpost we will look into paging feature of SQL Server. in Commerce we usually work with large set of data – millions of rows are fairly common, and it’s natural to load data by page. There is no point loading thousands, or even millions of rows in one go. First it’s not practical to display all of them. Second you’ll likely end up with an timeout exception and/or an out of memory exception. Even if you are lucky enough to get through, it’s still able to take your SQL Server instance to a knee, and transferring that much data over network will be another bottleneck for your system. So my friends, the best practice for loading data is to do it by batches, and to not load everything at once.

Continue reading “The art of paging” →

Fixing a stored procedure

At Episerver development team, we understand the importance of good performance. Who would not like a lightning fast website? We work hard to ensure the framework is fast, and we seize (almost) every opportunity to make it faster.

You know in Commerce 10.2 we introduced a new cart mode – serializable cart, and it’s proven to bring great performance compared to the “old/traditional” approach. Our own tests showed an improvement of 3-5x times faster. But can it be even faster? Probably yes.

And actually we did some improvements in later versions. In the scope of this blog post, we will just focus into a specific aspect – and to learn a little more about SQL Server performance optimization.

Continue reading “Fixing a stored procedure” →

Merging carts when a customer logs in

It’s quite common when a customer browses your site without logging in – either she/he intentionally does that, or just forget about logging in. The customer might add some items to carts and even checks out, but then is asked to log in or remember to log in. What would happen?

By default, Episerver Commerce will do as following:

Attach all orders made by that section to logged in customer. (I once placed an order in a famous retailer in Sweden without logging in, and then I asked their customer service to link that order to my account so I can track it easier. To my surprise, it cannot be done! That’s why I personally appreciate this feature.)
Merge all carts to existing carts, by name and market. So if I’m currently in US market and I added a item to that cart, and the US-cart linked to my account already have 2 other items, then when I log in, my US-cart will contain 3 items.
Merge all wishlist to existing wishlists, also per market. (The name for wishlist, by default, is fixed to “Wishlist”)

Continue reading “Merging carts when a customer logs in” →

Tale from inside TransactionScope

One of the last things you want to get from your Commerce site is that the order data is gone. What can be more confused than if your log shows that the cart has been converted into a purchase order, you even got the PO number, but after that, the order disappears? It’s nowhere to be found, even if you look into database. It’s kind of magic, but not the kind of magic you would want to have.

But everything happens for a reason. And actually it’s with a good reason: data consistency.

Episerver Commerce has the concept of TransactionScope. Simply put, it allows two or more database operations to be done as atomic: Either all of them succeed, or all of them will revert back. If a TransactionScope contains 3 operations A, B, C, then even if A, B succeeded, but C is yet to commit, and something goes wrong, then A and B would be reverted.

Continue reading “Tale from inside TransactionScope” →

Too much saves will kill you

… or at least, your website performance!

Recently I worked on two support cases from our customers as they see SQL Server errors, such as “System.Data.SqlClient.SqlException (0x80131904): The INSERT statement conflicted with the FOREIGN KEY constraint “FK_ShipmentEx_Shipment”. The conflict occurred in database “dbCommerce”, table “dbo.Shipment”, column ‘ShipmentId’“, or “System.Data.SqlClient.SqlException (0x80131904): The MERGE statement attempted to UPDATE or DELETE the same row more than once. This happens when a target row matches more than one source row. A MERGE statement cannot UPDATE/DELETE the same row of the target table multiple times. Refine the ON clause to ensure a target row matches at most one source row, or use the GROUP BY clause to group the source rows.”

These errors happened randomly, during the high load times – it seems to be affected by the concurrency level.

What was wrong? and why?

It took me a good amount of time, and good amount of hairs, too. The actual error is another one, and the one above is just the “by product”.

The cart system in Episerver Commerce suffers from a design flaw: it shares (almost) everything with the purchase orders. ShoppingCart is just another metaclass extended OrderGroup, so it’ll use the same OrderGroup, OrderForm, Shipment, LineItem and OrderAddress tables in the database, like PurchaseOrder and PaymentPlan. At first, it seems to be reasonable approach. But when you have hundreds, or thousands of customers visiting your website (and you would be happy to see that ;)) – problems start to appear.

Continue reading “Too much saves will kill you” →