Wednesday, December 16, 2015

Sitecore Commerce Connect - Cleaning up the Rusty Shopping Cart



In my last project, I really got to dig deep into Commerce Connect and can say that the Sitecore team did a fantastic job architecting the module.

As with all E-Commerce implementations, one of our requirements was to have a shopping cart that would persist for a period of time. The problem we were faced with was that the shopping cart needed to exist on the Sitecore side, and Commerce Connect expects the cart to be handled by the External Commerce System. Our External Commerce System was only responsible for handling the checkout process.

Reading through the Sitecore Connect Integration Guide, we discovered the "Storing a copy of the cart locally" section on page 17 that mentioned 2 sample cart repositories that ship with Commerce Connect:
  • EaStateCartRepository - for storing carts in MongoDB  (enabled by default)
  • EaStateSqlBasedCartRepository - for storing carts in SQL

Basically, what this section described was exactly what we wanted to do.

However, the documentation had a very specific caveat; "...these are only sample repositories and should not be used in a production scenario."

So, what could we wrong with the sample code? Surely if it shipped with the module and was enabled by default, it was OK?

Unfortunately, we discovered that the sample EaStateCartRepository had a significant flaw in it during our testing. A bug that could wreak havoc if put in production.

In this post, I will describe the issues with the sample cart repository, and how you can fix it so that it can be used in production.

Side note: I couldn't find a sample of the EaStateSqlBasedCartRepository mentioned in the documentation. Also, it was unfortunate that the sample repository didn't have any documentation describing it's functionality. So, you will need to get out your trusty decompiler to review the code.

About Commerce Connect

Before diving in, I want to take a step backwards to get some important facts highlighted about the module, as there has been a lot of confusion with partners wanting to understand comparisons between Commerce Connect and Commerce Server. This really isn't even an apples to oranges comparison. It's more like a cars to roads comparison.

What Commerce Connect IS NOT

  • A stand-alone E-Commerce solution.
  • A competitor or alternative to Sitecore Commerce Server.
  • An upgrade path for Sitecore Ecommerce Services.
  • A solution that provides out-of-the-box components like you would get with Active Commerce or uCommerce.

What Commerce Connect IS

  • A pluggable framework. 
  • A pass-through to an underlying E-Commerce platform or service. 
  • A common integration layer that external systems can be integrated with.
  • A means of bringing all the aspects of the Sitecore Experience Platform around user engagement to external E-Commerce systems.

With all this being said, if you are looking to implement a solution using the module, you REALLY NEED TO READ the following documentation in order to get a handle of all the features that the module provides and how to implement it correctly:

So, What's Wrong with the Cart Repository Anyway?

The sample cart repository issue that we found stemmed from the first two processors within the CreateOrResumeCart pipeline found in Sitecore.Commerce.Carts.config:

2:        Initiate the creation of a shopping cart and in the process to:   
3:         - Load persisted, potentially abandoned cart, if present;  
4:         - Call resumeCart pipeline to resume loaded cart;  
5:         - Call createCart pipeline to create cart if no cart was found on previous steps.  
6:     -->  
7:     <commerce.carts.createOrResumeCart>  
8:      <processor type="Sitecore.Commerce.Pipelines.Carts.CreateOrResumeCart.FindCartInEaState, Sitecore.Commerce">  
9:       <param ref="eaStateCartRepository" />  
10:      </processor>  
11:      <processor type="Sitecore.Commerce.Pipelines.Carts.CreateOrResumeCart.RunLoadCart, Sitecore.Commerce" />  
12:      <processor type="Sitecore.Commerce.Pipelines.Carts.CreateOrResumeCart.RunResumeCart, Sitecore.Commerce" />  
13:      <processor type="Sitecore.Commerce.Pipelines.Carts.CreateOrResumeCart.RunCreateCart, Sitecore.Commerce" />  
14:     </commerce.carts.createOrResumeCart>  
The FindCartInEaState processor retrieves a CartId for the current user, using the request.UserId shown below:

1:  public override void Process(ServicePipelineArgs args)  
2:  {  
3:     Assert.ArgumentNotNull((object) args, "args");  
4:     CreateOrResumeCartRequest request = (CreateOrResumeCartRequest) args.Request;  
5:     List<CartBase> list = Enumerable.ToList<CartBase>(Enumerable.Where<CartBase>(this.repository.GetByUserName(request.UserId, request.ShopName), (Func<CartBase, bool>) (c =>  
6:     {  
7:      if (c.Name == request.Name)  
8:       return c.CustomerId == request.CustomerId;  
9:      return false;  
10:     })));  
11:     if (list.Count == 1)  
12:      args.Request.Properties["CartId"] = (object) list[0].ExternalId;  
13:     else  
14:      args.Request.Properties["CartId"] = (object) null;  
15:    }  
16: }  
When the RunLoadCart processor runs, it initializes the LoadCartFromEaState pipeline.

1:  public override void Process(ServicePipelineArgs args)  
2:  {  
3:     Assert.ArgumentNotNull((object) args, "args");  
4:     Assert.ArgumentNotNull((object) args.Request, "args.Request");  
5:     object obj = args.Request.Properties["CartId"];  
6:     if (obj == null)  
7:      return;  
8:     CreateOrResumeCartRequest resumeCartRequest = (CreateOrResumeCartRequest) args.Request;  
9:     this.RunPipeline("commerce.carts.loadCart", new ServicePipelineArgs((ServiceProviderRequest) new LoadCartRequest(resumeCartRequest.ShopName, obj.ToString(), resumeCartRequest.UserId), args.Result), args.Request);  
10: }  
This pipeline calls the GetByCartId method of the EaStateCartRepository. It does not pass the UserId as an argument, despite the GetByCartId method requiring a CartId and a UserId to perform the lookup.

The EaStateCartRepository has a private string called lastAccessedUserName, which should be the contact id of the visitor / contact. This variable is set in several methods in the EaStateCartRepository, however it is only used in the GetByCartId method.

1:  public virtual Cart GetByCartId(string cartId, string shopName)  
2:  {  
3:     Assert.ArgumentNotNullOrEmpty(cartId, "cartId");  
4:     Assert.ArgumentNotNullOrEmpty(shopName, "shopName");  
5:     Type type = this.entityFactory.Create("Cart").GetType();  
6:     Cart cart1 = (Cart) null;  
7:     if (!string.IsNullOrEmpty(this.lastAcessedUserName))  
8:      cart1 = (Cart) Enumerable.SingleOrDefault<CartBase>(this.GetByUserName(this.lastAcessedUserName, shopName, type), (Func<CartBase, bool>) (cart => cart.ExternalId == cartId));  
9:     return cart1 ?? (Cart) Enumerable.SingleOrDefault<CartBase>(this.GetAll(shopName, type), (Func<CartBase, bool>) (cart => cart.ExternalId == cartId));  
10: }  
Looking at Sitecore.Commerce.Carts.config, you will notice that only one instance of the EaStateCartRepository will be created by the provider engine:

1:  <eaStateCartRepository type="Sitecore.Commerce.Data.Carts.EaStateCartRepository, Sitecore.Commerce" singleInstance="true">  
2:     <param ref="entityFactory" />  
3:     <param ref="eaPlanProvider" />  
4:    </eaStateCartRepository>  
What this means is that the lastAccessedUserName variable can be the id of a different contact.

In our investigation, the lastAccessedUserName not matching the current contact id (Tracker.Current.Contact.ContactId), caused the GetAll fallback method to fire (line 9 in the GetByCartId method above).

This method is massively expensive as it iterates through all users to retrieve their carts!

1:  /// <summary>  
2:  /// Gets all.  
3:  ///   
4:  /// </summary>  
5:  /// <param name="shopName">Name of the shop.</param><param name="cartType">Type of the cart.</param>  
6:  /// <returns>  
7:  /// The collection of carts.  
8:  ///   
9:  /// </returns>  
10:  private IEnumerable<CartBase> GetAll(string shopName, Type cartType)  
11:  {  
12:   Assert.ArgumentNotNullOrEmpty(shopName, "shopName");  
13:   ID itemId = this.eaPlanProvider.GetEaPlanId(shopName).Item1;  
14:   EngagementPlanItem engagementPlanItem = new EngagementPlanItem(Tracker.DefinitionDatabase.GetItem(itemId));  
15:   ContactManager contactManager = CommerceAutomationHelper.GetContactManager();  
16:   List<CartBase> list = new List<CartBase>();  
17:   foreach (CustomItemBase customItemBase in engagementPlanItem.States)  
18:   {  
19:       foreach (ID id in (IEnumerable<ID>) AutomationContactManager.GetStateContactsIdsPaged(customItemBase.ID))  
20:       {  
21:        Contact contact = contactManager.LoadContactReadOnly(id.Guid);  
22:        if (contact == null)  
23:        {  
24:            if (Tracker.Current.Contact.ContactId == id.Guid)  
25:             contact = Tracker.Current.Contact;  
26:            else  
27:             continue;  
28:        }  
29:        IEnumerable<CartBase> customDataOrNull = CommerceAutomationHelper.GetCustomDataOrNull<IEnumerable<CartBase>>(AutomationStateManager.Create(contact).GetCurrentStateInPlan(itemId), "commerce.carts");  
30:        if (customDataOrNull != null)  
31:            list.AddRange(customDataOrNull);  
32:       }  
33:   }  
34:   return (IEnumerable<CartBase>) list;  
35:  }  
As we ran tests with more and more users, the mismatch started happening all the time, and the expensive GetAll fallback method kept firing, causing the server's CPU usage to spike.


Fixing the Cart Repository 

How Sitecore Should Fix It

To permanently fix this issue, the Sitecore team should make the LoadCartFromEaState pipeline pass the GetByCartId method the ContactId and the lastAccessedUserName variable should be purged from the class.

My Fix

In my case, I created a new cart repository class that inherited from EaStateCartRepository. I then added a new GetCartByIdFallback method where I would retrieve the cart based on the current contact id (Tracker.Current.Contact.ContactId).

1:  private Cart GetCartByIdFallback(string cartId, string shopName, Type cartType)  
2:  {  
3:       if (Tracker.Current.Contact == null)  
4:       {       
5:            return null;  
6:       }  
8:       string contactId = Tracker.Current.Contact.ContactId.ToString();  
10:       List<CartBase> carts = this.GetByUserName(contactId, shopName, cartType).ToList();  
11:       Cart cart1 = (Cart) null;  
13:       if (carts.Any())  
14:       {  
15:            cart1 = (Cart) carts.SingleOrDefault(cart => cart.ExternalId == cartId);  
17:            if (cart1 != null)  
18:            {  
19:                 return cart1;  
20:            }  
21:       }  
23:       return null;  
24:  }  
Finally, I overrode the GetByCartId method to use my GetCartByIdFallback instead of the expensive GetAll fallback method if the cart couldn't be found for the lastAcessedUserName, or as mentioned above, the lastAcessedUserName didn't match the current contact id (Tracker.Current.Contact.ContactId).

1:  public override Cart GetByCartId(string cartId, string shopName)  
2:  {  
3:       Assert.ArgumentNotNullOrEmpty(cartId, "cartId");  
4:       Assert.ArgumentNotNullOrEmpty(shopName, "shopName");  
5:       Type type = this.entityFactory.Create("Cart").GetType();  
6:       Cart cart1 = (Cart)null;  
7:       if (!string.IsNullOrEmpty(this.lastAcessedUserName))  
8:       {  
9:            cart1 = (Cart)Enumerable.SingleOrDefault<CartBase>(this.GetByUserName(this.lastAcessedUserName, shopName, type), (Func<CartBase, bool>)(cart => cart.ExternalId == cartId));  
10:       }  
11:       return cart1 ?? GetCartByIdFallback(cartId, shopName, type);  
12:  }  

Final Configuration

To put my new cart repository into action, I simply had to modify the Sitecore.Commerce.Carts.config to use my new repository:

1:  <eaStateCartRepository type="CommerceTest.MyNewCartRepository, CommerceTest" singleInstance="true">  
2:   <param ref="entityFactory" />  
3:   <param ref="eaPlanProvider" />  
4:  </eaStateCartRepository>  

After this, the cart worked perfectly!