A New Rudder on Sitecore.Ship to Deploy Securely From Visual Studio Team Services

I fell in love with Kevin Obee's Sitecore.Ship module when it was first introduced to me by my Arke colleague and MVP, Patrick Perrone, and it has been part of my Continuous Deployment routine ever since.

If you are using Hedgehog's TDS, a build server of your flavor, and a touch of PowerShell and / or curl, Sitecore.Ship allows you sail code and content into Integration, Staging and Production environments in a breeze.

This post assumes that you are familiar with using TDS to generate .update packages, and have used the Sitecore.Ship module before. I intend to demonstrate how you can use my customizations to keep the module's REST service secure, and deploy from Visual Studio Team Services (VSTS) to your various environments.

All updated source code and documentation can be found on my GitHub fork: https://github.com/martinrayenglish/Sitecore.Ship

Security

Out of the box, Kevin's module allows you to enable remote deployments and add an access control list of machine IP addresses that can access the REST web service.

 <packageInstallation enabled="true" allowRemote="true" allowPackageStreaming="true" recordInstallationHistory="true">  
  <Whitelist>  
   <add name="local loopback" IP="127.0.01" />  
   <add name="Allowed machine 1" IP="10.20.3.4" />  
   <add name="Allowed machine 2" IP="10.40.4.5" />  
  </Whitelist>  
 </packageInstallation>  

This works really well for most use cases, but not when you are deploying from Visual Studio Team Services' Azure servers. Keep reading and I will explain why.

Visual Studio Team Services

Microsoft has done a great job with VSTS a.k.a. TFS in the cloud. They provide a nice suite of tools such as code repositories, continuous integration, bug and task tracking, and agile planning at the very wonderful price of free for up to 5 users.

Once your team grows beyond 5 for free, the increments are priced reasonably. If you are an MSDN subscriber, you aren't counted against the total number of user's in the pricing model. So if your company is a Microsoft shop with MSDN, you could be working with your client's VSTS account at no cost to them. Sweet!

You can check out Microsoft's pricing by navigating over to this page: https://www.visualstudio.com/pricing/visual-studio-team-services-pricing-vs.

Setting up the Build Definition In Visual Studio Team Services

VSTS provides you with a slew of options to help set up your build definitions for either Continuous Integration or Continuous Deployment.

In my example, I set up a build definition for a Staging environment with a few simple clicks. The basic steps to add a new definition are:

Build

Add a series of tasks used to from your build steps

Repository

Set the repository type (in my case Git) and branch that you intend to build from

Variables

Set the build configuration (such as "MyClientStaging"), and platform settings

Triggers

If you want to perform Continuous Integration and build on every commit to the repository using the branch set in the Repository Tab

General

Make sure that the "Default agent queue" is set to "Hosted"

In my example, I set the above-mentioned configurations and build steps to my fancy, and the end result looked something like this:

You will notice that there are 2 PowerShell script tasks that are executed after the build and tests have run.

About The Scripts

Both scripts were originally crafted by my Arke colleague Patrick Perrone. I simply added a few modifications to make things work in my environments.

Make sure you check out his Sitecore.Ship-PowerShell-Functions module that is available on GitHub: https://github.com/patrickperrone/Sitecore.Ship-PowerShell-Functions

The first script is used to deploy my TDS generated .update package from VSTS' Azure servers, over to my target machine(s) that have Sitecore.Ship installed on them.

The second script is used to execute an improved version of Pavel Veller's ConfigFileCommitter service shown by his post: http://jockstothecore.com/update-config-love/

Sitecore.Ship Whitelist VSTS Challenges

As I mentioned previously, the security considerations in Kevin's module work really well in most traditional environments where you are using a deployment server that is either on-premise or in the cloud but has a single, controllable IP address.

The challenge I faced building and deploying directly from VSTS, was that it was impossible to predict the IP address that the Azure server was deploying from, and thus impossible to lock it down using the Whitelist configuration.

First Stab - IP Address Ranges

After forking the Sitecore.Shop repository, I started working through the code so that I could lock things down when deploying from VSTS. After adding some logging I noticed a pattarn of IP Address ranges from the Azure servers, and went on my merry way to update the module to include support for IP Address ranges.

So, with this in place, I could now apply ranges to my Whitelist configuration like so:

 <packageInstallation enabled="true" allowRemote="true" allowPackageStreaming="true" recordInstallationHistory="true">  
  <Whitelist>  
   <add name="local loopback" IP="127.0.01" />  
   <add name="Allowed machine 1" IP="10.20.3.4" />  
   <add name="Allowed machine 2" IP="10.40.4.5" />  
   <add name="Allowed IP Range 1" IP="23.96.0.0-23.96.255.255" />  
   <add name="Allowed IP Range 2" IP="65.52.0.0-65.52.255.255" />  
  </Whitelist>  
 </packageInstallation>  

Problem solved? 

Not so much. 

After running several deployment tests, I discovered that the Azure IP ranges were just not predicable at all. Based on what I saw, I would end up having to open up a ton of ranges, and would just hope and pray that things didn't change and cause my deployments to blow up.

It was obvious that as I started opening up more and more ranges, the security aspect of my deployments was sinking, and I was opening up myself to those pesky Internet pirates who could sail right on in. (Hope you enjoyed this ;)

Second Stab - Authentication Token

Having recently worked with Sitecore's xDB Cloud REST API, I decided to take their approach and build in an Authentication Token security option.

With this approach, I added the ability set the name and value of a custom header via configuration. I could then add this token in the header of the request when running the deployment to the machines.

The updated configuration looked like this:

 <packageInstallation enabled="true" allowRemote="true" allowPackageStreaming="true" recordInstallationHistory="true"   
 authHeader="X-Ship-Auth" authToken="AC074C6EDBA518F807E0E3F2F36A8B512D9C5637744BE67CD60D271244AC523AAB9CF8DB7F7D3934205E5BD850B2768C7171C3C594D6C6BFCA3992CCCCA67148">  
  <Whitelist>  
   <add name="local loopback" IP="127.0.01" />  
   <add name="Allowed machine 1" IP="10.20.3.4" />  
   <add name="Allowed machine 2" IP="10.40.4.5" />  
   <add name="Allowed IP Range 1" IP="23.96.0.0-23.96.255.255" />  
   <add name="Allowed IP Range 2" IP="65.52.0.0-65.52.255.255" />  
  </Whitelist>  
 </packageInstallation>  

Testing and Implementation

Testing

When testing the authentication changes using a tool like Postman, all you would need to do is pass over the token value in a custom header that you set up on configuration when making the request.

Implementation

In my case, I modified the PowerShell deployment script to include the new custom header containing my token.

Final Note on Service Paths - "Services" vs "SitecoreShip"

Part of the module's configuration includes adding a NancyHttpRequestHandler to your web.config specifying the path of Ship service, along with telling Sitecore not to process things at that path via an IgnoreUrlPrefixes setting.

Web.config:

 <handlers>  
 <add name="Nancy" verb="*" type="Nancy.Hosting.Aspnet.NancyHttpRequestHandler" path="/services/*" />  
 </handlers>  

Ship.config:

 <settings>  
    <setting name="IgnoreUrlPrefixes" set:value="/services/|/sitecore/default.aspx|/trace.axd|/webresource.axd|/sitecore/shell/Controls/Rich Text Editor/Telerik.Web.UI.DialogHandler.aspx|/sitecore/shell/applications/content manager/telerik.web.ui.dialoghandler.aspx|/sitecore/shell/Controls/Rich Text Editor/Telerik.Web.UI.SpellCheckHandler.axd|/Telerik.Web.UI.WebResource.axd|/sitecore/admin/upgrade/|/layouts/testing" />  
 </settings>  

This turned out to be problematic for a recent project, where we actually had an item that needed to use the "services" path. They had content living in "/services/contact-us".

So, I modified the original service path from "services" to "sitecoreship" to overcome this issue, and hopefully prevent this conflict from happening on my future projects.

All updated source code and documentation can be found on my GitHub fork: https://github.com/martinrayenglish/Sitecore.Ship

Read More

Sitecore 8.x Component Datasource Item Locking Behavior

Background

A question was posed on the Sitecore Community Site around items in workflow and component datasource item locking, and it piqued my curiosity enough to want to dig in to understand Sitecore's behavior on datasource items when we have multiple content authors working within the same content scope.

After my review, I discovered some unexpected behavior that could pose a potential problem for content authors. Talking through the results with veteran MVP, Kamruz Jaman, he suggested a potential workaround that could will help provide relief for author's in this scenario.

Test Configuration

In my tests, I was working with Sitecore 8.1 Update 1, and created Author A and Author B who were both part of ​the sitecore\Author​ and ​sitecore\Designer roles. I made sure that my authors had the necessary read, write, delete etc permission for the content scope that they were working in.

Datasource Item Locking Behavior

Author A locks an the item with presentation that consists of a component with a datasource item. The item is locked but the datasource item is not locked.

Author A makes inline edits to the component's datasource item's content, and then clicks the save button. After saving the item, the datasource item is now locked.

Author B locks an item that has a component with the same datasource item that has been locked by Author A.

Author B cannot make inline edits to the component with the locked datasource item.

Author B can make inline edits to all other areas of the item that aren't linked to locked datasource items.

Two important things to note from above:

1. Author's cannot make changes to a component's datasource item when another author has locked the datasource item. One would expect this behavior.
2. The datasource item only gets locked by the author when the author has made changes to the datasource item and clicks the save button. Making changes to other items on the page and saving will not lock the datasource item.

Problem

So far, things work pretty much the way one would expect. The only oddity is that the datasource item only gets locked on save. This is something that we can live with though.

The problem that I identified from the abovementioned scenario was that when Author A had finished making edits and unlocked the item, the datasource item didn't get unlocked in with the item. It stayed locked!

Workaround - Automatic Unlocking

One would expect both items and datasource items in workflow to be unlocked when an author has clicked the unlock button after making their content changes.

As Kamruz pointed out, one way to deal with this is to have the automatic unlocking setting enabled. The setting can be found in the Sitecore.config file:

 <setting name="AutomaticUnlockOnSaved" value="false" />  

When the value of AutomaticUnLockOnSaved is set to true, then every item is automatically unlocked after saving.

Having this enabled would provide relief from the unexpected behavior that is there by default.

Read More

Sitecore xDB Cloud Edition: Using the REST API

As I mentioned in a previous post, one of the nice things about using xDB Cloud Edition is that once your licensing is in place, getting up and running is very easy.

It's important to note that you don't have direct access to the various collections. So, you can't connect using a tool like Robomongo or MongoVUE to get information about your instance. In the past, getting consumption information about your cloud instance would require you opening up a ticket with support.

All this has changed with the introduction of the REST API for the xDB Cloud service. To find more information about what the API has to offer, I recommend that you read through the REST API reference for the xDB Cloud service.

In this post, I will show you how to use the API with one of my favorite browser plugins, Postman, so that you can get useful information and manage various processes in your Cloud Instance.

Nexus Authentication Token

The first order of business is to use the SSO Encode Sitecore License endpoint (https://gateway-sso-scs.cloud.sitecore.net/api/License/Encode) to obtain a Nexus Authentication token.

In order to do this, you need to generate a POST request to the endpoint with your Sitecore license file in the body of the request.

After performing the post, you will see your Nexus token in the response:

Check xDB consumption

Once you have your Nexus token, hitting the other endpoints is a walk in the park.  All you need to do is pass over the token value in a custom header called "X-ScS-Nexus-Auth" when making the request.

One of the endpoints that you will be accessing regularly is the consumption one. This gives you very useful information such as instance size, number of contacts and interactions.

For this endpoint, you need to pass over the following parameters along with the token:

• licenseId – your Sitecore license ID (1 below)
• deploymentId – the unique identification of the deployment (2 below)
• year – the consumption year (3 below)
• month – the consumption month (3 below)

4 below is my generated Nexus token called "X-ScS-Nexus-Auth" that I have added to the HTTP header.

So for example, to get consumption information for October 2015, my GET request looks like this:

You will see that there is small bug in my particular instance, where it is returning "0GB" for size. At the time of writing this post, the Cloud Team was actively working to get this resolved.

If you want to use this data to create a fancy report, what you can do is convert the JSON data into CSV format using one of the free conversion sites like http://konklone.io/json/.

Using this site, I was able to get the data into Excel and create some cool looking graphs in a few clicks:

Other xDB Endpoints

Other endpoints that are available to you include:

Get Firewall settings
https://gateway-xdb-scs.cloud.sitecore.net/api/xdb/firewallsettings/{licenseId}/{deploymentId}

Get history processing status

https://gateway-xdb-scs.cloud.sitecore.net/api/xdb/historyProcessing/{licenseId}/{deploymentId}

Get xDB collection verification dataset

https://gateway-xdb-scs.cloud.sitecore.net/api/xdb/collection/{licenseId}/{deploymentId}

Get xDB set status

https://gateway-xdb-scs.cloud.sitecore.net/api/xdb/historyProcessing/{licenseId}/{deploymentId}

List xDB sets

https://gateway-xdb-scs.cloud.sitecore.net/api/xdb/{licenseId}

Trigger history processing

https://gateway-xdb-scs.cloud.sitecore.net/api/xdb/historyProcessing/{licenseId}/{deploymentId}

The Trigger history processing endpoint above is another one to note. This gives you the ability to trigger a rebuild of your cloud reporting database. Note that the HTTP method is a PUT for this.

Final Thoughts

With this API in place that gives us a good level of control of the xDB Cloud, we can't help but get even more excited about the additional self service APIs and the xConnect API, that will be released with 8.2 later this year.

Read More

Getting Started with Sitecore SPEAK 2.0

With the release of Sitecore 8.1, we now have the ability to work with SPEAK 2.0 which is promised to be less of a steep learning curve, and will give your fingers a bit of a rest when adding custom logic to your page code.

Because it is so new, most of the SPEAK related documentation that you find on the internet today is SPEAK 1.1 related. Most likely, if it doesn't specify a version, it's SPEAK 1.1. The only 2.0 post that you may find is by SPEAK guru, Mike Robbins, where he compares a Sitecore SPEAK 2.0 Component vs SPEAK 1.1. 

So to get started with 2.0, I recommend that you read through Mike's post along with the SPEAK changes document found on the 8.1 Initial Release page that describes the differences between 1.1 and 2.0. 

No SPEAK 2.0 Branches Yet

If you take the plunge and start building new apps using SPEAK 2.0, you will notice that there currently aren't any branches available in 2.0 yet.

What this means is that you have to build out your 2.0 pages manually. Never fear, it's actually not that painful.

SPEAK 2.0 Enabled Dashboard Page

In my POC, I started by building out a simple Dashboard Page. You can use my approach to build out any of the other SPEAK pages based on 2.0.

After talking through some things with one of our star developers, Sergey Perepechin, I got started and created a 1.1 version of the Dashboard that I would use as a guide by looking at the Presentation Details to make sure that I added the correct SPEAK renderings to the correct placeholders.

After this, I created a new item based on the Speak-DashboardPage template.

With this in place, I created a PageSettings item in its usual home; under the SPEAK page:

I then added the various SPEAK 2.0 enabled renderings to the page. The first and most important one I added was the PageCode rendering. For PageCode, I made sure that SpeakCoreVersion was set to Speak Core 2-0.

I worked through adding the rest of the renderings that the Dashboard Page requires, by using my 1.1 Dashboard's Presentation Details as a guide.

It is important to note that some of the rendering's names have change slightly in 2.0, so you will need to review the SPEAK changes document to make sure that you pick the correct ones. DashboardPageStructure is an example of one of these.

Slowly but surely, I matched my renderings and placeholders to the 1.1 version of my Dashboard, using the 2.0 enabled renderings:

The end result; a shiny new SPEAK 2.0 enabled Dashboard Page:

SPEAK 2.0 Page Code

The next thing I wanted to explore was implementing my own 2.0 enabled JavaScript page code. The changes to page code are highlighted in the "Page code changes" section within the  SPEAK changes document.

The new structure is like this:

 function(Speak) {  
   
  Speak.pageCode({  
   
  initialize: function() {},  
   
  initialized: function() {},  
   
  beforeRender: function() {},  
   
  render:function() {},  
   
  afterRender:function() {}  
   
 });  
   
 })(Sitecore.Speak);  

Easy enough, and as they mentioned, we now have a lot more hooks.

Adding a reference to a JavaScript library using SPEAK 2.0 

I am a fan of Sitecore.Services.Client, and have successfully implemented it's EntityService thanks to some great posts and videos by Mike Robbins. Seeing a pattern here? :)

If you haven't checked it out, make sure you at least give this post a read: http://mikerobbins.co.uk/2015/01/06/entityservice-sitecore-service-client/

Ok, so back to what I was trying to test. I wanted to take a function that I had used in a SPEAK 1.1 app's page code, where I implemented EntityService and made a fetchEntities call. 

This is a a sample snippet of the SPEAK 1.1 page code:

1:  require.config({  
2:    paths: {  
3:      entityService: "/sitecore/shell/client/Services/Assets/lib/entityservice"  
4:    }  
5:  });  
6:    
7:  define(["sitecore", "jquery", "underscore", "entityService"], function (Sitecore, $, _, entityService) {  
8:    var AuthorAdmin = Sitecore.Definitions.App.extend({  
9:    
10:      initialized: function () {  
11:    
12:        this.GetRecentlyDrafted();  
13:      },  
14:    
15:      initialize: function () { },  
16:    
17:      GetRecentlyDrafted: function () {  
18:    
19:        var datasource = this.RecentlyDraftedDataSource;  
20:    
21:        var workflowService = new entityService({  
22:          url: "/sitecore/api/ssc/Arke-Sitecore-AuthorAdmin-Controllers/WorkFlowItems"  
23:        });  
24:    
25:        var result = workflowService.fetchEntities().execute().then(function (workFlowItems) {  
26:          for (var i = 0; i < workFlowItems.length; i++) {  
27:            datasource.add(workFlowItems[i]);  
28:          }  
29:        });  
30:    
31:      }  
32:    });  
33:    
34:    return AuthorAdmin;  
35:  });  

After SPEAKING (pardon the pun) to Mike, and spending time going through the SPEAK 2.0 components in my 8.1 instance located at

C:\inetpub\wwwroot\{YourInstanceName}\Website\sitecore\shell\client\Business Component Library\version 2\Layouts\Renderings,

I was able to figure out the correct SPEAK 2.0 version of the page code with a reference to the entity service JavaScript library:

1:  (function (Speak) {  
2:    
3:    require.config({  
4:      paths: {  
5:        entityService: "/sitecore/shell/client/Services/Assets/lib/entityservice"  
6:      }  
7:    });  
8:    
9:    Speak.pageCode(["entityService"], function (entityService) {  
10:      return {  
11:        initialize: function () {  
12:    
13:          this.GetRecentlyDrafted();  
14:    
15:        },  
16:        GetRecentlyDrafted: function () {  
17:    
18:          var datasource = this.RecentlyDraftedDataSource;  
19:    
20:          var workflowService = new entityService({  
21:            url: "/sitecore/api/ssc/Arke-Sitecore-AuthorAdmin-Controllers/WorkFlowItems"  
22:          });  
23:    
24:          var result = workflowService.fetchEntities().execute().then(function (workFlowItems) {  
25:            for (var i = 0; i < workFlowItems.length; i++) {  
26:              datasource.add(workFlowItems[i]);  
27:            }  
28:          });  
29:        }  
30:      }  
31:    });  
32:  })(Sitecore.Speak);  

You will notice that I am accessing a DataSource component on line 18. This is actually a custom JSON SPEAK DataSource component that was developed by Anders Laub.

I really liked his approach and the component's purpose because it separates responsibility.

See his post on it's creation here: http://laubplusco.net/creating-simple-sitecore-speak-json-datasource/

SPEAK 2.0 JSON DataSource Component

My next move was to take Anders' component and convert it to SPEAK 2.0.  As highlighted in Mike's post Sitecore SPEAK 2.0 Component vs SPEAK 1.1, a major difference in 2.0 is that it includes a server side model that represents our rendering parameters for our component. No sweat!

The new SPEAK 2.0 JSON DataSource component was made up of these 3 parts:

Model - JsonDataSourceRenderingModel.cs

1:  using Sitecore.Mvc.Presentation;  
2:    
3:  namespace Arke.Sitecore.AuthorAdmin.Models  
4:  {  
5:    public class JsonDataSourceRenderingModel : SpeakRenderingModel  
6:    {  
7:      public string Json { get; set; }  
8:    }  
9:  }  

You will notice above that my model inherits from SpeakRenderingModel.

Within Sitecore Rocks, you have to specify the class in the Model field:

View - JsonDataSource.cshtml

 @model Arke.Sitecore.AuthorAdmin.Models.JsonDataSourceRenderingModel  
 <script @Model.HtmlAttributes type="text/x-sitecore-jsondatasource">  
 </script>  

Javascript - JsonDataSource.js

1:  (function (Speak) {  
2:    Speak.component([], {  
3:      name: "JsonDataSource",  
4:      initialize: function () {  
5:    
6:        this.Json = null;  
7:    
8:      },  
9:      add: function (data) {  
10:    
11:        var json = this.Json;  
12:    
13:        if (json === null) {  
14:          json = new Array();  
15:        }  
16:    
17:        var newArray = new Array(json.length + 1);  
18:    
19:        for (var i = json.length - 1; i >= 0; i--) {  
20:          newArray[i + 1] = json[i];  
21:        }  
22:    
23:        newArray[0] = data;  
24:        this.Json = newArray;  
25:      }  
26:    });  
27:  })(Sitecore.Speak);  

More to Come!

I hope that this post helps shed some light on getting started with SPEAK 2.0.  I will continue blabbering about SPEAK (love the puns) with my findings in the near future.

Watch this space!

Read More

Sitecore Commerce Connect - Cleaning up the Rusty Shopping Cart

Background

In my last project, I really got to dig deep into Commerce Connect and can say that the Sitecore team did a fantastic job architecting the module.

As with all E-Commerce implementations, one of our requirements was to have a shopping cart that would persist for a period of time. The problem we were faced with was that the shopping cart needed to exist on the Sitecore side, and Commerce Connect expects the cart to be handled by the External Commerce System. Our External Commerce System was only responsible for handling the checkout process.

Reading through the Sitecore Connect Integration Guide, we discovered the "Storing a copy of the cart locally" section on page 17 that mentioned 2 sample cart repositories that ship with Commerce Connect:

• EaStateCartRepository - for storing carts in MongoDB  (enabled by default)
• EaStateSqlBasedCartRepository - for storing carts in SQL

Basically, what this section described was exactly what we wanted to do.

However, the documentation had a very specific caveat; "...these are only sample repositories and should not be used in a production scenario."

So, what could we wrong with the sample code? Surely if it shipped with the module and was enabled by default, it was OK?

Unfortunately, we discovered that the sample EaStateCartRepository had a significant flaw in it during our testing. A bug that could wreak havoc if put in production.

In this post, I will describe the issues with the sample cart repository, and how you can fix it so that it can be used in production.

Side note: I couldn't find a sample of the EaStateSqlBasedCartRepository mentioned in the documentation. Also, it was unfortunate that the sample repository didn't have any documentation describing it's functionality. So, you will need to get out your trusty decompiler to review the code.

About Commerce Connect

Before diving in, I want to take a step backwards to get some important facts highlighted about the module, as there has been a lot of confusion with partners wanting to understand comparisons between Commerce Connect and Commerce Server. This really isn't even an apples to oranges comparison. It's more like a cars to roads comparison.

What Commerce Connect IS NOT

• A stand-alone E-Commerce solution.
• A competitor or alternative to Sitecore Commerce Server.
• An upgrade path for Sitecore Ecommerce Services.
• A solution that provides out-of-the-box components like you would get with Active Commerce or uCommerce.

What Commerce Connect IS

• A pluggable framework. 
• A pass-through to an underlying E-Commerce platform or service. 
• A common integration layer that external systems can be integrated with.
• A means of bringing all the aspects of the Sitecore Experience Platform around user engagement to external E-Commerce systems.

With all this being said, if you are looking to implement a solution using the module, you REALLY NEED TO READ the following documentation in order to get a handle of all the features that the module provides and how to implement it correctly:

• Sitecore Connect Developer Guide
• Sitecore Connect Integration Guide

So, What's Wrong with the Cart Repository Anyway?

The sample cart repository issue that we found stemmed from the first two processors within the CreateOrResumeCart pipeline found in Sitecore.Commerce.Carts.config:

1:  <!-- CREATE OR RESUME CART  
2:        Initiate the creation of a shopping cart and in the process to:   
3:         - Load persisted, potentially abandoned cart, if present;  
4:         - Call resumeCart pipeline to resume loaded cart;  
5:         - Call createCart pipeline to create cart if no cart was found on previous steps.  
6:     -->  
7:     <commerce.carts.createOrResumeCart>  
8:      <processor type="Sitecore.Commerce.Pipelines.Carts.CreateOrResumeCart.FindCartInEaState, Sitecore.Commerce">  
9:       <param ref="eaStateCartRepository" />  
10:      </processor>  
11:      <processor type="Sitecore.Commerce.Pipelines.Carts.CreateOrResumeCart.RunLoadCart, Sitecore.Commerce" />  
12:      <processor type="Sitecore.Commerce.Pipelines.Carts.CreateOrResumeCart.RunResumeCart, Sitecore.Commerce" />  
13:      <processor type="Sitecore.Commerce.Pipelines.Carts.CreateOrResumeCart.RunCreateCart, Sitecore.Commerce" />  
14:     </commerce.carts.createOrResumeCart>  

The FindCartInEaState processor retrieves a CartId for the current user, using the request.UserId shown below:

1:  public override void Process(ServicePipelineArgs args)  
2:  {  
3:     Assert.ArgumentNotNull((object) args, "args");  
4:     CreateOrResumeCartRequest request = (CreateOrResumeCartRequest) args.Request;  
5:     List<CartBase> list = Enumerable.ToList<CartBase>(Enumerable.Where<CartBase>(this.repository.GetByUserName(request.UserId, request.ShopName), (Func<CartBase, bool>) (c =>  
6:     {  
7:      if (c.Name == request.Name)  
8:       return c.CustomerId == request.CustomerId;  
9:      return false;  
10:     })));  
11:     if (list.Count == 1)  
12:      args.Request.Properties["CartId"] = (object) list[0].ExternalId;  
13:     else  
14:      args.Request.Properties["CartId"] = (object) null;  
15:    }  
16: }  

When the RunLoadCart processor runs, it initializes the LoadCartFromEaState pipeline.

1:  public override void Process(ServicePipelineArgs args)  
2:  {  
3:     Assert.ArgumentNotNull((object) args, "args");  
4:     Assert.ArgumentNotNull((object) args.Request, "args.Request");  
5:     object obj = args.Request.Properties["CartId"];  
6:     if (obj == null)  
7:      return;  
8:     CreateOrResumeCartRequest resumeCartRequest = (CreateOrResumeCartRequest) args.Request;  
9:     this.RunPipeline("commerce.carts.loadCart", new ServicePipelineArgs((ServiceProviderRequest) new LoadCartRequest(resumeCartRequest.ShopName, obj.ToString(), resumeCartRequest.UserId), args.Result), args.Request);  
10: }  

This pipeline calls the GetByCartId method of the EaStateCartRepository. It does not pass the UserId as an argument, despite the GetByCartId method requiring a CartId and a UserId to perform the lookup.

The EaStateCartRepository has a private string called lastAccessedUserName, which should be the contact id of the visitor / contact. This variable is set in several methods in the EaStateCartRepository, however it is only used in the GetByCartId method.

1:  public virtual Cart GetByCartId(string cartId, string shopName)  
2:  {  
3:     Assert.ArgumentNotNullOrEmpty(cartId, "cartId");  
4:     Assert.ArgumentNotNullOrEmpty(shopName, "shopName");  
5:     Type type = this.entityFactory.Create("Cart").GetType();  
6:     Cart cart1 = (Cart) null;  
7:     if (!string.IsNullOrEmpty(this.lastAcessedUserName))  
8:      cart1 = (Cart) Enumerable.SingleOrDefault<CartBase>(this.GetByUserName(this.lastAcessedUserName, shopName, type), (Func<CartBase, bool>) (cart => cart.ExternalId == cartId));  
9:     return cart1 ?? (Cart) Enumerable.SingleOrDefault<CartBase>(this.GetAll(shopName, type), (Func<CartBase, bool>) (cart => cart.ExternalId == cartId));  
10: }  

Looking at Sitecore.Commerce.Carts.config, you will notice that only one instance of the EaStateCartRepository will be created by the provider engine:

1:  <eaStateCartRepository type="Sitecore.Commerce.Data.Carts.EaStateCartRepository, Sitecore.Commerce" singleInstance="true">  
2:     <param ref="entityFactory" />  
3:     <param ref="eaPlanProvider" />  
4:    </eaStateCartRepository>  

What this means is that the lastAccessedUserName variable can be the id of a different contact.

In our investigation, the lastAccessedUserName not matching the current contact id (Tracker.Current.Contact.ContactId), caused the GetAll fallback method to fire (line 9 in the GetByCartId method above).

This method is massively expensive as it iterates through all users to retrieve their carts!

1:  /// <summary>  
2:  /// Gets all.  
3:  ///   
4:  /// </summary>  
5:  /// <param name="shopName">Name of the shop.</param><param name="cartType">Type of the cart.</param>  
6:  /// <returns>  
7:  /// The collection of carts.  
8:  ///   
9:  /// </returns>  
10:  private IEnumerable<CartBase> GetAll(string shopName, Type cartType)  
11:  {  
12:   Assert.ArgumentNotNullOrEmpty(shopName, "shopName");  
13:   ID itemId = this.eaPlanProvider.GetEaPlanId(shopName).Item1;  
14:   EngagementPlanItem engagementPlanItem = new EngagementPlanItem(Tracker.DefinitionDatabase.GetItem(itemId));  
15:   ContactManager contactManager = CommerceAutomationHelper.GetContactManager();  
16:   List<CartBase> list = new List<CartBase>();  
17:   foreach (CustomItemBase customItemBase in engagementPlanItem.States)  
18:   {  
19:       foreach (ID id in (IEnumerable<ID>) AutomationContactManager.GetStateContactsIdsPaged(customItemBase.ID))  
20:       {  
21:        Contact contact = contactManager.LoadContactReadOnly(id.Guid);  
22:        if (contact == null)  
23:        {  
24:            if (Tracker.Current.Contact.ContactId == id.Guid)  
25:             contact = Tracker.Current.Contact;  
26:            else  
27:             continue;  
28:        }  
29:        IEnumerable<CartBase> customDataOrNull = CommerceAutomationHelper.GetCustomDataOrNull<IEnumerable<CartBase>>(AutomationStateManager.Create(contact).GetCurrentStateInPlan(itemId), "commerce.carts");  
30:        if (customDataOrNull != null)  
31:            list.AddRange(customDataOrNull);  
32:       }  
33:   }  
34:   return (IEnumerable<CartBase>) list;  
35:  }  

As we ran tests with more and more users, the mismatch started happening all the time, and the expensive GetAll fallback method kept firing, causing the server's CPU usage to spike.

Yikes!

Fixing the Cart Repository 

How Sitecore Should Fix It

To permanently fix this issue, the Sitecore team should make the LoadCartFromEaState pipeline pass the GetByCartId method the ContactId and the lastAccessedUserName variable should be purged from the class.

My Fix

In my case, I created a new cart repository class that inherited from EaStateCartRepository. I then added a new GetCartByIdFallback method where I would retrieve the cart based on the current contact id (Tracker.Current.Contact.ContactId).

1:  private Cart GetCartByIdFallback(string cartId, string shopName, Type cartType)  
2:  {  
3:       if (Tracker.Current.Contact == null)  
4:       {       
5:            return null;  
6:       }  
7:    
8:       string contactId = Tracker.Current.Contact.ContactId.ToString();  
9:    
10:       List<CartBase> carts = this.GetByUserName(contactId, shopName, cartType).ToList();  
11:       Cart cart1 = (Cart) null;  
12:    
13:       if (carts.Any())  
14:       {  
15:            cart1 = (Cart) carts.SingleOrDefault(cart => cart.ExternalId == cartId);  
16:    
17:            if (cart1 != null)  
18:            {  
19:                 return cart1;  
20:            }  
21:       }  
22:    
23:       return null;  
24:  }  

Finally, I overrode the GetByCartId method to use my GetCartByIdFallback instead of the expensive GetAll fallback method if the cart couldn't be found for the lastAcessedUserName, or as mentioned above, the lastAcessedUserName didn't match the current contact id (Tracker.Current.Contact.ContactId).

1:  public override Cart GetByCartId(string cartId, string shopName)  
2:  {  
3:       Assert.ArgumentNotNullOrEmpty(cartId, "cartId");  
4:       Assert.ArgumentNotNullOrEmpty(shopName, "shopName");  
5:       Type type = this.entityFactory.Create("Cart").GetType();  
6:       Cart cart1 = (Cart)null;  
7:       if (!string.IsNullOrEmpty(this.lastAcessedUserName))  
8:       {  
9:            cart1 = (Cart)Enumerable.SingleOrDefault<CartBase>(this.GetByUserName(this.lastAcessedUserName, shopName, type), (Func<CartBase, bool>)(cart => cart.ExternalId == cartId));  
10:       }  
11:       return cart1 ?? GetCartByIdFallback(cartId, shopName, type);  
12:  }  

Final Configuration

To put my new cart repository into action, I simply had to modify the Sitecore.Commerce.Carts.config to use my new repository:

1:  <eaStateCartRepository type="CommerceTest.MyNewCartRepository, CommerceTest" singleInstance="true">  
2:   <param ref="entityFactory" />  
3:   <param ref="eaPlanProvider" />  
4:  </eaStateCartRepository>  

After this, the cart worked perfectly!

Read More

Getting on the 8.1 Train

Background

I recently got my development team on one of our projects updated from 8.0 Update-5 to 8.1. It was a pretty seamless process, obviously always easier when you are in development as opposed to performing an upgrade on a production site.

Content 

We use Hedgehog's Team Development for Sitecore (TDS), and so all of our content was serialized and available in source control.

Solr 

We are using Solr on the project, and after following these steps, everything was working like a champ:

• Added the required assemblies from the 8.1 Solr Support package from here: http://bit.ly/1NlFu4W. 
• Updated our Global.asax to include the Castle / Solr Support. 
• Ran Patrick Perrone's Solr switching script http://bit.ly/1YkXPUS. 
• Added the new sitecore_marketing_marketingdefinitions index. 
• Rebuilt all indexes to verify Solr and Sitecore were playing nice.   

MVC Areas 

We have been using a custom implementation to address the unsupported areas issue for quite some time. The switch over to Sitecore controller renderings with support for areas was straightforward. I just removed the pipeline processor in our custom implementation, changed the template type from our custom controller rendering to the new OOTB controller rendering with the area field populated and it worked perfectly.

NuGet and Such 

As most of us are aware, 8.1 supports the latest and greatest versions of MVC and Web Forms. So I had to make sure that my various modules and solution's NuGet Packages were up-to-date with version 5.2.3 of MVC, Razor 3.2.3 and WebPages 3.2.3. One thing I noticed that was interesting was the version of Microsoft.AspNet.Cors was still 5.1.2.

Glass Mapper 

We use Glass Mapper in our implementations, and I had no issues after installing the latest version (4.0.5.54 at the time of writing this) from our friend Michael Edwards: http://bit.ly/1lCnwSW

Castle Windsor 

Castle is our DI framework of choice. For some reason, when I got 8.1 up and running, I started getting the following error when running my IOC controllers:

Looks like you forgot to register the http module Castle.MicroKernel.Lifestyle.PerWebRequestLifestyleModule Add '<add name="PerRequestLifestyle" type="Castle.MicroKernel.Lifestyle.PerWebRequestLifestyleModule, Castle.MicroKernel" />' to the section on your web.config

 Adding the following entries to the web.config fixed the issue:

 <system.webServer>  
  <modules>  
   <add name="PerRequestLifestyle" type="Castle.MicroKernel.Lifestyle.PerWebRequestLifestyleModule, Castle.Windsor" />  
  <handlers>  
   <add name="PerRequestLifestyle" verb="*" path="*.castle" preCondition="managedHandler" type="Castle.MicroKernel.Lifestyle.PerWebRequestLifestyleModule, Castle.Microkernel"/>  

I am interested to understand why this has worked via Microsoft.Web.Infrastructure dynamically registering Castle before instead of me having to explicit register Castle in the web.config.

sc_site 

This parameter has always been around, but now, it is added to the url when you are in Preview or Experience Editor mode within your site(s). The problem I am having is that for some reason, it is always set to "website". We know that "website" is special in Sitecore and necessary for features to work correctly in a multi-site setup. But why is sc_site always "website"? I validated all my site configurations. Looks to be an annoying bug.

UPDATE: Thanks to Kamruz for pointing out the new <setting name="Preview.ResolveSite" value="false"/> config value in Sitecore.config. Changing the value to "true" fixed the issue.

Read More

How To Fix Sitecore Connector Version Issues After TDS Upgrade

Background

It is pretty obvious that if you are doing some form of Sitecore development, you are probably using Hedgehog's Team Development for Sitecore (TDS). It makes life so much easier when working on large projects with several team members.

The Upgrade Problem

I am running Visual Studio 2015, and ran into a peculiar problem after updating to the latest version of TDS. I had previously been running 5.1.0.17 and upgraded to version 5.1.0.20.

I was stuck in an endless loop where my connector was showing this error: "Warning: The version of the sitecore connector is from a different version of TDS." An addition, my output window was displaying this message: "Warning: Connector returned version 5.1.0.17. Looking for version 5.1.0.15."

 Warning: Connector returned version 5.1.0.17. Looking for version 5.1.0.15  
 Connection Test Start: 2015-11-05 09:13:54  
 Connection Test Pass: Content downloaded from http://hfedev/sitecore  
 Connection Test Pass: TDS connection test successful  
 Connection Test Message: Connector version local 5.1.0.15 and msbuild 5.1.0.15  
 Connection Test Pass: MSBuild version correct.  
 Connection Test Pass: Sitecore.Kernel.dll and Web.config found beneath G:\Inetpub\wwwroot\HFEDev\Website  
 Connection Test Pass: TDS service files are correctly installed.  
 Warning: Connector returned version 5.1.0.17. Looking for version 5.1.0.15.  
 Warning: Connector returned version 5.1.0.17. Looking for version 5.1.0.15.  
 Connection Test Failure: Failed to complete test.  
 Exception The given key was not present in the dictionary. (KeyNotFoundException):  
   at System.Collections.Generic.Dictionary`2.get_Item(TKey key)  
   at HedgehogDevelopment.SitecoreProject.VSIP.Utils.Support.CallServiceWrapper[T](TdsServiceSoapClient client, SitecoreProjectNode project, Func`2 clientCall)  
   at HedgehogDevelopment.SitecoreProject.VSIP.ProjectTests.AccessGuidTest.Execute()  
   at HedgehogDevelopment.SitecoreProject.VSIP.Dialogs.BuildTestDialog.RunTests()  
 Connection Test Finish: 2015-11-05 09:14:02  

This was pretty weird as I had just upgraded from version 5.1.0.17 to 5.1.0.20.

The Path To Solving The Problem

I started out by taking a bit of an invasive approach to fix the problem, as it had worked for me before after I ran into a similar issue. I navigated to my Sitecore instance, and deleted  the _DEV folder (that contains the TdsService.asmx) and then deleted the TDS assemblies from my bin folder. After this, I tried to install the connector again.

Unfortunately, this didn't help fix the issue, and I now had the following error:

I checked to make sure that my Access Guid in my various projects matched what was in the service's web.config, and they were in fact the same:

Getting Closer

After restarting Visual Studio, and trying my Sitecore Connector again, I started getting the same type of error message that I got initially, except now it was showing that it was looking for version 5.1.0.20 which was the version that should have been installed.

 Connection Test Start: 2015-11-05 09:43:44  
 Connection Test Pass: Content downloaded from http://hfedev/sitecore  
 Connection Test Pass: TDS connection test successful  
 Connection Test Message: Connector version local 5.1.0.20 and msbuild 5.1.0.20  
 Connection Test Pass: MSBuild version correct.  
 Connection Test Pass: Sitecore.Kernel.dll and Web.config found beneath G:\Inetpub\wwwroot\HFEDev\Website  
 Connection Test Pass: TDS service files are correctly installed.  
 Warning: Connector returned version 5.1.0.17. Looking for version 5.1.0.20.  
 Warning: Connector returned version 5.1.0.17. Looking for version 5.1.0.20.  
 Connection Test Failure: Failed to complete test.  
 Exception The given key was not present in the dictionary. (KeyNotFoundException):  
   at System.Collections.Generic.Dictionary`2.get_Item(TKey key)  
   at HedgehogDevelopment.SitecoreProject.VSIP.Utils.Support.CallServiceWrapper[T](TdsServiceSoapClient client, SitecoreProjectNode project, Func`2 clientCall)  
   at HedgehogDevelopment.SitecoreProject.VSIP.ProjectTests.AccessGuidTest.Execute()  
   at HedgehogDevelopment.SitecoreProject.VSIP.Dialogs.BuildTestDialog.RunTests()  
 Connection Test Finish: 2015-11-05 09:44:02  

The next thing I did was I add a new empty TDS project to my existing solution. Interestingly enough, the connector worked like a champ with no issues:

I then went ahead and tested the connection on the rest of my projects and they all worked too!

Yeah baby!

Problem solved? Well, not exactly.

The Final Solution

After successfully testing the connection on all of my existing TDS projects, I went ahead and deleted the empty TDS project. I am sure you can guess what happened next right? The other projects started failing to connect again!

So, what was different after I added the new TDS project? Mmmm. It had a different Access Guid than the other projects. That was weird though. How would the other projects connect then?

I went ahead and created another new, empty TDS project, configured it, did a test and made sure that everything passed. Then I copied the Access Guid from the new TDS project and replaced the Access Guids in all my projects with the new Guid. 

After doing this, I did a test on all the TDS projects in my solution, and every one of them worked like a champ.

Finally, I went ahead and I removed the empty TDS project. This time, after removing the empty project,  all my connections continued to work perfectly!

Read More