Sunday, July 8, 2018

Sitecore xDB: Performance Tuning your MongoDB Driver Configuration Settings


The Goal

Working with my team on a high-traffic client's Sitecore Commerce site, we were tasked with improving MongoDB connection management on the Content Delivery servers to help alleviate pressure on the servers and MongoDB, particularly during busy times of the day, and at times when traffic surges occur due to marketing campaigns or other real-world events.

The Key Settings

We confirmed that Sitecore ships with the default MongoDB driver settings that are actually set in the driver code.  You can view the default settings in the driver code, by following this GitHub link:

Working with mLab Support, we determined that our focus would be on the following:

Min Pool Size

We decided to increase the Min Pool Size from the default 0 to 20. The mLab team approved this suggestion on the basis that we observed the Content Delivery server's connection pools maxing out due to the amount of operations that were happening during the Sitecore startup process.

Max Pool Size

We increased our Max Pool Size from the default of 100 to 150 in order to better accommodate surges in connection demand. The purpose of this update was to lessen the chance of running out of connections altogether.

Connection Idle Time

We increased the Connection Idle Time from the default of 10 minutes to 25 minutes to reduce the need to create new connections during normal and high-traffic surges.

Connection Life Time

We dropped the default setting of 30 minutes down to 0 (no lifetime). This change was based on the default setting that could also be a contributing factor to our observed connection churning.

Per this thread, a MongoDB engineer (driver author) suggested that this setting is likely not needed:

The How

As Kam explains in his post, Sitecore exposes an empty updateMongoDriverSettings pipeline that you can hook into to modify configurations that are not available in the connection string.

I created a processor to add to this pipeline that alters the MongoClientSettings:

Finally, I added the following patch to add the processor to pipeline, allowing us to pass the updated MongoDB driver settings to the custom processor:

Final Note

A special thanks to Dan Read (Arke), Alex Mayle (Sogeti) and the mLab Support Team for their contributions.

Thursday, May 10, 2018

Sitecore Azure Search: Sharing a Search Service in a Development Environment



When developing on the Sitecore Platform using Azure Search, the cloud-based service behaves differently, and so it is best to develop against an actual search service to ensure that you uncover any unexpected behavior before pushing to a higher environment that is using Azure Search.

I put together a quick patch file that updates the Sitecore Index names, allowing your development team to share a Azure Search service.

Simply set the environment variable and drop this file in the "Include/zzz" folder.

Note: There is a platform dependency on the "sitecore_marketingdefinitions_master" and "sitecore_marketingdefinitions_web" naming so those 2 indexes are excluded from the patch.

In a development environment, we can live with sharing these indexes.

Update: Kudos to John Rappel for suggesting to use a variable to make the implementation simple.

Sunday, April 29, 2018

Exploring Sitecore Managed Cloud Part 1: Tiers, Sizing, Provisioning and Upgrades



I have been working within a client's Sitecore Managed Cloud environment for the last several months, and wanted to share some insights gained from my experience in a series of blog posts, this being the first.

Tier Configuration

Sitecore Managed Cloud Hosting offers a variety of tiers based on traffic volume.

Each tier has a recommended hardware configuration, as shown here:

When you commit to a tier, Sitecore's Managed Cloud Team will provision your Azure infrastructure to support that tier. You will have the ability to increase to the next tier when traffic increases are expected.

If traffic exceeds the threshold of the currently subscribed tier, an overage charge will be applied. From what I understand, the overage cost is about 25% greater than the additional cost of jumping to the next tier.

Tier Sizing and Overage

Sitecore Managed Cloud has the concept of an “included Azure spend” linked to each tier.

If you need to scale up or out, and the cost associated with your scaling goes above your “included Azure spend”,  it is up to you to pay an overage fee. This fee is based on an “overage multiplier” that Sitecore provides you with.

Here are some overage examples:

  • Additional Traffic beyond tier
  • Additional Web Apps Used (Add 1 or more Content Delivery Web App)
  • Exceed Storage Limits (loading large amounts of videos or pdfs)
  • Exceeding Storage Limits for xDB

XP–XSmall XP–Small XP–Medium XP–Large XP-XLarge
TRAFFIC (in visits/mo.) 0–100k 100k –200k 200k –1MM 1MM –5MM 5MM –10MM
Content Management 1 (B2) 1 (B2) 1 (B2) 1 (B2) 1 (B2)
Content Delivery 1 (B2) 2 (B2) 3 (B2) 4 (S2) 8 (S3)
Bandwidth 20 GB 40 GB 40 GB 60 GB 100 GB

The Azure App Service Plan pricing page provides details regarding what B2s and S3s are:

Sitecore recommends proactive increase of a given topology’s tiers when traffic increases are expected. However, the alternative overage charge poses only a moderate increase in cost to maintain site performance during unexpected, temporary spikes in volume. It also can serve as an indicator that advancement to a larger tier should be considered.

Infrastructure Provisioning

Before provisioning the new infrastructure, the Sitecore Managed Cloud Hosting Team will request that you provide them with the following  information:

  1. Sitecore Version (Eg: Sitecore XP 9.0 Update-1)
  2. Logical Name (Eg: MySuperSolution)
  3. Location of Deployment (Eg: East US)
  4. Location of your Microsoft Application Insights Resources (Eg: East US)
  5. Microsoft Live IDs who can access your Managed Cloud set
From my experience, provisioning will take about a day.

NOTE: As mentioned here: Sitecore provisions you with a temporary license file that is valid for one month. When the temporary license expires, Sitecore stops working, therefore it is important that you upload a valid permanent license.xml file as soon as possible.


Unfortunately, you as a partner or customer will be responsible for upgrades after the Sitecore Managed Cloud Hosting Team has provisioned your infrastructure.

Sitecore initially provisioned our infrastructure using Sitecore 9 Initial Release. After a couple weeks, Sitecore Update-1 was released, and we opened a ticket to request for them to provision all environments on the newer version.

The only reason they did this for us was because we hadn't deployed anything to the new environments, and they could simply delete the existing and provision the new ones using the updated Azure Web Packages.

So the point is - they will NOT upgrade your environment after you have deployed your custom solution into it.

Monday, April 2, 2018

SolrProvider SolrSearchFieldConfiguration Error After Upgrade from Sitecore 8.1 Update-3 to 8.2 Update-6



In a previous post, I wrote about upgrading an existing client's large, multisite 8.1 rev. 160519 (8.1 Update-3) instance to 8.2 rev. 171121 (8.2 Update-6).

After multiple rounds of testing, I deployed the upgraded instance to our client's Staging server (Windows Server 2008 R2). When navigating to the Content Editor, I ran into the following Solr related error:

Unable to cast object of type 'System.Collections.Concurrent.ConcurrentBag`1[Sitecore.ContentSearch.SolrProvider.SolrSearchFieldConfiguration]' to type 'System.Collections.Generic.IReadOnlyCollection`1[Sitecore.ContentSearch.SolrProvider.SolrSearchFieldConfiguration]'.

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

Exception Details: System.InvalidCastException: Unable to cast object of type 'System.Collections.Concurrent.ConcurrentBag`1[Sitecore.ContentSearch.SolrProvider.SolrSearchFieldConfiguration]' to type 'System.Collections.Generic.IReadOnlyCollection`1[Sitecore.ContentSearch.SolrProvider.SolrSearchFieldConfiguration]'.

Digging In

After reviewing the stack trace and decompiling the Sitecore.ContentSearch.SolrProvider.dll , I focused on the SolrFieldNameTranslator.StripKnownExtensions method in the SolrFieldNameTranslator class:

1:  public string StripKnownExtensions(string fieldName)  
2:    {  
3:     fieldName = this.StripKnownCultures(fieldName);  
4:     foreach (SolrSearchFieldConfiguration availableType in (IEnumerable<SolrSearchFieldConfiguration>) this.fieldMap.GetAvailableTypes())  
5:     {  
6:      if (fieldName.StartsWith("_", StringComparison.Ordinal))  
7:      {  
8:       if (!fieldName.StartsWith("__", StringComparison.Ordinal))  
9:        break;  
10:      }  
11:      string str = availableType.FieldNameFormat.Replace("{0}", string.Empty);  
12:      if (fieldName.EndsWith(str, StringComparison.Ordinal))  
13:       fieldName = fieldName.Substring(0, fieldName.Length - str.Length);  
14:      if (fieldName.StartsWith(str, StringComparison.Ordinal))  
15:       fieldName = fieldName.Substring(str.Length, fieldName.Length);  
16:     }  
17:     return fieldName;  
18:    }  
Looking at line 4 above, the GetAvailableTypes method is shown below:

  private readonly ConcurrentBag<SolrSearchFieldConfiguration> availableTypes = new ConcurrentBag<SolrSearchFieldConfiguration>();  
  public IReadOnlyCollection<SolrSearchFieldConfiguration> GetAvailableTypes()  
    return (IReadOnlyCollection<SolrSearchFieldConfiguration>) this.availableTypes;  

As you can see in the code, when calling the GetAvailableTypes method, it casts the ConcurrentBag<SolrSearchFieldConfiguration> type to IReadOnlyCollection<SolrSearchFieldConfiguration> and this is where the "Unable to cast object" error message came from:

Unable to cast object of type 'System.Collections.Concurrent.ConcurrentBag`1[Sitecore.ContentSearch.SolrProvider.SolrSearchFieldConfiguration]' to type 'System.Collections.Generic.IReadOnlyCollection`1[Sitecore.ContentSearch.SolrProvider.SolrSearchFieldConfiguration]'.


According to, the ConcurrentBag<T> class implements IReadOnlyCollection<T> interface in .NET Framework 4.5 and above. However, according to MSDN, in .NET Framework 4.0 it doesn't implement the IReadOnlyCollection<T> interface. The IReadOnlyCollection<T> interface was introduced in .NET Framework 4.5.

I went ahead and verified that the .NET Framework 4.5.2 was installed on the server. Next I checked my Web.config, and confirmed that the compilation targetFramework was set to "4.5.2" and that my httpRuntime targetFramework was set to "4.5.2":

    <compilation targetFramework="4.5.2"/>
    <httpRuntime targetFramework="4.5.2"/>

I then tried to repair the .NET framework, and even rebooted the server after the repair, and it didn't solve the problem.

Microsoft Security Update Woes

It was obvious that my 2 initial assumptions about not having the .NET Framework 4.5.2 installed or having a configuration issue were wrong.

As it turned out, .NET 4.5.2 did in fact contain the System.Collections.Concurrent.ConcurrentBag<T> class from System.dll which implemented the IReadOnlyCollection<T> interface, but there were a series of security updates that were rolled out by Microsoft, where the ConcurrentBag<T> did not implement the IReadOnlyCollection<T>. The security updates were built based on .NET framework 4.0 which did not have IReadOnlyCollection<T> interface at all.

In order to check whether one of the aforementioned security updates was installed, you could review the version of C:\Windows\Microsoft.NET\Framework\v4.0.30319\System.dll

For the regular .NET framework 4.5.2 this version looks like 4.5.2... , but if a security update was installed it will look like 4.0.30319.36388.

The Final Solution - So Simple It Hurt

Upgrading the .NET framework to 4.6.1 solved this issue because we were back to having the System.Collections.Concurrent.ConcurrentBag<T> class from System.dll which implemented the IReadOnlyCollection<T> interface.

Monday, February 26, 2018

My Upgrade Experience from Sitecore 8.1 Update-3 to 8.2 Update-6

I was assigned the task of upgrading an existing client's large, multisite 8.1 rev. 160519 (8.1 Update-3) instance to 8.2 rev. 171121 (8.2 Update-6).  This particular client wasn't ready to go all the way to version 9.0, but will do so in the near future.

It took me longer than anticipated to get things up and running, simply because I needed to perform some updates to their custom solution.

Side Notes

The Sitecore solution that I was upgrading was using Castle Windsor as the Inversion of Control container along with Glass Mapper.

Getting Ready

To get started, I navigated over to the site to arm myself with the files needed to perform the upgrade. The files that I downloaded from the site included:

  • Upgrade guide: Sitecore-8.2-Update-6-Upgrade-Guide.pdf
  • Sitecore update package: Sitecore 8.2 rev. 171121 (update package)
  • Configuration files for upgrade: Sitecore 8.2 rev. 171121 (config files)
  • Sitecore Update Installation Wizard: Sitecore Update Installation Wizard 2.0.2 rev. 170703
  • ZIP archive of the Sitecore site root folder: Sitecore 8.2 rev.

The software tools I use when performing upgrades are:

The road to 8.2 Update-6

These are the steps necessary to perform the upgrade:

Disabled xDB located in Sitecore instance \Website\App_Config\Include\Sitecore.Xdb.config
  • <setting name="Xdb.Enabled" value="false" />
  • <setting name="Xdb.Tracking.Enabled" value="false" />

The instance didn't have Email Experience and WFFM modules, so I didn't need disable their respective config files.

Ran the SQL database script called "CMS82_BeforeInstall.sql" located in \Sitecore 8.2 rev. 171121 (config files)\Database Upgrade Script on all Sitecore databases:
  • Core
  • Master
  • Web
  • Reporting (Analytics)

Installed the Sitecore Update Installation Wizard 2.0.2 rev. regular Sitecore package.

After it completed, I proceeded to install the "Sitecore 8.2 rev. 171121.update" update package using the installation wizard:  /sitecore/admin/UpdateInstallationWizard.aspx.

You will need to unzip the Sitecore 8.2 rev. 171121 (update package).zip in order to obtain the "update" file that Sitecore requires.

After clicking the "Analyze the package" button,  I opted NOT to install files as I preferred to start with a clean copy of the web root of Sitecore 8.2 Update-6.

I feel like this is a cleaner approach as it helps avoid having legacy cruft make its way into the new instance.

The package installation completed without any issues.

Instance Preparation and Comparing Files

I proceeded to stand up a clean version of Sitecore 8.2 Update-6 alongside my legacy 8.1 Update-3 instance and used the Beyond Compare app to compare the files. 

Apart from the Web.Config, the Sitecore.config was the next file where I saw the most differences.

Pro Tip: It is best practice is to move any differences that you find in vanilla config files to separate patch files. That way, life will be much easier for future upgrades.

Updating your Custom Solution

As some of the config files compared could exist in your custom solution, it is best to update the files in your solution as soon as you have completed your comparisons / merges on your Sitecore instance.

I worked in a new branch in source control, so that I could gradually update the files, and commit them as I made progress.

Sitecore 8.2 moved to .NET Framework version 4.5.2 from 4.5 in 8.1. So the target framework in each of the solution's projects needed to be updated:

NuGet Fun

The custom solution I was working with had all the Sitecore referenced assemblies in a single Nuget package, consumed via a custom feed. I opted to switch to the Sitecore public NuGet feed:

A lot of time was spent making sure the correct NuGet packages were loaded so that references where correct. As I was working with 8.2 rev. 171121, I matched my NuGet packages to the version by using the 8.2.171121, "NoReferences" packages.

As Jeremy Davis pointed out: "...the .NoReferences packages include the binary files but don’t have dependencies on other packages. So if you want to pick out individual DLLs without having to import all of the other assemblies that file depends on, choose these packages. It’s a bit more effort to manually add each of these that you need – but it means your project only refers to the specific files you want."

Note: When updating packages like WebGrease for example, it is important to match the assembly version in the Sitecore bin folder to the NuGet package versions.

Working with Solr

As I was using Solr as my search provider, I used Patrick's Powershell script to set my config files to use Solr.

The Sitecore instance was using the Single Instance Solr Configuration - Patch #391039, that I discussed in this post:

Support for Solr out-of-the box with this patch was added from Sitecore 8.2 Update-1 on, so I didn't have to include any configurations and files referencing this patch. Most of my work involved me changing my Solr index configurations

Sitecore.Support.ContentSearch.SolrProvider.SwitchOnRebuildSolrSearchIndex, Sitecore.Support.391039

Sitecore.ContentSearch.SolrProvider.SwitchOnRebuildSolrSearchIndex, Sitecore.ContentSearch.SolrProvider


 <index id="my_custom_master_index" type="Sitecore.Support.ContentSearch.SolrProvider.SwitchOnRebuildSolrSearchIndex, Sitecore.Support.391039">

 <index id="my_custom_master_index" type="Sitecore.ContentSearch.SolrProvider.SwitchOnRebuildSolrSearchIndex, Sitecore.ContentSearch.SolrProvider">

Bye Bye IsPageEditorEditing, Hello IsExperienceEditorEditing

As Darren mentioned in his post, Sitecore depreciated the variables IsPageEditor, and IsPageEditorEditing in Sitecore 8.0 Update 6, but kept the methods in all versions of 8.1. 

It would have been nice to have used the Obsolete attribute so that there wouldn't be such a surprise when upgrading to 8.2, and having all your usages of this method break your solution.

The fix was simple enough though. I performed a "find and replace" 



Problems with Castle Windsor

The solution I was working in was using Castle Windsor and Castle Core I opted to update Castle Windsor to version and Castle Core because I wanted the bug fixes and enhancements of the newer releases.

After deploying the updated assemblies to my upgraded Sitecore instance, I ran into the following error:

Could not load file or assembly 'Castle.Windsor, Version=, Culture=neutral, PublicKeyToken=407dd0808d44fbdc' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference.

Castle Core changed the AssemblyVersion attribute to only include major version numbers so that they could avoid assembly binding errors with future releases/upgrades of Castle Core:

In my case, the error was happening because I had assemblies that were compiled against the new AssemblyVersion strategy.

Applying the following assembly binding redirects in my Web.config, fixed the issue.

        <assemblyIdentity name="Castle.Core" publicKeyToken="407dd0808d44fbdc" culture="neutral" />
        <bindingRedirect oldVersion="" newVersion="" />

        <assemblyIdentity name="Castle.Windsor" publicKeyToken="407dd0808d44fbdc" culture="neutral" />
        <bindingRedirect oldVersion="" newVersion="" />

Minor Problem with Glass Mapper

Like Castle, I also opted to update Glass Mapper to a higher version. By doing so,  I ran into a small issue, similar to what is described here:

In my case, I discovered that I was simply missing the Glass.Mapper.Sc.Mvc references to the new assembly in the MVC 52 folder in the Nuget package, and the updated assembly in my Sitecore bin folder.

Minor Problem with WebGrease

After making my way through the above-mentioned problems, I ran into a WebGrease version issue. 

Inner Exception: Could not load file or assembly 'WebGrease, Version=, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)

The fix for this was to simply update my assembly binding redirect in the Web.config.

        <assemblyIdentity name="WebGrease" publicKeyToken="31bf3856ad364e35" culture="neutral" />
        <bindingRedirect oldVersion="" newVersion="" />

        <assemblyIdentity name="WebGrease" publicKeyToken="31bf3856ad364e35" culture="neutral" />
        <bindingRedirect oldVersion="" newVersion="1.6.5135.21930" />

Let Me Hear You Say "Hallelujah"!

After I completed all these updates and fixes, I was presented with a beautiful new instance of Sitecore 8.2 Update-6 where my site loaded beautifully and my logs were clean.

Per Sitecore's upgrade guide, I completed the following final steps:
  • Cleared the browser cache. 
  • Published the site. 
  • Rebuilt the search indexes and the link database. 
  • Redeployed marketing definitions. 

I made sure to review my Sitecore logs after performing all of these tasks, and was happy to report that they stayed error free.

Monday, December 4, 2017

Exploring Sitecore xConnect: Working with Contacts and the xConnect Client API



As I started exploring xConnect in XP 9, one of the questions I asked myself was how the change in the xDB contact and interaction model architecture would effect my existing Sitecore xDB implementations if we decided to upgrade. 

With this in mind, the focus of this post is on the changes to contact identification and updating contacts within Sitecore context, and what you need to know when you start working with the xConnect Client API in and outside of Sitecore context.

The xConnect documentation site was my initial point of reference, along with some guidance from Jason Wilkerson's series of posts

Working with Contacts

Identifying Contacts 

In XP 7.5 - 8.x, each xDB contact could be identified using a single, unique value. 

Your code looked like this:

 Tracker.Current.Contact.Identifiers.IdentificationLevel = ContactIdentificationLevel.Known;  

This changed in XP 9, as you now need to specify the source along with a unique value when identifying the contact.

 Tracker.Current.Session.IdentifyAs("corporateweb", "menglish");  
 //corporateweb is the source and menglish is the identifier.   

In XP 9, each contact can have multiple identifiers and sources within the new model. The magic lies in the ability to identify and merge contacts from all different sources together into a single contact.

Omnichannel contact identification and merging is a powerful thing!

Updating Contacts

In XP 7.5 - 8.x, updating contact information in xDB was achieved by using the Tracker Contact (Tracker.Current.Contact), and calling GetFacet using the Interface of the type and passing in the name of the Facet. 

Your code looked like this:

 var existingContact = Tracker.Current.Contact;  
 var personalFacet = existingContact.GetFacet<IContactPersonalInfo>("Personal");  
 personalFacet.FirstName = "Martin";  
 personalFacet.Surname = "English";  

This code will still run in XP 9, but it will no longer save the information to xDB. The update will only persist within session.

In XP 9, you need to use the xConnect Client API in order to update contact information.

 using (Sitecore.XConnect.Client.XConnectClient client = Sitecore.XConnect.Client.Configuration.SitecoreXConnectClientConfiguration.GetClient())  
           var webContactIdentifier = Tracker.Current.Contact.Identifiers.FirstOrDefault(t => t.Source == "corporateweb")?.Identifier;  
           var existingContact = client.Get<Sitecore.XConnect.Contact>(new IdentifiedContactReference("corporateweb", webContactIdentifier), new Sitecore.XConnect.ContactExpandOptions(PersonalInformation.DefaultFacetKey));  
           if (existingContact != null)  
                var personalFacet = existingContact.GetFacet<PersonalInformation>() ?? new PersonalInformation();  
                personalFacet.FirstName = "Martin";  
                personalFacet.LastName = "English";  
                client.SetFacet(existingContact, PersonalInformation.DefaultFacetKey, personalFacet);  
      catch (XdbExecutionException ex)  
           //Oops, something went wrong  

Some important things that you need to be aware of:

  • The xConnect Contact and Tracker Contact models are different. 
  • The legacy facet classes are available in XP 9, so your existing code won't break. 
  • When you update your Tracker facet's in session, the update will persist throughout session, but won't save to xDB.
  • On session end, the Tracker contact data is run through a series of conversion pipelines where it ends up in xConnect. 
  • When the web contact returns to the site, the Tracker contact is hydrated through another set of conversion pipelines using the contact's data stored in xConnect.
  • If you have been updating xDB contact data using the Tracker Contact and calling GetFacet, you will need to update your code to use the xConnect Client API in order to update contact information. 

Working with the xConnect Client API

xConnect Client API within Sitecore Context

If you are working within a Sitecore Context, using the xConnect client is really straightforward. You don't have to worry about endpoints or certificates, as all that is abstracted. An example of this is shown above.

xConnect Client API outside Sitecore Context

Unsecured Client Connection

The example code that Jason has on GitHub requires an untrusted client connection in order to work.


 private static XConnectClient GetClient()  
   var config = new XConnectClientConfiguration(new XdbRuntimeModel(CollectionModel.Model), new Uri("https://sc90.xconnect"), new Uri("https://sc90.xconnect"));  
   catch (XdbModelConflictException ex)  
    return new XConnectClient(config);  

In order to run this, you need to disable these two xml files: sc.XConnect.Security.EnforceSSLWithCertificateValidation.xml and sc.XConnect.Security.EnforceSSL.xml located at:  [location of your xConnect instance]\App_data\config\sitecore\CoreServices

If you don't, you will receive the following Sitecore.XConnect.XdbCollectionUnavailableException "The HTTP response was not successful: Unauthorized".

Making this type of adjustment is fine if you are writing some POC code, but it is obviously not recommended as you start writing code for your customers.

Secured Client Connection

In order to establish a trusted client connection, you need to add the security certificate info to the request.

The most important thing you will need is the xConnect client certificate thumbprint that is found in the validateCertificateThumbprint setting in the your xConnect AppSettings.config, located at [location of your xConnect instance]\App_Config\ or in the ConnectionStrings.config of your Sitecore instance. The "FindValue" part of each xConnect Connection String contains this value.

For example:

  <add name="xconnect.collection.certificate" connectionString="StoreName=My;StoreLocation=LocalMachine;FindType=FindByThumbprint;FindValue=ADC6D07F383B2E116CC7510F4681EA34EE822F22" />  

To see this value within the certificate itself, you can navigate to it within your Personal Certificates store, shown below:

Using this thumbprint, we can make a secure xConnect client connection using the following code sample:

 var certThumbprint = "adc6d07f383b2e116cc7510f4681ea34ee822f22";  
 var xConnectUrl = "https://sc90.xconnect";  
 var options = CertificateWebRequestHandlerModifierOptions.Parse($"StoreName=My;StoreLocation=LocalMachine;FindType=FindByThumbprint;FindValue={certThumbprint}");  
 var certificateModifier = new CertificateWebRequestHandlerModifier(options);  
 var clientModifiers = new List<IHttpClientModifier>();  
 var timeoutClientModifier = new TimeoutHttpClientModifier(new TimeSpan(0, 0, 20));  
 var collectionClient = new CollectionWebApiClient(new Uri($"{xConnectUrl}/odata"), clientModifiers, new[] { certificateModifier });  
 var searchClient = new SearchWebApiClient(new Uri($"{xConnectUrl}/odata"), clientModifiers, new[] { certificateModifier });  
 var configurationClient = new ConfigurationWebApiClient(new Uri($"{ xConnectUrl }/configuration"), clientModifiers, new[] { certificateModifier });  
 var config = new XConnectClientConfiguration(new XdbRuntimeModel(CollectionModel.Model), collectionClient, searchClient, configurationClient);  
 catch (Exception e)  
 return new XConnectClient(config);  

Wrap Up

I hope that this post has helped you understand some of the contact changes that xConnect presents us with, and also provides enough crumbs to get you started using the xConnect Client API.

Happy Exploring!

Sunday, October 15, 2017

Sitecore xDB: A Mechanic's Guide to Personalization Testing Troubleshooting



On my last few projects, I have experienced first-hand how marketers have leveraged the true power of Content Testing in the Experience Platform to truly gain some fantastic insights so that they can successfully optimize content in order to deliver an improved contextual customer experience.

Most of the tests I have experienced have been personalization-based, and I have helped various teams troubleshoot some glitches that have popped up along the way. I guess you can call me the "content testing mechanic".

In this post, I will provide some insight from my experiences to help other developers who face similar issues, get the issues resolved quickly.

Nuts and Bolts

The main entry point into content testing for users is workflow, and so the assumption is that you have some type of workflow in place to successfully launch tests.

For more information on this, please review Sitecore's documentation on Adding content testing to a workflow  as well as Jonathan Robbins' Sitecore 8 Content Testing post.

With a workflow in place, the following things happen under the covers when you launch a new test:

  • A new test item is created that contains all the information about the test. This can be found at this location: /sitecore/system/Marketing Control Panel/Test Lab.
  • The Final Renderings XML field will be updated with specific testing attributes that contain values with test reference information*.
  • The item that you are testing and the new test item will be published to the web database (based on workflow action).

* To view this, you will need to enable raw values and standard fields in the “View” section of the “View” tab

An example of the Final Renderings XML looks something like the below:

Looking at this XML, you will see that it contains a p:t attribute. This denotes a personalization test reference. More on this further down.

Springs that Pop Out

If after starting a personalization test, personalization on the component(s) that you are testing stops working and you don't see any data appearing in your Test Result dialogue, the most common error in your Sitecore logs will be the following "Evaluation of condition failed. Rule item ID: Unknown, condition item ID exception":

 ERROR Evaluation of condition failed. Rule item ID: Unknown, condition item ID: {4888ABBB-F17D-4485-B14B-842413F88732}  
 Exception: System.NullReferenceException  
 Message: Object reference not set to an instance of an object.  
 Source: Sitecore.ContentTesting  
   at Sitecore.ContentTesting.Pipelines.RenderingRuleEvaluated.TestingRule.Process(RenderingRuleEvaluatedArgs args)  
   at (Object , Object[] )  
   at Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args)  
   at Sitecore.Rules.RuleList`1.Run(T ruleContext, Boolean stopOnFirstMatching, Int32& executedRulesCount)  

At first glance, you might think that this error is simply because of the condition item with ID {4888ABBB-F17D-4485-B14B-842413F88732} that is not published to the web database.

 Unfortunately, this isn't the case.

Under the Hood

After working through Sitecore's testing code, the exception occurs while Sitecore evaluates one of the conditions associated with the personalization rule.

Sitecore.ContentTesting.Pipelines.RenderingRuleEvaluated.TestingRule                 Sitecore.Rules.Evaluate

The invocation that fails and causes the NullReferenceException is the 'rule.Condition.Evaluate(ruleContext)' where:

rule - is the Rule object instantiated from the rule XML definition ( from the presentation details )
Condition - is the root condition definition from the rule XML definitions
ruleContext - is the object containing additional data for the rule evaluation such as:
  • Item reference: this should be the page definition item. 
  • Test reference: this should be the test associated with the rendering.
  • The current MVC rendering object reference

My analysis determined that the most common cause of the error is due to old tests that are still part of the content item's configuration, that are either not stopped correctly, inactive or have been removed.

Fixing the Issue

The fix is to remove the bad/old test references from the item in question's Final Renderings XML field. 

My process to do this is the following:

  • Determine what item is throwing the testing exception.
  • Enable raw values and standard fields in the “View” section of the “View” tab.
  • Copy the Final Renderings XML value of the item and format it so that it is easy to read. This site does a nice job:
  • Paste you’re the XML into Visual Studio or another editor.
  • Locate the attributes in the XML that have a s:pt and remove the attributes.
  • Copy and paste the updated XML back into the item's Final Renderings field.
  • Save and publish.

After this, the errors will stop appearing in your logs. You will however need to launch your test again.

Final Gotcha

The exact same "Evaluation of condition failed. Rule item ID: Unknown, condition item ID" exception error mentioned above could also occur if content testing has been disabled.

In XP 8.1 and later, it is disabled when the ContentTesting.AutomaticContentTesting.Enabled setting is set to false in the App_Config\Include\ContentTesting\Sitecore.ContentTesting.config file.

This is a bit obscure, as one would think that there would be some other messaging in the logs indicating that this setting has been disabled.

Testing Diagnostics Page for your Toolbox

I have seen some cases where the ribbon of the Optimization tab shows a different number of active tests if compared to the active test list. An example of this is shown below:

One of the first things that you can try is to rebuild your sitecore_testing_index index. If this doesn't help, you can use the diagnostics page below to help troubleshoot the issue.

The page output will look similar to this:


Active filtered tests:
Test search result object checked for Item: sitecore://{703EED9B-C574-4310-AC47-EBCCB651F67E}?lang=en&ver=2, Test Item: sitecore://master/{57807f6f-a836-4132-8b2b-48124b0c4031}?lang=en&ver=1, Is Running: True, Is Cancelled: False

Test search result object checked for Item: sitecore://{0CA61CF3-D35A-4FE7-8CD6-90CF8F61179A}?lang=en&ver=9, Test Item: sitecore://master/{cb771932-d06d-4f42-9392-483fab3cbc1c}?lang=en&ver=1, Is Running: True, Is Cancelled: False
Configuration is null

Test search result object checked for Item: sitecore://{8C65AEB8-90A2-4348-BCDB-D8AB6CBA5974}?lang=en&ver=1, Test Item: sitecore://master/{0b298164-fcc5-4803-acda-5a321e8c2797}?lang=en&ver=1, Is Running: True, Is Cancelled: False

1 {57807F6F-A836-4132-8B2B-48124B0C4031}
2 {0B298164-FCC5-4803-ACDA-5A321E8C2797}
Active tests:
1 sitecore://master/{57807f6f-a836-4132-8b2b-48124b0c4031}?lang=en&ver=1
2 sitecore://master/{cb771932-d06d-4f42-9392-483fab3cbc1c}?lang=en&ver=1
3 sitecore://master/{0b298164-fcc5-4803-acda-5a321e8c2797}?lang=en&ver=1


The diagnostic output example above shows us that the issue lies with the test item with ID {cb771932-d06d-4f42-9392-483fab3cbc1c}.

To fix the issue, you will need to locate the test item with that ID and either set its "Is Running" field value to "No" or simply delete that item.

Make sure that these updates get published to your web database.