Monday, February 23, 2015

Syncing xDB Experience Profiles with Dynamics CRM


Intro and Prerequisites

Like other Sitecore enthusiasts, I have been extremely impressed with the new technology and architecture behind xDB

As Nick Wesselman puts it: "The Experience Database provides data structures and APIs to allow you to track an individual across devices, provided you can identify him/her by some unique identifier such as an email address. This is key, otherwise you just end up having a ton of anonymous contacts in your database."

If you haven't read his blog, I highly recommend it. It got me on the right path with all things xDB / Mongo:

Reading through his "One Month with Sitecore 7.5, Part 5: Persisting Contact Data to xDB" post where he talks about Contact Facets will help you better understand what I am doing below.

Also make sure you check out Martina Welander's posts as well: She has tons of good stuff in there.

The Goal

As you can see, the code for identifying a user is straightforward:

1:  var tracker = Sitecore.Analytics.Tracker.Current;  
2:  if (tracker == null || !tracker.IsActive || tracker.Contact == null)  
3:  {  
4:     return;  
5:  }  
6:  tracker.Session.Identify("some unique identifier");  

This is nice and all, but it doesn't help with populating my Experience Profile (xFile).

Wouldn't it be awesome if I could identify the user, and then check my CRM system to see if it contained profile data about the user, and if so, populate the xFile using that information?

Wouldn't it be even more awesome if after I identified the user, I could make sure that my xFile always kept in sync with my CRM system?

Having extensive Dynamics CRM development experience, I decided to write up a POC to make this happen.

NOTE: What I am doing here is a one-way sync from Dynamics CRM to Sitecore. We could very easily take this one step further with a more advanced contact form, and update data in CRM from Sitecore.

Identify User and Flush-To-Mongo Control

My first task was to put together a user control that would allow me to identify the user and then flush the session data to Mongo so that I could review the outcome.

The basic control consisted of 1 text box and 2 buttons. The text box would be used to capture the user's email address. Clicking the first button would identify the user using the code shown above, while clicking the second button would flush the session data to Mongo.

Flushing the session data can be achieved by simply making a call to end the session: Session.Abandon()

Here is the code-behind for my control:

1:   public partial class IdentifyUser : System.Web.UI.UserControl  
2:    {  
3:      private void Page_Load(object sender, EventArgs e)  
4:      {  
5:        btnIdentifyUser.Click += btnIdentifyUser_Click;  
6:        btnFlushToMongo.Click += btnFlushToMongo_Click;  
7:        lblIdentiedUser.Text = Tracker.Current.Contact.Identifiers.Identifier;  
8:      }  
9:      private void btnIdentifyUser_Click(Object sender, EventArgs e)  
10:      {    
11:        Tracker.Current.Session.Identify(txtCurrentUser.Text);  
12:        string myScript = "alert('Done!');";  
13:        Page.ClientScript.RegisterStartupScript(GetType(), new Guid().ToString(), myScript, true);  
14:      }  
15:      private void btnFlushToMongo_Click(Object sender, EventArgs e)  
16:      {  
17:        Session.Abandon();  
18:        string myScript = "alert('Flushed!');";  
19:        Page.ClientScript.RegisterStartupScript(GetType(), new Guid().ToString(), myScript, true);  
20:      }  
21:    }  

Nothing elegant about this at all (especially because I used WebForms over MVC). I simply slapped this on the out-of-the-box Sitecore demo page and the result looked like this:

You will notice that I also had a label on line 7 where I would populate the contact's identifier so that I could make sure that the tracker is working as expected.

Working with Dynamics CRM

Creating a Proxy Class

When coding for Dynamics CRM, you need to use the "CrmSvcUtil" command-line code generation tool, available in the CRM SDK, in order to generate early-bound classes that represent the entity data model in your CRM system.

So in short; this tool located in the bin folder of the SDK is used to create a proxy class from the CRM Web Service of your instance.

For more information about this, you can check out this link from Microsoft:

I named my generated class file "CrmModels.cs" and threw it into my project.

The xFileManager

I started putting together a manager class that I would use to facilitate finding a user / contact match in Dynamics CRM and then populate the xFile using the contact record's data.

I added a Dynamics "contact" entity property to my class called CrmContact. The plan was to populate this if I had a match in CRM based on email address, and then use it to hydrate my various Sitecore xFile contact facet objects.

Finding the contact in Dynamics CRM

The first method I added to my class was to make a call out to Dynamics to see if I had a contact match based on the identified user's email address. If I did find the user, it would set my CrmContact property to the match and return true.

1:  private bool FoundCrmContact(string emailAddress)  
2:      {  
3:        var connection = new CrmConnection("MTCCRMConnString");  
4:        var org = new OrganizationService(connection);  
5:        var crmDataContext = new CRMDataContext(org);  
6:        var contactMatch = crmDataContext.ContactSet.Where(t => t.EMailAddress1.Equals(emailAddress));  
7:        if (contactMatch.FirstOrDefault() != null)  
8:        {  
9:          CrmContact = contactMatch.FirstOrDefault();  
10:          return true;  
11:        }  
12:        return false;  
13:      }  

Syncing CRM contact data with the identified contact's facets

Next, I focused on writing a method that would populate the contact's various facets with the CRM data if a match was found.

Here is my method:

1:  public void SyncCrmXFile(Sitecore.Analytics.Tracking.Contact identifiedUser)  
2:      {  
3:        if (identifiedUser.Identifiers.IdentificationLevel == ContactIdentificationLevel.Known)  
4:        {  
5:          //Get contact unique indentifier, in our case the email address  
6:          var contactEmail = identifiedUser.Identifiers.Identifier;  
7:          if (FoundCrmContact(contactEmail))  
8:          {  
9:            #region Get Facets  
10:            var emailFacet = identifiedUser.GetFacet<IContactEmailAddresses>("Emails");  
11:            var addressFacet = identifiedUser.GetFacet<IContactAddresses>("Addresses");  
12:            var personalFacet = identifiedUser.GetFacet<IContactPersonalInfo>("Personal");  
13:            var phoneFacet = identifiedUser.GetFacet<IContactPhoneNumbers>("Phone Numbers");  
14:            IEmailAddress email = emailFacet.Entries.Contains("Work Email")   
15:              ? emailFacet.Entries["Work Email"]  
16:              : emailFacet.Entries.Create("Work Email");  
17:            IAddress address = addressFacet.Entries.Contains("Work Address")  
18:              ? addressFacet.Entries["Work Address"]  
19:              : addressFacet.Entries.Create("Work Address");  
20:            IPhoneNumber workPhone = phoneFacet.Entries.Contains("Work Phone")  
21:              ? phoneFacet.Entries["Work Phone"]  
22:              : phoneFacet.Entries.Create("Work Phone");  
23:            #endregion  
24:            #region Update Facets with CRM Contact Data  
25:            email.SmtpAddress = CrmContact.EMailAddress1;  
26:            emailFacet.Preferred = "Work Email";  
27:            address.StreetLine1 = CrmContact.Address1_Line1;  
28:            address.StreetLine2 = CrmContact.Address1_Line2;  
29:            address.PostalCode = CrmContact.Address1_PostalCode;  
30:            personalFacet.Title = CrmContact.Salutation;  
31:            personalFacet.FirstName = CrmContact.FirstName;  
32:            personalFacet.MiddleName = CrmContact.MiddleName;  
33:            personalFacet.Surname = CrmContact.LastName;  
34:            //personalFacet.Gender = CrmContact.GenderCode.Value;  
35:            personalFacet.JobTitle = CrmContact.JobTitle;  
36:            personalFacet.BirthDate = CrmContact.BirthDate;  
37:            workPhone.CountryCode = workPhone.Number = CrmContact.Telephone1;  
38:            #endregion  
39:            Sitecore.Diagnostics.Log.Info(string.Format("Successfully synced Dynamics CRM known user: {0} {1}", CrmContact.FirstName, CrmContact.LastName), this);  
40:          }  
41:        }  
42:      }  

In summary, the code above does the following:

  1. Accepts a Sitecore.Analytics.Tracking.Contact object as a parameter
  2. Makes sure that we had already identified the contact
  3. Pulls out the identified contact's email address and looks for a match in CRM
  4. If a match exists in CRM, it uses the data to populate / sync the identified contact's facets
I went ahead and added a call to the method in my IdentifyUser button click event handler just after the line where I identify the contact:

1:  Tracker.Current.Session.Identify(txtCurrentUser.Text);  
2:  var xFile = new XFileManager();  
3:  xFile.SyncCrmXFile(Tracker.Current.Contact);  

Identify, Flush and Review

With all these pieces in place, I was ready to do some testing. In my example below, I am using Stefan Edberg as my identified contact.

Side note

If you don't know who Stefan Edberg is, he was an awesome tennis player; one of my idols when growing up. He is currently the tennis coach of Roger Federer.

Here he is as a contact in our Dynamics CRM system:

So, I went ahead and identified Stefan using his email address. Things were starting to look good:

Next, I wanted to flush the session data to Mongo, and review what was populated in our Experience Profile dashboard:

Nice! There he is. Clicking on his name and then the details tab revealed the rest of the data that was populated using the CRM contact data:

Keeping things in sync

What I wanted to do next was make sure that my xFile always contained the most up-to-date data from my CRM system.

The key was finding the sweet spot in the "Session End" pipeline before the actual contact data gets flushed to Mongo.

Looking through some of the analytics config files, and digging a little bit around the analytics assemblies, I found my spot before the Sitecore.Analytics.Pipelines.SessionEnd.InitializeTracker processor.

So, my processor was simply:

1:   public class SyncCrm  
2:    {  
3:      public void Process(PipelineArgs args)  
4:      {  
5:        Assert.ArgumentNotNull(args, "args");  
6:        if (!AnalyticsSettings.Enabled)  
7:        {  
8:          return;  
9:        }  
10:        RunCrmSync();  
11:      }  
12:      private void RunCrmSync()  
13:      {  
14:        var xFile = new XFileManager();  
15:        xFile.SyncCrmXFile(Tracker.Current.Session.Contact);  
16:      }  
17:    }  

My config file looked like this:

1:  <configuration xmlns:patch="">  
2:   <sitecore>  
3:    <pipelines>  
4:     <sessionEnd>  
5:      <processor type="Arke.Sandbox.Xrm.Pipelines.Analytics.SyncCrm, Arke.Sandbox.Xrm" patch:before="processor[@type='Sitecore.Analytics.Pipelines.SessionEnd.InitializeTracker, Sitecore.Analytics']" />  
6:     </sessionEnd>  
7:    </pipelines>  
8:   </sitecore>  
9:  </configuration>  

Mission Complete

With this processor in place, every time the session ends, and before the data gets flushed to Mongo, there would be a call to Dynamics CRM to pull in the latest data associated with the contact so that their Sitecore xFile would always stay up-to-date.

Monday, February 16, 2015

Enable HTML tags in Single-line or Multi-line text fields - an oldie but a goodie

This comes up time and time again. When upgrading older 6.x Sitecore solutions to 7 and now 8, some content areas seem to be broken because html tags start being rendered on pages.


The GetTextFieldValue processor that was added to the renderField pipeline in 7 and above ensures that all text fields are output escaped when rendered through the renderField pipeline.

Yes, this is actually a really good security mechanism. However, if you have tech savvy Content Authors, you may want to turn this off.

This processor will help you do just that.


The key is the HttpUtility.HtmlDecode on line 20 below:

1:  using System.Web;  
2:  using Sitecore.Diagnostics;  
3:  using Sitecore.Pipelines.RenderField;  
4:  namespace MyProject.Framework.Pipelines.RenderField  
5:  {  
6:    /// <summary>  
7:    /// The get text field value.  
8:    ///   
9:    /// </summary>  
10:    public class CustomGetTextFieldValue  
11:    {   
12:      public void Process(RenderFieldArgs args)  
13:      {  
14:        Assert.ArgumentNotNull((object)args, "args");  
15:        string fieldTypeKey = args.FieldTypeKey;  
16:        if (fieldTypeKey != "text" && fieldTypeKey != "single-line text")  
17:        {  
18:          return;  
19:        }  
20:        args.Result.FirstPart = HttpUtility.HtmlDecode(args.Result.FirstPart);  
21:      }  
22:    }  
23:  }  

Config file for your include folder

 <configuration xmlns:patch="">  
    <processor type="MyProject.Framework.Pipelines.RenderField.CustomGetTextFieldValue, MyProject.Framework"  
       patch:instead="processor[@type='Sitecore.Pipelines.RenderField.GetTextFieldValue, Sitecore.Kernel']" />  

Sunday, February 15, 2015

Frustrated by hidden buttons when adding links in Sitecore 8?


Is anyone else constantly frustrated by the above picture?

In your site’s wwwroot folder, go to /Website/Sitecore/shell/client/speak/Assets/css/ and find the speak-default-theme.css file.

Edit the file by adding the following class:

 .sc-dialogFooter {
  z-index: 1050;

Make sure you clear your browser cache.

Voila! Your buttons are back!

Saturday, February 7, 2015

My Upgrade Experience from Sitecore 7.5 to 8.0 Update-1

I was assigned the task of upgrading an existing client's large, multisite Sitecore 7.5 rev. 141003 instance to the shiny new Sitecore 8 rev.150121 (Update-1).  Having performed several version upgrades in the past, I estimated that it would take me about half a day to complete this process.

It took me a little longer than this to get them up and running, only because I ran into some issues with some legacy modules and because rebuilding the search indexes simply took forever!

I am still having a couple of issues after the upgrade that I noted in my Outstanding Issues section. I plan to keep this section updated as I receive feedback from the ticket I currently have open with Sitecore.

Solution for publishing error.
Solution for error when rebuilding links database.

Getting Ready

To get started, I navigated over to the new and improved site to arm myself with the files needed to perform the upgrade. The files that I downloaded from the site included:

  1. 8.0 and 8.0 Update-1 upgrade packages
  2. 8.0 and 8.0 Update-1 config files for upgrade
  3. Upgrade guides for 8.0 and 8.0 Update-1
  4. Configuration file changes document for 8.0 and 8.0 Update-1
Note: You need to be logged into for these links to work.

I must say, the Sitecore team did a fantastic job on the documentation for the upgrade!

The road to 8.0 initial release

The short version

These are the steps necessary to perform the upgrade:

  1. Run SQL Server update scripts on Core, Master and Web databases
  2. Run SQL Server script on Analytics database
  3. Remove PageEditorHanldeNoLayout Processor from web.config
  4. Remove PageLevelTestItemResolver from Analytics.config
  5. Turn Analytics off: Set Analytics.Enabled to false in Analytics.config
  6. Make sure any A/B or MV tests are disabled
  7. Disable WFFM and ECM config files if you have the modules installed
  8. Install the upgrade package using the Update Installation Wizard
  9. Apply the configuration file changes described in Sitecore Configuration File Changes document
  10. Revert the changes from step 5-7


The configuration file changes document for 8.0 scared me at first, because it was 30 pages! But, I quickly realized that in most cases, I could simple do a copy and replace on most of the files.

The document notes; "Most customers have only applied a small number of changes (if any) to the Web.config file. In that case, we recommend that you simply download the clean config file using the link above and re-apply any changes that you've made to save time and to reduce the risk of making mistakes when adding the new settings, pipelines, etc. to the file."

We inherited this site from another vendor who made their updates directly to the web.config, instead of using separate patch files. So unfortunately, the small number of changes didn't apply to our case. I was forced to manually update the web.config in 36 different locations!

Bump in the road

While installing the upgrade package, I received the following error:
Installation resulted in "The following errors occured during file installation: Access to the path '{instancepath}\Website\bin\System.Web.Http.dll' is denied."

For some reason, the assembly in question was marked as "read-only". I simply unchecked the "read-only" checkbox in the file's properties, ran the upgrade package again, and it worked like a champ.

Testing 8.0 initial release

After updating the databases via scripts, installing the update package, and making the necessary config changes, I was ready to boot up the new version.

At first, things looked good. The new login page came up and I was able to successfully get to the new SPEAK dashboard.

However, when I clicked on any the buttons on the dashboard, I received the following error:
Could not load type 'Sitecore.Shell.Applications.WebEdit.Commands.WebEditCommand' from assembly 'Sitecore.Client, Version=, Culture=neutral, PublicKeyToken=null'.

Experience told me that there had to be some custom module causing the issue. Knowing that this particular site was using a "patched" version of the old 6.5 calendar module, I disabled the config file straight away, tried again, but still didn't have any luck. Other modules that the site used that I tried to disable without success included:

  1. Sitemap XML
  2. Page Rate
I decided to make a backup of everything in my instance's include folder, and then I went back over to and downloaded the zipped web root of the 8.0 initial release. I went ahead and removed everything that was in my instance's include folder, and then copied over all the files from the 8.0 zipped download's include folder.

This fixed my problem, and I was able to access all the areas of the platform. I also did a quick check in my log file, and it was error free.

Next, I loaded in the above-mentioned module's config files into my include folder. I kept my fingers and toes crossed, and to my delight, everything continued to work.

To 8.0 Update-1 and beyond

After getting the initial release up and running, I figured that upgrading to Update-1 would be a "walk in the park". It certainly was.

The short version

These are the steps necessary to perform the upgrade:

  1. Turn Analytics off: Set Analytics.Enabled to false in Analytics.config
  2. Disable WFFM and ECM config files if you have the modules installed
  3. Comment out the Sitecore.ContentTesting.Events.GenerateScreenshot,Sitecore.ContentTesting
    event handler in /App_Config/Include/ContentTesting/Sitecore.ContentTesting.config
  4. Install the upgrade package using the Update Installation Wizard
  5. Apply the configuration file changes described in Sitecore Configuration File Changes document
  6. Revert the changes from step 1-3
I was presently surprised to see that there was only 1 change to the web.config this time. As for the rest of the configuration files; I was able to simply do a copy and replace from the config files zip that I had previously downloaded.

Outstanding Issues

After doing some extensive testing, these are the following issues that are outstanding in our upgraded 8.0 Update-1 instance. 

I plan to keep this section updated with feedback from the ticket I currently have open with Sitecore.

Error when rebuilding the links database for Master and Web:

Job started: RebuildLinkDatabasesIndex|System.FormatException: Unrecognized Guid format.
   at System.Guid.GuidResult.SetFailure(ParseFailureKind failure, String failureMessageID, Object failureMessageFormatArgument, String failureArgumentName, Exception innerException)
   at System.Guid.TryParseGuid(String g, GuidStyles flags, GuidResult& result)
   at System.Guid..ctor(String g)
   at Sitecore.Analytics.Data.TrackingField.<>c__DisplayClass21.<get_Events>b__20(XElement e)
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at Sitecore.Analytics.Data.TrackingField.ValidateLinks(LinksValidationResult result)
   at Sitecore.Links.ItemLinks.AddLinks(Field field, List`1 links, ItemLinkState linkState)
   at Sitecore.Links.ItemLinks.GetLinks(ItemLinkState linkState, Boolean allVersions, Boolean includeStandardValuesLinks)
   at Sitecore.Links.LinkDatabase.UpdateReferences(Item item)
   at Sitecore.Links.LinkDatabase.RebuildItem(Item item)
   at Sitecore.Links.LinkDatabase.RebuildItem(Item item)
   at Sitecore.Links.LinkDatabase.RebuildItem(Item item)
   at Sitecore.Links.LinkDatabase.RebuildItem(Item item)
   at Sitecore.Links.LinkDatabase.RebuildItem(Item item)
   at Sitecore.Links.LinkDatabase.RebuildItem(Item item)
   at Sitecore.Links.LinkDatabase.RebuildItem(Item item)
   at Sitecore.Links.LinkDatabase.Rebuild(Database database)
   at Sitecore.Shell.Applications.Dialogs.RebuildLinkDatabase.RebuildLinkDatabaseForm.Builder.Build()|Job ended: RebuildLinkDatabasesIndex (units processed: )


WFFM problem items were causing the errors.

Using SQL Profiler, we identified the problem items:

/sitecore/system/Modules/Web Forms for Marketers/Sample form
/sitecore/system/Modules/Web Forms for Marketers/Website
/sitecore/templates/Web Forms for Marketers/Form/_Standard Values

The site had the module installed but wasn't using it for anything. I think our client had intentions to start building some custom forms, but just never got around to doing it.

Knowing that we will be using it in the near future for a Dynamics CRM integration project, I went ahead and installed the latest version of the module, overwriting the problem items in the tree.

After doing this, I was able to successfully rebuild my links databases. Yay!

Error when publishing some items:

Job started: Publish to 'web'|#Exception: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.Exception: Could not resolve type name: Sitecore.Publishing.Pipelines.PublishVersion.Processors.Re
                                moveOtherVersions, Sitecore.Kernel (method: Sitecore.Configuration.Factory.CreateType(XmlNode configNode, String[] parameters, Boolean assert)).
   at Sitecore.Diagnostics.Error.Raise(String error, String method)
   at Sitecore.Configuration.Factory.CreateType(XmlNode configNode, String[] parameters, Boolean assert)
   at Sitecore.Configuration.Factory.CreateFromTypeName(XmlNode configNode, String[] parameters, Boolean assert)
   at Sitecore.Configuration.Factory.CreateObject(XmlNode configNode, String[] parameters, Boolean assert, IFactoryHelper helper)
   at Sitecore.Pipelines.CorePipelineFactory.GetObjectFromType(String type, XmlNode processorNode)
   at Sitecore.Pipelines.CoreProcessor.GetMethod(Object[] parameters)
   at Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args)
   at Sitecore.Publishing.PublishHelper.PublishVersionToTarget(Item sourceVersion, Item targetItem, Boolean targetCreated)
   at Sitecore.Publishing.Pipelines.PublishItem.PerformAction.ExecuteAction(PublishItemContext context)
   at Sitecore.Publishing.Pipelines.PublishItem.PerformAction.Process(PublishItemContext context)
   at (Object , Object[] )
   at Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args)
   at Sitecore.Publishing.Pipelines.PublishItem.PublishItemPipeline.Run(PublishItemContext context)
   at Sitecore.Publishing.Pipelines.Publish.ProcessQueue.ProcessEntries(IEnumerable`1 entries, PublishContext context)
   at Sitecore.Publishing.Pipelines.Publish.ProcessQueue.Process(PublishContext context)
   at (Object , Object[] )
   at Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args)
   at Sitecore.Publishing.Pipelines.Publish.PublishPipeline.Run(PublishContext context)
   at Sitecore.Publishing.Publisher.PublishWithResult()
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
   at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
   at (Object , Object[] )
   at Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args)
   at Sitecore.Jobs.Job.ThreadEntry(Object state)


Error in the web.config file:

       <publishVersion help="Processors should derive from Sitecore.Publishing.Pipelines.PublishItem.PublishVersionProcessor">
moveOtherVersions, Sitecore.Kernel" />

You should concatenate the rows with the type name:

       <publishVersion help="Processors should derive from Sitecore.Publishing.Pipelines.PublishItem.PublishVersionProcessor">
type="Sitecore.Publishing.Pipelines.PublishVersion.Processors.RemoveOtherVersions, Sitecore.Kernel" />