Sitecore xDB: A Mechanic's Guide to Personalization Testing Troubleshooting

Background

On my last few projects, I have experienced first-hand how marketers have leveraged the true power of Content Testing in the Experience Platform to truly gain some fantastic insights so that they can successfully optimize content in order to deliver an improved contextual customer experience.

Most of the tests I have experienced have been personalization-based, and I have helped various teams troubleshoot some glitches that have popped up along the way. I guess you can call me the "content testing mechanic".

In this post, I will provide some insight from my experiences to help other developers who face similar issues, get the issues resolved quickly.

Nuts and Bolts

The main entry point into content testing for users is workflow, and so the assumption is that you have some type of workflow in place to successfully launch tests.

For more information on this, please review Sitecore's documentation on Adding content testing to a workflow  as well as Jonathan Robbins' Sitecore 8 Content Testing post.

With a workflow in place, the following things happen under the covers when you launch a new test:

• A new test item is created that contains all the information about the test. This can be found at this location: /sitecore/system/Marketing Control Panel/Test Lab.
• The Final Renderings XML field will be updated with specific testing attributes that contain values with test reference information*.
• The item that you are testing and the new test item will be published to the web database (based on workflow action).

* To view this, you will need to enable raw values and standard fields in the “View” section of the “View” tab

An example of the Final Renderings XML looks something like the below:

Looking at this XML, you will see that it contains a p:t attribute. This denotes a personalization test reference. More on this further down.

Springs that Pop Out

If after starting a personalization test, personalization on the component(s) that you are testing stops working and you don't see any data appearing in your Test Result dialogue, the most common error in your Sitecore logs will be the following "Evaluation of condition failed. Rule item ID: Unknown, condition item ID exception":

 ERROR Evaluation of condition failed. Rule item ID: Unknown, condition item ID: {4888ABBB-F17D-4485-B14B-842413F88732}  
 Exception: System.NullReferenceException  
 Message: Object reference not set to an instance of an object.  
 Source: Sitecore.ContentTesting  
   at Sitecore.ContentTesting.Pipelines.RenderingRuleEvaluated.TestingRule.Process(RenderingRuleEvaluatedArgs args)  
   at (Object , Object[] )  
   at Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args)  
   at Sitecore.Rules.RuleList`1.Run(T ruleContext, Boolean stopOnFirstMatching, Int32& executedRulesCount)  
   

At first glance, you might think that this error is simply because of the condition item with ID {4888ABBB-F17D-4485-B14B-842413F88732} that is not published to the web database.

 Unfortunately, this isn't the case.

Under the Hood

After working through Sitecore's testing code, the exception occurs while Sitecore evaluates one of the conditions associated with the personalization rule.

Sitecore.ContentTesting.Pipelines.RenderingRuleEvaluated.TestingRule                 Sitecore.Rules.Evaluate

The invocation that fails and causes the NullReferenceException is the 'rule.Condition.Evaluate(ruleContext)' where:

rule - is the Rule object instantiated from the rule XML definition ( from the presentation details )
Condition - is the root condition definition from the rule XML definitions
ruleContext - is the object containing additional data for the rule evaluation such as:

• Item reference: this should be the page definition item. 
• Test reference: this should be the test associated with the rendering.
• The current MVC rendering object reference

My analysis determined that the most common cause of the error is due to old tests that are still part of the content item's configuration, that are either not stopped correctly, inactive or have been removed.

Fixing the Issue

The fix is to remove the bad/old test references from the item in question's Final Renderings XML field. 

My process to do this is the following:

• Determine what item is throwing the testing exception.
• Enable raw values and standard fields in the “View” section of the “View” tab.
• Copy the Final Renderings XML value of the item and format it so that it is easy to read. This site does a nice job: https://www.freeformatter.com/xml-formatter.html
• Paste you’re the XML into Visual Studio or another editor.
• Locate the attributes in the XML that have a s:pt and remove the attributes.
• Copy and paste the updated XML back into the item's Final Renderings field.
• Save and publish.

After this, the errors will stop appearing in your logs. You will however need to launch your test again.

Final Gotcha

The exact same "Evaluation of condition failed. Rule item ID: Unknown, condition item ID" exception error mentioned above could also occur if content testing has been disabled.

In XP 8.1 and later, it is disabled when the ContentTesting.AutomaticContentTesting.Enabled setting is set to false in the App_Config\Include\ContentTesting\Sitecore.ContentTesting.config file.

This is a bit obscure, as one would think that there would be some other messaging in the logs indicating that this setting has been disabled.

Testing Diagnostics Page for your Toolbox

I have seen some cases where the ribbon of the Optimization tab shows a different number of active tests if compared to the active test list. An example of this is shown below:

One of the first things that you can try is to rebuild your sitecore_testing_index index. If this doesn't help, you can use the diagnostics page below to help troubleshoot the issue.

The page output will look similar to this:

~~~~~~~~~~~~~~

Active filtered tests:
Test search result object checked for Item: sitecore://{703EED9B-C574-4310-AC47-EBCCB651F67E}?lang=en&ver=2, Test Item: sitecore://master/{57807f6f-a836-4132-8b2b-48124b0c4031}?lang=en&ver=1, Is Running: True, Is Cancelled: False

Test search result object checked for Item: sitecore://{0CA61CF3-D35A-4FE7-8CD6-90CF8F61179A}?lang=en&ver=9, Test Item: sitecore://master/{cb771932-d06d-4f42-9392-483fab3cbc1c}?lang=en&ver=1, Is Running: True, Is Cancelled: False
Configuration is null

Test search result object checked for Item: sitecore://{8C65AEB8-90A2-4348-BCDB-D8AB6CBA5974}?lang=en&ver=1, Test Item: sitecore://master/{0b298164-fcc5-4803-acda-5a321e8c2797}?lang=en&ver=1, Is Running: True, Is Cancelled: False

1 {57807F6F-A836-4132-8B2B-48124B0C4031}
2 {0B298164-FCC5-4803-ACDA-5A321E8C2797}
Active tests:
1 sitecore://master/{57807f6f-a836-4132-8b2b-48124b0c4031}?lang=en&ver=1
2 sitecore://master/{cb771932-d06d-4f42-9392-483fab3cbc1c}?lang=en&ver=1
3 sitecore://master/{0b298164-fcc5-4803-acda-5a321e8c2797}?lang=en&ver=1

~~~~~~~~~~~~~~~

The diagnostic output example above shows us that the issue lies with the test item with ID {cb771932-d06d-4f42-9392-483fab3cbc1c}.

To fix the issue, you will need to locate the test item with that ID and either set its "Is Running" field value to "No" or simply delete that item.

Make sure that these updates get published to your web database.