Sunday, September 30, 2018

Sitecore Azure PaaS: Updating Your License File For All Deployed App Service Instances

Standard

Background

If you are provisioning a new set of Sitecore environments on your own, or if the Sitecore Managed Cloud Hosting Team provisions your environments for you, you will most likely be using a temporary license file that is valid for 1 month while you are waiting for your permanent license file .

When the temporary license expires, your Sitecore instances will stop working. Therefore, it is important that you upload a valid permanent license.xml file as soon as it is available.

File Locations

In an XP Scaled environment, there are many different App Services and locations where the license.xml file will need to be updated.

I created a list of the App Service roles and the license file locations for your reference:

App Service Role License File Location
xc-search  \App_data & \App_data\jobs\continuous\IndexWorker\App_data
ma-ops  \App_data & \App_data\jobs\continuous\AutomationEngine\App_Data
cd  \App_data
cm  \App_data
ma-rep  \App_data
prc  \App_data
rep  \App_data
xc-collect  \App_data
xc-refdata  \App_data

Updating the License File

The easiest way to update the file is to use the Debug console in the Kudu "Advanced Tools" in your App Service Instance, or an FTPS client to connect directly to the App's filesystem.


Thursday, September 20, 2018

Sitecore GeoIP - A Developer's Guide To What Has Changed In Sitecore 9

Standard
In my previous post, I took a dive into the 8.x version of the Sitecore GeoIP service from a developer's point of view. Sitecore 9 introduced great improvements to xDB, GeoIP being one of those features.

In this post, I intend to help developers understand what has changed in Sitecore 9 GeoIP.  Like my previous post, the purpose is to arm developers with the necessary details to understand what is happening under the hood, so that they can successfully troubleshoot a problem if one arises.


Reference Data

One of the first things that I discovered when diving into version 9 is the use of a series of "ReferenceDataClientDictionaries" that are exposed to us as "KnownDataDictionaries".

As inferred by the name, these are known collections of things that are used to store common data, one being IP Geolocation data. The data is ultimately stored in a SQL database, so that it can be referenced throughout the Experience Platform.

There is a new pipeline in Sitecore 9 that initializes these dictionaries, as shown here:

Config: 

<initializeKnownDataDictionaries patch:source="Sitecore.Analytics.Tracking.config">
<processor type="Sitecore.Analytics.DataAccess.Pipelines.InitializeKnownDataDictionaries.InitializeKnownDataDictionariesProcessor, Sitecore.Analytics.DataAccess"/>
<processor type="Sitecore.Analytics.XConnect.DataAccess.Pipelines.InitializeKnownDataDictionaries.InitializeDeviceDataDictionaryProcessor, Sitecore.Analytics.XConnect" patch:source="Sitecore.Analytics.Tracking.Database.config"/>
</initializeKnownDataDictionaries>

Processor Code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
namespace Sitecore.Analytics.DataAccess.Pipelines.InitializeKnownDataDictionaries
{
  public class InitializeKnownDataDictionariesProcessor : InitializeKnownDataDictionariesProcessorBase
  {
    public override void Process(InitializeKnownDataDictionariesArgs args)
    {
      Condition.Requires<InitializeKnownDataDictionariesArgs>(args, nameof (args)).IsNotNull<InitializeKnownDataDictionariesArgs>();
      GetDictionaryDataPipelineArgs args1 = new GetDictionaryDataPipelineArgs();
      GetDictionaryDataPipeline.Run(args1);
      Condition.Ensures<DictionaryBase>(args1.Result).IsNotNull<DictionaryBase>("Check configuration, 'getDictionaryDataStorage' pipeline  must set args.Result property with instance of DictionaryBase type.");
      args.LocationsDictionary = new LocationsDictionary(args1.Result);
      args.ReferringSitesDictionary = new ReferringSitesDictionary(args1.Result);
      args.GeoIpDataDictionary = new GeoIpDataDictionary(args1.Result);
      args.UserAgentsDictionary = new UserAgentsDictionary(args1.Result);
      args.DeviceDictionary = new DeviceDictionary(args1.Result);
    }
  }
}

If you look at line 13 above, the GeoIpDataDictionary object being created is inherited from Sitecore's new ReferenceDataDictionary.

This is the glue between GeoIP and the new Reference Data "shared storage" mechanism.

Here is what the code looks like:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
namespace Sitecore.Analytics.DataAccess.Dictionaries
{
  public class GeoIpDataDictionary : ReferenceDataDictionary<Guid, GeoIpData>
  {
    public GeoIpDataDictionary(DictionaryBase dictionary, int cacheSize)
      : base(dictionary, "GeoIpDataDictionaryCache", XdbSettings.GeoIps.CacheSize * cacheSize)
    {
      this.ReadCounter = AnalyticsDataAccessCount.DataDictionariesGeoIpsReads;
      this.WriteCounter = AnalyticsDataAccessCount.DataDictionariesGeoIpsWrites;
      this.CacheHitCounter = AnalyticsDataAccessCount.DataDictionariesGeoIpsCacheHits;
      this.DataStoreReadCounter = AnalyticsDataAccessCount.DataDictionariesGeoIpsDataStoreReads;
      this.DataStoreReadTimeCounter = AnalyticsDataAccessCount.DataDictionariesGeoIpsDataStoreReadTime;
      this.DataStoreWriteTimeCounter = AnalyticsDataAccessCount.DataDictionariesGeoIpsDataStoreWriteTime;
    }

    public GeoIpDataDictionary(DictionaryBase dictionary)
      : this(dictionary, XdbSettings.GeoIps.CacheSize)
    {
    }

    public override TimeSpan CacheExpirationTimeout
    {
      get
      {
        return TimeSpan.FromSeconds(600.0);
      }
    }

    public override Guid GetKey(GeoIpData value)
    {
      return value.Id;
    }

    public string GetKey(Guid id)
    {
      return id.ToString();
    }
  }
}


Notice on line 25 that this object is cached for 10 minutes. More on this below.

Reference Data Storage and the GeoIP Lookup Flow

You may be wondering how this Reference Data feature changes what you know about the GeoIP flow from previous versions of the platform.

Let's review the steps:

  • Sitecore runs the CreateVisits pipeline. Within this pipeline, there is a processor called UpdateGeoIpData that fires a method called GeoIpManager.GetGeoIpData within Sitecore.Analytics.Tracking.CurrentVisitContext that initiates the GeoIP lookup for the visitor's interaction.

  • Sitecore performs a GeoIP data lookup in the GeoIP memory cache.
    • NOTE: Cache expiration is set to 10 seconds => TimeSpan.FromSeconds(10.0)

Sitecore.Analytics.Lookups.GeoIpCache:

1
2
3
4
5
6
7
8
    public void Add(GeoIpHandle handle)
    {
      Assert.ArgumentNotNull((object) handle, nameof (handle));
      if (this.cache.Count >= this.maxCount)
        this.Scavenge();
      this.cache.Add(handle.Id, (object) handle, TimeSpan.FromSeconds(10.0));
      AnalyticsTrackingCount.GeoIPCacheSize.Value = (long) this.cache.Count;
    }

  • If the GeoIP data IS in the GeoIP memory cache, then it will attach it to the visitor's interaction.

  • If the GeoIP data IS NOT in the GeoIP memory cache, it performs a lookup in the Reference Data's GeoIpDataDictionary (KnownDictionaries) memory cache.
    • NOTE: Cache expiration is set to 10 minutes => TimeSpan.FromSeconds(600.0). See above for the 10 minute CacheExpirationTimout property on the Sitecore.Analytics.DataAccess.Dictionaries.GeoIpDataDictionary class.

  • If the GeoIP data IS in the Reference Data's GeoIpDataDictionary memory cache, it attaches it to the visitor's interaction and adds it to the GeoIP memory cache.

  • If the GeoIP data IS NOT in the Reference Data's GeoIpDataDictionary memory cache, it performs a lookup in the SQL ReferenceData database and if found, stores the result in the Reference Data's GeoIpDataDictionary cache and GeoIP memory cache, and then attaches it to the visitor's interaction.

  • If the GeoIP data IS NOT in the SQL ReferenceData database, it performs a lookup using the Sitecore Geolocation service and stores the result in the SQL ReferenceData database, the Reference Data's GeoIpDataDictionary cache and GeoIP memory cache, and then attaches it to the visitor's interaction.

Reference Data Storage in SQL

By using SQL Server Management Studio, and opening up the ReferenceData database's DefinitionTypes table, you can see the different types of reference data that is being stored. The GeoIp data type name as you can see below, is called "Tracking Dictionary - GeoIpData".


By looking at the Definitions table, you can see that the data is stored as a Binary data type:


The following SQL Query will return the top 100 GeoIP reference data results:

1
2
3
4
SELECT TOP 100 [xdb_refdata].[DefinitionTypes].Name, [xdb_refdata].[Definitions].Data, [xdb_refdata].[Definitions].IsActive, [xdb_refdata].[Definitions].LastModified, [xdb_refdata].[Definitions].Version
FROM [xdb_refdata].[Definitions]
INNER JOIN [xdb_refdata].[DefinitionTypes] ON [xdb_refdata].[DefinitionTypes].ID = [xdb_refdata].[Definitions].TypeID
WHERE [xdb_refdata].[DefinitionTypes].Name = 'Tracking Dictionary - GeoIpData'



Changes to the GeoIpManager class

Finally, I wanted to provide a glimpse of the changes in the GeoIpManager class that I referenced in my previous post.

By comparing the 8.x version of the GeoIpManager code to 9, you can see the usage of the KnownDataDictionaries.GeoIPs dictionary instead of the Tracker.Dictionaries.GeoIpData (ContactLocation class) from 8.x:



Final Words

I hope that this information helps developers understand more about Reference Data and the updated GeoIP Lookup Flow in Sitecore 9.

As always, feel free to comment or reach me on Slack or Twitter if you have any questions.


Thursday, August 16, 2018

Sitecore GeoIP - What Is Happening Under The Hood In 8.x?

Standard

Background

Most posts explain how Sitecore's GeoIP service works from a high-level point of view.

In this post, I intend to take the explanation a few steps deeper, so that developers can understand all the pieces that make this process work. The goal is to arm developers with the necessary details to successfully troubleshoot a problem if one arises.


Visitor Interaction - Start of visitor's session

  • Visitor visits Sitecore website, and this is regarded as a new interaction. Sitecore's definition of an interaction is ".. any point at which a contact interfaces with a brand, either online or offline". In our case, this is a new visitor session on the website.

  • Sitecore runs the CreateVisits pipeline. Within this pipeline, there is a processor called UpdateGeoIpData that fires a method called GeoIpManager.GetGeoIpData within Sitecore.Analytics.Tracking.CurrentVisitContext that initiates the GeoIP lookup for the visitor's interaction.

  • Within the GeoIP lookup logic, Sitecore will use the visitor's IP address to generate a unique identifier (GUID) based on the visitor's IP address. Eg. 192.168.1.100 => fd747022-dd48-b1ca-1312-eb4ba55030b2. 

NOTE: Sitecore performs all GeoIP lookups using this unique identifier. You can see this id by looking inside your MongoDB's GeoIPs collection. The field is named _id and this is the unique naming convention that MongoDB uses across all of its content. See my previous post for a snapshot.

  • Sitecore performs a GeoIP data lookup in memory cache.

  • If the GeoIP data IS in memory cache, then it will attach it to the visitor's interaction.

  • If the GeoIP data IS NOT in memory cache, it performs a GeoIP lookup in the MongoDB Analytics database's GeoIps collection.

  • If the GeoIP data IS in the MongoDB Analytics database's GeoIps collection, it attaches it to the visitor's interaction and stores the result in memory cache.

  • If the GeoIP data IS NOT in the GeoIps collection, it performs a lookup using the Sitecore Geolocation service and stores the result in memory cache and attaches it to the visitor's interaction.

NOTE: After a successful lookup, the GeoIP data is stored in the Tracker.Current.Interaction.GeoData (ContactLocation class)

GeoIP Data Cache

  • When the GeoIP data is obtained, it is added to a dictionary object that is part of the Sitecore Tracker so that it can be referenced via the Tracker.Current.Interaction.GeoData (shown above).

  • The odd thing that I noticed was that the cache expiration was set to 10 seconds (by default)
          Code reference:
          Sitecore.Analytics.Data.Dictionaries.TrackingDictionary
          private readonly TimeSpan defaultCacheExpirationTimeout = TimeSpan.FromSeconds(10.0);

GeoIP Data - End of visitor's session

  • At the end of the visitor's interaction / session, Sitecore will run the CommitSession pipeline.

  • Like the CreateVisits pipeline, there is a processor called UpdateGeoIpData that fires a method called GeoIpManager.GetGeoIpData (with the exact same code as in the CreateVisits pipeline). This initiates the GeoIP lookup flow once again (Cache / MongoDB / GeoIP Service).

  • Seems like the intention here is to confirm the visitor's GeoData before storing the data in MongoDB that will ultimately make it's way to the reporting database.

More To Come

Next, I intend to dig into Sitecore's GeoIP code for the 9.x series, and talk about the differences identified in that implementation.

Friday, July 27, 2018

Sitecore xDB - GeoIP and Contention Dynamics in MongoDB

Standard

Background

In my previous post, I discussed how our team has been diligently working to alleviate pressure on the our servers and MongoDB, on a high-traffic client's Sitecore Commerce site.

We use mLab to host our Experience Database, and while monitoring the telemetry of cluster, we noticed a series of contention indicators related to the increased number of queries and connections during high-traffic surges during the day.

In our scenario, our client's site has a lunchtime traffic surge between 11am and 3pm every day.

Contention Dynamics

Overall, our MongoDB was not being over-taxed in terms of overall capacity, as we were not using up all the RAM and CPU, but the telemetry charts did show what looked like pretty clear contention.

We noticed a certain pattern and volume in the site's traffic that lead to contention dynamics on our MongoDB nodes. The contention would eventually start to affect the Sitecore Content Delivery servers, which were obviously also dealing with that day’s peak load of web lunchtime traffic.

We were seeing a surge in connections with data reads (as reflected in MongoDB metric) such as the count of Queries (Operations Per Second) and the Returned documents count (Docs Affected Per Second). This was leading to a high degree of contention, as reflected in various other MongoDB metrics (CPU time, queues, disk I/O, page faults).


Our initial theory supported the idea the root cause of this contention in MongoDB was caused by high volume of lunchtime traffic in Sitecore, but in an indirect way.

GeoIP and MongoDB

Having troubleshooted Sitecore's GeoIP service before, I had a pretty good understanding of the flow.

If you need some insight, I suggest reading Grant Killian's post: https://grantkillian.wordpress.com/2015/03/09/geoip-resolution-for-sitecore-explained

In summary, the flow looks like this:
  • Visitor visits Sitecore website
  • Sitecore performs a GeoIP information lookup from the memory cache using the visitor's IP address
  • If the GeoIP information IS in memory cache then it uses it in the visitor's interaction
  • If the GeoIP information IS NOT in memory cache, it performs a GeoIP lookup in the MongoDB Analytics database's GeoIps collection
  • If the GeoIP information IS in the MongoDB Analytics database's GeoIps collection, it uses it in the visitor's interaction and stores the result in memory cache
  • If the GeoIP information IS NOT in the GeoIps collection, it performs a lookup using the Sitecore Geolocation service and stores the result in memory and uses it in the visitor's interaction

Our high-traffic site makes heavy use of GeoIP, as the Home Page is personalized based on the visitor's location and local time. 

There had to be a correlation between the high-traffic, GeoIP and the activity we were seeing on our MongoDB cluster. 

The item that stood out at me was the highlight above - the GeoIP lookup against the MongoDB Analytics GeoIps collection.



Running a record count query against the GeoIps collection, we discovered that it contained 7.4 million records! This confirmed our theory that the MongoDB GeoIp collection was heavily populated and being used for the lookups to hydrate the visitor's interaction and memory cache.

As a side note, if you crack open the interaction collection, you can see how Sitecore ties the GeoIP data from the lookup to the visitor's interaction (this is old news):


GeoIP Cache Settings

After digging into the code, we discovered that Sitecore's GeoIP service uses the cache called LegacyLocationList to store the GeoIP lookup data after is has been returned from either MongoDB or the GeoLocation service.

The naming of the cache is what caught us by surprise. One would think that a "legacy" cache would no longer be used.

If you crack open the Sitecore.CES.GeoIp.LegacyLocation.dll with your favorite .NET Decompiler  and you will see the following:


We started monitoring this legacy location cache closely, and discovered that it was in fact hitting capacity and clearing frequently during our lunchtime traffic surge. This had a direct relationship with the contention we were seeing on our MongoDB nodes during that period of time.

It was obvious to us at this point, that the 12MB default size of this cache was not enough to handle all that GeoIP lookup data!



GeoIP Cache Size Updates and Results

Our team decided to increase the LegacyLocationList cache size to 20MB via a simple patch update:

 <setting name="CES.GeoIp.LegacyLocation.Caching.LegacyLocationListCacheSize">  
     <patch:attribute name="value">20MB</patch:attribute>  
 </setting>  
After our deployment, we monitored the cluster's telemetry closely. It was apparent by looking at the connection count, that there was an instant improvement resulting from the increased cache size.

Before the deployment of the cache setting change (LegacyLocationList cache default set to 12MB), we were averaging around 400 connections during the traffic surge.


After the deployment (increase the LegacyLocationList cache size to 20MB), our connection count was only averaging around 200!


Over the course of several weeks, our team was happy to report that during our lunchtime traffic surges, there was a dramatic reduction in connections with Data Reads, Operations Per Second, Docs Affected Per Second, CPU time, queues, disk I/O, page faults on our MongoDB cluster.

Another positive step towards our overall goal of improving MongoDB connection management on our Content Delivery servers.

Final Note

Another special thanks to Dan Read (Arke), Alex Mayle (Sogeti) for their contributions.

Sunday, July 8, 2018

Sitecore xDB: Performance Tuning your MongoDB Driver Configuration Settings

Standard

The Goal

Working with my team on a high-traffic client's Sitecore Commerce site, we were tasked with improving MongoDB connection management on the Content Delivery servers to help alleviate pressure on the servers and MongoDB, particularly during busy times of the day, and at times when traffic surges occur due to marketing campaigns or other real-world events.

The Key Settings

We confirmed that Sitecore ships with the default MongoDB driver settings that are actually set in the driver code.  You can view the default settings in the driver code, by following this GitHub link:
https://github.com/mongodb/mongo-csharp-driver/blob/v2.0.x/src/MongoDB.Driver/MongoDefaults.cs

Working with mLab Support, we determined that our focus would be on the following:

Min Pool Size

We decided to increase the Min Pool Size from the default 0 to 20. The mLab team approved this suggestion on the basis that we observed the Content Delivery server's connection pools maxing out due to the amount of operations that were happening during the Sitecore startup process.

Max Pool Size

We increased our Max Pool Size from the default of 100 to 150 in order to better accommodate surges in connection demand. The purpose of this update was to lessen the chance of running out of connections altogether.

Connection Idle Time

We increased the Connection Idle Time from the default of 10 minutes to 25 minutes to reduce the need to create new connections during normal and high-traffic surges.

Connection Life Time

We dropped the default setting of 30 minutes down to 0 (no lifetime). This change was based on the default setting that could also be a contributing factor to our observed connection churning.

Per this thread, a MongoDB engineer (driver author) suggested that this setting is likely not needed:
https://stackoverflow.com/questions/32816076/what-is-the-purpose-of-the-maxconnectionlifetime-setting

The How

As Kam explains in his post, Sitecore exposes an empty updateMongoDriverSettings pipeline that you can hook into to modify configurations that are not available in the connection string.

I created a processor to add to this pipeline that alters the MongoClientSettings:

Finally, I added the following patch to add the processor to pipeline, allowing us to pass the updated MongoDB driver settings to the custom processor:

Final Note

A special thanks to Dan Read (Arke), Alex Mayle (Sogeti) and the mLab Support Team for their contributions.

Thursday, May 10, 2018

Sitecore Azure Search: Sharing a Search Service in a Development Environment

Standard

Background

When developing on the Sitecore Platform using Azure Search, the cloud-based service behaves differently, and so it is best to develop against an actual search service to ensure that you uncover any unexpected behavior before pushing to a higher environment that is using Azure Search.

I put together a quick patch file that updates the Sitecore Index names, allowing your development team to share a Azure Search service.

Simply set the environment variable and drop this file in the "Include/zzz" folder.


Note: There is a platform dependency on the "sitecore_marketingdefinitions_master" and "sitecore_marketingdefinitions_web" naming so those 2 indexes are excluded from the patch.

In a development environment, we can live with sharing these indexes.

Update: Kudos to John Rappel for suggesting to use a variable to make the implementation simple.

Sunday, April 29, 2018

Exploring Sitecore Managed Cloud Part 1: Tiers, Sizing, Provisioning and Upgrades

Standard

Background

I have been working within a client's Sitecore Managed Cloud environment for the last several months, and wanted to share some insights gained from my experience in a series of blog posts, this being the first.



Tier Configuration

Sitecore Managed Cloud Hosting offers a variety of tiers based on traffic volume.

Each tier has a recommended hardware configuration, as shown here: https://doc.sitecore.net/sitecore_experience_platform/setting_up_and_maintaining/sitecore_on_azure/deploying/sitecore_configurations_and_topology_for_azure

When you commit to a tier, Sitecore's Managed Cloud Team will provision your Azure infrastructure to support that tier. You will have the ability to increase to the next tier when traffic increases are expected.

If traffic exceeds the threshold of the currently subscribed tier, an overage charge will be applied. From what I understand, the overage cost is about 25% greater than the additional cost of jumping to the next tier.

Tier Sizing and Overage

Sitecore Managed Cloud has the concept of an “included Azure spend” linked to each tier.

If you need to scale up or out, and the cost associated with your scaling goes above your “included Azure spend”,  it is up to you to pay an overage fee. This fee is based on an “overage multiplier” that Sitecore provides you with.

Here are some overage examples:

  • Additional Traffic beyond tier
  • Additional Web Apps Used (Add 1 or more Content Delivery Web App)
  • Exceed Storage Limits (loading large amounts of videos or pdfs)
  • Exceeding Storage Limits for xDB

TIER 1 TIER 2 TIER 2 TIER 4 TIER 5
XP–XSmall XP–Small XP–Medium XP–Large XP-XLarge
TRAFFIC (in visits/mo.) 0–100k 100k –200k 200k –1MM 1MM –5MM 5MM –10MM
Content Management 1 (B2) 1 (B2) 1 (B2) 1 (B2) 1 (B2)
Content Delivery 1 (B2) 2 (B2) 3 (B2) 4 (S2) 8 (S3)
Bandwidth 20 GB 40 GB 40 GB 60 GB 100 GB

The Azure App Service Plan pricing page provides details regarding what B2s and S3s are: https://azure.microsoft.com/en-us/pricing/details/app-service/

Sitecore recommends proactive increase of a given topology’s tiers when traffic increases are expected. However, the alternative overage charge poses only a moderate increase in cost to maintain site performance during unexpected, temporary spikes in volume. It also can serve as an indicator that advancement to a larger tier should be considered.

Infrastructure Provisioning

Before provisioning the new infrastructure, the Sitecore Managed Cloud Hosting Team will request that you provide them with the following  information:

  1. Sitecore Version (Eg: Sitecore XP 9.0 Update-1)
  2. Logical Name (Eg: MySuperSolution)
  3. Location of Deployment (Eg: East US)
  4. Location of your Microsoft Application Insights Resources (Eg: East US)
  5. Microsoft Live IDs who can access your Managed Cloud set
From my experience, provisioning will take about a day.

NOTE: As mentioned here: Sitecore provisions you with a temporary license file that is valid for one month. When the temporary license expires, Sitecore stops working, therefore it is important that you upload a valid permanent license.xml file as soon as possible.

Upgrades

Unfortunately, you as a partner or customer will be responsible for upgrades after the Sitecore Managed Cloud Hosting Team has provisioned your infrastructure.

Sitecore initially provisioned our infrastructure using Sitecore 9 Initial Release. After a couple weeks, Sitecore Update-1 was released, and we opened a ticket to request for them to provision all environments on the newer version.

The only reason they did this for us was because we hadn't deployed anything to the new environments, and they could simply delete the existing and provision the new ones using the updated Azure Web Packages.

So the point is - they will NOT upgrade your environment after you have deployed your custom solution into it.


Monday, April 2, 2018

SolrProvider SolrSearchFieldConfiguration Error After Upgrade from Sitecore 8.1 Update-3 to 8.2 Update-6

Standard

Background

In a previous post, I wrote about upgrading an existing client's large, multisite 8.1 rev. 160519 (8.1 Update-3) instance to 8.2 rev. 171121 (8.2 Update-6).

After multiple rounds of testing, I deployed the upgraded instance to our client's Staging server (Windows Server 2008 R2). When navigating to the Content Editor, I ran into the following Solr related error:

Unable to cast object of type 'System.Collections.Concurrent.ConcurrentBag`1[Sitecore.ContentSearch.SolrProvider.SolrSearchFieldConfiguration]' to type 'System.Collections.Generic.IReadOnlyCollection`1[Sitecore.ContentSearch.SolrProvider.SolrSearchFieldConfiguration]'.

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

Exception Details: System.InvalidCastException: Unable to cast object of type 'System.Collections.Concurrent.ConcurrentBag`1[Sitecore.ContentSearch.SolrProvider.SolrSearchFieldConfiguration]' to type 'System.Collections.Generic.IReadOnlyCollection`1[Sitecore.ContentSearch.SolrProvider.SolrSearchFieldConfiguration]'.



Digging In

After reviewing the stack trace and decompiling the Sitecore.ContentSearch.SolrProvider.dll , I focused on the SolrFieldNameTranslator.StripKnownExtensions method in the SolrFieldNameTranslator class:

1:  public string StripKnownExtensions(string fieldName)  
2:    {  
3:     fieldName = this.StripKnownCultures(fieldName);  
4:     foreach (SolrSearchFieldConfiguration availableType in (IEnumerable<SolrSearchFieldConfiguration>) this.fieldMap.GetAvailableTypes())  
5:     {  
6:      if (fieldName.StartsWith("_", StringComparison.Ordinal))  
7:      {  
8:       if (!fieldName.StartsWith("__", StringComparison.Ordinal))  
9:        break;  
10:      }  
11:      string str = availableType.FieldNameFormat.Replace("{0}", string.Empty);  
12:      if (fieldName.EndsWith(str, StringComparison.Ordinal))  
13:       fieldName = fieldName.Substring(0, fieldName.Length - str.Length);  
14:      if (fieldName.StartsWith(str, StringComparison.Ordinal))  
15:       fieldName = fieldName.Substring(str.Length, fieldName.Length);  
16:     }  
17:     return fieldName;  
18:    }  
Looking at line 4 above, the GetAvailableTypes method is shown below:

  private readonly ConcurrentBag<SolrSearchFieldConfiguration> availableTypes = new ConcurrentBag<SolrSearchFieldConfiguration>();  
   
  public IReadOnlyCollection<SolrSearchFieldConfiguration> GetAvailableTypes()  
   {  
    return (IReadOnlyCollection<SolrSearchFieldConfiguration>) this.availableTypes;  
   }  
   

As you can see in the code, when calling the GetAvailableTypes method, it casts the ConcurrentBag<SolrSearchFieldConfiguration> type to IReadOnlyCollection<SolrSearchFieldConfiguration> and this is where the "Unable to cast object" error message came from:

Unable to cast object of type 'System.Collections.Concurrent.ConcurrentBag`1[Sitecore.ContentSearch.SolrProvider.SolrSearchFieldConfiguration]' to type 'System.Collections.Generic.IReadOnlyCollection`1[Sitecore.ContentSearch.SolrProvider.SolrSearchFieldConfiguration]'.


Troubleshooting

According to docs.microsoft.com, the ConcurrentBag<T> class implements IReadOnlyCollection<T> interface in .NET Framework 4.5 and above. However, according to MSDN, in .NET Framework 4.0 it doesn't implement the IReadOnlyCollection<T> interface. The IReadOnlyCollection<T> interface was introduced in .NET Framework 4.5.

I went ahead and verified that the .NET Framework 4.5.2 was installed on the server. Next I checked my Web.config, and confirmed that the compilation targetFramework was set to "4.5.2" and that my httpRuntime targetFramework was set to "4.5.2":

<configuration>
  <system.web>
    <compilation targetFramework="4.5.2"/>
    <httpRuntime targetFramework="4.5.2"/>
  </system.web>
</configuration>

I then tried to repair the .NET framework, and even rebooted the server after the repair, and it didn't solve the problem.

Microsoft Security Update Woes

It was obvious that my 2 initial assumptions about not having the .NET Framework 4.5.2 installed or having a configuration issue were wrong.

As it turned out, .NET 4.5.2 did in fact contain the System.Collections.Concurrent.ConcurrentBag<T> class from System.dll which implemented the IReadOnlyCollection<T> interface, but there were a series of security updates that were rolled out by Microsoft, where the ConcurrentBag<T> did not implement the IReadOnlyCollection<T>. The security updates were built based on .NET framework 4.0 which did not have IReadOnlyCollection<T> interface at all.

In order to check whether one of the aforementioned security updates was installed, you could review the version of C:\Windows\Microsoft.NET\Framework\v4.0.30319\System.dll

For the regular .NET framework 4.5.2 this version looks like 4.5.2... , but if a security update was installed it will look like 4.0.30319.36388.

The Final Solution - So Simple It Hurt

Upgrading the .NET framework to 4.6.1 solved this issue because we were back to having the System.Collections.Concurrent.ConcurrentBag<T> class from System.dll which implemented the IReadOnlyCollection<T> interface.




Monday, February 26, 2018

My Upgrade Experience from Sitecore 8.1 Update-3 to 8.2 Update-6

Standard
I was assigned the task of upgrading an existing client's large, multisite 8.1 rev. 160519 (8.1 Update-3) instance to 8.2 rev. 171121 (8.2 Update-6).  This particular client wasn't ready to go all the way to version 9.0, but will do so in the near future.

It took me longer than anticipated to get things up and running, simply because I needed to perform some updates to their custom solution.


Side Notes

The Sitecore solution that I was upgrading was using Castle Windsor as the Inversion of Control container along with Glass Mapper.

Getting Ready

To get started, I navigated over to the dev.sitecore.net site to arm myself with the files needed to perform the upgrade. The files that I downloaded from the site included:

  • Upgrade guide: Sitecore-8.2-Update-6-Upgrade-Guide.pdf
  • Sitecore update package: Sitecore 8.2 rev. 171121 (update package)
  • Configuration files for upgrade: Sitecore 8.2 rev. 171121 (config files)
  • Sitecore Update Installation Wizard: Sitecore Update Installation Wizard 2.0.2 rev. 170703
  • ZIP archive of the Sitecore site root folder: Sitecore 8.2 rev. 171121.zip

The software tools I use when performing upgrades are:

The road to 8.2 Update-6

These are the steps necessary to perform the upgrade:

Disabled xDB located in Sitecore instance \Website\App_Config\Include\Sitecore.Xdb.config
  • <setting name="Xdb.Enabled" value="false" />
  • <setting name="Xdb.Tracking.Enabled" value="false" />

The instance didn't have Email Experience and WFFM modules, so I didn't need disable their respective config files.

Ran the SQL database script called "CMS82_BeforeInstall.sql" located in \Sitecore 8.2 rev. 171121 (config files)\Database Upgrade Script on all Sitecore databases:
  • Core
  • Master
  • Web
  • Reporting (Analytics)

Installed the Sitecore Update Installation Wizard 2.0.2 rev. 170703.zip regular Sitecore package.

After it completed, I proceeded to install the "Sitecore 8.2 rev. 171121.update" update package using the installation wizard:  /sitecore/admin/UpdateInstallationWizard.aspx.

You will need to unzip the Sitecore 8.2 rev. 171121 (update package).zip in order to obtain the "update" file that Sitecore requires.


After clicking the "Analyze the package" button,  I opted NOT to install files as I preferred to start with a clean copy of the web root of Sitecore 8.2 Update-6.

I feel like this is a cleaner approach as it helps avoid having legacy cruft make its way into the new instance.


The package installation completed without any issues.

Instance Preparation and Comparing Files

I proceeded to stand up a clean version of Sitecore 8.2 Update-6 alongside my legacy 8.1 Update-3 instance and used the Beyond Compare app to compare the files. 

Apart from the Web.Config, the Sitecore.config was the next file where I saw the most differences.

Pro Tip: It is best practice is to move any differences that you find in vanilla config files to separate patch files. That way, life will be much easier for future upgrades.

Updating your Custom Solution

As some of the config files compared could exist in your custom solution, it is best to update the files in your solution as soon as you have completed your comparisons / merges on your Sitecore instance.

I worked in a new branch in source control, so that I could gradually update the files, and commit them as I made progress.

Sitecore 8.2 moved to .NET Framework version 4.5.2 from 4.5 in 8.1. So the target framework in each of the solution's projects needed to be updated:


NuGet Fun

The custom solution I was working with had all the Sitecore referenced assemblies in a single Nuget package, consumed via a custom feed. I opted to switch to the Sitecore public NuGet feed: https://doc.sitecore.net/sitecore_experience_platform/82/developing/developing_with_sitecore/sitecore_public_nuget_packages_faq

A lot of time was spent making sure the correct NuGet packages were loaded so that references where correct. As I was working with 8.2 rev. 171121, I matched my NuGet packages to the version by using the 8.2.171121, "NoReferences" packages.

As Jeremy Davis pointed out: "...the .NoReferences packages include the binary files but don’t have dependencies on other packages. So if you want to pick out individual DLLs without having to import all of the other assemblies that file depends on, choose these packages. It’s a bit more effort to manually add each of these that you need – but it means your project only refers to the specific files you want."

Note: When updating packages like WebGrease for example, it is important to match the assembly version in the Sitecore bin folder to the NuGet package versions.

Working with Solr

As I was using Solr as my search provider, I used Patrick's Powershell script to set my config files to use Solr.

The Sitecore instance was using the Single Instance Solr Configuration - Patch #391039, that I discussed in this post: http://sitecoreart.martinrayenglish.com/2016/09/bulletproofing-your-sitecore-solr-and.html

Support for Solr out-of-the box with this patch was added from Sitecore 8.2 Update-1 on, so I didn't have to include any configurations and files referencing this patch. Most of my work involved me changing my Solr index configurations

From:
Sitecore.Support.ContentSearch.SolrProvider.SwitchOnRebuildSolrSearchIndex, Sitecore.Support.391039

To:
Sitecore.ContentSearch.SolrProvider.SwitchOnRebuildSolrSearchIndex, Sitecore.ContentSearch.SolrProvider

Example

From:
 <index id="my_custom_master_index" type="Sitecore.Support.ContentSearch.SolrProvider.SwitchOnRebuildSolrSearchIndex, Sitecore.Support.391039">

To:
 <index id="my_custom_master_index" type="Sitecore.ContentSearch.SolrProvider.SwitchOnRebuildSolrSearchIndex, Sitecore.ContentSearch.SolrProvider">

Bye Bye IsPageEditorEditing, Hello IsExperienceEditorEditing

As Darren mentioned in his post, Sitecore depreciated the variables IsPageEditor, and IsPageEditorEditing in Sitecore 8.0 Update 6, but kept the methods in all versions of 8.1. 

It would have been nice to have used the Obsolete attribute so that there wouldn't be such a surprise when upgrading to 8.2, and having all your usages of this method break your solution.

The fix was simple enough though. I performed a "find and replace" 

From:
Sitecore.Context.PageMode.IsPageEditorEditing

To:
Sitecore.Context.PageMode.IsExperienceEditorEditing

Problems with Castle Windsor

The solution I was working in was using Castle Windsor 3.3.0.51 and Castle Core 3.3.3.58. I opted to update Castle Windsor to version 4.1.0.0 and Castle Core 4.2.1.0 because I wanted the bug fixes and enhancements of the newer releases.

After deploying the updated assemblies to my upgraded Sitecore instance, I ran into the following error:

Could not load file or assembly 'Castle.Windsor, Version=3.3.0.0, Culture=neutral, PublicKeyToken=407dd0808d44fbdc' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference.

Castle Core changed the AssemblyVersion attribute to only include major version numbers so that they could avoid assembly binding errors with future releases/upgrades of Castle Core: https://github.com/castleproject/Core/issues/307

In my case, the error was happening because I had assemblies that were compiled against the new AssemblyVersion strategy.

Applying the following assembly binding redirects in my Web.config, fixed the issue.

<dependentAssembly>
        <assemblyIdentity name="Castle.Core" publicKeyToken="407dd0808d44fbdc" culture="neutral" />
        <bindingRedirect oldVersion="0.0.0.0-999.999.999.999" newVersion="4.0.0.0" />
</dependentAssembly>

<dependentAssembly>
        <assemblyIdentity name="Castle.Windsor" publicKeyToken="407dd0808d44fbdc" culture="neutral" />
        <bindingRedirect oldVersion="0.0.0.0-999.999.999.999" newVersion="4.0.0.0" />
</dependentAssembly>

Minor Problem with Glass Mapper

Like Castle, I also opted to update Glass Mapper to a higher version. By doing so,  I ran into a small issue, similar to what is described here: https://github.com/mikeedwards83/Glass.Mapper/issues/244

In my case, I discovered that I was simply missing the Glass.Mapper.Sc.Mvc references to the new assembly in the MVC 52 folder in the Nuget package, and the updated assembly in my Sitecore bin folder.

Minor Problem with WebGrease

After making my way through the above-mentioned problems, I ran into a WebGrease version issue. 

Inner Exception: Could not load file or assembly 'WebGrease, Version=1.5.2.14234, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)

The fix for this was to simply update my assembly binding redirect in the Web.config.

From:
<dependentAssembly>
        <assemblyIdentity name="WebGrease" publicKeyToken="31bf3856ad364e35" culture="neutral" />
        <bindingRedirect oldVersion="0.0.0.0-1.5.2.14234" newVersion="1.5.2.14234" />
      </dependentAssembly>

To:
<dependentAssembly>
        <assemblyIdentity name="WebGrease" publicKeyToken="31bf3856ad364e35" culture="neutral" />
        <bindingRedirect oldVersion="0.0.0.0-1.6.5135.21930" newVersion="1.6.5135.21930" />
      </dependentAssembly>

Let Me Hear You Say "Hallelujah"!

After I completed all these updates and fixes, I was presented with a beautiful new instance of Sitecore 8.2 Update-6 where my site loaded beautifully and my logs were clean.

Per Sitecore's upgrade guide, I completed the following final steps:
  • Cleared the browser cache. 
  • Published the site. 
  • Rebuilt the search indexes and the link database. 
  • Redeployed marketing definitions. 

I made sure to review my Sitecore logs after performing all of these tasks, and was happy to report that they stayed error free.