Sitecore Azure PaaS - Fixing "No owin.Environment item was found in the context" When Application Initialization Configured

Background

On Azure PaaS, it is well known that Application Initialization can assist in “warming up” your Sitecore application when scaling up or swapping slots, so that your App Services instances aren't thrown into rotation to receive traffic when they aren't ready. 

Unfortunately, since Sitecore 9.1, there is a nasty Owin error that prevents this from working correctly and this in turn impacts the healthy signaling of your App Services. 

Strangely, the error would disappear after about 2 minutes.

Sitecore Support Response

After talking with Sitecore Support engineers, they ended up telling us that using <applicationInitialization> had not been tested for Sitecore, and that they don't have any official guidelines on configuring Azure App Initialization.

Finally, they said that they were not able to provide us with any assistance.

Cause

I started deciphering the problem on my own. 

Looking at this error, the key to understand its root was on line 6: the BeginDiagnostics processor.

1:  [InvalidOperationException: No owin.Environment item was found in the context.]  
2:    System.Web.HttpContextExtensions.GetOwinContext(HttpContext context) +100  
3:    Sitecore.Owin.Authentication.Publishing.PreviewManager.get_User() +15  
4:    Sitecore.Owin.Authentication.Publishing.PreviewManager.GetShellUser() +23  
5:    Sitecore.Pipelines.HttpRequest.BeginDiagnostics.ResolveUser() +41  
6:    Sitecore.Pipelines.HttpRequest.BeginDiagnostics.Process(HttpRequestArgs args) +43  
7:    (Object , Object ) +15  

The BeginDiagnostics processor in the httpRequestBegin pipeline is used for Sitecore's diagnostic tools. This determines what happens if you try to run in Debug Mode in the page editor.

In this case, the PreviewManager User property was trying to return the HttpContext.Current.User which was not available yet, and was returning null and the error “No owin. Environment item was found in the context.” was thrown.

The Fix

After determining what was causing this error, the fix was simple. The processor isn't needed on Content Delivery instances as Debug Mode is only used on the Content Management instances.

Removing the processor via a simple patch fixed the error:

Read More

Sitecore xDB Resharding Tool - Unlock xDB Storage and Performance Limitations by Increasing Collection Shards Without Data Loss

Background

There is currently no way for you to increase the number of xDB collection database shards for an existing deployment, without starting from scratch and losing all your data.

The release of my colleague Vitaly Taleyko's Sitecore xDB Resharding Tool solves this problem for all of us, as it provides us with a migration path from an old shard collection to a new one.

https://github.com/pblrok/Sitecore.XDB.ReshardingTool

The inability to increase shards after deployment is a major problem for enterprise customers using the platform, who may not be aware of how quickly the collection databases will grow over time.

If you are a Sitecore veteran, you have experienced this rapid collection growth in MongoDB. As soon as the platform is "turned on", it starts collecting interactions and events for both anonymous and identified contacts.

Putting a Sitecore environment into Production, means opening up the flood gates for a massive amount of data that you don't have much control over.

xDB Search Index

On the xDB index side of the house, Sitecore has filtered the interactions and contact data in the xDB index to identified contacts only (by default). This is good for simple customers who aren't doing much with the platform.

However, if you have millions of contacts, you will have the same index problems as you may have faced when dealing with anonymous contact data in previous versions mentioned in this blog post from a while ago. There is a solve for this, that I may touch on in a later post.

The 2 Shard SQL Problem

Out of the box, if you install the Sitecore platform using default scripts and SIF or if Sitecore Managed Cloud has set up your environment in Azure, you will have 2 SQL shard collection databases.

The value proposition of Sitecore xDB is the ability to store any events tied to an interaction from any channel in xDB. They have provided a robust set of APIs to allow this.

The problem is that storing hundreds of gigabytes or even terabytes of data requires a very carefully planned strategy, or else, you will fail. It is just a matter of when.

I always have the following scene from Evan Almighty in my head when I talk about this problem:

The bottom line - if you have an enterprise deployment, and are using xDB, the out of the box 2 shard collection database configuration is not enough!

New Deployments

If you are new to the Sitecore platform, you can fix this by using the Shard Map Manager Tool to increase the number of shards. This great post by Kelly Rusk explains how: http://thebitsthatbyte.com/what-is-and-how-to-use-the-sitecore-9-shard-map-manager-tool

Existing / Live Deployments

Bad news if you have an existing deployment.

You will hit bottlenecks as you store more contact, interaction and event data. With limited CPU, storage capacity and memory, database performance will start to suffer and query performance and routine maintenance will slow down.

When it comes to adding resources to support database operations, vertical scaling (aka scaling up which is very easy to do on Azure) has its own set of limits and eventually reaches a point of diminishing returns.

The Negative Ripple Effect on xDB

I have seen cases where due to the massive amount of data stored in the xDB collection shards over time, xConnect Search will fail to keep the xDB index in sync, and xConnect search will stop working.

After this happens, the only option is to rebuild your xDB index, but because of the poor xDB collection database performance, it will take days, if not weeks if you are lucky.

Or, it will simply keep failing.

How Increasing Shards Helps

Adding additional collection shards to xDB means additional SQL compute capacity to serve incoming queries in the distributed configuration, and thus faster query response times and index builds.

Additional shards will increase total cluster storage capacity, speed up processing, and offer higher availability at a much lower cost than vertical scaling.

How the xDB Resharding Tool Helps

As I started working with Vitaly to architect this tool, our first idea was to use the Data Exchange Framework that is used to power the xDB Migration Tool. We had used a customized version of this tool when we migrated from our Sitecore 8.2 deployment to our current 9.1.

We decided to pivot, because we wanted a lightweight tool that could run on any Windows-based machine, and would run directly against SQL and as a result, be much more efficient!

The Beauty of the Tool Part 1: Migrating Your Data

This tool allows you to reshard your Sitecore xDB Collection Databases without any data loss.

What does this mean exactly?!

This tool allows you to migrate your current SQL xDB analytics collection database shards to a new set of xDB analytics collection database shards without losing any of your data.

So, for example if you have 2 shard databases, and want to move up to 4 shard databases, this tool will allow you to migrate over.

For this example, you would set up 4 new shards using SIF (as shown in Vitaly's GitHub readme doc), or use the Shard Map Manager Tool, and then point the tool at your old shards and the new shards and voila! Your data will get migrated over!

The Beauty of the Tool Part 2: Resume Mode

Another fantastic feature that Vitaly added to this tool is "resume mode". If there is a glitch in the migration process, or you need to stop it manually for some reason and resume later, it will remember where it left off and pick the migration right back up!

Battle-Tested and Ready For Download!

This tool has been tested, and I can say that it works, and works well!

You can check the full source code out on Vitaly's GitHub: https://github.com/pblrok/Sitecore.XDB.ReshardingTool

You can download your copy today using this link: https://github.com/pblrok/Sitecore.XDB.ReshardingTool/raw/master/ToolReleases/win-x64.zip

Read More

Sitecore Azure PaaS: Updating Your License File For All Deployed App Service Instances

Background

If you are provisioning a new set of Sitecore environments on your own, or if the Sitecore Managed Cloud Hosting Team provisions your environments for you, you will most likely be using a temporary license file that is valid for 1 month while you are waiting for your permanent license file .

When the temporary license expires, your Sitecore instances will stop working. Therefore, it is important that you upload a valid permanent license.xml file as soon as it is available.

File Locations

In an XP Scaled environment, there are many different App Services and locations where the license.xml file will need to be updated.

I created a list of the App Service roles and the license file locations for your reference:

App Service Role	License File Location
xc-search	\App_data & \App_data\jobs\continuous\IndexWorker\App_data
ma-ops	\App_data & \App_data\jobs\continuous\AutomationEngine\App_Data
cd	\App_data
cm	\App_data
ma-rep	\App_data
prc	\App_data
rep	\App_data
xc-collect	\App_data
xc-refdata	\App_data

Updating the License File

The easiest way to update the file is to use the Debug console in the Kudu "Advanced Tools" in your App Service Instance, or an FTPS client to connect directly to the App's filesystem.

Read More

Exploring Sitecore Managed Cloud Part 1: Tiers, Sizing, Provisioning and Upgrades

Background

I have been working within a client's Sitecore Managed Cloud environment for the last several months, and wanted to share some insights gained from my experience in a series of blog posts, this being the first.

Tier Configuration

Sitecore Managed Cloud Hosting offers a variety of tiers based on traffic volume.

Each tier has a recommended hardware configuration, as shown here: https://doc.sitecore.net/sitecore_experience_platform/setting_up_and_maintaining/sitecore_on_azure/deploying/sitecore_configurations_and_topology_for_azure

When you commit to a tier, Sitecore's Managed Cloud Team will provision your Azure infrastructure to support that tier. You will have the ability to increase to the next tier when traffic increases are expected.

If traffic exceeds the threshold of the currently subscribed tier, an overage charge will be applied. From what I understand, the overage cost is about 25% greater than the additional cost of jumping to the next tier.

Tier Sizing and Overage

Sitecore Managed Cloud has the concept of an “included Azure spend” linked to each tier.

If you need to scale up or out, and the cost associated with your scaling goes above your “included Azure spend”,  it is up to you to pay an overage fee. This fee is based on an “overage multiplier” that Sitecore provides you with.

Here are some overage examples:

• Additional Traffic beyond tier
• Additional Web Apps Used (Add 1 or more Content Delivery Web App)
• Exceed Storage Limits (loading large amounts of videos or pdfs)
• Exceeding Storage Limits for xDB

	TIER 1	TIER 2	TIER 2	TIER 4	TIER 5
	XP–XSmall	XP–Small	XP–Medium	XP–Large	XP-XLarge
TRAFFIC (in visits/mo.)	0–100k	100k –200k	200k –1MM	1MM –5MM	5MM –10MM
Content Management	1 (B2)	1 (B2)	1 (B2)	1 (B2)	1 (B2)
Content Delivery	1 (B2)	2 (B2)	3 (B2)	4 (S2)	8 (S3)
Bandwidth	20 GB	40 GB	40 GB	60 GB	100 GB

The Azure App Service Plan pricing page provides details regarding what B2s and S3s are: https://azure.microsoft.com/en-us/pricing/details/app-service/

Sitecore recommends proactive increase of a given topology’s tiers when traffic increases are expected. However, the alternative overage charge poses only a moderate increase in cost to maintain site performance during unexpected, temporary spikes in volume. It also can serve as an indicator that advancement to a larger tier should be considered.

Infrastructure Provisioning

Before provisioning the new infrastructure, the Sitecore Managed Cloud Hosting Team will request that you provide them with the following  information:

1. Sitecore Version (Eg: Sitecore XP 9.0 Update-1)
2. Logical Name (Eg: MySuperSolution)
3. Location of Deployment (Eg: East US)
4. Location of your Microsoft Application Insights Resources (Eg: East US)
5. Microsoft Live IDs who can access your Managed Cloud set

From my experience, provisioning will take about a day.

NOTE: As mentioned here: Sitecore provisions you with a temporary license file that is valid for one month. When the temporary license expires, Sitecore stops working, therefore it is important that you upload a valid permanent license.xml file as soon as possible.

Upgrades

Unfortunately, you as a partner or customer will be responsible for upgrades after the Sitecore Managed Cloud Hosting Team has provisioned your infrastructure.

Sitecore initially provisioned our infrastructure using Sitecore 9 Initial Release. After a couple weeks, Sitecore Update-1 was released, and we opened a ticket to request for them to provision all environments on the newer version.

The only reason they did this for us was because we hadn't deployed anything to the new environments, and they could simply delete the existing and provision the new ones using the updated Azure Web Packages.

So the point is - they will NOT upgrade your environment after you have deployed your custom solution into it.

Read More