A New Rudder on Sitecore.Ship to Deploy Securely From Visual Studio Team Services

I fell in love with Kevin Obee's Sitecore.Ship module when it was first introduced to me by my Arke colleague and MVP, Patrick Perrone, and it has been part of my Continuous Deployment routine ever since.

If you are using Hedgehog's TDS, a build server of your flavor, and a touch of PowerShell and / or curl, Sitecore.Ship allows you sail code and content into Integration, Staging and Production environments in a breeze.

This post assumes that you are familiar with using TDS to generate .update packages, and have used the Sitecore.Ship module before. I intend to demonstrate how you can use my customizations to keep the module's REST service secure, and deploy from Visual Studio Team Services (VSTS) to your various environments.

All updated source code and documentation can be found on my GitHub fork: https://github.com/martinrayenglish/Sitecore.Ship

Security

Out of the box, Kevin's module allows you to enable remote deployments and add an access control list of machine IP addresses that can access the REST web service.

 <packageInstallation enabled="true" allowRemote="true" allowPackageStreaming="true" recordInstallationHistory="true">  
  <Whitelist>  
   <add name="local loopback" IP="127.0.01" />  
   <add name="Allowed machine 1" IP="10.20.3.4" />  
   <add name="Allowed machine 2" IP="10.40.4.5" />  
  </Whitelist>  
 </packageInstallation>  

This works really well for most use cases, but not when you are deploying from Visual Studio Team Services' Azure servers. Keep reading and I will explain why.

Visual Studio Team Services

Microsoft has done a great job with VSTS a.k.a. TFS in the cloud. They provide a nice suite of tools such as code repositories, continuous integration, bug and task tracking, and agile planning at the very wonderful price of free for up to 5 users.

Once your team grows beyond 5 for free, the increments are priced reasonably. If you are an MSDN subscriber, you aren't counted against the total number of user's in the pricing model. So if your company is a Microsoft shop with MSDN, you could be working with your client's VSTS account at no cost to them. Sweet!

You can check out Microsoft's pricing by navigating over to this page: https://www.visualstudio.com/pricing/visual-studio-team-services-pricing-vs.

Setting up the Build Definition In Visual Studio Team Services

VSTS provides you with a slew of options to help set up your build definitions for either Continuous Integration or Continuous Deployment.

In my example, I set up a build definition for a Staging environment with a few simple clicks. The basic steps to add a new definition are:

Build

Add a series of tasks used to from your build steps

Repository

Set the repository type (in my case Git) and branch that you intend to build from

Variables

Set the build configuration (such as "MyClientStaging"), and platform settings

Triggers

If you want to perform Continuous Integration and build on every commit to the repository using the branch set in the Repository Tab

General

Make sure that the "Default agent queue" is set to "Hosted"

In my example, I set the above-mentioned configurations and build steps to my fancy, and the end result looked something like this:

You will notice that there are 2 PowerShell script tasks that are executed after the build and tests have run.

About The Scripts

Both scripts were originally crafted by my Arke colleague Patrick Perrone. I simply added a few modifications to make things work in my environments.

Make sure you check out his Sitecore.Ship-PowerShell-Functions module that is available on GitHub: https://github.com/patrickperrone/Sitecore.Ship-PowerShell-Functions

The first script is used to deploy my TDS generated .update package from VSTS' Azure servers, over to my target machine(s) that have Sitecore.Ship installed on them.

The second script is used to execute an improved version of Pavel Veller's ConfigFileCommitter service shown by his post: http://jockstothecore.com/update-config-love/

Sitecore.Ship Whitelist VSTS Challenges

As I mentioned previously, the security considerations in Kevin's module work really well in most traditional environments where you are using a deployment server that is either on-premise or in the cloud but has a single, controllable IP address.

The challenge I faced building and deploying directly from VSTS, was that it was impossible to predict the IP address that the Azure server was deploying from, and thus impossible to lock it down using the Whitelist configuration.

First Stab - IP Address Ranges

After forking the Sitecore.Shop repository, I started working through the code so that I could lock things down when deploying from VSTS. After adding some logging I noticed a pattarn of IP Address ranges from the Azure servers, and went on my merry way to update the module to include support for IP Address ranges.

So, with this in place, I could now apply ranges to my Whitelist configuration like so:

 <packageInstallation enabled="true" allowRemote="true" allowPackageStreaming="true" recordInstallationHistory="true">  
  <Whitelist>  
   <add name="local loopback" IP="127.0.01" />  
   <add name="Allowed machine 1" IP="10.20.3.4" />  
   <add name="Allowed machine 2" IP="10.40.4.5" />  
   <add name="Allowed IP Range 1" IP="23.96.0.0-23.96.255.255" />  
   <add name="Allowed IP Range 2" IP="65.52.0.0-65.52.255.255" />  
  </Whitelist>  
 </packageInstallation>  

Problem solved? 

Not so much. 

After running several deployment tests, I discovered that the Azure IP ranges were just not predicable at all. Based on what I saw, I would end up having to open up a ton of ranges, and would just hope and pray that things didn't change and cause my deployments to blow up.

It was obvious that as I started opening up more and more ranges, the security aspect of my deployments was sinking, and I was opening up myself to those pesky Internet pirates who could sail right on in. (Hope you enjoyed this ;)

Second Stab - Authentication Token

Having recently worked with Sitecore's xDB Cloud REST API, I decided to take their approach and build in an Authentication Token security option.

With this approach, I added the ability set the name and value of a custom header via configuration. I could then add this token in the header of the request when running the deployment to the machines.

The updated configuration looked like this:

 <packageInstallation enabled="true" allowRemote="true" allowPackageStreaming="true" recordInstallationHistory="true"   
 authHeader="X-Ship-Auth" authToken="AC074C6EDBA518F807E0E3F2F36A8B512D9C5637744BE67CD60D271244AC523AAB9CF8DB7F7D3934205E5BD850B2768C7171C3C594D6C6BFCA3992CCCCA67148">  
  <Whitelist>  
   <add name="local loopback" IP="127.0.01" />  
   <add name="Allowed machine 1" IP="10.20.3.4" />  
   <add name="Allowed machine 2" IP="10.40.4.5" />  
   <add name="Allowed IP Range 1" IP="23.96.0.0-23.96.255.255" />  
   <add name="Allowed IP Range 2" IP="65.52.0.0-65.52.255.255" />  
  </Whitelist>  
 </packageInstallation>  

Testing and Implementation

Testing

When testing the authentication changes using a tool like Postman, all you would need to do is pass over the token value in a custom header that you set up on configuration when making the request.

Implementation

In my case, I modified the PowerShell deployment script to include the new custom header containing my token.

Final Note on Service Paths - "Services" vs "SitecoreShip"

Part of the module's configuration includes adding a NancyHttpRequestHandler to your web.config specifying the path of Ship service, along with telling Sitecore not to process things at that path via an IgnoreUrlPrefixes setting.

Web.config:

 <handlers>  
 <add name="Nancy" verb="*" type="Nancy.Hosting.Aspnet.NancyHttpRequestHandler" path="/services/*" />  
 </handlers>  

Ship.config:

 <settings>  
    <setting name="IgnoreUrlPrefixes" set:value="/services/|/sitecore/default.aspx|/trace.axd|/webresource.axd|/sitecore/shell/Controls/Rich Text Editor/Telerik.Web.UI.DialogHandler.aspx|/sitecore/shell/applications/content manager/telerik.web.ui.dialoghandler.aspx|/sitecore/shell/Controls/Rich Text Editor/Telerik.Web.UI.SpellCheckHandler.axd|/Telerik.Web.UI.WebResource.axd|/sitecore/admin/upgrade/|/layouts/testing" />  
 </settings>  

This turned out to be problematic for a recent project, where we actually had an item that needed to use the "services" path. They had content living in "/services/contact-us".

So, I modified the original service path from "services" to "sitecoreship" to overcome this issue, and hopefully prevent this conflict from happening on my future projects.

All updated source code and documentation can be found on my GitHub fork: https://github.com/martinrayenglish/Sitecore.Ship