Sunday, April 11, 2021

Sitecore Publishing Service - Using Sitecore PowerShell Extensions To Move Publishing Jobs To The Top Of The Queue

Standard

Background

In my previous post, I provided a way to get a job queue report using PowerShell Extensions (SPE). In this post, I am going to show how you can use the output from the report to promote publishing jobs to the top of the queue using SPE.

Large Publishing Queue

You may ask, well why? Sitecore's Publishing Service is a great improvement over the out-of-the-box publishing mechanism, and is pretty fast at publishing items.

That is indeed true, however when working with extremely large sites with several hundred content authors and multiple publishing targets, the queue can become extremely long. I have seen it grow to upwards of several thousands items, and publishing taking several hours.

This is problematic if you have something urgent that needs to be published, as the job could be sitting in the queue for hours!

The Solution

As you saw in my last post, it is pretty simple to access the SQL publishing queue database table using SPE. As the operations of a queue make it a first-in-first-out (FIFO) data structure based on the "Queued" datetime field,  I discovered that simply updating target job's datetime field to a smaller value, would instantly move the job higher in the queue.


So, my final logic was this:

  • Get smallest Queued datetime of the jobs sitting in the queue that still needed to be published
  • Subtract 2 minutes from the value
  • Update the queued datetime of job that I want to promote to the top of the queue with this new smaller datetime value
  • Done! My job was popped to the top!

Now, this is the perfect pairing with the job queue report from my previous post. You can use the report to find the job and its id that you want to promote, and then use the id to run the script to promote the job!

I recommend following this guide to convert this into your own SPE custom module for your solution: Modules - Sitecore PowerShell Extensions

 




I hope you find this script another useful add to your PowerShell toolbox.

Saturday, March 20, 2021

Sitecore Publishing Service - Publishing Job Queue Report Using Sitecore PowerShell Extensions

Standard

Background

Sitecore's Publishing Service only allows you to see a maximum of 10 items at a time within the Queued or Recent jobs reports within the dashboard.

This is not ideal if you need to see how many total items are in the queue, need to get an estimate of how long it will take to get your publish live or quite simply need to do any type of analysis or troubleshooting.

Usually, you will have to talk to a DevOps person who has access to your Sitecore Master database, and get them to write a somewhat complex SQL query against your Publishing_JobQueue table to get you the information that you need.

It is a bit complex due to the fact that most of the key information is stored in an XML field called "Options" within this particular table.


PowerShell For The Win

After spending a bit of time formulating a decent SQL query that would get the key information that we were after, I decided to take it one step further by incorporating it into a PowerShell script that could be generated on demand from within the Sitecore console, and also output a searchable and downloadable report.

A clear win for our Authoring Admins and DevOps teams!

I hope you find this script a useful add to your PowerShell toolbox.



Saturday, March 13, 2021

Sitecore Content Hub - Set up SAML-based SSO in Azure AD using an App Registration

Standard

Background

In this post, I will show you how to create and configure an Azure Application Registration in your tenant to allow Sitecore Content Hub users to successfully authenticate against your Azure Active Directory.

Options

The Content Hub team's preferred set up option is to create an Enterprise application within your Azure AD, but unfortunately for us, our DevOps would not allow this due to very strict security constraints that we had to abide by. This is the main reason that we had to go the App Registration route.

We initially tried to get the App Registration working using Microsoft Provider SSO, but could not get the proper Group claims working correctly.

As a result, we focused on configuring SAML Auth within our App Registration, and were able to get all the claims needed to successfully get SSO authentication working with this approach.

Set up within Azure

Within your Azure Portal, find App registrations and click on the New Registration button. Give it a name, and leave the default options selected, and click Register.


Within the newly created registration, go to the Authentication menu option within the Manage section.

Click "Add a platform", and then select "Web".


Set your Redirect URIs to be the Content Hub portal url. You will be able to add additional URIs after the initial set up. For now, I will use a default one.

Make sure you check the Access tokens and ID tokens boxes within the Implicit grant and hybrid flows section.




After this, click the Configure button.

Next, go to the Token configuration menu option within the Manage section.

Click Add group claim, and check the Security group box. Confirm that the Group ID radio option is selected within the ID, Access and SAML options.

Click the Add button.


Next, click Add optional claim.

Within Token type, select SAML, and check the email Claim box. Click Add.


When prompted, check the "Turn on the Microsoft Graph email permission" box to allow the claims to appear in the token. Click Add.


Next, go to the Expose an API menu option within the Manage section. Click Add a scope, and it will generate an Application ID URI for you. 

Make note of this, as you will need it for the Content Hub side.

Click Save and continue.


After is has been created, you can click the Cancel button.




Go to the Overview menu option, and click Endpoints. Go to the Federation metadata document XML url and make note of it

Then, copy and past it into a new browser tab.





Make note of the entityID.

Your set of notes should look similar to this:


Set up within Content Hub

Log into your Content Hub portal. Click on Manage, and then go to Settings.



Within Settings, go to PortalConfiguration, and select the Authentication menu option. Change the view to Text as it's easier to work with.

Within the ExternalAuthenticationProviders, saml XML config, set the key values to what you saved in your notes. Make sure you set the provider_name and add some basic messages.



Example:
 ExternalAuthenticationProviders": {  
   "global_username_claim_type": "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name",  
   "global_email_claim_type": "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress",  
   "google": [],  
   "Microsoft": [],  
   "saml": [  
    {  
     "metadata_location": "https://login.microsoftonline.com/8ac76c91-e7f1-41ff-a89c-3553b2da2c17/federationmetadata/2007-06/federationmetadata.xml",  
     "sp_entity_id": "api://c8696890-1d5f-479b-9df1-154e8f315165",  
     "idp_entity_id": "https://sts.windows.net/8ac76c91-e7f1-41ff-a89c-3553b2da2c17/",  
     "password": null,  
     "certificate": null,  
     "binding": "HttpRedirect",  
     "authn_request_protocol_binding": null,  
     "is_enabled": true,  
     "provider_name": "martinSamlNewLocal",  
     "messages": {  
      "signIn": "Martin SAML SSO Test"  
     },  
     "authentication_mode": "Passive"  
    }  
   ],  
   "sitecore": [],  
   "ws_federation": [],  
   "yandex": []  
  }  

Click Save, and you are done!

You are now ready to test out your authentication using your shiny, new authentication button.

Users with more than 200 groups

We found a limitation with SSO authentication group claims in Azure AD https://docs.microsoft.com/en-us/azure/active-directory/hybrid/how-to-connect-fed-group-claims wherein if there are more than 200 groups associated to a user, then the SSO authentication will provide a graph link instead of passing in the group claims. 

There is currently no solution for this problem. We are handling these handful of users via manual security set up.

Sunday, November 29, 2020

Sitecore PowerShell Extensions - Find Content Items Missing From Sitecore Indexes

Standard

Over the course of the last month, we ran into data inconsistencies between what was in our content databases compared to our Solr indexes.

We have content authors from around the globe and content creation happens around the clock by authors via the Experience Editor and imports via external sources.

Illegal Characters Causing Index Issues

As mentioned by this KB article https://kb.sitecore.net/articles/592127, index documents are submitted Solr in XML format, and if your content contains and “illegal” characters that cannot be converted to XML, all documents in the batch submission will fail.  

When you perform an index rebuild or re-index a portion of your tree, Sitecore will submit 100 documents in a batch to Solr. How is the related to the character issue? If you perform an index rebuild and have a single bad character in one of your items in the batch, none of the 100 docs in that batch will make it into your Solr index. 

What makes this especially difficult to troubleshoot is that item batches contain different items every time. So, what could be missing from your index during one rebuild, could be different during the next rebuild.  

There is a good Stack Exchange article that explains all of this, and kudos to Anders Laub who provides a pretty decent fix for this issue: https://sitecore.stackexchange.com/questions/18832/wildly-inconsistent-index-data-after-rebuilds 

PowerShell Index Item Check Script

There are several other reasons why content could be missing from your Sitecore Indexes, and so I needed to come up with a way to identify would could be missing.

PowerShell Extensions for the win!

I decided to create a PowerShell script to do just that - check for items in a selected target database that are missing in selected index, and produce a downloadable report.

What’s nice is that I strapped on an interactive dialog making it friendly for Authors or DevOps to make their comparison selections.



If you are newish to PowerShell Extensions, this could also help you understand how powerful it truly is, and serve as a guide to build your own scripts that you can use daily!

Saturday, November 14, 2020

The Curious Case of Sitecore's Enforce Version Presence Permission Denied - The Fix

Standard

In a previous post, I shared a baffling enforce version presence & language fallback issue, that lead my team down a rabbit hole until we discovered that it was indeed a critical Sitecore.Kernel bug that impacted all versions, including the recent 10.0 version.

Since then, the Sitecore product team has provided a fix for this issue, which will be part of the upcoming 10.1 release.  

Until then, you can open a support ticket and reference bug #416301 and request help with your specific Sitecore version if you run into this problem.


How was it fixed?

As previously mentioned, the piece of code responsible for the permission denied error was within the GetAncestorAccess within the Sitecore.Security.AccessControl.ItemAuthorizationHelper class which is part of the Sitecore.Kernel. 

Within this method, regardless of the value that was being returned for the security checks, the key/value combo was stored in AccessResultCache and resulted in the permission denied error being thrown the next time the item was requested for a different language.

To correct this problem, a EnforceVersionPresenceDisabler "using statement wrapper" was added within the GetAccess method that is responsible for returning the "access allowed for an operation on an item".  See line 26 below. 

The switcher disables the Enforce Version Presence functionality, more specifically, it bypasses the functionality that enforces the relevant translated language version of the item to be be available for it to be returned from the API.

This was the key in corrected the access issue related to the extranet\anonymous user, and enforce version presence logic.

Saturday, November 7, 2020

Sitecore Publishing Service - Publishing Sub Items of Related Items

Standard

Background

We ran into an issue with our Sitecore 9.1 and Publishing Service 4.0 environment where when a page item with a rendering was being published, the corresponding data source of the rendering was not published fully.

To be more specific, if the rendering referred to a data source item that had multiple levels of items, then only the root of the data source was being published but not the child items.

A good example would be a navigation rendering that had a data source item with a lot of children. Content authors were making updates to all the link items within Experience Editor, but they were not being published.

This was happening for both manual publishing and publishing through workflow.


Configuration Updates

In our research, we discovered that publishing service allows you to specify the templates of the items you wish to publish as descendants of related items. 

Adding the following node to sc.publishing.relateditems.xml did trick (after a restart):

It is very important to note that the the template nodes need to have unique names in order for this to work. 

In other words: DatasourceTemplate1, DatasourceTemplate2, DatasourceTemplate3 etc. 

So as you can imagine, if you want to include a lot of data source item templates, the list in your configuration can get extremely large!

Final Words

I hope that this information helps developers who face a similar issue, as I could not find anything online about this related publishing configuration.

As always, feel free to comment or reach me on Slack or Twitter if you have any questions.                                                    

Saturday, October 31, 2020

Control Tracking JavaScript Using Sitecore Rule-based Configuration

Standard

Background

Being able to control tracking JavaScript via Environment and Server role is a common problem that Sitecore developers are faced with. For example, your client doesn't want their Production Google Tag Manager or agency delivered tracking pixel scripts firing on any server / app instance other than their Production Content Delivery as it will spoil analytics.

Most of the time, developers will add some type of "if statement" code in Layouts or Renderings to help facilitate this, but this could be difficult to control and maintain based on the number of scripts you end up adding to you site(s). 

In addition, if you are using SXA and HTML snippets in your metadata partial design to house the scripts, this becomes even harder.

Post-Processing HTML

I wanted to focus on finding the sweet spot in Sitecore where I could inspect the entire HTML output after it had been glued together by the various pipelines, and then remove the target script from the HTML before it was transmitted to the browser.

Sitecore's httpRequestProcessed pipeline gives is the entry point, where we can leverage the MVC framework's result filter to manipulate the HTML.

I told my content authors to add a new attribute called "data-xp-off" to their scripts that I would use as the flag to determine if the script would be removed from the page.

For example:
<script data-xp-off>some tracking stuff</script>



Writing the Code

The first step was to create a new HttpRequest processer and associated configuration to inject into the httpRequestProcessed pipeline. Within this, I was able to access the HttpContext response filter object where I could perform the targeted script removal.

As you can see by the config, you can use whatever rule-based role config to apply the processor.

Next, I created a class based on the System.IO Stream class, where I overrode the Flush method. Within this new Flush method, I removed the script using a regular expression (based on the existence of the data-xp-off attribute within the html), and then wrote it to the response.

You will notice that I also included the "noscript" and "style" tags as an option for the filtering which was a bonus.

So you may ask me; "Martin, why did you not use the powers of the Html Agility Pack to perform your HTML manipulation?".  To be honest, that was my first approach. I wrote this code:

I discovered that the InnerHtml returned by the Agility Pack was making unintentional changes to my HTML markup, and that caused problems with client-side heavy components.  Digging into Sitecore's code, I discovered that they used the regular expression approach when injecting their Experience Editor "chrome" elements, and so I went down that path too.