Publish to pre-production web database. Part 2.

A while back I shared a technique of publishing to a pre-production database via a custom ExtendedPublishProvider. As much as I like the clarity of this approach, as Kamsar mentioned in the comments, “…it has a downside: if you run a smart publish site to your staging database it will "unstage" any items that workflow staged into them with the current published version, ignoring the workflow state.”

Valid point.

After some internal brainstorming on the subject with our US based tech team, here is another approach to the problem that eliminates the drawback mentioned above.

What you can actually do is plug in a custom WorkflowProvider which will refer to a custom implementation of Sitecore.Workflows.Simple.Workflow class. This class has a method IsApproved(Item item) that is called from all publishing operations. We are going to override that and check if the item we are checking approval for is currently in a “semi-final” workflow state and if the database we publish to matches the designated “pre-production” database. Pretty simple, huh?

Here is the code.

1. First you need a custom version of the WorkflowProvider implementation:

using Sitecore.Data;
using Sitecore.Data.Items;
using Sitecore.Diagnostics;
using Sitecore.Workflows;
namespace Sitecore.Starterkit.Workflow
{
   public class WorkflowProvider : Sitecore.Workflows.Simple.WorkflowProvider
   {
      public WorkflowProvider(string databaseName, HistoryStore historyStore)
         : base(databaseName, historyStore)
      {
      }
      public override IWorkflow GetWorkflow(Item item)
      {
         Assert.ArgumentNotNull(item, "item");
         string workflowID = GetWorkflowID(item);
         if (workflowID.Length > 0)
         {
            // customization
            return new AdvancedWorkflow(workflowID, this);
            // customization
         }
         return null;
      }
      private static string GetWorkflowID(Item item)
      {
         Assert.ArgumentNotNull(item, "item");
         WorkflowInfo workflowInfo = item.Database.DataManager.GetWorkflowInfo(item);
         if (workflowInfo != null)
         {
            return workflowInfo.WorkflowID;
         }
         return string.Empty;
      }
      public override IWorkflow GetWorkflow(string workflowID)
      {
         Assert.ArgumentNotNullOrEmpty(workflowID, "workflowID");
         Error.Assert(ID.IsID(workflowID), "The parameter 'workflowID' must be parseable to an ID");
         if (Database.Items[ID.Parse(workflowID)] != null)
         {
            // customization
            return new AdvancedWorkflow(workflowID, this);
            // customization
         }
         return null;
      }
      public override IWorkflow[] GetWorkflows()
      {
         Item item = this.Database.Items[ItemIDs.WorkflowRoot];
         if (item == null)
         {
            return new IWorkflow[0];
         }
         Item[] itemArray = item.Children.ToArray();
         IWorkflow[] workflowArray = new IWorkflow[itemArray.Length];
         for (int i = 0; i < itemArray.Length; i++)
         {
            // customization
            var wfId = itemArray[i].ID.ToString();
            workflowArray[i] = new AdvancedWorkflow(wfId, this);
            // customization
         }
         return workflowArray;
      }
   }
}

2. Then you will need to attach it to the “master” database:

<!-- master -->
<database id="master" singleInstance="true" type="Sitecore.Data.Database, Sitecore.Kernel">
   <param desc="name">$(id)</param>
   ...
   <workflowProvider hint="defer" type="Sitecore.Starterkit.Workflow.WorkflowProvider, Sitecore.Starterkit">
      <param desc="database">$(id)</param>
      <param desc="history store" ref="workflowHistoryStores/main" param1="$(id)" />
    </workflowProvider>

 

3. Since the custom WorkflowProvider is referring to AdvancedWorkflow implementation, you need the code for that:

using System;
using Sitecore.Data;
using Sitecore.Data.Items;
using Sitecore.Data.Managers;
using Sitecore.Diagnostics;
using Sitecore.Globalization;
using Sitecore.SecurityModel;
using Sitecore.Workflows;
using Version = Sitecore.Data.Version;
namespace Sitecore.Starterkit.Workflow
{
   public class AdvancedWorkflow : Sitecore.Workflows.Simple.Workflow
   {
      private readonly WorkflowProvider _owner;
      public AdvancedWorkflow(string workflowID, WorkflowProvider owner)
                             : base(workflowID, owner)
      {
         _owner = owner;
      }
      private Database Database
      {
         get
         {
            return _owner.Database;
         }
      }
      public override bool IsApproved(Item item)
      {
         var result = base.IsApproved(item);
         if (!result && 
            Context.Site.Name.Equals("publisher",
                                     StringComparison.InvariantCultureIgnoreCase))
         {
            var stateItem = GetStateItem(item);
            if (stateItem != null &&
                MatchTargetDatabase(stateItem) &&
                IgnoreWorkflow(stateItem))
            {
               result = true;
            }
         }
         return result;
      }
      protected virtual bool MatchTargetDatabase(Item stateItem)
      {
         if (Context.Job != null && !String.IsNullOrEmpty(Context.Job.Name))
         {
            var target = TargetDatabase(stateItem);
            return Context.Job.Name.Equals(
                     String.Format("Publish to '{0}'", target), StringComparison.InvariantCultureIgnoreCase);
         }
         return false;
      }
      protected virtual string TargetDatabase(Item stateItem)
      {
         var publishTargetId = stateItem["Semi-Final Target Database"];
         var publishTargetItem = PublishActionHelper.GetItemById(publishTargetId);
         if(publishTargetItem != null)
         {
            return PublishActionHelper.GetFieldValue(publishTargetItem, "Target database");
         }
         return String.Empty;
      }
      protected virtual bool IgnoreWorkflow(Item stateItem)
      {
         return stateItem["Semi-Final"] == "1";
      }
      private Item GetStateItem(Item item)
      {
         string stateID = GetStateID(item);
         if (stateID.Length > 0)
         {
            return GetStateItem(stateID);
         }
         return null;
      }
      private Item GetStateItem(ID stateId)
      {
         return ItemManager.GetItem(stateId, Language.Current, Version.Latest, Database, SecurityCheck.Disable);
      }
      private Item GetStateItem(string stateId)
      {
         ID id = MainUtil.GetID(stateId, null);
         return id == (ID)null ? null : this.GetStateItem(id);
      }
      private string GetStateID(Item item)
      {
         Assert.ArgumentNotNull(item, "item");
         WorkflowInfo workflowInfo = item.Database.DataManager.GetWorkflowInfo(item);
         if (workflowInfo != null)
         {
            return workflowInfo.StateID;
         }
         return string.Empty;
      }
   }
}

All the magic is happening within “IsApproved” method that we override. We check if both “MatchTargetDatabase()” and “IgnoreWorkflow()” methods return true. The majority of other methods in this class are here because the derived class could not inherit those.

4. I reference PublishActionHelper class here too, it provides some utility methods for me:

 public class PublishActionHelper
   {
      public static Database Db
      {
         get { return Context.ContentDatabase ?? Context.Database ?? Factory.GetDatabase("master"); }
      }
      public static string GetFieldValue(Item item, string fieldName)
      {
         return item[fieldName] ?? String.Empty;
      }
      public static Item GetItemById(string id)
      {
         if (ID.IsID(id))
         {
            return Db.GetItem(ID.Parse(id));
         }
         return null;
      }
      public static string GetTargetDatabaseName(string targetId)
      {
         var publishingTarget = Db.SelectSingleItem(targetId);
         return publishingTarget["Target database"] ?? String.Empty;
      }
      public static IEnumerable<Item> GetItemsFromMultilist(Item carrier, string fieldName)
      {
         var multilistField = carrier.Fields[fieldName];
         if(FieldTypeManager.GetField(multilistField) is MultilistField)
         {
            return ((MultilistField)multilistField).GetItems();
         }
         return new Item[0];
      }
   }

5. Finally, you will need to have the extended version of the WorkflowState template where you can set the needed flags and settings:

I created my own “Semi Final State” template which inherits from standard /System/Workflow/State template.

This solution has been provided to a customer, and appears to be working fine in production for a couple of months now.

Let me know what you think!

Exception in alarm clock event subscriber [Common Errors]

Greetings! As you are going into production, hopefully you allocate a few days for exception sweep up. This is applicable to any website running on Sitecore CMS or any platform, really.

One of the common exceptions that I see in customer’s log files these days is related to “alarm clock” event subscriber and can be encountered quite frequently.

Here is the error stack trace that you may be seeing in your logs:

ERROR Exception in alarm clock event subscriber.
Exception: System.Reflection.TargetInvocationException
Message: Exception has been thrown by the target of an invocation.
Source: mscorlib
   at System.RuntimeMethodHandle._InvokeMethodFast(Object target, Object[] arguments, SignatureStruct& sig, MethodAttributes methodAttributes, RuntimeTypeHandle typeOwner)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture, Boolean skipVisibilityChecks)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Sitecore.Reflection.ReflectionUtil.InvokeMethod(MethodInfo method, Object[] parameters, Object obj)
   at Sitecore.Configuration.Factory.AssignProperties(Object obj, Object[] properties)
   at Sitecore.Configuration.Factory.AssignProperties(XmlNode configNode, String[] parameters, Object obj, Boolean assert, Boolean deferred, IFactoryHelper helper)
   at Sitecore.Configuration.Factory.CreateObject(XmlNode configNode, String[] parameters, Boolean assert, IFactoryHelper helper)
   at Sitecore.Pipelines.CorePipelineFactory.GetObjectFromType(String type, XmlNode processorNode)
   at Sitecore.Pipelines.CoreProcessor.GetMethod(Object[] parameters)
   at Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args)
   at Sitecore.Services.AlarmClock.Heartbeat_Beat(Object sender, EventArgs e)

Nested Exception

Exception: System.InvalidOperationException
Message: Category does not exist.
Source: System
   at System.Diagnostics.PerformanceCounterLib.GetCategorySample(String machine, String category)
   at System.Diagnostics.PerformanceCounterCategory.GetCounterInstances(String categoryName, String machineName)
   at Sitecore.Diagnostics.PerformanceCounters.PerformanceCounter.GetInstanceName()
   at Sitecore.Pipelines.HealthMonitor.HealthMonitor.AddCounter(XmlNode configNode)

================================

After a quick research in our knowledge base, I’ve discovered an answer. Apparently, this has been addressed in Sitecore CMS 6.2.0 rev.100104 (Also known as 6.2.0 Update-1).

For 6.0/6.1 instances, you can comment out the following part of your web.config file as a temporary workaround:

<processor type="Sitecore.Pipelines.HealthMonitor.HealthMonitor, Sitecore.Kernel" method="LogCounterStatus">
   <counters hint="raw:AddCounter">
    <!--  <counter category="Process" name="Private Bytes" />
      <counter category="Process" name="Virtual Bytes" />
      <counter category="Process" name="Page File Bytes" />
      <counter category=".net CLR Memory" name="# Bytes in all Heaps" />
      <counter category=".net CLR Memory" name="% Time in GC" />
      <counter category=".net CLR Memory" name="Large Object Heap size" />
      <counter category=".net CLR Loading" name="Bytes in Loader Heap" />
      <counter category=".net CLR Loading" name="Current Assemblies" />-->
   </counters>
</processor>

If you are still on 6.0 or 6.1, start thinking about upgrading. It is especially easy to do it from 6.1! Check the official steps here.

Publish to pre-production web database

Update 3/11/2010: Correction » the last parameter of the ExtendedPublishOptions constructor should be “false”.  Please see below for details.

Update 9/2/2010: Addition » published an alternative solution to this problem here.

In my experience with enterprise level implementations, there is often a need for a separate “stage environment” for final content preview or a pre-production phase of workflow.
It is also often the case that the environmental limitations or restrictions make the content delivery part of the authoring environment suitable for such purpose. For example, your e-commerce infrastructure could not be available from the authoring instance. One of the possible ways to approach such a requirement is to configure a stand-alone staging instance of Sitecore which will be used for content delivery instance and is commonly configured exactly like one of the servers in the web farm. There is one exception though – the database that is used for content delivery is generally different from the one that is used in production content delivery.
With this approach, now the question is how to plug in workflow in a way that content could go via this “stage” environment and then only upon final approval will be able to get to “production”.
Now there is a problem that you cannot get get items published until they are in final workflow state. You could have two workflow states “Staged” and “Published”, both marked as final and with auto publish actions connected.
I don’t like this approach since it goes against the workflow nature – only one state should be marked as final.
Alternatively, you can plug in a “copy item” workflow action to the “Staged” workflow state where you can programmatically copy an item from master database to stage web.
While this approach does seem legitimate, it works around the need of publishing thus any additional processing you may be having in publish(item) pipeline would not work.
I looked into the option of having publishing process ignore workflow when publishing to stage. While this seemed like a dangerous path to go, I have soon discovered that the PublishHelper’s GetVersionToPublish method already accepts this notion of passing a “requireapproval” flag to the underlying publishing mechanism. Since it is always passes “true”, I started looking into ways to make this flag dynamic and figured that setting it on the level of PublishOptions could be a good idea. For example, from the code of my workflow action I would be able to define whether I want to have workflow respected or not.
Here is what you will need to do to make it work.
First, override the default PipelinePublishProvider and plug it into the web.config:

<publishManager defaultProvider="default" enabled="true">
    <providers>
        <clear />
         <!--<add name="default" type="Sitecore.Publishing.PipelinePublishProvider, Sitecore.Kernel" />-->
         <add name="default" type="SCUSAINC.Publishing.ExtendedPublishProvider, SCUSAINC.Publishing" />
     </providers>
</publishManager>

Secondly, you will need to override the CreatePublishHelper class:

public class ExtendedPublishProvider : PipelinePublishProvider
    {
        public override PublishHelper CreatePublishHelper(PublishOptions options)
        {
            Assert.ArgumentNotNull(options, "options");
            if (options is ExtendedPublishOptions)
                return new ExtendedPublishHelper(options as ExtendedPublishOptions);

            return new PublishHelper(options);
        }
    }

After that, introduce two more classes – ExtendedPublishHelper and ExtendedPublishOptions:

public class ExtendedPublishHelper : PublishHelper
    {
        private readonly ExtendedPublishOptions _options;

        public ExtendedPublishHelper(ExtendedPublishOptions options)
            : base(options)
        {
            _options = options;
        }

        public override Item GetVersionToPublish(Item sourceItem)
        {
            Assert.ArgumentNotNull(sourceItem, "sourceItem");
            if (Options is ExtendedPublishOptions)
            {
                return sourceItem.Publishing.GetValidVersion(Options.PublishDate, _options.RequireApproval);
            }

            return sourceItem.Publishing.GetValidVersion(Options.PublishDate, true);
        }
    }

public class ExtendedPublishOptions : PublishOptions
    {
        public ExtendedPublishOptions(Database sourceDatabase, Database targetDatabase, PublishMode mode, Language language, DateTime publishDate, bool requireApproval)
            : base(sourceDatabase, targetDatabase, mode, language, publishDate)
        {
            RequireApproval = requireApproval;
        }

        public bool RequireApproval { get; set; }
    }

Now you are ready to launch publishing with workflow settings completely ignored. For example, this is how you can do it from the workflow action:

var database = Factory.GetDatabase(databaseName);

var options = new ExtendedPublishOptions(dataItem.Database, database, PublishMode.SingleItem, dataItem.Language, DateTime.Now, false)
{
         RootItem = dataItem,
         Deep = true;
};

new Publisher(options).PublishAsync();

3/11/2010: Correction » the last parameter of the ExtendedPublishOptions constructor should be “false”.
Instead of hardcoding it, I’d suggest having a parameter on the level of a workflow action which you can read and pass on.

private static bool RequireApproval(string parameters)
{
         return WebUtil.ParseUrlParameters(parameters)["RequireApproval"] == "true";
 }

public void Process(WorkflowPipelineArgs args)
{
                ...
               var requireApproval = RequireApproval(innerItem["parameters"]);
               var options = new ExtendedPublishOptions(dataItem.Database, database, PublishMode.SingleItem, dataItem.Language, DateTime.Now, requireApproval)
              
                ...
           
 }

That’s it!
Tested on Sitecore 6.1. Expected to work with 6.2.

Sitecore Media Library Policeman. Free shared source module.

Greetings! It is my pleasure to announce the very first module for Sitecore CMS I developed in collaboration with Ivan Sharamok finally made it to Shared Source Library.

The idea behind the module belongs to a customer who was willing to take full control over the Sitecore Media Library in their multi site solution. Managing more than 15 sites on one instance, it is important to restrict permissions on different areas of the content tree, as well as /media library branch.

I’ve seen too many situations in my practice where this important aspect has been neglected and hundreds of media files ended up in the root of media library, causing all sorts of performance and compatibility issues.

This module will hopefully provide a good starting point for what you can do in terms of policing your Media Library.

Instead of writing a bunch of boring docs, I decided to put a boring video on YouTube.

Enjoy!

Sitecore Logging Part 4. Integration with Log4Net Dashboard.

In my older posts I was showing how to teach Sitecore CMS to log events into a SQL Server database, today it is time to reveal the last part of this blog series.
Having everything in SQL is great, and it already enables you run queries, but wouldn’t it better to have some sort of a visual dashboard showing you all new events, highlighted by severity and type, and also let you filter those?

Read more »