Meet Your New Best Friends in Sitecore 7: SearchLog and CrawlingLog

It’s super important to understand how both search and indexing work under the hood, and for that reason, the CrawlingLog and SearchLog facilities should be very helpful in mastering Sitecore 7.

Read more »

Have ‘/sitecore/content’ in your URLs? Time to fix it.

Alright, I’ve been wanting to blog about this for years now. Some recent posts on twitter inspired me. Finally. If you following ‘sitecore’ on twitter, you may be seeing a few tweets that apparently have nothing to do with Sitecore, it’s products and services, so when you look at the shortened URL in the tweet you see something like this:

http://www.site.com/en/sitecore/content/site1/news/blabla.aspx

Read more »

Windows Authentication for Sitecore. Repost

Back in 08 I posted this walkthrough on how to configure Sitecore to use Windows Authentication for SQL connection and effectively remove the username and password from the connectionStrings.config file.
Here is a quick repost of the walkthrough with a few tweaks for SQL 2008 / Windows 2008 (R2) and Sitecore 6.4.x/6.5 (some steps are not required any more):

Read more »

Teach Sitecore to ignore a directory

This is a really quick post dedicated to the following scenario. You have either a standalone asp.net web app or legacy website. You want to host it underneath the web root with Sitecore instance configured and have Sitecore completely ignore the whole path (“/webapp”).

The issue you may be having is that during the request, Sitecore would try to “take over” and resolve a dynamic item “webapp” from the content tree which obviously does not exist. So you would get a “document not found” error*:

*Depending on the version of IIS, you may be experiencing different behavior.

There is a quick way to resolve this. Simply add the path to your standalone website to the “IgnoreUrlPrefixes” list:

   1: <!--  IGNORE URLS

   2:       Set IgnoreUrlPrefixes to a '|' separated list of url prefixes that should not be

   3:       regarded and processed as friendly urls (ie. forms etc.)

   4: -->

   5: <setting name="IgnoreUrlPrefixes" 
               value="/sitecore/default.aspx|/trace.axd|.....|/webapp" />

Now if your are not feeling nerdy today, stop reading and go fix it!

For the ones who continue reading, I can tell that the “ignore url magic” happens within the <httpRequestBegin /> pipeline which is invoked when the request goes through Sitecore.

So before the ItemResolver is hit, the “IgnoreList” processor would read the value of the “IgnoreUrlPrefixes” setting and figure out whether the pipeline needs to be aborted or not:

   1: <httpRequestBegin>

   2:    ...

   3:    <processor type="Sitecore.Pipelines.HttpRequest.IgnoreList, Sitecore.Kernel" />

   4:    <processor type="Sitecore.Pipelines.HttpRequest.SiteResolver, Sitecore.Kernel" />

   5:    ...

   6:    <processor type="Sitecore.Pipelines.HttpRequest.ItemResolver, Sitecore.Kernel" />

   7: </httpRequestBegin>

That’s all folks!

How to verify HTML cache is working

Greetings,

Here is a quick guide about how to verify that your presentation components (XSLTs, WebControls or Sublayouts) are getting cached.

For more details on how the HTML caching works, see section “3.4.2 Managed Web Site Caches” within Cache Configuration Reference and Chapter 4 within “Presentation Component Reference”.

1. Enable caching settings for your presentation control

Depending on your needs and implementation specifics, you can apply caching on different levels:

- via Layout Details dialog:

- on the definition item of the presentation control:

- within the markup on the control itself if it is bound statically:

   1: <form id="mainform" method="post" runat="server">

   2:     <div id="MainPanel">

   3:       <sc:XslFile ID="sampleXsl"

   4:                   Path="/xsl/sample rendering.xslt"

   5:                   Cacheable="true"

   6:                   VaryByData="true"

   7:                   runat="server" />

   8:       <sc:placeholder key="content" runat="server" /> 

   9:     </div>

  10: </form>

More details about it can be found within Chapter 4 of the “Presentation Component Reference” document. Here is one important snippet from this doc:

When you dynamically bind a rendering to a placeholder using layout details, cache settings explicitly defined in layout details override cache settings defined in the rendering definition item.
Cache settings defined in the definition item apply only when no caching settings exist in the Caching section in the Control Properties dialog.

This post does not explain the difference between cache variation settings like VaryByData or VaryByDevice. Consult the official documentation mentioned above in order to find appropriate strategy for your site.

2. Verify that the change to the cache setting got published to the “web” database.

For that, you can use the database selector at the bottom right corner and then open Content Editor to browse the content of the published database.

In a distributed server scenario, such things as cache clearing may be factoring in. So make sure that the data cache is properly cleared after publishing.

3. Use Sitecore Debugger to explore Trace

After you confirm that the cache setting were published, verify that Sitecore is caching it using Sitecore Debugger. The article is for Sitecore 5.3 but most concepts are still valid for your version.

3.1 To start the debugger, click on the “Debug” item in the menu:

3.2 When the debugger is launched, make sure to disable “Rendering Information” feature, otherwise caching won’t work within the debugger:

3.3. Locate the page where the presentation component is placed.

3.4. Refresh the page a couple of times.

3.5. Scroll down to the “Trace” section and locate your rendering:

If the control is cached, you should see the highlighted (using cache) string.

4. Rendering Statistics page

Another way to confirm is to launch the “Rendering Statistics” page: /sitecore/admin/stats.aspx where you can see the number of times your rendering is fetched from cache vs. total render count:

As you may have guessed, the value within the “From cache” column should not be zero.

5. Verify that HTML caching is turned on.

If the steps above indicate that presentation component is not getting cached, consult you’re the cache page: /sitecore/admin/cache.aspx

5.1 Find the current site that you’re running. By default, it should be “website”.
5.2 Find the column for html cache of your website.
5.3 Verify that the MaxSize column is not set to zero.

6. Verify the configuration

If the cache page shows zero, then it’s time to review your configuration.

Within web.config, make sure that:

6.1 Default HTML cache size is not set to zero:

   1: <setting name="Caching.DefaultHtmlCacheSize" value="5MB"/>

6.2 Locate your website definition within the <sites /> section and make sure that cacheHtml is set to “true” and the value of “htmlCacheSize” is not set to zero:

   1: <site name="website" ... cacheHtml="true" htmlCacheSize="10MB" />

6.3 Locate the <cacheSizes /> section next to the <sites /> section. Make sure that the html cache is not set to zero here too:

   1: <cacheSizes>

   2:    <sites>

   3:       <website>

   4:          <html>10MB</html>

   5:          ...

   6:       </website>

   7:    </sites>

   8: </cacheSizes>

This should be more or less sufficient in order to troubleshoot html cache issues.

HTH.

Disable WebDAV in Sitecore

If you are not sure what WebDAV is, please refer to this document on Sitecore Developer Network.
Briefly, quoting this doc:

WebDAV allows users of Web clients such as browsers to manage files on Web servers using Web protocols such as HTTP or HTTPS. With WebDAV, users can drag-and-drop files between the Windows desktop and the browser.

I would recommend disabling this feature on the Content Delivery side as WebDAV is not expected to be used there.

Read more »

Securing Sitecore Admin

Greetings,

One of the frequent questions I am hearing besides development related stuff is configuration related. In Sitecore world, there is always plenty of options available for you in terms of configuring your production environment. Not to get carried away, but this is really a critical aspect, especial for large enterprises. When your product cannot be flexible enough to be decoupled in components, this may represent quite a challenge. Systems forcing large footprint are more difficult to maintain, backup, secure, etc.
With Sitecore, you can pretty much create a lightweight Content Delivery instance by cutting down the configuration and files to mere 50 Mb quite with a little bit of effort. This will create a more manageable and secure environment, but what if you don’t want to go through this exercise?

A quick and proven way to handle this it rely on native IIS securing features. With IIS7 you can do that even easier. What you can do is simply deny access to /sitecore folder based on IP restrictions.

1. Make sure you have “IP Security” feature installed for IIS:

2. Locate your site in IIS, select /sitecore folder:

3. On the Features view, select “IP Address and Domain Restrictions”:

4. Configure any allow/deny rules you want:

Isn’t it easy?

Optimizing Sitecore Link Database Rebuild

Greetings! One of the topics I covered during my Dreamcore presentation on Data Access Techniques with Sitecore CMS was Link Database. While it is a great feature that enables fast and efficient retrieval of item relations, there is one caveat, it requires some maintenance and configuration to be working properly in multi server environment. Many customers face this challenge, and while there is a number of ways you can tackle this, the approach I want to share today seems like the most appropriate to me.

The idea behind the scenes is to separate the Link Database used on the Content Delivery side from the one used on the Content Management side by introducing a second link database. This database will be updated during item publishing. This way if CMS decides to update its own Link DB, the one used on the front-end will not be affected.

So, here is what you need to do:

1. Add the FrontEndLinkDatabase definition below the default <LinkDatabase> in web.config file of the CMS server:

<!-- LINK DATABASE -->
<LinkDatabase type="Sitecore.Data.$(database).$(database)LinkDatabase,
                    Sitecore.Kernel">
   <param connectionStringName="core" />
</LinkDatabase>
<!-- FRONT END LINK DATABASE -->
<FrontEndLinkDatabase type="Sitecore.Data.$(database).$(database)LinkDatabase,
                            Sitecore.$(database)">
   <param desc="connection" ref="connections/web" />
</FrontEndLinkDatabase>

2. Compile the following code that contains two Sitecore event handlers (publish:itemprocessed and publish:itemprocessing)

using System;
using Sitecore.Configuration;
using Sitecore.Links;
using Sitecore.Publishing;
using Sitecore.Publishing.Pipelines.PublishItem;
using Sitecore.Diagnostics;
namespace Sitecore.SharedSource.LinkDatabaseRebuilder
{
   public class EventHandler
   {
      protected static LinkDatabase FrontEndLinkDatabase
      {
         get
         {
            return (Factory.CreateObject("FrontEndLinkDatabase", true) 
                     as LinkDatabase);
         }
      }
      public string Database { get; set; }
      public void OnItemProcessed(object sender, EventArgs args)
      {
         var context = ((ItemProcessedEventArgs)args).Context;
         Assert.IsNotNull(context, 
         "Cannot get PublishItem context");
         Assert.IsNotNull(FrontEndLinkDatabase,
         "Cannot resolve FrontEndLinkDatabase from config");
         if (context.PublishOptions.TargetDatabase.Name.Equals(Database))
         {
            var item = context.PublishHelper.GetTargetItem(context.ItemId);
            // if an item was not unpublished, 
            // the call below will reintroduce the reference
            // removed within OnItemProcessing method
            if (item != null)
            {
               FrontEndLinkDatabase.UpdateReferences(item);
            }
         }
      }
      public void OnItemProcessing(object sender, EventArgs args)
      {
         var context = ((ItemProcessingEventArgs)args).Context;
         Assert.IsNotNull(context,
         "Cannot get PublishItem context");
         Assert.IsNotNull(FrontEndLinkDatabase, 
         "Cannot resolve FrontEndLinkDatabase from config");
         if (context.PublishOptions.TargetDatabase.Name.Equals(Database))
         {
            if (context.Action == PublishAction.DeleteTargetItem)
            {
               var item = context.PublishHelper.GetTargetItem(context.ItemId);
               Assert.IsNotNull(item, "Source item cannot be found");
               FrontEndLinkDatabase.RemoveReferences(item);
            }
         }
      }
   }
}

3. Last thing to do is to define the two event handler mapping in web.config.

<events>
   <event name="publish:itemProcessing">
    <handler type="Sitecore.SharedSource.LinkDatabaseRebuilder.EventHandler, 
                   Sitecore.SharedSource.LinkDatabaseRebuilder" method="OnItemProcessing">
      <database>web</database>
     </handler>
   </event>
   <event name="publish:itemProcessed">
     <handler type="Sitecore.SharedSource.LinkDatabaseRebuilder.EventHandler,
                    Sitecore.SharedSource.LinkDatabaseRebuilder" method="OnItemProcessed">
       <database>web</database>
      </handler>
   </event>
 </events>

That’s all you need to do. For the sake of simplicity, here is the link to the whole VS project where you can find the auto-include file.

While this solution has been tested on 6.2, it is expected to work on all Sitecore 6 installations.

Let me know if it does not ;-)

Sitecore Logging Part 4. Integration with Log4Net Dashboard.

In my older posts I was showing how to teach Sitecore CMS to log events into a SQL Server database, today it is time to reveal the last part of this blog series.
Having everything in SQL is great, and it already enables you run queries, but wouldn’t it better to have some sort of a visual dashboard showing you all new events, highlighted by severity and type, and also let you filter those?

Read more »

Sitecore Logging Part 2. Dealing with the Exceptions

As as follow up to the initial post about SQL based logging in Sitecore, here is another quick tip about how to include exception stack trace into your log database.

Read more »

Sitecore Logging. Quick update.

Remember in the last post about SQL based logging I mentioned that there is an internal buffer that log4net has before it dumps all into the database.
While it seems to be quite useful when running in production, in development environment you would want to see immediate messages in the logs, especially when troubleshooting.
In order to do that, simply add the “bufferSize” section for your ADONetAppender and set the value to “1”:

<appender name="ADONetAppender_SqlServer" type="log4net.Appender.ADONetAppender" >
<bufferSize value="1" />

If the buffer is not explicitly set, it will be defaulted to 512 :-)

Happy logging!

Sitecore Logging. Write it to SQL!

Today I have been trying to make Sitecore CMS log to the database instead of file system. After hitting a few roadblocks, I’ve managed to find an answer via our helpful support portal.

Now it is not a secret that we are using an open source logging framework called log4net and that it is possible to configure it to use SQL database for storage instead of the file system.

This could come extremely handy due to a couple of reasons.

1. By default, file system logging means that Sitecore frequently creates a new file during system restart with a new timestamp, which may not be really what you expect, but this is by design.

2. Parsing of text files is an extremely creative and involving process. There are tools and solutions to facilitate in that but, I prefer writing SQL queries to get my data.

There are definitely more, but you catch my drift…

So how to get it working? As I mentioned, our support folks, or I should say, solution consultants, came up with 3 easy steps.

1. Create a table to store your logs. I’d recommend creating a separate database for that called “SitecoreSystem” as you don’t want to add a custom table to your Sitecore database, believe me!

CREATE TABLE [dbo].[Log] ( 
[ID] [int] IDENTITY (1, 1) NOT NULL ,
[Date] [datetime] NOT NULL ,
[Thread] [varchar] (255) NOT NULL ,
[Level] [varchar] (20) NOT NULL ,
[Logger] [varchar] (255) NOT NULL ,
[Message] [varchar] (4000) NOT NULL 
) ON [PRIMARY]

2. Define your own “SQL based appender” in <log4net> section of web.config.

<appender name="ADONetAppender_SqlServer" type="log4net.Appender.ADONetAppender, Sitecore.Logging" >
<param name="ConnectionType" value="System.Data.SqlClient.SqlConnection, System.Data, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" />
<param name="ConnectionString" value="user id=sa;password=sitecorerocks;Data Source=.\\sql2008;Database=SitecoreSystem" />
<param name="CommandText" value="INSERT INTO Log ([Date],[Thread],[Level],[Logger],[Message]) VALUES (@log_date, @thread, @log_level, @logger, @message)" />
<param name="Parameter">
<param name="ParameterName" value="@log_date" />
<param name="DbType" value="DateTime" />
<param name="Layout" type="log4net.Layout.PatternLayout">
<param name="ConversionPattern" value="%d{yyyy&apos;-&apos;MM&apos;-&apos;dd HH&apos;:&apos;mm&apos;:&apos;ss&apos;.&apos;fff}" />
</param>
</param>
<param name="Parameter">
<param name="ParameterName" value="@thread" />
<param name="DbType" value="String" />
<param name="Size" value="255" />
<param name="Layout" type="log4net.Layout.PatternLayout">
<param name="ConversionPattern" value="%t" />
</param>
</param>
<param name="Parameter">
<param name="ParameterName" value="@log_level" />
<param name="DbType" value="String" />
<param name="Size" value="50" />
<param name="Layout" type="log4net.Layout.PatternLayout">
<param name="ConversionPattern" value="%p" />
</param>
</param>
<param name="Parameter">
<param name="ParameterName" value="@logger" />
<param name="DbType" value="String" />
<param name="Size" value="255" />
<param name="Layout" type="log4net.Layout.PatternLayout">
<param name="ConversionPattern" value="%c" />
</param>
</param>
<param name="Parameter">
<param name="ParameterName" value="@message" />
<param name="DbType" value="String" />
<param name="Size" value="4000" />
<param name="Layout" type="log4net.Layout.PatternLayout">
<param name="ConversionPattern" value="%m" />
</param>
</param>
</appender>

3. Reference it now in <root> section, comment out the standard file based appender:

<root>
<priority value="INFO"/>
<appender-ref ref="ADONetAppender_SqlServer" />
<!--<appender-ref ref="LogFileAppender"/>-->
</root>

Troubleshooting

1. Make sure to double back slash the SQL name, e.g.  (.\\sqlexpress) in ConnectionString.

2. If you don’t see any entries in the Log database straight after the change, don’t panic. There is a threshold which should be exceeded before anything is written to the database. To test, launch tree serialization process – as this creates a lot of log entries, you will definitely see those unless something is wrong.

3. If something is wrong, add the following to your web.config. And if you don’t see anything written to the database, check the log file specified below.

<system.diagnostics>
<trace autoflush="true" indentsize="4">
<listeners>
<add name="myListener" type="System.Diagnostics.TextWriterTraceListener" initializeData="D:\logs\error.log" />
<remove name="Default" />
</listeners>
</trace>
</system.diagnostics>

 Open questions

- Maintainability
Now the most interesting part. You will probably need to create a scheduled agent that cleans up the Log table periodically for obvious reasons.

What next?

You can take it further and add custom parameters to your logs. Let’s leave that till next time.

Share if you think it is useful. Cheers!

Sitecore Friendly URLs and plus sign

One of the customers recently asked about using the plus sign in URLs, for example: http://localhost/news+events.aspx

There are at least two ways to accomplish this:
1. Allow “+” in item name via a configuration change
For this you will need to adjust the regular expression in the “ItemNameValidation” setting in web.config and make sure + sign is not  present in the “InvalidItemNameChars” setting.
2. Rely on the display name instead.

Read more »

Sitecore startup time - something that could improve it

One of the things any Sitecore developer had to deal with is startup time. There are tons of reasons why an application pool may restart (file change notification, config change, etc.) so this generally happens pretty often, especially on a development machine.
So I decided to spend some time on the problem and engaged our brilliant technical support into this little project.

Read more »

Sitecore 6 quick tip. Email as user name for authentication

It is pretty common to have email served as username for authentication in web systems for both visitors and internal users. Sitecore Content Management System is no exception.

With 5.3 it was pretty easy since users were just items – adjust the regex of the “ItemNameValidation” setting, make sure you don’t have @ and dots in the “InvalidItemNameChars” setting and you are pretty much set.

Now as you know, Sitecore 6 has different rules for security so you will need to do the following to make it work:
1. Make sure the membership provider treats email as a unique attribute for users so you don’t end up with more than one user attached to the same email:
<add name="sql" type="System.Web.Security.SqlMembershipProvider" connectionStringName="core" applicationName="sitecore" minRequiredPasswordLength="1" minRequiredNonalphanumericCharacters="0" requiresQuestionAndAnswer="false" requiresUniqueEmail="true" maxInvalidPasswordAttempts="256" />

2. Put email into both username and email properties (fields) of a user during the registration (can be handled via code).

3. Introduce the following entry into the web.config’s <settings> section. The “value” attribute parameter contains the regular expression used in the Create User dialog within User Manager. This regex should allow emails, otherwise Sitecore will fallback on a default regex that does not allow it.
<setting name="AccountNameValidation" value=".+" />

4. If you want to handle the case when email needs to be changed, either provide an extranet form for the profile section on your website or you can even take it one step forward – modify the EditUser dialog within User Manager to have this ability.

Happy coding!

No such host is known

Real quick tip. Such error may appear when you DNS is down or unreachable. To fix this, add your hostname that is in the bindings for IIS website to the hosts file under C:\Windows\System32\drivers\etc, point it to the local IP and that that should help till the issue with DNS is resolved:
127.0.0.1      yoursite.com