Using Content Enrichment Web Service Callout in SharePoint 2013 Preview

SharePoint 2013 Preview release intoduced a new functionality called content enrichment web service callout. It provides the ability to inspect and manipulate managed property values for each item before it’s added to the search index. Prior to SharePoint 2013, the only way to accomplish something similar was in FAST Search for SharePoint by extending the item processing pipeline. Clients using SharePoint server search were out of luck as the functionality was not available to them.

The process of building and configuring a web service callout is relatively straight forward. These are the high-level steps to follow:

  1. Build a web service by implementing the IContentProcessingEnrichmentService interface. Add logic to manipulate managed property values.
  2. Run PowerShell commands to configure the callout web service endpoint address, input and output managed properties, trigger condition and a few other things.
  3. Execute a full crawl.

In this blog post I’ll show an example of developing a web service that populates a new managed property value which is then used as a refiner on the search results page. Let’s say we have a number of project sites in SharePoint where each site contains information about a specific bike model.

Each bike model belongs to a product category such as Mountain Bikes, Road Bikes and Touring Bikes. We’d like to be able to refine search results by product category but unfortunately that metadata is not available in SharePoint at this point. What we are going to do next is create a new managed property called ProductCategory and build a web service to populate the managed property values based on our custom business logic. The ProductCategory managed property can then be used as a refiner on the search results page.

To create the managed property, navigate to Central Administration > Search Service Application > Search Schema > New Managed Property.

  • Property name: ProductCategory
  • Type: Text
  • Searchable: checked
  • Queryable: checked
  • Retrievable: checked
  • Refinable: Yes – active
  • Token Normalization: checked

In Visual Studio 2012, create the web service: New Project > Visual C# > WCF > WCF Service Application.

Delete the Service1 created by default or rename it to EnrichmentService. Delete the IService1 or IEnrichmentService interface.

Add an assembly reference to C:\Program Files\Microsoft Office Servers\15.0\Search\Applications\External\microsoft.office.server.search.contentprocessingenrichment.dll.

Open EnrichmentService.svc.cs, add the following using statements:

using Microsoft.Office.Server.Search.ContentProcessingEnrichment;
using Microsoft.Office.Server.Search.ContentProcessingEnrichment.PropertyTypes;

Replace the class implementation:

public class EnrichmentService : IContentProcessingEnrichmentService
{
    private Dictionary<string, string> productModels = new Dictionary<string, string>()
    {
        {"mountain-100", "Mountain Bikes"},
        {"mountain-500", "Mountain Bikes"},
        {"road-150", "Road Bikes"},
        {"road-450", "Road Bikes"},
        {"touring-1000", "Touring Bikes"},
        {"touring-2000", "Touring Bikes"}
    };

    public ProcessedItem ProcessItem(Item item)
    {
        ProcessedItem processedItem = new ProcessedItem();
        processedItem.ItemProperties = new List<AbstractProperty>();

        AbstractProperty pathProperty = item.ItemProperties.Where(p => p.Name == "Path").FirstOrDefault();
        if (pathProperty != null)
        {
            Property<string> pathProp = pathProperty as Property<string>;
            if (pathProp != null)
            {
                foreach (var productModel in productModels)
                {
                    if (pathProp.Value.Contains(productModel.Key))
                    {
                        Property<string> modelProp = new Property<string>()
                        {
                            Name = "ProductCategory",
                            Value = productModel.Value
                        };
                        processedItem.ItemProperties.Add(modelProp);
                    }
                }
            }
        }

        return processedItem;
    }
}

Now the web service is ready and the next step is to configure SharePoint to call the web service during the crawl. That is done using PowerShell. To minimize the performance impact of the web service callout, we only want it to be called under a certain condition – this condition is defined in the Trigger property. More information about the syntax can be found in the Trigger expression syntax article on MSDN. The expected input and output managed properties are configured via the InputProperties and OutputProperties. When debugging the web service, the DebugMode property value can be set to $true in which case SharePoint will ignore the InputProperties value and will send all available managed properties for each item to the service. Any managed property values returned by the web service in debug mode are ignored by SharePoint.

$ssa = Get-SPEnterpriseSearchServiceApplication
$config = New-SPEnterpriseSearchContentEnrichmentConfiguration
$config.DebugMode = $false
$config.Endpoint = "http://localhost:64401/EnrichmentService.svc"
$config.FailureMode = "WARNING"
$config.InputProperties = "Path"
$config.OutputProperties = "ProductCategory"
$config.SendRawData = $false
$config.Trigger = 'StartsWith(Path,"http://intranet.contoso.com/adventureworks/models/")'
Set-SPEnterpriseSearchContentEnrichmentConfiguration –SearchApplication $ssa –ContentEnrichmentConfiguration $config

Finally, launch the web EnrichmentService created earlier and start a new full crawl. Once the crawl is complete, the ProductCategory managed property should be populated and searchable:

The final step is to add a Product Category search refiner. Edit the search results page, edit the Refinement web part, click the Choose Refiners… button within the Properties for Search Refinement section, select the ProductCategory managed property in the Available refiners list and press the Add > button. Move the ProductCategory to the top of the Selected refiners list, then scroll down and set the Display name to Product Category and save your changes.

Run a search for “bike” and you should now be able to refine the search results by the product categories:

References:

  • Thony Yin

    Very helpful information, thanks for sharing.

  • Miran

    Nice article!
    How would you handle multiple search applications? How to get search application name in the content enrichment service?

    Thanks!

    • Vassili Altynikov

      Hi Miran,

      I don’t believe there’s a way to retrieve the search application name but you can easily get the content source name (ContentSource managed property) and/or the SharePoint site collection URL (SPSiteUrl managed property).

  • Ludo

    Hello,

    I tried to do the same but my web service is never called ? Even if the configuration is correct.
    Do you have any ideas why ?

    • Vassili Altynikov

      Hi Ludo,

      I’d suggest checking the following:
      * Use Get-SPEnterpriseSearchContentEnrichmentConfiguration to confirm that your configuration has been successfully applied
      * Check the Trigger expression and verify that it’s valid
      * Check the crawl log to see if there are relevant messages there

  • Mark Slavik

    This is great. One issue that I cant seem to over come is how do you tell what has just happened to the “item”. For example can you tell if the items is in the pipeline because it has been deleted or would you create a service that just dealt with deletes as an action?

    • Vassili Altynikov

      Hi Mark, this is a great question. I highly doubt that the web service will get called when the item is being deleted but there doesn’t appear to be a way to distinguish between inserts and updates.

  • Pingback: SharePoint Development Material(Links) | @Coding BlackHole()