SharePoint 2007 search - custom keyword search

Question

I'm writing some code against the search API in SharePoint 2007, and am seeing some interesting behaviour. I'm using the KeywordQuery 'model' rather than FullTextSqlQuery, since this matches the advanced search form our client wants.

I'm using the following code to execute the search - this builds a keyword search query in the same format as the OOTB web parts:

        ResultTableCollection resultTables = null;

        using (KeywordQuery keywordQuery = new KeywordQuery(SPContext.Current.Site))
        {
            string fullQuery = string.Empty;

            // TODO: refactor..
            string allWords = HttpUtility.UrlDecode(queryString[AllWordsQsParam]);
            string exactWords = HttpUtility.UrlDecode(queryString[ExactPhraseParam]);
            string excludeWords = HttpUtility.UrlDecode(queryString[NoneWordsQsParam]);
            string anyWords = HttpUtility.UrlDecode(queryString[AnyWordsQsParam]);

            if (!string.IsNullOrEmpty(allWords))
            {
                fullQuery += getFormattedTermString(' ', '+', allWords);
            }
            if (!string.IsNullOrEmpty(exactWords))
            {
                fullQuery += exactWords;
            }
            if (!string.IsNullOrEmpty(anyWords))
            {
                fullQuery += anyWords;
            }
            if (!string.IsNullOrEmpty(excludeWords))
            {
                fullQuery += getFormattedTermString(' ', '-', excludeWords);
            }

            keywordQuery.QueryText = fullQuery;
            keywordQuery.ResultTypes = ResultType.RelevantResults;

            resultTables = keywordQuery.Execute();
        }

After this, I load into a DataTable and eventually get XML to transform. Currently I'm using the default XSL from the CoreResultsWebPart, but I'm finding the produced XML is not what this XSL is expecting:

• element names are not all lowercase (not too difficult to amend the XSL to match up, but curious)
• the hit highlighting properties are in a different format, possibly due to an encoding issue somewhere. As an example, HitHighlightingSummary does not contain child XML elements as it does in an OOTB search - instead, there is one text node as a child which effectively contains an 'InnerXml' of the child elements. Effectively all of the angled brackets have been encoded e.g. '<' to 'ampersand lt semi-colon' (had to write it that way due to SO encoding, but you hopefully get my drift).

This latter means the xsl:apply-templates for the hit highlighting stuff is not able to process the XML, since it's really a string rather than a tree.

I've verified that it isn't the later steps of loading into a DataTable/obtaining XML/transforming which is the issue - it's coming back from the search API like this.

Any ideas why?? I'm not immediately seeing something I'm doing wrong with the KeywordQuery class, but could it be something there?

Answer 1

You will find that the SearchResultHiddenObject encodes the contents of the HitHighlighted fields in the XML document before sending it to be transformed - it does a few other things too, e.g. changing the name of the PATH element to URL!!! All these "quirks" need to be accounted for when performing custom search and rendering when wanting to use the OOTB (or modified) XSLT's.

Take a look at the AppendHitsToXmlDocument method in the SearchResultHiddenObject for more information.

Alternatively, i will dig out the code I did that replicated this entire procedure to have custom search with OOTB xslt's.

Updated with code:

Found the code: (you probably would have known this already but hey!)

oResDoc is my XmlDocument object containing the results returned from the API.

foreach (XmlElement oHhx in oResDoc.GetElementsByTagName("HITHIGHLIGHTEDPROPERTIES"))
{
    oHhx.InnerXml = System.Web.HttpUtility.HtmlDecode(oHhx.InnerXml);
}

In terms of the first point, the uppercasing of the elements, I hit this too. I originally started lowercasing all the element names, but abandoned this approach and just modified the XSLT.

Answer 2

Did you try reflector against the oob web parts? Maybe there's an interstitle xml transformation going on?

Answer 3

Regarding your second point, you are saying that the HitHighlightingSummary column in the ResultTable you get from SharePoint contains HtmlEncoded values? Perhaps this is by design, and it's your responsibility to HtmlDecode these values when you turn it into XML?