0

I'm creating web site to compare prices and products. I decided to use SharePoint Search 2010 to crawl the content of web site to find certain products and get data like title, price, description and image for that product.

Sample html code:

<body>
   <h2>ACER TM8172 11.6/I3-380UM/4/320/BT/3G/7P</h2>
   <br /><br />
   <table>
      <tr>
         <td valign="top">
        <img src="images/acer_TM8172.jpg" alt="ACER TM8172" /><br />
     </td>
     <td valign="top"><br />
        <b>Price:</b> 669.- &euro; <br />
        <b>Availability:</b> Stock (29)<br />
        <b>Description:</b><span>The range of notebooks ...</span>              
     </td>
      </tr>
   </table>
</body>

SharePoint Search have many crawl properties (89) for web content. I can retrieve values from tags like title, path, h1, h2 and so on but not let's say form table or paragraph tag.

I made VS console application project to test values I could retrieve using KeywordQuery. Something like this:

KeywordQuery query = new KeywordQuery(SSAProxy); 
query.SelectProperties.Add("Title");
query.QueryText = "acer";
query.ResultsProvider = SearchProvider.SharepointSearch;
query.ResultTypes |= ResultType.RelevantResults;
ResultTableCollection searchResults = query.Execute();
..

Everything works but I retrieve only those values which properties I add (e.g Title). Is there a way to retrieve the rest of the body text (for which the crawl property is not exist) so I can still filter out those values I need?

I know that one solution is to implement custom IFilter for that case, but I really don't wanna do that. Or maybe there is a third option?

Improve this question

1 Answer 1

Reset to default
1

By default KeywordQuery returns only these properties 1) WorkId 2) Rank 3) 4) Title 5) Author 6) Size 7) Path 8) Description 9) Write 10) SiteName 11) CollapsingStatus 12) HitHighlightedSummary 13) HitHighlightedProperties 14) ContentClass 15) IsDocument 16) PictureThumbnailURL

If you want your own you have to use the SelectProperties (http://msdn.microsoft.com/en-us/library/microsoft.office.server.search.query.keywordquery.selectproperties.aspx) however, one important trick: As soon as you use SelectProperties all the others are automatically removed, so all these mentioned above must be added MANUALLY otherwise they will not exist.

Corey Roth explains it very clearly here http://www.dotnetmafia.com/blogs/dotnettipoftheday/archive/2008/02/19/how-to-use-the-moss-enterprise-search-keywordquery-class.aspx

FAST includes an XML Mapper which relies on XPath to target Content Extraction, see here http://msdn.microsoft.com/library/ff795813.aspx

Hope it helps, C:\Marius

Improve this answer
2
  • Thank you Marius! SelectProperties option doesn't help me, because there isn't proper crawl properties for some HTML tags. If you look the HTML code example above, then I can use 'Web:4(Text)' property to retrieve HTML Heading 2 but I cannot retrieve the text within span tags. How can I retrieve text within span tags?
    – group
    Commented Apr 25, 2012 at 8:29
  • Yes, you are right. For advanced content/property extraction I would rather consider FAST.
    – Marius Constantinescu
    Commented Apr 25, 2012 at 12:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.