Tell me more ×
SharePoint Stack Exchange is a question and answer site for SharePoint enthusiasts. It's 100% free, no registration required.
up vote 1 down vote favorite

I've been working on a step for F4SP 2010, where I had to get the HTML content from the Site crawled of Sharepoint, and get a div into it.

My question is not how to do that.

I have the main problem that I do not know what crawled property uses the crawler to save theHTML content.

Could anyone help me identifying this property?

share|improve this question

4 Answers

up vote 2 down vote

Html content is usually in the crawled property:

Property Set: 11280615-f653-448f-8ed8-2915008789f2 Variant type: 31 Name: html

(http://msdn.microsoft.com/en-us/library/ff795815.aspx)

You can also add a "Spy" stage to examine what crawled properties are available in the pipeline - and see if you can find the one with the correct content. Or enable FFDDumper.

Also remember that crawled property names are case sensitive inside the pipeline.

share|improve this answer
I tried this yesterday but I got no answer, it's true that html has a property in the Spy, but I added to the pipelineextensibility.xml, and I got nothing, tomorrow, I'll try again, and report what I see. Thanks! – Amedio Jan 26 '12 at 20:36
Mikael, I tried you Logger class to log the input result with a simple log console application like static void Main(string[] args) { Logger.WriteLogFile(args[0], "-input"); Logger.WriteLogFile(args[1], "-output"); } But there is nothing in the log files when I add <CrawledProperty propertySet="11280615-f653-448f-8ed8-2915008789f2" varType="31" propertyName="html"/> – Amedio Jan 27 '12 at 9:42
1  
Then the html is in some other crawled property. At least in the "data" one which is base64 encoded.. but perhaps another as well. If you look through the spy log, see if you can find something which resembles the html your are looking for. – Mikael - SharePoint MVP Jan 27 '12 at 12:27
Maybe my solution is to get the byte array of the data property, and saving it to disk, so I can see if it has the html? – Amedio Jan 27 '12 at 13:08
That was the clue, maybe I was using the wrong encoding to parse b64 resulting bytes to string. Thanks :) – Amedio Jan 27 '12 at 13:34
up vote 1 down vote

It depends on which column was used to store the HTML content in the list for which you want to get the data. With default Publishing Sites, the column name is PublishingPageContent, so I assume the crawled property is ows_PublishingPageContent.

share|improve this answer
I tried with that property, but had no data returned. – Amedio Jan 25 '12 at 9:24
Is the property configured to include content in the index, and do you have pages that use that field with content in them? – James Love Jan 25 '12 at 9:26
I have the initial configuration, I'm so new with FAST, and they told me to create a stage in pipelineextensibility to get the breadcrumb in HTML format from the page being crawled, so, consider I'm silly and maybe I'm not doing something I need to do. By the moment, I tried de special crawled property 'data' in FAST, which saves binary from the page on a base64 string, but that's the problem, is the binary, and de 'body' property only saves the content parsed, and not complete. – Amedio Jan 25 '12 at 9:28
1  
data is binary, and html has "html" if present. The binary is base64 encoded so can be decoded easily. – Mikael - SharePoint MVP Jan 27 '12 at 7:40
up vote 1 down vote

I believe that the data that you get from crawled property is IEnumerable. So, I wonder that how you convert it into byte[] and then string?

I have tried this with no success because the "invalid character in a Base-64" is retuned.

XDocument inputDoc = XDocument.Load(args[0]);
            var data = from cp in inputDoc.Descendants("CrawledProperty")
                       where new Guid(cp.Attribute("propertySet").Value).Equals(FMTID_SummaryInformation_DATA) &&
                       cp.Attribute("propertyName").Value == "data" &&
                       cp.Attribute("varType").Value == "31"
                       select cp.Value;

byte[] html = System.Convert.FromBase64String(data);

string dataHtml = Encoding.UTF8.GetString(html);

Edit: Finally, I figured it out using this...

var data = from cp in inputDoc.Descendants("CrawledProperty")
           where new Guid(cp.Attribute("propertySet").Value).Equals(FMTID_SummaryInformation_DATA) &&
               cp.Attribute("propertyName").Value == "data" &&
               cp.Attribute("varType").Value == "31"
           select cp.Value;

string datahtml_base64 = "";

foreach (string datas in data)
{
    datahtml_base64 += datas;
}

byte[] html = System.Convert.FromBase64String(datahtml_base64);
string dataHtml = Encoding.UTF8.GetString(html);
share|improve this answer
I figured it out by using this – Amy Feb 28 '12 at 10:08
up vote 0 down vote accepted

I solved it taking the 'data' crawled property and passing from b64 to byte[] and then using Encoding.UTF8.GetString(byte[]) I could have the html.

Thanks for the two answerers

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.