Take the 2-minute tour ×
SharePoint Stack Exchange is a question and answer site for SharePoint enthusiasts. It's 100% free, no registration required.

I have a problem with navigation elements being crawled on our SharePoint site. These elements come from a custom control that's used for building the site navigation, so the output is just html that might look like this simple example:

<div class="nav_root">
  <ul class="nav_root_wrap">
    <li class="SomeLink twoline first" id="SomeLink">
      <a href="someLink.aspx">some link</a>
    </li>
    <li class="AnotherOption twoline second" id="AnotherOption">
      <div class="flyoutMenu">
        <a href="#">more links</a>
         ...
      </div>
    </li>
  </ul>
</div>

It looks like adding the "noindex" class to the elements in the html should do the trick. However, because I don't have very much control over the css classes that get applied to this content on the server-side, I'm wondering if I can add some jquery like below to apply the css classes on the client-side.

$('.nav_root *').addClass("noindex");

Does anyone know if this should work in SharePoint 2010's Enterprise Search (i.e. not FAST Search Server)? I can't tell if the search crawler is going to allow javascript to run before looking at the content of the page.

Update: I'm not having any luck getting this to work. This seems to indicate that the content of the page is crawled without javascript being taken into account. I really hope that I'm just doing something wrong, so please let me know if this is not accurate.

Update2: It sounds like some folks have had luck creating a user control that prevents content from rendering for a particular user agent. Certainly an interesting option, though it's not what I'd hoped for: http://underthehood.ironworks.com/2010/05/sharepoint-search-partial-page-exclusion.html

share|improve this question
    
Can you add a wrapper around that control on the masterpage and apply the noindex class to the wrapper and not the actuanl navigation elements? –  Luis Oct 3 '13 at 22:13
    
Unfortunately, SharePoint requires "noindex" to be on each nested div. It's a good thought, and a bit frustrating that it doesn't work that way. However, I've verified this myself, and this link also refers to it: blogs.msdn.com/b/markarend/archive/2010/06/07/… –  atcrawford Oct 4 '13 at 2:18
add comment

3 Answers

up vote 4 down vote accepted

As far as I'm aware, the crawler won't 'run' the javascript on the generated page, therefore this technique probably won't actually work. Can't you add a noindex class to the container div of the nav control on the masterpage?

share|improve this answer
    
I wish adding the noindex class to a container div would do the trick, but I guess the SharePoint team felt like that would be too easy or something :) –  atcrawford Oct 4 '13 at 2:25
    
Looks like that might work with FAST Search Server though, per this response to another question: sharepoint.stackexchange.com/a/46800/2995 –  atcrawford Oct 4 '13 at 2:32
    
Marking this as the answer since it most closely addresses my question in the title. For the underlying problem, I actually wound up using the solution that I pointed out in "Update 2"; here's another link to the blog post. –  atcrawford Oct 12 '13 at 5:47
add comment

You can use the following code if you do not want the webpart to be crawled(if the navigation control is a separate webpart).

protected override void CreateChildControls()
{
if (!string.IsNullOrEmpty(Request.UserAgent))
{
  if (Request.UserAgent.ToUpperInvariant().IndexOf("MS SEARCH") > -1) 
  {
    return;
  }
}

}

Reference : How to prevent a SharePoint web part from being crawled

share|improve this answer
    
This technique is pretty cool actually - you can have a control with only this code in it, and wrap things on the masterpage you don't want indexed with this control, and it shouldn't crawl them. Used a similar technique on a project a few years ago and it worked like a charm. –  James Love Oct 12 '13 at 9:43
add comment

I've definitely seen this operate in reverse: I had a bear of a time getting content to show because of that darn noindex class.

Your issue looks like you're using the class (.) operator instead of the id (#) operator. I think your JQuery should look like this:

$("#nav_root").addclass("noindex");

If you just want to block the entire page and don't care that this process might take an extra second or two to load, you could just call every element on the page (warning: often several thousand on an SP page):

$("*").addclass("noindex");

The other thing you can take advantage of is that SharePoint won't crawl any information hidden behind a div tag with the "display:none;" element applied to it. You could just add the above style tag to your page if for some reason the noindex class continues to not work and then add some sort of toggle switch to display the data for end users.

share|improve this answer
    
I actually used the class operator intentionally as we have 3 generated navigation elements on each page (top, left, and bottom). I was able to verify that the javascript was correct by looking at the page in the browser. –  atcrawford Oct 4 '13 at 2:20
    
The part of your answer about "dispaly:none" is very interesting though... I might be able to hide the nav elements by default and use javascript to reveal them to normal users. I'll have to test out. –  atcrawford Oct 4 '13 at 2:22
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.