SharePoint Stack Exchange is a question and answer site for SharePoint enthusiasts. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

The crawler crawls everything successfully except the below URL http://localhost/alums/lists/324324_classnotes/brown,%20harold%20jr.%20'74 where it gives an error in the crawl log like "Content for this URL is excluded by the server because a no-index attribute".

I do not have any crawl rules setup for any of my content sources.

Any ideas?

Thanks, Michael

Update: I checked the source of the page and I found out below tag in it. <META NAME="ROBOTS" CONTENT="NOHTMLINDEX"/>

Is it possible if this is blocking the search to work?

share|improve this question
See Anders' comment on the Crawl Rules for the site (In Site Settings->Search Visibility). – James Love Aug 27 '10 at 16:45

Check the source of the page and see if theres a

<META name="robots" content= "noindex">

Tag in the header. This could be set by the <SharePoint:RobotsMetaTag ID="RobotsMetaTag1" runat="server" /> in the master page.

Have you checked the crawl rules both for the site (Search visibility) and for the pages library (index settings on doclib)?

share|improve this answer

in crawl rules untick,

"Follow links on the URL without crawling the URL itself"

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.