I have a simple HTML documentation site with a TOC column on the left side that includes all the links on the page. The URLs are structured http://documentation/section1_subsection1_subsection2.htm

When I set Sharepoint to crawl the site, only 41 pages are indexed (out of 5000+). I don't get any crawl errors, it's like Sharepoint doesn't see all the other pages, even though they're listed as html links right in the page.

Sharepoint seems to be stopping halfway through http://documentation/section2_subsection3, so it's seeing some of the links, but not others..

Content source settings:

start address: `http://documentation`
only crawl within the server of each start address
incremental crawl: every night at 6pm
full crawl: every Sunday

I did set a crawl rule for http://documentation/*, but that didn't make a difference.

there is no noindex set on the site.

Another fun part of this is that I recently had to recreate the search service application - Before I recreated it, all of the content in the source was crawled.

  • It might not be getting to the other pages because there are no links to them possibly? If you have an affected page not showing in the index, can you track if there are any incoming links to the page? Nov 4 '15 at 18:40
  • The index page has links to every single page in the documentation. Some are getting indexed, some aren't. Nov 4 '15 at 18:46
  • Oh derp, i see that you said that, sorry. Nov 4 '15 at 18:56

I realized that the site links are loaded when the link is clicked - There's an XML sitemap file, so I'm working on figuring out how to set up a BCS connector.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.