The requirements: exclude any subfolders in a document library on search results.

Crawling Rules set up in the following order:

  1. https://MySite.*(/.+)*/.*[.].* use regexp checked Include all item in this path checked

  2. https://MySite.*/.*(/.+)* use regexp checked exclude All items in this path checked

After a Index reset and then a full crawl, I could see the in the Crawling log, only 1 item https://MySite.xxx.com is listed as successfully crawled.

All tests done on the Crawling Rules page was as expected. so:

  • https://mysite/doclib will not be crawled.
  • https://mysite.xxx.com/doclib1/subfolder will be not be crawled.
  • https://mysite.xxx.com/doclib1/subfolder/doc.doc Will be crawled.

But why none of the actual documents or files are not crawled at all?

I have seen a lot of posts regarding use the technique, so the include for the specific type of item first then the excluding rule for the rest of the paths.

What have I done wrong here? or is it a https issue I wonder?

  • To clarify, if you remove those crawl rules then crawling works successfully and the entire site is crawled? – Mike2500 May 12 '15 at 14:19
  • Yes. And if I just remove the 2nd one and without reset index. Then after a full crawl then a lot of items deleted with the message "Deleted by the gatherer ( This item was deleted because its parent was deleted. )" – xun May 12 '15 at 14:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.