0

My search crawl is not working for my local site. On my Crawl Log I see a warning:

Item not crawled due to one of the following reasons: Preventive crawl rule; Specified content source hops/depth exceeded; URL has query string parameter; Required protocol handler not found; Preventive robots directive. ( This item was deleted because it was excluded by a crawl rule. )

Important information:

  • I just moved all my content databases to a newly installed Sharepoint on a new server
  • I am running Sharepoint Server 2019 Standard
  • When I change the address in the content source to full FQDN, the search works partially with many 401 warnings
  • The address in question is https
  • Same address with sps3 gets crawled successfully with no issues
  • There is no robots.txt file in the site

I tried and checked everything I could think of, including:

  • I checked that the default access account has access to the website
  • I browsed the website ( using a browser ) while being logging in using the default access account
  • I made sure there are no crawl rules
  • I created a new crawl rule to include the affected URL
  • I did "index reset"
  • I disabled loopbackCheck
  • I used Fiddler to monitor the traffic, the crawler is not even accessing the site

If you have any suggestions, please advice.

0

After two days of trying and failing, I solved the issue by cleaning up the content databases from missing features.

I had to manually uninstall all missing features using:

stsadm -o uninstallfeature -id <GUID> -force

After doing that the crawler started crawling again. But I don't know which feature in particular was causing the issue.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.