0

We have hosted a web application that has 108687 records and when crawled the content source, 104,314 successes and there are 3788 errors from the crawl log.

All these 3788 errors are due to the below error message.

The crawler could not communicate with the server. Check that the server is available and that the firewall access is configured correctly.

We did a google search and gone through below link https://social.technet.microsoft.com/Forums/office/en-US/f378c45f-fa8b-48f3-88bb-739f3cd27830/the-crawler-could-not-communicate-with-the-server-check-that-the-server-is-available-and-that-the?forum=sharepointadminprevious

did modification on the registry editor and the given configuration already exists.

on permission front , we could successfully crawl rest of the data's.

Could you please advise on what is the cause and solution?

Improve this question
8
  • 1
    I am having recently very same problem and open case with Premier support and getting nowhere for last 2 weeks. My full crawl sometimes crawls all content, sometimes half and sometimes none finishing on the top level error. Are your errors consistent, meaning with each crawl it is always same set of documents? My latest finds are pointing to some kind of corruption of the "Content Source" after I have removed some sites from original CS and added it to new one I don't experience any more issues, will be doing larger test over the weekend as the data corpus is quite big. Will update on Monday.
    – Marek Sarad
    Jan 13, 2017 at 14:21
  • 1
    John one more question, on what CU are you running the farm?
    – Marek Sarad
    Jan 13, 2017 at 14:31
  • My content source is web application especially HTML pages with contents. The errors are inconsistent. I tried troubleshooting with Fiddler, we are getting error 500 for those pages even though those pages are responding when we browse separately. This is something weird behavior in Search. Regarding CU , I am not sure about it, any guidance on how to identify would be great!
    – John Zachariah
    Jan 15, 2017 at 13:36
  • 1
    To get the build version you are running execute the PS in Sharepoint Shell: $farm = Get-SPFarm $farm.BuildVersion
    – Marek Sarad
    Jan 16, 2017 at 8:12
  • used this command Get-SPFarm | Select BuildVersion and the result is BuildVersion ------------ 16.0.4327.1000
    – John Zachariah
    Jan 16, 2017 at 13:35

1 Answer 1

Reset to default
1

I was getting intermittent issues as well; after checking the usual suspects: Loopback, Balancing config, SuperReader, SuperUser, is the default zone crawled etc we still had no resolution. It came out that the issue was corrupted content source (index reset didn't resolve issue either) recreating content source resolved problem. To delete content source you need to

  1. Disable crawl
  2. Edit existing content source.
  3. Change the existing start addresses to something like https://xywz.com
  4. Delete Content Source (important for SP2013 if you delete content source without changing the start addresses you will not be able to create new with the old start addresses, SP somewhere saves that info and only way to recover from that is to rebuild search service.)
  5. Now create new content source, add the original start addresses to the CS and start a full crawl.

Disclaimer: I state that it was corrupted content source as the recreation solved the issue (and index reset did not), I have no other proof or way to test that the content source is corrupted.

If you still facing the problem at this point only point left is to rebuild Search service completely from scratch.

Edit: Regarding the build version.

You are on very early build, not event RTM. If index recreation doesn't work I would first update. You version is saying that you are on the SP2016 RC. You are more than one year behind the updates. Update to latest CU and see if you are still facing the issue. As usual Todd's blog is very helpful

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.