SharePoint 2013 Search Crawl Issue with https web application and local DNS

Question

We have a Multiple server SharePoint farm with 2 Wfes and 3 App servers.

We have configured SharePoint search and have a web application under 443 port https enabled. and we are accessing the root site of the web application using a local DNS that is added an entry in the host file with the one of the WFE's IP,

and can access the site successfully from crawler App server and when trying to run the Search Full Crawl it fails with the below error message.

An unrecognized HTTP response was received from the server when attempting to crawl this item, verify whether the item can be accessed using your browser ("Error from SharePoint site: WebExceptionStatus: SendFailure The underlying connection was closed: An unexpected error occurred on a send." Search ID = "")

and I'm getting the below msgs logged in verbose level ULS log of the search components.

CHttpAccessorHelper::InitRequestInternal - unexpected status (404) on request for 'https://siteintra/robots.txt' Authentication 1.  [httpacchelper.cxx:746]  search\native\gather\protocols\http\httpacchelper.cxx

SetSTSErrorInfo ErrorMessage = Error from SharePoint site: WebExceptionStatus: SendFailure The underlying connection was closed: An unexpected error occurred on a send. hr = 80041204  [sts3util.cxx:6994]  search\native\gather\protocols\sts3\sts3util.cxx



Couldn't retrieve server https://siteintra policy, hr = 80041204            [sts3util.cxx:1871]  search\native\gather\protocols\sts3\sts3util.cxx



STS3::StoreCachedError: Object initialization failed.  Message:  "Error from SharePoint site: WebExceptionStatus: SendFailure The underlying connection was closed: An unexpected error occurred on a send." HR: 80041204  [sts3util.cxx:7089]  search\native\gather\protocols\sts3\sts3util.cxx



Server siteintra security initialization failed, hr = 80041204 error Message Error from SharePoint site: WebExceptionStatus: SendFailure The underlying connection was closed: An unexpected error occurred on a send.  [sts3util.cxx:1597]  search\native\gather\protocols\sts3\sts3util.cxx



CSTS3Accessor::InitURLType fails, Url https://siteintra, hr=80041204        [sts3acc.cxx:297]  search\native\gather\protocols\sts3\sts3acc.cxx



FLTRDMN: URL https://siteintra Errorinfo is "Error from SharePoint site: WebExceptionStatus: SendFailure The underlying connection was closed: An unexpected error occurred on a send."  [fltrsink.cxx:566]  search\native\mssdmn\fltrsink.cxx

and when I tried to crawl the Central administration site the full crawl is successfully done and the crawled items appear in the search, which makes sure that the search is configured correctly.

here I have tried solutions suggested related to this issues:

1.disable loopback check 2.checking all the permission level of the Default Content Access Account in Search Service Application, User Profile Service Application, and Web Application User Policy level 3.Accessing the site pages with the Search Account

nothing helps here as still the full crawl fails in this situation.

here I suspect whether the local DNS which is not in our DNS server will make any issues in the crawling.

Your inputs are highly appreciated here.

Answer 1

You need to ensure that the crawler can get to the destination SharePoint site without any certificate or access denied errors. Resolve all certificate errors by opening the site FROM the crawl server AS the crawl account. This is how the crawl server is "seeing" your site. If there are errors it will not be able to crawl.

Another alternative is to create the default zone without SSL. Then extend the web application to a second zone with SSL. Crawl the default non ssl zone and let AAM handle the translation. This works but really only masks the issue with your certificates. It is always preferable to get the certificates right from the start.

Answer 2

My ideas:

• Check if your SSL-Certificate is placed in CentralAdmin -> Security -> Manage Trust. This is SharePoint owns Certificate-Store and sometimes it is required to place your SSL-Certificate there.
• Check your Content-Sources. I have seen entries like https://siteintra:80. The port was wrong because the site was recently changed from HTTP to HTTPS.
• Temporary change the service-account. CentralAdmin -> Security -> Configure Service Accounts -> Select the Service Application Pool of search -> Change it to Farm-Account. Immediately after that, revert it to the previous Service-Account.
• Check your Windows-Eventlog on Search- and WFE-Server. Windows Schannel-Service is responsible for opening SSL-Connections and it logs stuff only there (cannot be seen in ULS-Log).
• Add SharePoint URLs to HOSTS-File on Search-Server and point it directly to your WFE IP-Address. This will bypass DNS and Loadbalancers.