"Object not found" error in search service crawl Logs

Question

I have a search service application that won't crawl correctly.

The crawl account has all the right access to the farm and databases. THe AAM are as follows:

- http://machinename - default 

- https://sp.contoso.com - internet

I try crawling "http://machinename" as content source, I get:

Error - The object was not found.

I try crawling "https://sp.contoso.com" as content source, I get:

Deleted(Warning) Item not crawled due to one of the following reasons: Preventive crawl rule; Specified content source hops/depth exceeded; URL has query string parameter; Required protocol handler not found; Preventive robots directive. ( The item was deleted because it was either not found or the crawler was denied access to it. )

Any ideas?

Answer 1

Ah rats. It was my doing. I had a host header in IIS linked to port 80 that was supposed to be entailed in the certificate for SSL. I deleted the host header in the IIS Manager for port 80, and could navigate to the site and the crawl worked successfully.

Answer 2

Verify the search account is registered correctly, if you are using claims the search account should have the claims prefix ( 0#.w ) as part of the logon name. If you only see the normal domain or computer prefix without that above, than the search account is only mapped to the windows account, not the claim, so essentially they will be seen as different users with different or non-existent permissions.

Whats the purpose of crawling via the AAM address vs. default zone? In my experience, the default zone over HTTP get's you past a lot of issues. When you access search via the AAM URL all your links will be updated properly, so it's not required to search via the URL you are using.

Answer 3

After making changes to the crawl accts and/or content sources I would often get the error Required protocol handler not found. After restarting the Search Service I would find that the crawl woud than perform as required. Worth a shot to see what happens when you restart the search services.