1

We're using Search Server 2010 Express to crawl our own sites that do not use SharePoint. There are several other content sources that are crawled without issue, but this one is giving me the following error.

The start address ****** cannot be crawled.

Context: Application 'Search_Service_Application', Catalog 'Portal_Content'

Details: Access is denied. Verify that either the Default Content Access Account has access to this repository, or add a crawl rule to crawl this repository. If the repository being crawled is a SharePoint repository, verify that the account you are using has "Full Read" permissions on the SharePoint Web Application being crawled.
(0x80041205)

Here's what I have tried so far.

  • Following the this for the LoopbackCheck fix http://support.microsoft.com/kb/896861
    • Safe method using BackConnectionHostNames didn't work
    • DisableLoopbackCheck set to 1 also didn't work
    • Server has been rebooted after BackConnectionHostNames key added
  • Added crawl rule that specified a different content access account
    • Re-added the same content access account with "Do not allow Basic Authentication" checked and unchecked
    • Used different domain admin account with same result
  • Updated the Default Content Access account
    • Re-added the same account and also tried with a different domain admin account
  • Came across mentions of UserProfile Service Application, but I don't see this on my service applications list
  • This is the only content source that's giving me this issue
    • I don't think it's an issue with access privileges with the account being used
    • This shouldn't be an issue with connecting to SQL Server since the other content sources work fine
  • Can access the site from the server while logged in as the content access account in IE

I've gone through most of the comments in the below two links along with much time spent reading Google search results. Most seem to point the the LoopbackCheck fix as the solutoin to this error, but this has not worked for me.

Search Crawl - Access Denied

http://littletalk.wordpress.com/2011/03/21/access-is-denied-verify-that-either-the-default-content-access-account-has-access-to-this-repository-or-add-a-crawl-rule-to-crawl-this-repository-if-the-repository-being-crawled-is-a-sharepoint-rep/

I would appreciate any help in this since it's been driving me nuts for two days now.

EDIT: The account used for this is not a domain account. It's just a local account on the server that's in the admin group. But like I said, the other content sources crawl without issue using this setup.

Improve this question
4
  • If these are content sources that do not use SharePoint what kind are they that you are trying to crawl? Most of the errors and troubleshooting you are doing seem more geared towards SP issues
    – MichaelF
    Commented Dec 5, 2012 at 18:08
  • @MichaelF These are just normal .Net based websites we've built. I'm not familiar with Sharepoint, just trying to search for solutions to the errors I see in the crawl log.
    – Jerry
    Commented Dec 5, 2012 at 19:21
  • What Content Source type did you select and what is the Hostname you provided? Since this is a .Net you should have something like Web Site or something non-SharePoint specific and the Hostname URL should be http://<hostname> for the site
    – MichaelF
    Commented Dec 5, 2012 at 20:32
  • @MichaelF The content type is "Web Sites" and the URL is subdomain.domain.com.
    – Jerry
    Commented Dec 5, 2012 at 20:36

2 Answers 2

Reset to default
1

One thing you can check is the robots.txt file of the site you are trying to crawl. Here is a thread on that subject (as well as a few other things you can investigate): http://social.technet.microsoft.com/Forums/en/sharepointadminprevious/thread/9ae514a1-3d19-4d85-b458-6be9743e1d7b

Improve this answer
5
  • Thanks for the response mikey. I've seen that link when dealing with another issue. All the other subsites have a robots.txt file and the crawler has no issues or errors when crawling their content.
    – Jerry
    Commented Dec 5, 2012 at 17:34
  • 1
    You can perhaps use fiddler to capture the crawl attempt and get a better look at what the server response is for that site and how it differs from the others.
    – mikey
    Commented Dec 5, 2012 at 18:07
  • I installed fiddler, but it doesn't seem to capture any of the crawler attempts. It shows up my activity on the sharepoint central admin area, but nothing the crawl service is doing.
    – Jerry
    Commented Dec 5, 2012 at 20:01
  • Try installing Fiddler on the Crawler machine, you need it to be scanning from where the crawls come from
    – MichaelF
    Commented Dec 5, 2012 at 20:28
  • @MichaelF The install is on the server where the search is setup on and the crawls initiate from. Does fiddler pick up on traffic generated by a service rather then a user?
    – Jerry
    Commented Dec 5, 2012 at 20:35
0

You will need to make sure that the account used for the crawl in 2010 has "full read" permission to the content via the external site Web Application Policy if the external content is a sharepoint site.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.