Tell me more ×
SharePoint Stack Exchange is a question and answer site for SharePoint enthusiasts. It's 100% free, no registration required.
up vote 4 down vote favorite

I am not able to crawl website that has STS (security token service) based authentication using FAST Search for SharePoint 2013 . I tried Form credentials and Cookie based authentication in Crawl Rules but following error comes:

Error: Access is denied. You may not have sufficient privileges to perform the operation.

What is the right way to crawl an external web that is secured with STS based authentication?

share|improve this question

2 Answers

up vote 1 down vote

You need to create a new crawler rule with default crawler account if it already has read permission on external websites, if not then you can specify a new crawler rule with a crawler account that has at least read permission on external content your crawler will crawl.

Since you getting Access denied error seems like your crawler account doesn't have permissions to crawl external websites regardless of authentication type.

Other then that,

Ensure that the domain account that is used for the default content access account or any other content access account is not the same domain account that is used by an application pool associated with any Web application that you crawl. Doing so can cause unpublished content in SharePoint sites and minor versions of files (that is, history) in SharePoint sites to be crawled and indexed.

Source

Hope it helps.

share|improve this answer
As I already mentioned, I tried Form based and Cookie based crawler rule. In both cases, I specified an account that has read permission on external content. When we make crawler rule it authenticate account credentials with external sites and if it is successful then it creates rule. In my case when I created rule, account credentials were successfully authenticated with external content and than rule was created. Note that there is Test option on Crawl Rule page, that takes URL as input and highlight which rule is applicable. Test is also passed. – Jamil Feb 15 at 5:18
I saw following text on your source link: Source "If you are using claims-based authentication, make sure that that Windows authentication is enabled on any Web applications to be crawled." Does it mean Windows Authentication must be enabled on external website if it uses claim based authentication for crawling? – Jamil Feb 15 at 6:01
up vote 0 down vote accepted

I explored web in order to solve this issue but it seems it is not possible to crawl an external website using SharePoint search that has STS based authentication and don't have Windows authentication enabled. Here are few references:

"If you are using claims-based authentication, make sure that that Windows authentication is enabled on any Web applications to be crawled." Source

"SharePoint can only crawl using a Windows NTLM authentication account. Your content source must authorize the Windows account sent as part of the crawl request in order to access the document content. Though claims authentication is supported in SharePoint 2010, the gatherer is still not a claims-aware application and will not access a content source that has claims authentication only." Source

"No, it needs NTLM/Kerberos in order to crawl the Web App." Source

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.