0

I have a redundant search topology with 2 search servers and 10 WFEs on top of SharePoint 2019, my web application is extended to Intranet zone, the reason the web application is extended is due to ADFS integration, so the Default zone has ADFS trusted identity provider only enabled, and the Intranet zone has NTLM authentication only enabled.

In AAM, we set the Default zone to web application URL, and in the Inranet zone we set it to the extended application URL, knowing that the Intranet zone URL is HTTP not HTTPS.

In content source I'm using the URL that exists in Intranet zone.

Once I start full crawl, it can crawl many items but many content pages (ASPX pages) inside my SharePoint site throw the error message below in crawl log:

"The SharePoint item being crawled returned an error when attempting to download the item (SearchID=xxxxxxx")"

Therefore those items that have the error above will not be crawled and will not be retrieved in the search result.

Any help would be appreciated.

2
  • We need to first learn what the error is. Can you check the crawl log for more details? If there is no error message there, check the ULS logs. Commented Dec 30, 2022 at 17:27
  • After looking into the ULS logs we found that there's a web part added to the pages that the crawler was trying to index, this web part had an error "object reference not set to an instance of an object", we fixed the issue of this web part and ran full crawl again and now those internal pages are indexed. Commented Jan 2, 2023 at 6:26

1 Answer 1

0

After checking ULS logs, we found that there's an Issue with a web part added to those internal pages (basically added to the page layout attached to those pages),the web part was throwing exception "Object reference not set to an instance of an object", we fixed the issue of this web part and then started full crawl, and then the crawler was able to crawl those pages and index them.

It seems the error of that web part prevented the crawler to crawl and index those pages.

1
  • Also note that for search to “crawl properly” the default zone should be NTLM and the one defined in the content source. The “Intranet” zone should be the one extended for ADFS. If not done this way, the URLs in the index will not be recorded correctly and may cause issues. Ref: learn.microsoft.com/en-us/sharepoint/search/…
    – Greg W
    Commented Jan 3, 2023 at 3:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.