Hot answers tagged crawling
You can define crawler impact rules so you can reduce the load SharePoint will put on an external site. What type of content? Is this static HTML pages or something different? You may want to be sure your crawler is also configured to crawl complex URL's. You can also adjust the SharePoint crawler timeouts and retry events - but these are server settings ...
Only top voted, non community-wiki answers of a minimum length are eligible