SharePoint Stack Exchange is a question and answer site for SharePoint enthusiasts. It only takes a minute to sign up.
Sign up to join this community
I am seeing thousands of warnings in my crawl log for a login page for a public website that is using a complex URL.
Is there any benefit then to create an exclusion crawl rule for this login page specifically?
Will search crawl faster or not take a long if I use the exclusion crawl rule for the login page?
By default, the crawler will not follow complex URLs, which are URLs that contain a question mark followed by additional parameters — for example, http://contoso/page.aspx?x=y. If you enable the crawler to follow complex URLs, this can cause the crawler to gather many more URLs than is expected or appropriate. This can cause the crawler to gather unnecessary links, fill the crawl database with redundant links, and result in an index that is unnecessarily large.
Performance will improve if you create exclusion rules. And it is one of the best practice
https://technet.microsoft.com/en-us/library/dn535606.aspx#BKMK_UseCrawlRules
Exclusion rules are useful and help in performance.
Avoid crawling less relevant content by excluding one or more URLs. This also helps to reduce the use of server resources and network traffic, and to increase the relevance of search results.Creating crawl rules to exclude URLs is particularly useful when planning start addresses for external content because the impact on resource usage is not under the control of people in your organization.
