This question is difficult to describe, so bear with me please :)
I'm trying to work out SharePoint 2010 searching and scopes. I have a site that contains 369 sites, and 5563 total pages within those sites. The sites are all publishing sites, so all the pages are within /Pages libraries. I've setup 9 custom scopes so that my users can search specific areas of the site, but the number of items in each scope doesn't come close to the total pages
My Content Source is my site's URL, e.g.
https://mysite.com, and the Search Administration page, System Status section, shows 98,480 searchable items.
The default 'All Sites' scope shows 21600 items.
Having run a powershell script to get ALL the pages, I see that there are, for example, 851 publishing pages within the site and subsites of
https://mysite.com/us, but the scope I setup using this exact URL only shows 102 items. Another example is the site and subsites of
https://mysite/support/version2, which has 1218 pages (according to the powershell script), but only 650 items according to the scope using the same URL.
Clearly something is wrong.
Again, my search's content source uses
https://mysite.com as the 'Start Address', and I have the following crawl rules (please excuse the bold.. want to use * in my rules):
- https://mysite.com/*/Documents/* (exclude)
- https://mysite.com/*/_layouts/* (exclude)
- https://mysite.com/*/_vti_bin/* (exclude)
- https://mysite.com/* (include - crawl content as http pages)
The first three rules are to remove items from the search results that are shown to my users, while the final rule is so that pages within sites that have Anonymous Access turned off can still be seen in search results.
Given all this, is there a reason why my scopes are only showing a fraction of the items they are supposed to? These rules should not impact the pages within my Pages libraries, should they? It did seem that before I added the 2nd and 3rd rules (layouts & _vti_bin), that the scopes had way more items in them, but then the 'searchable items' was around 400,000 instead of 98,480, and most of these were what i call 'internal system pages' rather than content pages that we actually want people to see.
I've completely lost on this, and could really use some advice - anything!
Thanks for reading :)