This is a difficult question

I am trying to figure out how SharePoint 2010 Search and Scope works. I have a site that contains 369 sites, and 5563 total pages within those sites. The sites are all publishing sites, so all the pages are within Pages libraries. I've setup 9 custom scopes so that my users can search specific areas of the site, but the number of items in each scope doesn't come close to the total pages

My Content Source is my site's URL, e.g. https://mysite.com, and the Search Administration page, System Status section, shows 98,480 searchable items.

The default All Sites scope shows 21600 items.

Having run a powershell script to get ALL the pages, I see that there are, for example, 851 publishing pages within the site and subsites of https://mysite.com/us, but the scope I setup using this exact URL only shows 102 items. Another example is the site and subsites of https://mysite/support/version2, which has 1218 pages (according to the powershell script), but only 650 items according to the scope using the same URL.

Clearly something is wrong.

Again, my search's content source uses https://mysite.com as the 'Start Address', and I have the following crawl rules

The first three rules are to remove items from the search results that are shown to my users, while the final rule is so that pages within sites that have Anonymous Access turned off can still be seen in search results.

Given all this, is there a reason why my scopes are only showing a fraction of the items they are supposed to?

These rules should not impact the pages within my Pages libraries, should they?

It did seem that before I added the 2nd and 3rd rules (layouts & _vti_bin), that the scopes had way more items in them, but then the 'searchable items' was around 400,000 instead of 98,480, and most of these were what I call 'internal system pages' rather than content pages that we actually want people to see.

I've completely lost on this, and could really use some advice - anything!

1 Answer 1


Have your scopes been "compiled" (updated)? Check the scopes update status in Search Administration.

The second thing to check here is that the pages are in your index. Check your crawl logs, or search for specific pages using the All Sites scope. Perhaps the crawl rules have prevented the crawler from finding your pages - if you set it to crawl as http it will only find pages that are ultimately linked to one of the start addresses.

If there is a page that you can't find in the index, but should be there, enter its URL in the crawl rules configuration page to see if one of the crawl rules is excluding it. If the page is in the index and the scopes are all updated then there must be a problem with the scope rule.

  • The scopes have updated. The update status is "Idle". I set the scopes a while back, so they are already in place when I do my crawl. Shouldn't be an issue with that, right? I checked the crawl logs, putting in https://mysite.com/us (used in my e.g. above), and the second page, /us/authoring.aspx, has a green icon (for success), an Item ID, and the URL shows 'Crawled'. However, doing a search with the 'US' scope (matchingstring = mysite.com/us), does not show this page when searching for 'authoring'. I tested the URL in the rules page, and I'm told the url WILL be crawled.
    – QMKevin
    Mar 11, 2012 at 15:04
  • (ran out of space in previous comment) Looking at the scope, this is what I have: Title="English (US) site", Update status="Ready", Rule (just 1 rule) = "Folder: mysite.com/us" + Behavior: Include. I'm going to try using Hostname instead of folder, but any other advice on what a rule should look like to a scope there the desired content is anything within mysite.com/us and it's children, would be appreciated.
    – QMKevin
    Mar 11, 2012 at 15:09
  • When you look at the scope do you see the expected number of items in the Item Count next to the rule? You can also check in Search Administration for "Scopes needing update" - should be zero.
    – SPDoctor
    Mar 12, 2012 at 13:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.