Take the 2-minute tour ×
SharePoint Stack Exchange is a question and answer site for SharePoint enthusiasts. It's 100% free, no registration required.

I'm planning the indexing and search topology for a new customer.

This customer wants to store approximately 10 million documents in the SharePoint farm on an annual basis. These documents are to be retained for 2-5 years (or more in certain cases). Now my reading on TechNet it looks like I need an index partition per 10 million items.

It looks to me that I'm going to need Query Server 1 server per 10 Million items which should be mirrored for redundancy, so for ~50M items I should be looking at 6 Query Servers each hosting their own index partiton and a mirror index partition, is this correct or is this overkill?

share|improve this question
add comment

1 Answer

up vote 5 down vote accepted

If they are looking to add 10m items per year and retain content for 2-5 years, then I would probably design the initial system to support 30m items and grow it as needed in 2-3 years. Business and technology changes may require a different topology at that point.

One Crawl Server / Index Partition can support up to 10m items. In many cases you will want to plan on that being smaller since the amount of time it takes to do a full crawl will be directly related to the hardware capabilities and the number of items in the index. I typically try and plan for 5m per index; if anything this gives you a buffer.

You also need one or more query component for each Index Partition. If you look at the farm that is why you see the query component duplicated across multiple boxes for each index partition.

Following the logic above if you plan for 6 Index partitions you would need 6 crawl servers and at least six web front ends, supporting six primary and six redundant query components.

With all of that said, I push pretty hard for FAST with any installation where you know there will be more than 10m items in the index.

Update to address comment: A single WFE is a bad idea unless the system really isn't critical or only has a small number of users. There are a number of reasons to load balance that have little to do with scalability.

It is also very important to understand the impact that crawling has on the system overall. In most small 1-3 server environments with any real content, the biggest performance issues tend to come from Search because the crawl hammers the system's ram and cpu. The Web Front End is overwhelmed which then impacts regular use of the content. Your six crawl servers cannot be serviced by a single web server. The number of WFEs will limit the number of query servers and index servers you can support.

share|improve this answer
    
Thanks Mike, I was planning on pushing for FAST. Just got confirmation that we need to migrate historical data (which was used to generate the sizings) into the system :|, 6 WFE though? I was planning on 2 Query Servers : 1 WFE –  GavinB Nov 23 '11 at 19:53
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.