1

In our SharePoint farm it has 4 big Content Source Web Applications around 3 TB of documents and daily users upload documents around 5 - 10,000 documents, and i need to run incremental crawling every day every 30 minutes and full crawling every weekend,

  1. Can i create separate Search Service Applications for big content sources is good idea?
  2. What is the best practise to run full crawl and incremental crawl like above scenario in our sharepoint farm?

This farm has two separate crawl servers Below is the number of searchable items: enter image description here

3
  • How many documents so far in the index?
    – Waqas Sarwar MVP
    Commented May 8, 2016 at 21:54
  • 1
    Why are you running a full crawl so frequently? Consider going a few months between full crawls as at the rate of documents added, even if you sustain your DPS, it will become untenable to perform a full crawl (perhaps on the order of months to complete, which I've encountered previously).
    – user6024
    Commented May 8, 2016 at 22:53
  • now 12,000,000 searchable items in index, @Trevor can you explain more about run full crawl ? now we have three content sources in this daily 5000 documents upload by users
    – adilahmed
    Commented May 9, 2016 at 7:08

1 Answer 1

1

1) Unless you are splitting your content into different farms, adding more processes (new Service Application) to run on the same farm (servers) will only increase the load.

2)

  • a) Full crawl - needs to be run only when you are aware of changes to document properties. E.g. there is a requirement to add new managed properties or change existing managed properties.
  • b) Incremental Crawl - This depends on the content refresh rate expected by the business. Here is where you have to intelligently segregate your content stores and update the incremental crawl schedule. For archives and records, once a day should be sufficient. Again, this is based on business needs.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.