0

We're facing slow crawl times on our main content source and I'm not sure what the cause is.

The farm (SP2010) consists of 2 WFE's and 3 APP servers. On all APP servers, crawl components have been deployed. The index / query components are located on the WFE's. Any comments on the architecture are welcome. The content source crawls two SharePoint web applications which together contain about a million items.

Now what I'm seeing is that the initial performance is fine. In the first few hours, almost everything is crawled and propagated into the index. But then everything seems to slow down to almost a halt (0.05 items a second). Eventually everything finishes and the incrementals begin. These run very irregular, some times it finishes in 15 minutes and other times it would take several hours to finish.

Query performance is ok and my users can find their stuff, but because of the running full crawl, incrementals are not started. Due to migrations the content was pretty dynamic some months ago, but it's stable now.

So my question: what could be causing this lag, or what's a good way to find out?

Farm is patched untill december 2013 patches. Search filter pack has been updated to the latest version as well.

Hardware specs: 2 WFE's which both host a query component. 3 APP's which all host a crawl component. Index is spread over all servers.

All servers are equal in specs: 12GB RAM, Xeon quad core @ 2GHz.

Update April 4th: I splitted up the content source in order to be able to narrow down the problem. I found out that the culprit is our most locked down web application. The weird thing is that on this web application, we are enforcing the usage of AD groups, so there are almost no individual accounts in place other then some administrative ones. According to what I read on security only crawls, those would be prevented by using AD groups. So that's weird and leaves me to think the security only crawl is not the culprit here.

What I DO see, is quite a few filter related errors:

  • "The filter daemon did not respond within the timeout limit."
  • "The filtering process could not be initialized when trying to process this item. Verify that the file extension is a known type and the item is not corrupt."
  • "The item may be too large or corrupt. You may also verify that you have the latest version of this IFilter"

Could have something to do with it, although the sum of these is only about 100 errors. I wouldn't expect 100 timeouts to be the cause of days of delay.

Improve this question
1
  • 1
    It may be also useful for use to know what the Hardware Specifications are for each of the servers (cores, GB RAM, Free disk on database server).
    – Cameron Verhelst
    Commented Mar 4, 2014 at 12:49

3 Answers 3

Reset to default
1

We have found the solution. After a lot of scoping, we narrowed it down to a web application. I then went on, finding out which site within that web application was the culprit. It turned out to be a single site collection in which over 500k MSG items were stored. These folks (one specific department) are using SharePoint as a mail storage system, because they didn't like the retention settings on the Exchange environment. I think the number of very small items is the main cause of the problem, and perhaps issues the crawler has indexing these files.

Improve this answer
0

If you are crawling SharePoint content check the database servers. Is there enough memory, and disk space. Also check if the speed of your (database) drives is not the bottleneck.

When crawling remote content (ex. file shares) check where the content is hosted and if the connection between the content and the crawl server is fast enough.

The items that you are indexing do they contain custom code? (web parts, user controls, custom master pages or page layouts, ...) is all the custom code properly disposed of, no memory leaks etc...

Improve this answer
1
  • The hardware performance cannot be the issue. This is because initially, the crawl is fast and indexes a lot of items. It's only at the last part of the crawl that everything slows down to a halt and seems to be hanging. If this was a hardware bottleneck; wouldn't the entire process be a lot slower? We're not indexing remote content, no customizations either.
    – Jasper
    Commented Mar 27, 2014 at 14:20
0

It could be that there are issues with the security only crawl. These crawls happen when users are added or removed from SharePoint groups and lists. These constitute updated ACLs and need to get into the index.

Take a look at this post to see if your symptoms fit: http://blogs.msdn.com/b/russmax/archive/2009/02/09/troubleshooting-security-only-crawl.aspx

Crawler Issue with SharePoint 2010 and Windows Server 2008 R2

64K ACL Limit

Improve this answer
1
  • I see a lot of security changes, but a full crawl is also slow. In the blog you link to, they seem to scope this problem to incremental crawls only. Doesnt that imply this cannot be the problem? The crawl issue doesn't apply to our situation either.
    – Jasper
    Commented Mar 27, 2014 at 14:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.