0

My organization has been attempting to use FAST Search for Sharepoint 2010 as a search solution for our DMS. I am currently utilizing a .Net indexing connector to index a database of ~ 8 million records and their accompanying files.

Previously, I was having a memory issue on the crawl box, which included the incremental crawls being unable to complete. After months of troubleshooting, that issue seemed to be resolved with a hotfix (http://support.microsoft.com/kb/2601211). The entire issue is illustrated in another thread (http://goo.gl/3CuZUY).

However, on the following incremental, I began to receive new warnings regarding batch time outs in the logs and an atrocious crawl rate (0.1 dps). This has also caused the crawls to be unable to complete, even after running for hundreds of hours. Yet again, I am at a loss.

I have been working with a Microsoft support rep for over two months, but they have been unable to offer assistance for any of our issues. I have also been monitoring the performance counters on both servers for quite some time, and (other than a high 'batches ready' number on the crawl box) I have yet to find a smoking gun. Thus, any suggestion would be greatly appreciated.

Environmental and log information detailed below:

SharePoint Environment

  • 1x WFE
  • 1x Database Server
  • 1x Application Server (32GB RAM)
  • 1x Index Server (FAST) (16GB RAM, 8 CPU Cores, 5 Document Processors)

Logs

======[FAST BOX]======

DOCLOG

  • WARNING Document conversion failed: External process timeout reached (300 seconds) (warning code 0)
  • INFO Processor "IFilterConverter" ran for 300s

PSCtrl Statistics

  • IFilterConverter (System Time)229.6 (User Time)2670.5 (Real Time)778625.3

======[CRAWL BOX]======

ULS

  • Timeout while feeding batch with 1 docs after 60.0s. Attempted 2 times: Could not submit content:operation set timed out
    [documentsubmitterworkerthread.cpp:492] d:\office\source\search\native\gather\plugins\contentpi\documentsubmitterworkerthread.cpp
  • Content Exception after 151.8s: Could not submit content:WinHttpReceiveResponse failed. Url:'http:// [FAST SERVER]:13391/processing::session/5.2/1406916153000000043/process' Error:'12002'
  • Timed out batch. Waiting 300 seconds before retrying [documentsubmitterworkerthread.cpp:521] d:\office\source\search\native\gather\plugins\contentpi\documentsubmitterworkerthread.cpp
Improve this question

1 Answer 1

Reset to default
0

We had a similar issue with FAST and we opened a support case as well. We were indexing 7 TB worth of data totaling around 3 million items. They said that disk IO was our major issue causing the timing out so we had to order new FAST servers. We ended up going with:

  • 96 GB of RAM
  • 2 x 6 Core Xeon CPUs
  • 8 x 300 GB HD - 2 mirrored for OS, and other 6 in a RAID 50 for a little over 1 TB of space for the index

We got two of those to build our FAST farm and once we put them in place we haven't had any timeout issues with FAST.

With the new servers we increased our document processors to 24 and when we did our full indexing over the weekends it would take a little over 24 hours to complete.

Unfortunately, with FAST, it seems like the best answer is to throw more hardware at it.

Improve this answer
3
  • Thanks, Steve. I am afraid you are correct with needing to beef up the hardware -- although your scenario seems a bit extreme for what I think would be considered a small(ish) deployment. Unfortunately, I have to over-justify obtaining more resources, so I had my fingers crossed for an explanation on why this was affecting the incrementals so much.
    – BJenkins
    Jul 23, 2014 at 19:06
  • I would consider our environment medium-large. Even though we don't have lots of items, we work with very large files. These large pdfs and other files were causing very slow indexing and thus the need for larger hardware. I would say it might be a little overkill but it still takes 24 hours to index. We basically wanted to ensure we didn't get any more timeout issues. We plan on re-purposing these as search/application servers for our 2013 environment as well.
    – Steve Lineberry
    Jul 23, 2014 at 19:51
  • You also didn't mention your drive speed for your server specs. That has a HUGE amount to do with it.
    – Steve Lineberry
    Jul 23, 2014 at 19:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.