My organization has been attempting to use FAST Search for Sharepoint 2010 as a search solution for our DMS. I am currently utilizing a .Net indexing connector to index a database of ~ 8 million records and their accompanying files.
Previously, I was having a memory issue on the crawl box, which included the incremental crawls being unable to complete. After months of troubleshooting, that issue seemed to be resolved with a hotfix (http://support.microsoft.com/kb/2601211). The entire issue is illustrated in another thread (http://goo.gl/3CuZUY).
However, on the following incremental, I began to receive new warnings regarding batch time outs in the logs and an atrocious crawl rate (0.1 dps). This has also caused the crawls to be unable to complete, even after running for hundreds of hours. Yet again, I am at a loss.
I have been working with a Microsoft support rep for over two months, but they have been unable to offer assistance for any of our issues. I have also been monitoring the performance counters on both servers for quite some time, and (other than a high 'batches ready' number on the crawl box) I have yet to find a smoking gun. Thus, any suggestion would be greatly appreciated.
Environmental and log information detailed below:
SharePoint Environment
- 1x WFE
- 1x Database Server
- 1x Application Server (32GB RAM)
- 1x Index Server (FAST) (16GB RAM, 8 CPU Cores, 5 Document Processors)
Logs
======[FAST BOX]======
DOCLOG
- WARNING Document conversion failed: External process timeout reached (300 seconds) (warning code 0)
- INFO Processor "IFilterConverter" ran for 300s
PSCtrl Statistics
- IFilterConverter (System Time)229.6 (User Time)2670.5 (Real Time)778625.3
======[CRAWL BOX]======
ULS
- Timeout while feeding batch with 1 docs after 60.0s. Attempted 2 times: Could not submit content:operation set timed out
[documentsubmitterworkerthread.cpp:492] d:\office\source\search\native\gather\plugins\contentpi\documentsubmitterworkerthread.cpp - Content Exception after 151.8s: Could not submit content:WinHttpReceiveResponse failed. Url:'http:// [FAST SERVER]:13391/processing::session/5.2/1406916153000000043/process' Error:'12002'
- Timed out batch. Waiting 300 seconds before retrying [documentsubmitterworkerthread.cpp:521] d:\office\source\search\native\gather\plugins\contentpi\documentsubmitterworkerthread.cpp