Exception: The search request was unable to execute on FAST Search Server

Question

I evaluate an application which will use search technology. In our SharePoint 2010 farm we use FAST Search for SharePoint 2010 as our search engine.

There are 1424 results for following query:

MyManagedProperty:"SomeText*"

The application uses pagination. At index 1413 the search query results in an error. It does not matter whether the query was sent over the SOAP operations Query or QueryEx. I also observe the same behavior within the FAST Search Center site collection (UI) when I paginate far enough.

I have debugged the Query Result Server with FS4SP Query Logger. A search with the range

<Range>
  <StartAt>1411</StartAt>
  <Count>10</Count>
</Range>

has the following XML

<?xml version="1.0" encoding="utf-8" ?> 
<SEGMENTS>
  <SEGMENT NAME="webcluster">
    <RESULTPAGE>
      <!-- omitted other tags for readability -->
      <RESULTSET FIRSTHIT="1411" LASTHIT="1420" HITS="2" TOTALHITS="1424"
                 MAXRANK="4927" TIME="0.0000">
        <HIT NO="1411" RANK="708" SITEID="1102844735582508041" MOREHITS="0" FCOCOUNT="0">
          <!-- omitted the FIELD tags for readability -->
        </HIT>
        <HIT NO="1412" RANK="703" SITEID="213822451840282734" MOREHITS="0" FCOCOUNT="0">
          <!-- omitted the FIELD tags for readability -->
        </HIT>
      </RESULTSET>
      <PAGENAVIGATION>
        <PREVPAGE FIRSTHIT="1401" LASTHIT="1410" URL="..." /> 
        <NEXTPAGE FIRSTHIT="1421" LASTHIT="1424" URL="..." /> 
      </PAGENAVIGATION>
    </RESULTPAGE>
  </SEGMENT>
</SEGMENTS>

A search with the range

<Range>
  <StartAt>1421</StartAt>
  <Count>10</Count>
</Range>

has the following XML

<?xml version="1.0" encoding="utf-8" ?> 
<SEGMENTS>
  <SEGMENT NAME="webcluster">
    <RESULTPAGE>
      <!-- omitted other tags for readability -->
      <EMPTYRESULTSET /> 
      <PAGENAVIGATION>
        <PREVPAGE FIRSTHIT="1411" LASTHIT="1420" URL="..." /> 
      </PAGENAVIGATION>
    </RESULTPAGE>
  </SEGMENT>
</SEGMENTS>

For both ranges a bunch of exceptions are logged into the ULS.

Exception #1

Exception caught in QueryService class.
Exception message: The search request was unable to execute on FAST Search Server..
Stack: at Microsoft.Office.Server.Search.Administration.SearchServiceApplicationProxy.ThrowBackwardCompatibleException(FaultException`1 ex)
at Microsoft.Office.Server.Search.Administration.SearchServiceApplicationProxy.DoSpLoadBalancedUriWsOp[T](WebServiceBackedOperation`1 webServiceCall, Int32 timeoutInMilliseconds, Int32 wcfTimeoutInMilliseconds, String operationName)
at Microsoft.Office.Server.Search.Administration.SearchServiceApplicationProxy.DoWebServiceBackedOperation[T](String operationName, Int32 timeoutInMilliseconds, Int32 wcfTimeoutInMilliseconds, WebServiceBackedOperation`1 webServiceCall)
at Microsoft.Office.Server.Search.Administration.SearchServiceApplicationProxy.Execute(QueryProperties properties)
at Microsoft.Office.Server.Search.Query.Query.Execute()
at Microsoft.Office.Server.Search.Query.QueryService.DoQuery(QueryMethod queryMethod, String queryXml, String& domain, String& queryId, Int32& startAt, Boolean& fStandardResults, StringCollection& querySuggestions)
at Microsoft.Office.Server.Search.Query.QueryService.QueryEx(String queryXml).

Exception #2

AbstractSearchGateway::Search--Exception when searching for MyManagedProperty:"SomeText*".
Ex: System.InvalidOperationException: Search operations are not allowed once the search engine is disposed.
at Microsoft.SharePoint.Search.Extended.Query.Result.DocSumPool.Get(Int32 index)
at Microsoft.SharePoint.Search.Extended.Query.Result.QueryResult.GetDocument(Int32 index)
at Microsoft.Office.Server.Search.Query.Gateway.FastResultExtractor.ExtractItemsAsDataTable(IQueryResult result, Int32 from, Int32 to, SPUrlZone urlZone, StringCollection alternateAccessMapProperties, StringCollection selectProperties, Boolean excludeBodyProperty)
at Microsoft.Office.Server.Search.Query.Gateway.FastResultExtractor.ExtractResults(IQueryResult queryResult, SearchRequest searchRequest, Boolean excludeBodyProperty)
at Microsoft.Office.Server.Search.Query.Gateway.FastSearchGateway.ExecuteSearch(SearchRequest request, String queryString)
at Microsoft.Office.Server.Search.Query.Gateway.AbstractSearchGateway.Search(SearchRequest request)

Exception #3

FASTQueryInternal::Execute--Unexpected exception System.InvalidOperationException trying to search for 'MyManagedProperty:"SomeText*"' with FAST backend: Search operations are not allowed once the search engine is disposed.
System.InvalidOperationException: Search operations are not allowed once the search engine is disposed.
at Microsoft.SharePoint.Search.Extended.Query.Result.DocSumPool.Get(Int32 index)
at Microsoft.SharePoint.Search.Extended.Query.Result.QueryResult.GetDocument(Int32 index)
at Microsoft.Office.Server.Search.Query.Gateway.FastResultExtractor.ExtractItemsAsDataTable(IQueryResult result, Int32 from, Int32 to, SPUrlZone urlZone, StringCollection alternateAccessMapProperties, StringCollection selectProperties, Boolean excludeBodyProperty)
at Microsoft.Office.Server.Search.Query.Gateway.FastResultExtractor.ExtractResults(IQueryResult queryResult, SearchRequest searchRequest, Boolean excludeBodyProperty)
at Microsoft.Office.Server.Search.Query.Gateway.FastSearchGateway.ExecuteSearch(SearchRequest request, String queryString)
at Microsoft.Office.Server.Search.Query.Gateway.AbstractSearchGateway.Search(SearchRequest request)
at Microsoft.Office.Server.Search.Query.FASTQueryInternal.ExecuteSearch(SearchRequest request, ResultTableCollection rtc)
at Microsoft.Office.Server.Search.Query.FASTQueryInternal.Execute(QueryProperties properties)

If a query with a range starting at 1425 is executed then an XML with <EMPTYRESULTSET /> on the QR Server is returned and no exceptions are logged in the ULS.

Why does FAST Search ommit the hits 1413 through 1424 when there are 1424 total hits?

Why in the first XML is the first hit 1411 and the last hit 1420 when there are only 2 hits? I would expect either

<RESULTSET FIRSTHIT="1411" LASTHIT="1420" HITS="10" TOTALHITS="1424" ...>

with 10 <HIT> elements or

<RESULTSET FIRSTHIT="1411" LASTHIT="1412" HITS="2" TOTALHITS="1424" ...>

Does anyone knows where the error comes from and how to solve it?

UPDATE

Resetted the index on the FAST Search Content SSA and deleted the collection on the FAST Server afterwards. Built up the index again. Did not changed anything.

Answer 1

After opening a support case the engineers figured out that this happens because of design limitations. If there are more than 1000 results and TrimDuplicates is turned on (which by default is) then the total hit count is not reliable.

I understand this as follows. There are 1345 results. If we do not count the duplicates then there are 1286 results. FAST Search tells that there are 1310 results. This happens because of the combination of more than 1000 results and duplicate trimming. So a query for the next 10 hits which starts at 1281 will result in the error. There is no result summary with hit number 1287.

The one supported workaround is to turn off duplicate trimming in the query.

<TrimDuplicates>false</TrimDuplicates>

You can experience the same behaviour in the UI, too. The workaround here is to uncheck the Remove Duplicate Results option in the Search Core Results Webpart settings. Maybe there is be a bug. If Remove Duplicate Results is already turned off you have to turn it on, save, and turn it off again.