Take the 2-minute tour ×
SharePoint Stack Exchange is a question and answer site for SharePoint enthusiasts. It's 100% free, no registration required.

I am writing a custom indexing connector using changelog incremental crawl approach.

The full crawl is working perfectly. All atems are indexed and queryable.

The problem is that incremental crawl is not working: crawler starts full crawl and ChangedIdEnumerator and DeletedIdEnumerator methods are not called.

I have tried the Timestamp-Based Incremental Crawl with my connector and it works perfect. But we need changelog incremental crawl because where are millions of items in the external system and they are changing.

I have tried changelog crawl with .NET connector and it works wonderful. It is described very good in article http://msdn.microsoft.com/en-us/library/gg512147.aspx but only for for .NET connectors .

But we need custom connector because our external system doesn't have static interfaces.

............ I will try to describe my connector.

  1. Modell has next stereotypes: SpecificFinder, Finder, ChangedIdEnumerator, DeletedIdEneumerator, StreamAccessor ... ProductConnectorCustom.ProductModelCustom.ConnectorLobUri, ProductConnectorCustom, Version=1.0.0.0, Culture=neutral, PublicKeyToken=a8b3943b0419acb9 ProductConnectorCustom.ProductModelCustom.ConnectorNamingContainer, ProductConnectorCustom, Version=1.0.0.0, Culture=neutral, PublicKeyToken=a8b3943b0419acb9 ProductConnectorCustom.ProductModelCustom.BdcConnector, ProductConnectorCustom, Version=1.0.0.0, Culture=neutral, PublicKeyToken=a8b3943b0419acb9 Name

        <!--        SpecificFinder          -->
        <Method Name="ReadItem" IsStatic="false">
          <Parameters>
            <Parameter Direction="In" Name="id">
              <TypeDescriptor TypeName="System.String" IdentifierName="ID" Name="ID" />
            </Parameter>
            <Parameter Direction="Return" Name="returnParameter">
              <TypeDescriptor TypeName="ProductConnectorCustom.ProductModelCustom.Product, ProductConnectorCustom, Version=1.0.0.0, Culture=neutral, PublicKeyToken=a8b3943b0419acb9" Name="Product">
                <TypeDescriptors>
                  <TypeDescriptor TypeName="System.String" IdentifierName="ID" Name="ID" />
                  <TypeDescriptor TypeName="System.String" Name="Manufacturer" />
                  <TypeDescriptor Name="Name" TypeName="System.String" />
                </TypeDescriptors>
              </TypeDescriptor>
            </Parameter>
          </Parameters>
          <MethodInstances>
            <MethodInstance Type="SpecificFinder" ReturnParameterName="returnParameter" Default="true" Name="ReadItem" DefaultDisplayName="Read Product">
            </MethodInstance>
          </MethodInstances>
        </Method>
    
        <!--        AssociationNavigator          -->          
        <Method Name="GetProductsList" LobName="GetProductsList" IsStatic="false">
          <Parameters>
            <Parameter Name="productsList" Direction="In">
              <TypeDescriptor Name="productsList" TypeName="System.String" IdentifierEntityNamespace="ProductConnectorCustom.ProductModelCustom" IdentifierEntityName="Product" IdentifierName="ID" ForeignIdentifierAssociationName="GetProductsListInstance" ForeignIdentifierAssociationEntityName="Product" ForeignIdentifierAssociationEntityNamespace="ProductConnectorCustom.ProductModelCustom"/>
            </Parameter>
            <Parameter Name="Return" Direction="Return">
              <TypeDescriptor Name="Return" TypeName="ProductConnectorCustom.ProductModelCustom.Product[], ProductConnectorCustom, Version=1.0.0.0, Culture=neutral, PublicKeyToken=a8b3943b0419acb9" IsCollection="true">
                <TypeDescriptors>
                  <TypeDescriptor Name="Item" TypeName="ProductConnectorCustom.ProductModelCustom.Product, ProductConnectorCustom, Version=1.0.0.0, Culture=neutral, PublicKeyToken=a8b3943b0419acb9">
                    <TypeDescriptors>
                      <TypeDescriptor Name="ID" TypeName="System.String" IdentifierEntityNamespace="ProductConnectorCustom.ProductModelCustom" IdentifierEntityName="Product" IdentifierName="ID" />
                    </TypeDescriptors>
                  </TypeDescriptor>
                </TypeDescriptors>
              </TypeDescriptor>
            </Parameter>
          </Parameters>
          <MethodInstances>
            <Association Name="GetProductsListInstance" Type="AssociationNavigator" ReturnParameterName="Return" ReturnTypeDescriptorName="Return">
              <Properties>
                <Property Name="DirectoryLink" Type="System.String">x</Property>
              </Properties>
              <SourceEntity Name="Product" Namespace="ProductConnectorCustom.ProductModelCustom" />
              <DestinationEntity Name="Product" Namespace="ProductConnectorCustom.ProductModelCustom" />
            </Association>
          </MethodInstances>
        </Method>           
    
        <!--        Finder          -->
        <Method Name="ReadList" IsStatic="false">
          <Parameters>
            <Parameter Direction="Return" Name="returnParameter">
              <TypeDescriptor TypeName="System.Collections.Generic.IEnumerable`1[[ProductConnectorCustom.ProductModelCustom.Product, ProductConnectorCustom, Version=1.0.0.0, Culture=neutral, PublicKeyToken=a8b3943b0419acb9]]" IsCollection="true" Name="ProductList">
                <TypeDescriptors>
                  <TypeDescriptor Name="Product" TypeName="ProductConnectorCustom.ProductModelCustom.Product, ProductConnectorCustom, Version=1.0.0.0, Culture=neutral, PublicKeyToken=a8b3943b0419acb9">
                    <TypeDescriptors>
                      <TypeDescriptor Name="ID" TypeName="System.String" IdentifierName="ID" />
                      <TypeDescriptor Name="Manufacturer" TypeName="System.String" />
                      <TypeDescriptor Name="Name" TypeName="System.String" />
                    </TypeDescriptors>
                  </TypeDescriptor>
                </TypeDescriptors>
              </TypeDescriptor>
            </Parameter>
          </Parameters>
          <MethodInstances>
              <MethodInstance Type="Finder" ReturnParameterName="returnParameter" Default="true" Name="ReadList" DefaultDisplayName="Product List"></MethodInstance>
          </MethodInstances>
        </Method>
    
        <!--        ChangedIdEnumerator          -->
        <Method Name="ReadChangedIds" IsStatic="false">
          <Properties>
            <Property Name="LastRunDate" Type="System.DateTime">2013-01-01 00:00:00Z</Property>
          </Properties>
          <FilterDescriptors>
            <FilterDescriptor Name="LastRunDate" Type="InputOutput">
              <Properties>
                <Property Name="SynchronizationCookie" Type="System.String">ChangedItemCookie</Property>
              </Properties>
            </FilterDescriptor>
            <FilterDescriptor Name="Timestamp" Type="Timestamp" />
          </FilterDescriptors>
          <Parameters>
            <Parameter Name="LastRunDate" Direction="InOut">
              <TypeDescriptor Name="LastRunDateTypeDescriptor" TypeName="System.DateTime" IsCollection="false" AssociatedFilter="LastRunDate">
                <Interpretation>
                  <NormalizeDateTime LobDateTimeMode="Local" />
                </Interpretation>
              </TypeDescriptor>
            </Parameter>
            <Parameter Name="returnIds" Direction="Return">
              <TypeDescriptor Name="DeletedIds" TypeName="System.Collections.Generic.IEnumerable`1[System.String]" IsCollection="true">
                <TypeDescriptors>
                  <TypeDescriptor Name="ID" TypeName="System.String" IdentifierName="ID" />
                </TypeDescriptors>
              </TypeDescriptor>
            </Parameter>
          </Parameters>
          <MethodInstances>
            <MethodInstance Name="ReadChangedIds" Type="ChangedIdEnumerator" Default="true" ReturnParameterName="returnIds"></MethodInstance>
          </MethodInstances>
        </Method>
    
  2. Code

Implemented are the SystemUtility based on StructuredRepositorySystemUtility, NamingContainer implements INamingContainer, CustomLobUri class derives from LobUri.

............

I have tried all things but it doesn't work!

Have somebody ever tried to write something like that?

share|improve this question
add comment

3 Answers

I also used changelog incremental crawl approach and it worked for me.

Here's an example:

<Method Name="ReadIncrementalList" IsStatic="false">
                        <FilterDescriptors>
                            <FilterDescriptor Name="LastCrawl" Type="InputOutput">
                                <Properties>
                                    <Property Name="SynchronizationCookie" Type="System.String">ChangedItemCookie</Property>
                                </Properties>
                            </FilterDescriptor>
                            <FilterDescriptor Name="timestamp" Type="Timestamp" />
                        </FilterDescriptors>
                        <Parameters>
                            <Parameter Name="lastCrawlDate" Direction="InOut">
                                <TypeDescriptor Name="LastCrawlDate" TypeName="System.DateTime" IsCollection="false" AssociatedFilter="LastCrawl">
                                    <Interpretation>
                                        <NormalizeDateTime LobDateTimeMode="Local" />
                                    </Interpretation>
                                </TypeDescriptor>
                            </Parameter>
                            <Parameter Name="returnIds" Direction="Return">
                                <TypeDescriptor Name="returnIds" TypeName="System.Collections.Generic.IEnumerable`1[System.String]" IsCollection="true">
                                    <TypeDescriptors>
                                        <TypeDescriptor Name="id" TypeName="System.String" IdentifierName="IncId" />
                                    </TypeDescriptors>
                                </TypeDescriptor>
                            </Parameter>
                        </Parameters>
                        <MethodInstances>
                            <MethodInstance Name="ReadIncrementalListInstance" Type="ChangedIdEnumerator" ReturnParameterName="returnIds">

                            </MethodInstance>
                        </MethodInstances>
                    </Method>
                    <Method Name="GetDeletedIds" IsStatic="false">
                        <FilterDescriptors>
                            <FilterDescriptor Name="LastCrawl" Type="InputOutput">
                                <Properties>
                                    <Property Name="SynchronizationCookie" Type="System.String">DeletedItemCookie</Property>
                                </Properties>
                            </FilterDescriptor>
                            <FilterDescriptor Name="timestamp" Type="Timestamp" />
                        </FilterDescriptors>
                        <Parameters>
                            <Parameter Name="lastCrawlDate" Direction="InOut">
                                <TypeDescriptor Name="LastCrawlDate" TypeName="System.DateTime" IsCollection="false" AssociatedFilter="LastCrawl">
                                    <Interpretation>
                                        <NormalizeDateTime LobDateTimeMode="Local" />
                                    </Interpretation>
                                </TypeDescriptor>
                            </Parameter>
                            <Parameter Name="DeletedIds" Direction="Return">
                                <TypeDescriptor Name="DeletedIds" TypeName="System.Collections.Generic.IEnumerable`1[System.String]" IsCollection="true">
                                    <TypeDescriptors>
                                        <TypeDescriptor Name="Identifier1" TypeName="System.String" IdentifierName="IncId" />
                                    </TypeDescriptors>
                                </TypeDescriptor>
                            </Parameter>
                        </Parameters>
                        <MethodInstances>
                            <MethodInstance Name="GetDeletedIdsInstance" Type="DeletedIdEnumerator" ReturnParameterName="DeletedIds">

                            </MethodInstance>
                        </MethodInstances>
                    </Method>
share|improve this answer
    
Hi Alex! Thank you for your answer. Do you use custom and not .Net connector? I have compared yours and mine models and I can't found differences. –  Mitka May 13 '13 at 12:31
    
If crawl starts, it calls first SpecificFinder. I have defined AssociationNavigator to forward this call to Finder. Do you use something like this? May be problem is here? Thank you, Dmitry –  Mitka May 13 '13 at 12:46
    
Although the article doesn't have your specific scenario, it explains step by step how to achieve parent/child relationship using Visual Studio BCS solution: blog.johnnyt.me/2013/03/crawling-with-fast-and-sharepoint-2013 –  Alex Choroshin May 14 '13 at 18:16
add comment

I have fond solution for the problem:

The identity parameter should not be set in the Initialize method in the class derived from LobUri. I'm getting ChangedIdEnumerator and DeletedIdEnumerator called now.

share|improve this answer
add comment

I think the main problem (at least for SharePoint 2013, is that after the IdEnumerator, ChangedIdEnumerator and DeletedIdEnumerator, the Specific finder is never invoked.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.