I'm trying to set up a Sharepoint search service using a Content Source based on an external BCS data source.
In the BDC model, I implemented the operations I needed, adding a SpecificFinder, an IdEnumerator, a ChangedIdEnumerator, and a DeletedIdEnumerator.
When I crawl the source, everything works fine. My BDC methods are called, the crawl log doesn't report any error, and any new/deleted/modified items can be seen in the log viewer.
The problem is, even if some items are marked as deleted in the index, they still show up in search results.
For instance, considering the following sequence of events :
- Full crawl
- Add new item to the external content source
- Incremental crawl (new item shows up in results at that point)
- Remove item from external content source
- Invremental crawl
The deleted item, though marked as deleted during crawling, will keep on showing up in search results forever, as long as I only do incremental crawls.
When I do a full crawl instead, it seems the SpecificFinder for the deleted item is called and the crawler somehow realizes that the content is not available any more, removing it from the index and subsequent searches. So doing a full crawl appears to be a workaround, however I can't afford doing them too often since they take 20 minutes or so to complete.
Has anyone had that problem before ? Is it normal to have to perform full crawls in a production Sharepoint environment on a regular basis to get rid of that kind of issue ?