Duplicates being created in SharePoint Record Centres

Question

Just wondering if someone can point me in the right direction in relation to a problem I have been grappling with for about 12 months. We have a SharePoint 2010 system. We have set up numerous team sites along with 10 record centre sites. On a weekly basis (weekend) we transfer documents that have not been modified in 2 years to one of the 10 record centres using site collection information policies. The whole process works well except for the fact that for about 12000 documents in a system of almost 2 million total records, duplicates are created each week in Record Centre. In an effort to trouble shoot the problem I disabled all information policy, expiration and content organiser timer jobs in central admin to stop the weekly transfer of records. To my amazement, although the record transfer did not occur that week, i.e. no new documents were created in record centre, those documents that were known to duplicate in the past continued to duplicate. Interestingly the duplicates get created at approximately the sametime they would have been created had I not disabled the timer jobs. I repeated this several times over the last 12 months with the exact same results. The transfer doesn't occur but the same 12000 or so duplicates get created without fail.

I ran an SQL script against the AuditData table for one of the record Centre databases in question and noted that duplicates that were created appeared to be the result of a record submission: the EventData column specified that the records had been created as the result of a Records Respository Submission. What is puzzling is that the documents that apparently were submitted as duplicates no longer exist in the team sites they supposedly came from (I checked AllDocs, AllDocStreams and the RecylingBin to make sure none of them existed) and have not existed for about 5 years. This raises the question where are the documents coming from? The only thing I can think of is that somehow the event receivers in the record Centre libraries are playing up or have become corrupted. But if that is the case what triggers them to misbehave at almost the exact time a normal transfer would have occurred regardless of whether the timer jobs are disabled or enabled.

I restored one of the record centre databases to another environment to see if I could replicate the problem there but the behaviour did not repeat. This would suggest the event receivers might not be the issue. I was wondering if anyone has come across this kind of behaviour before or has suggestions on where I should look next as I have completely run out of ideas.

Answer 1

I have an answer to my own question on the mysterious issue of document duplicates being created in our Record Centres. In a nutshell, the documents which appeared to come out of thin air were coming from another SharePoint environment. The other evironment had been a long forgotten restore of our Production system made 5 years earlier. The restored environment served as a test and therefore did not have its record centres configured. It did, however, have have fully functioning information management policies and record centre timer jobs. This meant a weekly transfer of documents was being made to the Production farm record centres.

The reason why the same duplicates were being made each week was because communication between the Restore and Production systems appeared to be one way only. This meant although documents in the Restore system got moved to Production, the normal process of replacing the original document with a record stub in the Restore system (Move and Leave a Link) never occurred. This meant the following week, the exact same documents were still up for transfer and thus becoming an endless loop. On the other hand, where Move only information policies had been implemented, only a single duplicate was created due to the fact the transferred document was deleted from the source.

So how did I identify that duplicates were coming from an external system? I disabled a content organiser rule for one of the content types associated with one of the duplicates. This resulted in the document getting trapped in a record Centre Drop Off Library. It was then a matter of examining SP properties of the document and identifying that the original URL of the document reflected a different source system than the Production system. I should point out the critical property that identifies the document source (where it came from) is _vti_RoutingExistingProperties. This property is cleared when the document is routed to its final destination and this is why the problem was so hard to troubleshoot.

The moral of the story might be: if you restore a SharePoint Production system delete the system as soon possible once it has served its purpose. Alternatively disable any processes or transfers that should not run.