A duplicate is created when the workflow fails to complete correctly. Often the workflow created the Doc Set and one or two sub-folders (it should create 5 sub-folders). I checked when duplicates are created, it seems that SP tries to run the incomplete workflows again at specific intervals. The same pattern of times is seen throughout the duplicates. Perhaps this suggests there is insufficient system resource available to complete the workflow at that time - Doc Set creation is demanding on SP System resource?.
There is an external data source, coming from a separate financial system, which is hooked up to a SP list (Suppliers List). A timer job adds new items to the SP Suppliers List when new suppliers are added in the Financial system. The Doc Set creation workflow is triggered when the Suppliers List has a new item>Create Doc Set+sub-folders in the 'Quality Suppliers' List.
List A. Suppliers-Financial System (external data list - 'new items imported')
List B. Suppliers List (Native Sharepoint list)
List C. Quality Suppliers List (Document Set List - for storing 4 classes of documentation)
Current Process Overview
Financial System (new item added) > this instantly updates
List A(new item created by Windows Task Scheduler running a Powershell script).
The same .ps script in
List Atriggers new item creation in
When a new item is created in
List Bit triggers a create item SP workflow in
List C> duplicates are made as the workflow fails to fully complete.
Sometimes the workflow fails to complete the first time (as described above).
Change the start time of the Doc Set creation workflow? Maybe make it initiate overnight instead. 'Inserting a workflow pause' will not work.
Or perhaps trigger Doc Set creation by a Timer Job instead? Since I don't need the Doc Sets to be created instantly.
Or, going right back to the start of the chain of lists, modify the start time of the chain in
List A. Change the timer job so it does not run instantly and instead runs overnight where available system resource should not be a constraint.