Does it take longer to move a file into a fuller document library?

Question

I'm seeing a pattern that appears to be that it takes longer to move files into a document library that has a lot of content.

We have a custom timer job that takes documents from various libraries on a particular site, renames them, and moves them to the Drop Off Library so that the Content Organizer can route them to a different site. We have a large initial amount of data that we are uploading to this site via web services. As we watched our custom timer job progress, it seemed to be taking a very long time to complete the process.

We have extensive logging, so I cracked open some log files to see what was going on. What I saw was that when the timer job started (and the Drop Of Library was empty), it would take one to two seconds per file to move it to the Drop Off Library. Much later (days later), when the Drop Off Library had over 12K files in it, each new file being moved there was taking 20-30 seconds.

Has anyone else seen this kind of behavior? Can anyone explain what might be happening?

---

As it turned out, it was because of a bug in a third-party event receiver that did in fact iterate over all of the items in the library.

Answer 1

A lot depends on the implementation of your custom timer job. If it is not programmed and tested property, then such slowdowns will occur.

I would recommend reviewing the source code to ensure that it follows Microsoft Best Practices for SharePoint development.

For example the following line of code runs very slow and should be avoided, as it enumerates all items of the list to count them:

list.Items.Count

while this property only gets the count, which is much more effective:

list.ItemCount

When reviewing the code, start with this MSDN article on the Best Practices when handling large lists: http://msdn.microsoft.com/en-us/library/office/ee557257.aspx