3

I am inserting some test data into my single server sharepoint farm. I noticed some performance decay after you insert a bunch of items.

Here is the powershell script:

Add-PSSnapin Microsoft.SharePoint.PowerShell 

# Set the variables
$WebURL = "http://win-d9fm7ip9r36/sites/adventureworks-cycles-inc/operati"
$DocLibName = "Shared Documents"

# Get a variable that points to the folder
$Web = Get-SPWeb $WebURL
$List = $Web.GetFolder($DocLibName)
$Files = $List.Files

For ($i=20001; $i -le 200000; $i++) {
    $FilePath = "c:\testdir\File$i.txt"
    echo "file $i" > $FilePath

    # Get just the name of the file from the whole path
    $FileName = $FilePath.Substring($FilePath.LastIndexOf("\")+1)

    # Load the file into a variable
    $File= Get-ChildItem $FilePath

    # Upload it to SharePoint
    $Files.Add($DocLibName +"/" + $FileName,$File.OpenRead(),$false)
}
$web.Dispose()

It runs very fast at first, then starts to slow down. After a couple hours of running, it becomes extremely slow. Only manages to add a 10,000 in a 8 hour period.

The whole box becomes pretty slow after running that script for a while. The virtual machine has 2 cpu's and they are both 100% while inserting list items.

I noticed that if i start inserting into a new list, it gets faster temporarily and then eventually slow again.

So it seems like something is slow about inserting list items into a list with lots of items.

Any idea how I can eliminate this CPU churn?

2

Instead of $Files.Add use the list.AddItem();

The big problem is this object $Files = $List.Files. It means you hold information about all files in the list in memory changing to list.AddItem() should really help

Have a look at this other thread for more insights: Best practice to add list item in SharePoint Server OM?

Another element that might be slowing you down is crawler. If you are adding new content and crawler starts to crawl same library it may significantly impact performance. In case of migration best practice is to pause crawler until you complete migration.

  • None of these helped me. I'm trying to add several hundred thousand list items to document libraries. These did not cover any way to make this faster. – Nicholas DiPiazza Feb 12 at 15:12
  • i added a bunch of clarification to the ticket. please see if that helps – Nicholas DiPiazza Feb 12 at 18:20
  • Well looking at your comment it seems simply your machine doesn't have enough resources. It looks you are executing your script on local farm that has only 2 processors. minimum required for Sharepoint server is 4 so it seems you might be just running out of resources. I updated as well my answer with some search guidance. – Marek Sarad Feb 13 at 8:36
  • your big problem is this object $Files = $List.Files. It means you hold information about all files in the list in memory changing to list.AddItem() should really help. As per previous comment you might be facing some limitation due to VM resources. – Marek Sarad Feb 13 at 8:41
  • yep that's the answer. you should add that into your response. and i'll make it accepted answer. when i removed the $Files = $List.Files its much better now – Nicholas DiPiazza Feb 13 at 17:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.