Moving large document libraries to new site collections

Question

I have a site collection with a dozen or so document libraries in it (total size about 300GB's). The goal is to move each of these document libraries into it's own site collection (and content database).

I've seen posts like Exporting sites to a new content database (talks about moving whole sites) and Restructuring a Very Large Document Library (restructuring a large document library). I found an article online that specifically adresses this issue, http://pravahaminfo.blogspot.com/2011/05/how-to-move-document-library-from-one.html, but the method of exporting the list as a template with content and then importing it on the new site only works up to 10 MB's (500 Mb's max if you change it) and my smallest document library is about 5GB's.

Right now I am considering exporting the list as a template WITHOUT content, importing it, and then using code to move the files. I looked into using Explorer view to move the files but the document libraries I have with over 20,000 folders in the root won't open in Explorer view...

Suggestions on how I should move forward? I'm currently looking for a method that will work in 2007 BUT we are migrating to 2010 in the near future so if the time/headache savings are big enough I will wait.

Answer 1

I am going to provide an answer that will work in 2010. I will need to depend on others who are more versed in MOSS for that; however, I believe there is a similar stsadm command for the Powershell I will provide below.

The Export-SPWeb command is able to be defined for a site, list or library. I have used this extensively to move a troubling list from production to dev/test so that I may work on a specific criteria.

Here is the TechNet Article

An example:

Add-PSSnapin Microsoft.SharePoint.PowerShell
$URL = "http://iwanttotest.com/MyManagedPath/SiteName"
$DocLibrary = "http://iwanttotest.com/apps/site/DocLibraryName"
Export-SPWeb -Identity $URL -Path C:\SiteBackup.cmp -ItemURL $DocLibrary -IncludeUserSecurity -IncludeVersions

Please note this must be executed on a server within the SP Farm so you may add the Powershell snapin.