2

I need to migrate the documents of an existing library to a new library. The requirements of the migration is that I maintain versions and audit data (who/when created the document and last modified it).

The Export-SPWeb and Import-SPWeb cmdlets only seem to work on a list level or higher. This means that the old list gets created on import and includes custom views and what-not. But I want only the items to be migrated.

It is also my understanding that those cmdlets are ultimately using classes and methods in the Microsoft.SharePoint.Deployment namespace.

Does anybody know how to do this?

Update

I have been researching the Content Migration API and here is what I have found.

Exporting Just List Items

$ver = $host | select version
if($Ver.version.major -gt 1) {$Host.Runspace.ThreadOptions = "ReuseThread"}
if(!(Get-PSSnapin Microsoft.SharePoint.PowerShell -ea 0))
{
  Write-Progress -Activity "Loading Modules" -Status "Loading Microsoft.SharePoint.PowerShell"
  Add-PSSnapin Microsoft.SharePoint.PowerShell
}

[string]$sourceSiteUrl = "http://some.server.com"
[string]$sourceListName = "SomeList"

$fileLocation = $pwd


$site = New-Object Microsoft.SharePoint.SPSite($sourceSiteUrl)
$web = $site.OpenWeb()
$list = $web.Lists[$sourceListName]

$settings = New-Object Microsoft.SharePoint.Deployment.SPExportSettings
$settings.BaseFileName = "test.cmp"
$settings.CommandLineVerbose = $true
$settings.ExportMethod = [Microsoft.SharePoint.Deployment.SPExportMethodType]::ExportAll
$settings.FileCompression = $true
$settings.FileMaxSize = [Int32]::MaxValue
$settings.IncludeSecurity = [Microsoft.SharePoint.Deployment.SPIncludeSecurity]::All
$settings.IncludeVersions = [Microsoft.SharePoint.Deployment.SPIncludeVersions]::CurrentVersion
$settings.LogFilePath = "$pwd\log.txt"
$settings.OverwriteExistingDataFile = $true
$settings.SiteUrl = $sourceSiteUrl
$settings.FileLocation = $fileLocation

ForEach ($i In $list.Items) {
  $eo = New-Object Microsoft.SharePoint.Deployment.SPExportObject
  $eo.Id = $i.UniqueId
  $eo.Type = [Microsoft.SharePoint.Deployment.SPDeploymentObjectType]::ListItem

  $settings.ExportObjects.Add($eo)
}

$settings.Validate()

$export = New-Object Microsoft.SharePoint.Deployment.SPExport($settings)
$export.Run()

This script will export all items (including documents) from the specified list. The key is adding each item to the ExportObjects collection of the SPExportSettings object.

Importing Just List Items

$ver = $host | select version
if($Ver.version.major -gt 1) {$Host.Runspace.ThreadOptions = "ReuseThread"}
if(!(Get-PSSnapin Microsoft.SharePoint.PowerShell -ea 0))
{
  Write-Progress -Activity "Loading Modules" -Status "Loading Microsoft.SharePoint.PowerShell"
  Add-PSSnapin Microsoft.SharePoint.PowerShell
}

[string]$destSiteUrl = "http://some.server.com"    # or http://some.other.server.com
[string]$destListName = "SomeOtherList"

$fileLocation = $pwd

$site = New-Object Microsoft.SharePoint.SPSite($destSiteUrl)
$web = $site.OpenWeb()
$list = $web.Lists[$destListName]

$settings = New-Object Microsoft.SharePoint.Deployment.SPImportSettings
$settings.BaseFileName = "test.cmp"
$settings.CommandLineVerbose = $true
$settings.FileCompression = $true
$settings.LogFilePath = "$pwd\in-log.txt"
$settings.UserInfoDateTime = [Microsoft.SharePoint.Deployment.SPImportUserInfoDateTimeOption]::ImportAll
$settings.SiteUrl = $destSiteUrl
$settings.WebUrl = $destSiteUrl
$settings.FileLocation = $fileLocation
$settings.RetainObjectIdentity = $false

$settings.Validate()

$import = New-Object Microsoft.SharePoint.Deployment.SPImport($settings)

$oe = Register-ObjectEvent -InputObject $import -EventName Started -SourceIdentifier ImportStarted -Action {
  $ro = $eventArgs.RootObjects
  ForEach ($io in $ro) {
    if ($io.Type -eq [Microsoft.SharePoint.Deployment.PDeploymentObjectType]::ListItem) {
      $io.TargetParentUrl = $list.RootFolder.ServerRelativeUrl
    }
  }
}

$import.Run()

Get-EventSubscriber | Unregister-Event

This SHOULD perform the import on just the items but is not working correctly.

The problem seems to be around the Register-ObjectEvent code. When an object gets imported into SharePoint, the object needs to have a parent url. Because I want to import into a list other than the source list, I need to re-parent the list items. According to the document, I should be using the Started event of the SPImport object to find the "orphaned" objects and point them to the desired list. For some reason, the Started event is not being raised or if it is, PS is not executing my handler code.

  • This doesn't directly answer your question because this script does not preserve the features you requested, but the author indicates he believes it may be possible to enhance the script to do so. I'm providing in in case it forms a starting point for you though. blogs.msdn.com/b/rcormier/archive/2012/11/16/… Hope it's of some help. – user24313 Apr 4 '14 at 19:11
  • @user24313 - I actually looked at that blog posting (among many others) before I posted this question. I even tried modifying it to retain user and timestamp data but to no avail. I am in the middle of reading up on and working with the classes in the Content Migration Object Model and will post the solution when I am done. – Jason Apr 4 '14 at 19:30
  • Extracting and importing the files with PowerShell should be straight forward. Although what you can do is to use Export-SPWeb, extract the .cmp-file (7zip works), then add the .dat-files to wherever you want (their metadata is stored in Metadata.xml – eirikb Apr 4 '14 at 19:37
  • @eirikb - As I stated, the normal export/import for SP does not work. It does not allow you to export just the items. The lowest level you can go is the list itself, plus views, which means the list gets re-created on import. I just want the items so I can place them into a new list. – Jason Apr 4 '14 at 20:28
  • @Jason, what I meant by extracting the .cmp-file was to use PowerShell to manually add the stuff you want from the XML file. Although if you are doing manual work you might as well just Export-csv the item data and save the files, although the cmp-file is kind of this, as it has all the data – eirikb Apr 4 '14 at 20:35
2

Because I could not get an answer to the event handling in PowerShell, the posted Import PowerShell script does not work. As a solution, I merged the two scripts and ported them to C#. Here is the code.

using System;
using Microsoft.SharePoint;

namespace MigrateArticles {

   public class Program {

      #region Shared Constants

      protected static readonly string LogFile = System.IO.Path.Combine(Properties.Settings.Default.WorkFolder, "migration-log.txt");

      #endregion

      #region Private Methods

      protected static void ExportContent() {
         try {
            // clean up file location and re-create it
            if (System.IO.Directory.Exists(Properties.Settings.Default.WorkFolder)) {
               System.IO.Directory.Delete(Properties.Settings.Default.WorkFolder, true);
            }

            System.IO.Directory.CreateDirectory(Properties.Settings.Default.WorkFolder);

            var site = new Microsoft.SharePoint.SPSite(Properties.Settings.Default.SourceSiteUrl);
            var web = site.OpenWeb();
            var list = web.Lists[Properties.Settings.Default.SourceListName];

            // define the export settings
            var settings = new Microsoft.SharePoint.Deployment.SPExportSettings();
            settings.BaseFileName = Properties.Settings.Default.BaseFileName;
            settings.CommandLineVerbose = true;
            settings.ExportMethod = Microsoft.SharePoint.Deployment.SPExportMethodType.ExportAll;
            settings.FileCompression = false;
            settings.FileMaxSize = Int32.MaxValue;
            settings.IncludeSecurity = Microsoft.SharePoint.Deployment.SPIncludeSecurity.All;
            settings.IncludeVersions = Microsoft.SharePoint.Deployment.SPIncludeVersions.All;
            settings.LogFilePath = LogFile;
            settings.OverwriteExistingDataFile = true;
            settings.SiteUrl = Properties.Settings.Default.SourceSiteUrl;
            settings.FileLocation = Properties.Settings.Default.WorkFolder;

            // by default the entire site will be exported...
            //   loop through and specifically add each of the list items so only those get exported
            foreach (Microsoft.SharePoint.SPListItem i in list.Items) {
               var eo = new Microsoft.SharePoint.Deployment.SPExportObject();
               eo.Id = i.UniqueId;
               eo.Type = Microsoft.SharePoint.Deployment.SPDeploymentObjectType.ListItem;

               settings.ExportObjects.Add(eo);
            }

            // validate the settings, set up the export, and kick off the job
            settings.Validate();

            var export = new Microsoft.SharePoint.Deployment.SPExport(settings);

            export.Run();
         }
         catch (System.IO.FileNotFoundException ex) {
            Console.WriteLine();
            Console.WriteLine("The source site '{0}' was not found. Aborting migration.", Properties.Settings.Default.SourceSiteUrl);
            System.Environment.Exit(1);
         }
         catch (System.ArgumentNullException ex) {
            Console.WriteLine();
            Console.WriteLine("The source list '{0}' was not found. Aborting migration.", Properties.Settings.Default.SourceListName);
            System.Environment.Exit(1);
         }
         catch (System.Exception ex) {
            Console.WriteLine();
            Console.WriteLine("Unexpected exception during export. Aborting migration. Exception -- ", ex.Message);
            System.Environment.Exit(1);
         }
      }

      protected static void MapContent() {
         var manifestFileName = System.IO.Path.Combine(Properties.Settings.Default.WorkFolder, "Manifest.xml");

         // make sure manifest file exists
         if (!System.IO.File.Exists(manifestFileName)) {
            Console.WriteLine();
            Console.WriteLine("The 'manifest.xml' file does not exist. Aborting migration.");
            System.Environment.Exit(1);
         }

         // back it up!
         var manifestBackupFileName = System.IO.Path.Combine(Properties.Settings.Default.WorkFolder, "Manifest.ORIG.xml");
         System.IO.File.Copy(manifestFileName, manifestBackupFileName, true);

         // load it up
         var doc = new System.Xml.XmlDocument();
         doc.Load(manifestFileName);

         var root = doc.DocumentElement;

         var mgr = new System.Xml.XmlNamespaceManager(doc.NameTable);
         mgr.AddNamespace(@"x", @"urn:deployment-manifest-schema");

         // remove legacy content type
         var node = doc.SelectSingleNode(@"//x:SPObject[@Id='a2aa69e8-a1d8-0117-9744-146aeb47a14b']", mgr);
         if (node != null) {
            root.RemoveChild(node);
         }

         // remove folder
         node = doc.SelectSingleNode(@"//x:SPObject[@Id='cf3362c4-1e1b-43a5-8715-0e1a5a7b167']", mgr);
         if (node != null) {
            root.RemoveChild(node);
         }

         // remove site
         node = doc.SelectSingleNode(@"//x:SPObject[@Id='6430e1f7-e66f-4280-8b7c-91e15b57a29d']", mgr);
         if (node != null) {
            root.RemoveChild(node);
         }

         // write it back
         doc.Save(manifestFileName);
      }

      protected static void ImportContent() {
         try {
            var site = new Microsoft.SharePoint.SPSite(Properties.Settings.Default.DestinationSiteUrl);
            var web = site.OpenWeb();
            var list = web.Lists[Properties.Settings.Default.DestinationListName];

            var ct = list.ContentTypes[Properties.Settings.Default.ArticleContentTypeName];

            // define the import settings
            var settings = new Microsoft.SharePoint.Deployment.SPImportSettings();
            settings.BaseFileName = Properties.Settings.Default.BaseFileName;
            settings.CommandLineVerbose = true;
            settings.FileCompression = false;
            settings.LogFilePath = LogFile;
            settings.UserInfoDateTime = Microsoft.SharePoint.Deployment.SPImportUserInfoDateTimeOption.ImportAll;
            settings.SiteUrl = Properties.Settings.Default.DestinationSiteUrl;
            settings.WebUrl = Properties.Settings.Default.DestinationSiteUrl;
            settings.RetainObjectIdentity = false;
            settings.FileLocation = Properties.Settings.Default.WorkFolder;

            // validate settings, set up the import, define event handlers, and kick off the job
            settings.Validate();
            var importer = new Microsoft.SharePoint.Deployment.SPImport(settings);

            importer.Started += (sender, e) => {
               // we need to re-aprent each of the orphaned list items
               var ro = e.RootObjects;
               foreach (Microsoft.SharePoint.Deployment.SPImportObject io in ro) {
                  if (io.Type == Microsoft.SharePoint.Deployment.SPDeploymentObjectType.ListItem) {
                     io.TargetParentUrl = list.RootFolder.ServerRelativeUrl;
                  }
               }
            };

            importer.ObjectImported += (sender, e) => {
               // we need to change the Content Type of the list item after it has been added
               if (e.Type == Microsoft.SharePoint.Deployment.SPDeploymentObjectType.ListItem) {
                  var item = list.GetItemByUniqueId(e.TargetId);
                  if (item != null) {
                     item["ContentTypeId"] = ct.Id;
                     item.SystemUpdate();
                  }
               }
            };

            importer.Run();
         }
         catch (System.IO.FileNotFoundException ex) {
            Console.WriteLine();
            Console.WriteLine("The destination site '{0}' was not found. Aborting migration.", Properties.Settings.Default.DestinationSiteUrl);
            System.Environment.Exit(1);
         }
         catch (System.ArgumentNullException ex) {
            Console.WriteLine();
            Console.WriteLine("The detination list '{0}' was not found. Aborting migration.", Properties.Settings.Default.DestinationListName);
            System.Environment.Exit(1);
         }
         catch (System.Exception ex) {
            Console.WriteLine();
            Console.WriteLine("Unexpected exception during import. Aborting migration. Exception -- ", ex.Message);
            System.Environment.Exit(1);
         }
      }

      #endregion

      #region Public Methods

      public static void Main(string[] args) {
         if (Properties.Settings.Default.DoExport) {
            ExportContent();
         }

         if (Properties.Settings.Default.DoImport) {
            MapContent();
            ImportContent();
         }
      }

      #endregion

   }

}
2

The PowerShell Register-ObjectEvent adds events to a queue and processes them asynchronously, i.e. after the import has run, so it changes the value of the parent too late to be of any use. Also, there is a typo in "PDeploymentObjectType".

You can add a synchronous event handler with the following:

$import.add_Started({
    param($Source, $EventArgs)
    Write-Host "SYNC EVENT: Import Started"
    Write-Host "SYNC Value of list: '$($list.RootFolder.ServerRelativeUrl)'"
    $ro = $EventArgs.RootObjects
    ForEach ($io in $ro) {
        if ($io.Type `
            -eq [Microsoft.SharePoint.Deployment.SPDeploymentObjectType]::ListItem) {
            $io.TargetParentUrl = $list.RootFolder.ServerRelativeUrl
        }
    }
})

Also add output from the Register-ObjectEvent to see how it fires later. You will also see that the $list variable has the value from the shell (usually $null), not the script.

$oe = Register-ObjectEvent -InputObject $import -EventName Started `
    -SourceIdentifier ImportStarted -Action {
    Write-Host "ASYNC EVENT: Import Started, happens after event has completed"
    Write-Host "ASYNC Value of list: '$($list.RootFolder.ServerRelativeUrl)'"
}
0

Jason,

Can you please try the suggestion given in following question:

How can you move a document with its version history to a new library?

It says

Just moved (drag & drop) a file in Explorer View from one document library to another, and it did retain previous versions!

Obviously both libraries need to have versioning enabled.

Try it yourself.

EDIT: note that it ONLY retains the versions if you MOVE the files. It does not do so when you perform a COPY!

  • Moving it does retain versions (along with timestamps) but the author for each version is set to the person who performs the move (me in this case). I also need the original person along with the versions and timestamp. – Jason Apr 4 '14 at 20:26
  • @Jason get yourself a developer :) a custom script will do.. – Arsalan Adam Khatri Apr 4 '14 at 23:30
  • I am a developer :-) If you re-read the updated posting from last week you will see that the import custom script (via PwerShell) does not quite work because of the event handler. – Jason Apr 7 '14 at 14:11
  • @Jason did we ever found a way of doing this. I am stuck in a similar situation. – Sudhir Kesharwani Apr 14 '14 at 19:44
  • @SudhirKesharwani - I did. As you see in the updated posting from last week, there are two PowerShell scripts. The first one will perform the export whereas the second will perform the import. The only downside was that I could not get the import script to work 100% because I could not get the Started event to be recognized. That being said, I did port the PS scripts over to a C# console app that did work. I will post it later and you will see that the C# app is almost identical to the PS scripts except for syntax. – Jason Apr 14 '14 at 19:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.