Downloading Bulk of data(150 GB) using powershell or C#

Question

I have a task to download bulk of data(150 GB) from SharePoint document libraries? What will be more efficient and faster,Doing it using a powershell script or using a c# code to download files?

Answer 1

use this powershell script to download Data from SharePoint Document Library.it takes around 2 minutes to download 1Gb data.

if((Get-PSSnapin "Microsoft.SharePoint.PowerShell") -eq $null)
{
    Add-PSSnapin Microsoft.SharePoint.PowerShell
}

#Script settings

$webUrl = "http://xyz:1000/"

$docLibraryName = "Documents"
$docLibraryUrlName = "mydocs"

$localFolderPath = "C:\\foldername\\mydocs"

#Open web and library

$web = Get-SPWeb $webUrl

$docLibrary = $web.Lists[$docLibraryName]

$files = ([System.IO.DirectoryInfo] (Get-Item $localFolderPath)).GetFiles()

ForEach($file in $files)
{

    #Open file
    $fileStream = ([System.IO.FileInfo] (Get-Item $file.FullName)).OpenRead()

    #Add file
    $folder =  $web.getfolder($docLibraryUrlName)

    write-host "Copying file " $file.Name " to " $folder.ServerRelativeUrl "..."
    $spFile = $folder.Files.Add($folder.Url + "/" + $file.Name, [System.IO.Stream]$fileStream, $true)
    write-host "Success"

    #Close file stream
    $fileStream.Close();
}

#Dispose web

$web.Dispose()

Answer 2

You best bet is to write PowerShell script which iterates through the lists and their items, and eventually download them all. You can use System.IO.FileStream to write the bytes.

Answer 3

I would certainly use C# for it.

Debugging alone is a major asset for bulk operations and obviously you will have more control over the process.

I prefer using PowerShell for administrative or repetitive tasks only, focusing on simple things.

Answer 4

There will be no difference in speed - I would choose the solution you feel more comfortable with writing.

Answer 5

The answer from SharePointer does the opposite thing. It uploads local files to a Document Library and not from.

I've found a working sample here: http://blogs.msdn.com/b/varun_malhotra/archive/2012/02/08/sharepoint-2010-powershell-download-all-files-in-document-library-to-network-share-file-share.aspx

######################## Start Variables ########################
######################## Varun's Script######################
$destination = "C:\\tools\\Folder"
$webUrl = ""
$listUrl = ""
##############################################################

$web = Get-SPWeb -Identity $webUrl
$list = $web.GetList($listUrl)

function ProcessFolder {
    param($folderUrl)
    $folder = $web.GetFolder($folderUrl)
    foreach ($file in $folder.Files) {
        #Ensure destination directory
        $destinationfolder = $destination + "/" + $folder.Url 
        if (!(Test-Path -path $destinationfolder))
        {
            $dest = New-Item $destinationfolder -type directory 
        }
        #Download file
        $binary = $file.OpenBinary()
        $stream = New-Object System.IO.FileStream($destinationfolder + "/" + $file.Name), Create
        $writer = New-Object System.IO.BinaryWriter($stream)
        $writer.write($binary)
        $writer.Close()
        }
}

#Download root files
ProcessFolder($list.RootFolder.Url)
#Download files in folders
foreach ($folder in $list.Folders) {
    ProcessFolder($folder.Url)
}