3

I have an external partner account that wants to download a large GIS data set from our SharePoint. The data set is about 13500 files and is around 43 gb in size, and is housed within a folder. When they try to download it, it's downloaded as a zip file, but it's never able to fully complete, even though the browser says the download is complete. Sometimes the zip file is only a couple of GBs and won't open.

enter image description here

Other times, the download manages to get to around 20 gb, and the zip file can be opened, but half the files haven't been downloaded and I get a bunch of text files in the zip with error messages in them.

enter image description here

Each text file has an error message inside: "This file cannot be downloaded.

ExceptionType: ZipMeTAException.

CorrelationId: 7dcb10e9-be4f-4edb-b24b-d469dcaaf844,

UTC DateTime: 1/29/2020 10:09:47 PM"

I've tried downloading the folder myself and never got past 20 gb before the download "completed". Is there any reason why it's not possible to download the whole folder, and is there a better method to download such a huge number of files that would work for an external partner?

1

If the folder contains 13500 files then "Open with Explorer" will not work because you exceed the List View Threshold LVT (default 5000). Increasing the LVT is not possible for SharePoint Online. Alternatives:

  1. Use PowerShell (search for scripts that provide this capability). Check upfront if the scripts can handle large lists.
  2. Use tools that can download large number of items and that support LVT. e.g. here
  3. Use OneDrive for Business to sync the files

Paul

1
  • Regarding 3rd option 'Use OneDrive for Business to sync the files', is it possible to sync document library to local machine AND THEN COPY the 'local version' of the files to another destination on local machine eg 'C:\Users\Me\Downloads\My Copied Files' so that I can rename copied files and upload them to another document library? I tried to download the folder (Step 1) and it would only download 20GB of a 50GB folder, so syncing seems like my only option. I have the 'Files On Demand' feature turned on in OneDrive, so not sure if I can copy synced folders/files like that. Sep 21 at 7:39
0

You can return to Classic view in the library, and try to use "Open with Explorer" feature to open the folder, then copy and paste the files into your local. enter image description here

Have a try and let's know the result.

1
  • 1
    I tried that, but when opening a folder that should have about 13500 files inside, the folder instead was empty of any files, even with reloading the folder.
    – gtew234
    Jan 31 '20 at 23:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.