0

I have a custom WebPart that I have developed that has a ZIP function using JSZip for downloading file from the Document library. This is working fine for most cases, however when the document library is large, the process is failing due to memory issues.

I've had a look around at the different options of JSZip such as nodestream but this requires 'fs' which can't be used in a WebPart (browser).

Just wondering if anyone else has managed this. I can't generate the ZIP file all in memory so need a way of streaming the zip to the local machine as it is being created...I think!

Thanks

Improve this question

2 Answers 2

Reset to default
2

You're going to need a server side process. Zipping the file inside a browser process is going to eventually crap out if the file is big enough no matter what you do.

Your other option is to push back on the requirement. Using a zip file in this way isn't really helpful as you still need to download all the bytes. The the user will have to unzip the file anyway in order to make any use of it. Might as well just download the file and be done with it.

Edit - If you have lots of files and a lot of bytes, say over 100MB, you might want an asynchronous process. Here's what I might do in that case:

  1. button on page pings a web service with email of user and folder path
  2. web service drops this data onto an azure queue
  3. Azure function picks up the queue item, zips the files, dumps them somewhere (maybe an Azure blob) and retrieves a download URL
  4. The Azure function emails the link to the zip back to the requesting user.
Improve this answer
4
  • Thank you. Agree that the requirement should be pushed back on, but it's for our clients that like to have upward of 2000 files and folders stored, and requested an easy way to download them in one go. A ZIP file seemed the easiest way to get them everything with the folder structure intact. We are now thinknig of using a seperate on prem webservice that can be triggered to create the zip.
    – Bentura
    Commented Dec 21, 2023 at 14:00
  • I see that I use the word 'file' in my post. I meant files. So there would be multiple files and folders.
    – Bentura
    Commented Dec 21, 2023 at 14:01
  • @Bentura I made an edit with a quick and dirty arch suggestion given your comments.
    – Derek Gusoff
    Commented Dec 22, 2023 at 14:57
  • Thank you for this. We did consider an azure function or even something in house. We are worried that Azure will become quite expensive when dealing with GBs.
    – Bentura
    Commented Jan 3 at 13:40
1

Another approach is to use multiple disk spanning, i.e., create multiple zip files.

We use this approach (link) to create separate zip files of up to 1 GB. This obviously limits the size of an individual file to 1 GB. Start downloading content and when the 1 GB limit is approached download the zip file and start downloading more files that will end up in the next zip file and so forth. This works well.

Improve this answer
1
  • Thanks for this. Will add to our list of options.
    – Bentura
    Commented Jan 3 at 13:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.