Downloading large binary files and upload it to SharePoint document library using REST

Question

I am building a SharePoint hosted app that communicates With a REST service, also built by us.

The REST service provides a method to get a file (image or video file, typically 4Mb - 50Mb, but also needs to support larger files, up to hundreds of Mb)

Now I would like the SharePoint app to request a file from the service and store it in a document Library in the hostweb in SharePoint Online.

I have found a few samples on how to do this when uploading a local file, but I am not sure how to handle the binary file in Javascript when it comes from a service.

1.Any Clues on how to do this?

2.How will this perform With large files? Will the file always have to be temporary stored in the browser JavaScript context (as a variable or similar) or is there any way to pipe it directly from the REST service and into the document Library?

Since I am in Control of the REST service, I have the option to set the encoding to what I like, base64 for example.

Any help appreciated!

Answer 1

After some more Research I was able to find an answer.

Most of the examples I found online was regarding file upload from local Client, which implies handling of the local file, e.g. by the File API. Since I am Reading the file from a service, I don't need to consider this, and the result turned out to be simpler than I thought.

The rough process is as follows:

1. Download the data in base64 format
2. Convert the base64 string to an Uint8Array
3. From the array, construct a string using the fromCharCode function
4. Use the constructed string as the content in a REST Call to SharePoint

The code below is based on this excellent article: http://blogs.msdn.com/b/uksharepoint/archive/2013/04/20/uploading-files-using-the-rest-api-and-client-side-techniques.aspx

Regarding file sizes, it Works on large files as well (tested on a 100Mb+ file), but of course it takes a bit of time because of the Conversion process as well as the upload. Improvement suggestions are welcome, but for now I am just happy it Works! :)

function AlternativeUploadFile() {
    //contains a small image
    var base64String = "iVBORw0KGgoAAAANSUhEUgAAABQAAA";    
    base64String += "AOCAIAAACgpqunAAAAAXNSR0IArs4c6";
    base64String += "QAAAARnQU1BAACxjwv8YQUAAAAJcEhZ";
    base64String += "cwAADsMAAA7DAcdvqGQAAACPSURBVDh";
    base64String += "PY/z//z8DuYAJSpMFhrbmidbWE++AWX";
    base64String += "cQTMIAGNogsC2NIW0bEk0UgGn+f3uCl";
    base64String += "dWE29vSgASMDzUeahjQWAiAqUDSDJWF";
    base64String += "qgRqhTkAah7CWARA0ozsZLg1YAARhbo";
    base64String += "FpgQIcIc2kqKZniABlfyjQPZtrRZEgE";
    base64String += "KkwQAlsJDsBrkXEQRIagYqbTMwAACPN";
    base64String += "/ICUra1KwAAAABJRU5ErkJggg==";

    var byteArray = new Uint8Array(atob(base64String).split("").map(function (c) {
        return c.charCodeAt(0);
    }));
    var fileData = '';

    for (var i = 0; i < byteArray.byteLength; i++) {
        fileData += String.fromCharCode(byteArray[i]);
    }

    var docLibrary = $('#docLibraryNameInput').val();
    PerformUpload(docLibrary, "success.png", '', fileData);

}

function PerformUpload(libraryName, fileName, folderName, fileData) {
    var url;

    // if there is no folder name then just upload to the root folder
    if (folderName == "") {
        url = appWebUrl + "/_api/SP.AppContextSite(@TargetSite)/web/lists/getByTitle(@TargetLibrary)/RootFolder/Files/add(url=@TargetFileName,overwrite='true')?" +
            "@TargetSite='" + targetSiteUrl + "'" +
            "&@TargetLibrary='" + libraryName + "'" +
            "&@TargetFileName='" + fileName + "'";
    }
    else {
    // if there is a folder name then upload into this folder
        url = appWebUrl + "/_api/SP.AppContextSite(@TargetSite)/web/lists/getByTitle(@TargetLibrary)/RootFolder/folders(@TargetFolderName)/files/add(url=@TargetFileName,overwrite='true')?" +
           "@TargetSite='" + targetSiteUrl + "'" +
           "&@TargetLibrary='" + libraryName + "'" +
           "&@TargetFolderName='" + folderName + "'" +
           "&@TargetFileName='" + fileName + "'";
    }

    // use the request executor (cross domain library) to perform the upload
    var reqExecutor = new SP.RequestExecutor(appWebUrl);
    reqExecutor.executeAsync({
        url: url,
        method: "POST",
        headers: {
            "Accept": "application/json; odata=verbose",
            "X-RequestDigest": digest
        },
        contentType: "application/json;odata=verbose",
        binaryStringRequestBody: true,
        body: fileData,
        success: function (x, y, z) {
            alert("Success! Your file was uploaded to SharePoint.");
        },
        error: function (x, y, z) {
            alert("Oooooops... it looks like something went wrong uploading your file.");
        }
    });
}

Answer 2

I have a solution to this problem below using Flow so that the call can be called from any web page.

.docx files uploaded to SharePoint through REST API being corrupted. Other file types okay