Using Javascript REST API to do asynchronous batch POST to multiple SharePoint lists

Question

I'm in a bit of a quandry as I'm just learning how to do development in SharePoint 2016 / SharePoint Online and using Javascript to perform REST calls. I have a form that is using REST to successfully pull data from a SharePoint list using the following:

function getListItems(webURL, listName, successFunction, successParams, failFunction) {

    // REST Call        
    $.ajax({
        url: webURL + "/_api/web/lists/GetByTitle('" + listName + "')/items?$top=1000",
        type: "GET",
        headers: { "ACCEPT": "application/json;odata=verbose" },
        success: function(data) {
            successFunction(data, successParams);
            },
        error: function(error) {
            failFunction(error);
        }
    });

};

I also have the following to submit data to a single SharePoint list, which isn't quite working (it submits to the list, but calls the fail function). Could someone help me to understand why it is failing?

// Get List Item Type metadata
function GetItemTypeForListName(name) {
    return "SP.Data." + name.charAt(0).toUpperCase() + name.split(" ").join("").slice(1) + "ListItem";
}

function submitListItems(webURL, listName, jsonObj, successFunction, failFunction) {

    // REST Call to create new list item

    var jsonMeta = {'__metadata': {'type': GetItemTypeForListName(listName)}};

    // Merge the two JSON lists to include SP metadata information
    jsonObj = $.extend(jsonMeta, jsonObj);

    $.ajax({
        url: webURL + "/_api/web/lists/GetByTitle('" + listName + "')/items",
        type: 'POST',
        contentType: 'application/json;odata=verbose',
        data: JSON.stringify(jsonObj),
        headers: { 
                    'ACCEPT': 'application/json;odata=verbose',
                    'X-RequestDigest': $('#__REQUESTDIGEST').val() 
        },
        success: function(data) {
            successFunction(data, jsonObj);
        },
        error: function (error) {
            console.log(JSON.stringify(error));
            failFunction(error);
        }
    });
};

The form I am working with is best designed to submit data to two separate lists, with the lists having a 1:Many relationship on a common field primary key that is created during the submit process for list1, and the lists would join on list1.pk_id_1 = list2.sk_id1 (if SharePoint could do list joins). Thus my two lists are set up as:

list1: 
   PK_ID_1   item1 item2 item3 item4 ...
   PK_ID_2   item1 item2 item3 item4 ...
   PK_ID_3   item1 item2 item3 item4 ...

list2:
   PK_1   SK_ID_1   item1a item2a item3a item4a
   PK_2   SK_ID_1   item1b item2b item3b item4b ...
   PK_3   SK_ID_1   item1c item2c item3c item4c ...
   PK_4   SK_ID_2   item1a item2a item3a item4a ...
   PK_5   SK_ID_2   item1b item2b item3b item4b ...

Since the form is tied to a single submit button, I need to somehow set up or modify the above submitListItems function to submit two separate JSON objects to two separate lists, at the same time, and then report on the success or fail status. If either fail, it needs to roll back the changes on the one that did succeed and report back to the user to resubmit.

How can I best go about this? I had started reading into batch processing at this link, but I need to modify the above structure to submit two separate JSON objects to their respective lists.

Additionally, the JSON object I am passing the function works fine for the items going into list1, being a simple:

var jsonObj = {
    key1: value,
    key2: value,
    key3: value,
    key4: value
};

Answer 1

Are you talking about classic SP Forms? If so, you can create a function called

PreSaveAction(){...}

This function will be called if the user clicks on submit. You can do some stuff here. If this function returns true, the normal submitting process is initiated (submit and redirect). If it returns false, the submit will not be executed.

Be aware that additional REST-Operations tend to be executed asynchronously. You would need to return false and handle the whole submitting process manually.

Be also aware that if you handle the submitting process manually, you could overwrite changes made in the meantime. You would need to check this as well.

There is the possibility of doing batch requests, but I don't know if it implements transaction management (I would think it doesn't). I wouldn't recommend doing a batch request for two modifications and just chain the operations. You probably will just get one changeset per batch request instead of #operation changesets.

As for the error you get on submitting: You should check the error message. I think they are often quite explanatory in SP. I would guess your itemtype is wrong. Be aware that spaces are replaced by 'x005f' for example (There are other replacements!). Your GetItemTypeForListName()-function should only work for simple names. You can check the itemtype by doing a get request for the list in question, it should be returned with all the metadata.

As for a solution to your problem: I tend to just simply chain the operations in simple javascript style: Next operation is always executed in the success-callback of the previous operation. The normal javascript callback hell. Not very efficient, but it might suffice for your needs. List REST-operations tend to be quite fast compared to anything else in SP, so it wouldn't be that noticeable.

Works, if your operations don't get too complex.

Answer 2

Here is an implementation based on some research by modifying this and combining it with this (specifically here) and trial and error. I have a JSON object and a JSON array of objects that I need to add to my respective lists. The first object needs to be passed directly in using AJAX REST calls, while the second needs to be processed as a batch using AJAX REST calls:

// Get List Item Type metadata
function GetItemTypeForListName(name) {
    return "SP.Data." + name.charAt(0).toUpperCase() + name.split(" ").join("").slice(1) + "ListItem";
}

function submitListItems(listName, jsonObj, _async) {

  var jsonMeta = {'__metadata': {'type': GetItemTypeForListName(listName)}};

    // Merge the two JSON lists to include SP metadata information
    jsonObj = $.extend(jsonMeta, jsonObj);

  // Update form digest for X-Request-Digest
  UpdateFormDigest(_spPageContextInfo.webServerRelativeUrl, _spFormDigestRefreshInterval);

  var baseRequest = {
              url: "",
              type: "",
              contentType: "",
              data: ""
          };

    var request = baseRequest;
    request.type = 'POST';
    request.contentType = 'application/json;odata=verbose';
    request.data = JSON.stringify(jsonObj);
    request.url = _spPageContextInfo.webAbsoluteUrl + "/_api/web/lists/GetByTitle('" + listName + "')/items";
    request.headers = {
        ACCEPT: 'application/json;odata=verbose',
        'X-RequestDigest': $("#__REQUESTDIGEST").val()

    };
    return $.ajax(request);
};

  /*
 * @name generateUUID
 * @description
 * Generates a GUID-like string, used in OData HTTP batches.
 * 
 * @returns {string} - A GUID-like string.
 */
function generateUUID() {
  var d = new Date().getTime();
  var uuid = 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, function (c) {
    var r = (d + Math.random() * 16) % 16 | 0;
    d = Math.floor(d / 16);
    return (c == 'x' ? r : (r & 0x7 | 0x8)).toString(16);
  });
  return uuid;
};

function submitBatchListItems(listName, jsonArr, _async) {

  // Add SP metadata information to jsonArr array
  jsonArr = jsonArr.map(x => ({'__metadata': {'type': GetItemTypeForListName(listName)}, ...x}));

// Update form digest for X-Request-Digest
  UpdateFormDigest(_spPageContextInfo.webServerRelativeUrl, _spFormDigestRefreshInterval);

  // generate a batch boundary
  var batchGuid = generateUUID();

  // creating the body
  var batchContents = new Array();
  var changeSetId = generateUUID();

  // get current host
  var temp = document.createElement('a');
  temp.href = _spPageContextInfo.webAbsoluteUrl;
  var host = temp.hostname;

  // for each record in array...
  for (var recordIndex = 0; recordIndex < jsonArr.length; recordIndex++) {

      var jsonRecord = jsonArr[recordIndex];

      // create the request endpoint
      var endpoint = _spPageContextInfo.webAbsoluteUrl + '/_api/web/lists/getbytitle(' + listName + ')/items';

      // create the changeset
      batchContents.push('--changeset_' + changeSetId);
      batchContents.push('Content-Type: application/http');
      batchContents.push('Content-Transfer-Encoding: binary');
      batchContents.push('');
      batchContents.push('POST ' + endpoint + ' HTTP/1.1');
      batchContents.push('Content-Type: application/json;odata=verbose');
      batchContents.push('');
      batchContents.push(JSON.stringify(jsonRecord));
      batchContents.push('');
  }

  // END changeset to create data
  batchContents.push('--changeset_' + changeSetId + '--');

  // generate the body of the batch
  var batchBody = batchContents.join('\r\n');

  // start with a clean array
  batchContents = new Array();

  // create batch for creating items
  batchContents.push('--batch_' + batchGuid);
  batchContents.push('Content-Type: multipart/mixed; boundary="changeset_' + changeSetId + '"');
  batchContents.push('Content-Length: ' + batchBody.length);
  batchContents.push('Content-Transfer-Encoding: binary');
  batchContents.push('');
  batchContents.push(batchBody);
  batchContents.push('');

  // create request in batch to get all items after all are created
  endpoint = _spPageContextInfo.webAbsoluteUrl + '/_api/web/lists/getbytitle(' + listName + ')/items?$orderby=Title';

  batchContents.push('--batch_' + batchGuid);
  batchContents.push('Content-Type: application/http');
  batchContents.push('Content-Transfer-Encoding: binary');
  batchContents.push('');
  batchContents.push('GET ' + endpoint + ' HTTP/1.1');
  batchContents.push('Accept: application/json;odata=verbose');
  batchContents.push('');

  batchContents.push('--batch_' + batchGuid + '--');

  batchBody = batchContents.join('\r\n');

  // create the batch
  console.debug(batchBody);

  // create the request endpoint 
  var endpoint = _spPageContextInfo.webAbsoluteUrl + '/_api/$batch';

  // batches need a specific header
  var batchRequestHeader = {
  'X-RequestDigest': $("#__REQUESTDIGEST").val(),
  'Content-Type': 'multipart/mixed; boundary="batch_' + batchGuid + '"'
  };

  // Create AJAX request and return resultant state
  var baseRequest = {
      url: "",
      type: "",
      headers: "",
      data: ""
  };

  var request = baseRequest;
  request.url = endpoint;
  request.type = 'POST';
  request.headers = batchRequestHeader;
  request.data = batchBody;

  return $.ajax(request);

};

With using the below to call the above functions as needed:

function submitForm() {

var list1DataInfo = $.Deferred();
var list2DataInfo = $.Deferred();

var dfdList1DataFiles;
var dfdList2DataFiles;

// Collect data into JSON object
var jsonData = packageFormData();
var jsonAmountData = packageFormDataAmounts();

// Submit the data to the list tables

dfdList1DataFiles = submitListItems('list1Name', jsonData, true);
dfdList2DataFiles = submitBatchListItems('list2Name', jsonAmountData, true);

dfdList1DataFiles.done(function(pdoDataResponse) {
    console.log("dfdList1DataFiles.done");
    list1DataInfo.resolve(pdoDataResponse);
});

dfdList2DataFiles.done(function(pdoAmountResponse) {
    console.log("dfdList2DataFiles.done");
    list2DataInfo.resolve(pdoAmountResponse);
});

dfdList1DataFiles.fail(function(error) {
    console.log("dfdList1DataFiles.fail");
    console.log(JSON.stringify(error));
    list1DataInfo.fail(error);
});

dfdList2DataFiles.fail(function(lerror) {
    console.log("dfdList2DataFiles.fail");
    console.log(JSON.stringify(lerror));
    list2DataInfo.fail(lerror);
});

$.when(list1DataInfo, list2DataInfo)
    .then(function(list1Response, list2Response) {
      submitRecordSuccess(list1Response, list2Response)
    }, function(error, lerror) {
      submitRecordFail(error, lerror);
    });
};

This have the effect of submitting to the two separate lists without having to chain the AJAX calls.