Rest to read more than 5000 items from document library

Question

Below is my code to read items from a document library.

var url = _spPageContextInfo.webAbsoluteUrl + "/_api/web/lists/getbytitle('DocumentList')/items?$select=customerID";
$.ajax({  
    url: url,  
    method: "GET",  
    headers: {  
        "Accept": "application/json; odata=verbose"  
    },  
    success: function(data) {  
        $.each(data.d.results, function(index, item) {
            arrayCustomerID[index] = item.customerID;
        });  
    },  
    error: function(data) {  
        console.log("error");  
    }  
});  

This work for the normal case but what should I do so that it works if I have more than 5000 items in the document library.

Answer 1

Please refer below code :

Here we are doing recursive call to the GetListItems() function. This will fetch data in the bunch of 1000 items and concat it to response variable.

 var url = _spPageContextInfo.webAbsoluteUrl + "/_api/web/lists/getbytitle('DocumentList')/items?$select=customerID&$top=1000";
    var response = response || [];  // this variable is used for storing list items
    function GetListItems(){
        return $.ajax({
            url: url,  
            method: "GET",  
            headers: {  
                "Accept": "application/json; odata=verbose"  
            },
            success: function(data){
                response = response.concat(data.d.results);
                if (data.d.__next) {
                    url = data.d.__next;
                    GetListItems();
                }
                $.each(response, function(index, item) {
                    arrayCustomerID[index] = item.customerID;
                });
            },
            error: function(error){
            }
        });
    }

Answer 2

THIS IS WORKING !!! SOLVED

If your list or library is having more than 5000 items, then you need to perform a repeated ajax call function filtering 5000 items each time. But to know how many times to call the ajax function, we must do in Dynamic way.

1. First call a ajax function with parameters "/_api/Web/Lists/GetByTitle(ListName)/Items?$orderby=Id desc&$top=1". Now you will get the latest added "Id".
2. Now Divide the Id returned by the ajax by 5000. (Id/5000) So that you will get a result of how many times you need to perform the ajax.
3. Now you can perform the ajax function repeatedly be filtering every 5000 items, with filter like, $filter=Id ge 0 and Id le 5000, $filter=Id ge 5000 and Id le 10000, ........ , etc.
4. You can have a foreach loop to perform the ajax repeatedly with a dynamic filter, $filter=Id ge (5000*LoopValue) and Id le (5000*(LoopValue+1)).

Also make sure, the ajax to have async:true, to avoid performance issue. You can store the returned data in array and perform further actions, so that you could store more than 5000 data to perform your functions.

Since, there is a threshold limit for there kinda issues, we could use this kind of alternatives to make it possible.