Sharepoint Rest API, retrieve more than 5000 files in folder

Question

I am trying to get the list of all files contained in a sharepoint folder, however, because there is more than 5000 files I cannot get any results.

I understand that I cannot query more than 5000 items. Then I tried to limit the number of elements returned by my query but unsuccessfully (I tried to do like the getbytitle() function)

When there is less than 5000 files, I successfuly get the list of files in a folder when I run this query:

web/GetFolderByServerRelativeUrl('Shared%20Documents')/Files

But when there is more than 5000 files, I cannot have any results. I tried this:

web/GetFolderByServerRelativeUrl('Shared%20Documents')/Files?$top=100

web/GetFolderByServerRelativeUrl('Shared%20Documents')/Files?$limit=100

It seems that the parameters 'limit' and 'top' are just ignored by sharepoint and I always get the "SPQueryThrottledException" error.

Is it possible to do what i'm trying to do with the sharepoint API?

Answer 1

You could use data.d.__next to get the next items for overcome the List View Threshold Limitation:

<script type="text/javascript">
 var url = _spPageContextInfo.webAbsoluteUrl + "/_api/web/lists/getbytitle('DocumentList')/items?$top=1000";
var response = response || [];  // this variable is used for storing list items
function GetListItems(){
    return $.ajax({
        url: url,  
        method: "GET",  
        headers: {  
            "Accept": "application/json; odata=verbose"  
        },
        success: function(data){
            response = response.concat(data.d.results);
            console.log(response);
            if (data.d.__next) {
                url = data.d.__next;
                GetListItems();
            }
            
        },
        error: function(error){
           console.log(error);
        }

    });
}
</script>

Rest to read more than 5000 items from document library

Answer 2

You should include a filter on an indexed column so that it retrieves less than 5000 items