How do i dynamically load a large list based on a filter?

Question

I have written some javascript that generates a link with filters for a list of items. It has a couple of drop downs to choose certain filter values. This is all good until the list reaches 5k items. Now i cant apply the filter because it has reached the list view threshold even thought the list after the filter is applied is below the threshold.

How do i handle loading only the items that pass the filter criteria, instead of loading all items and then apply the filter?

I can do this in the view settings under filtering, but i dont want to create a new view for each possibility for my drop downs

Answer 1

Try this, it's developed using CSOM, u can change it as per your script. filteritems variable stores the filtered result.

IEnumerable<Microsoft.SharePoint.Client.ListItem> filteritems = null;
ListItemCollectionPosition position = null;
var page = 1;
List<Microsoft.SharePoint.Client.ListItem> items = new List<Microsoft.SharePoint.Client.ListItem>();

try
{
    do
    {
        CamlQuery camlQuery = new CamlQuery();
        camlQuery.ViewXml = @"<View Scope='RecursiveAll'><Query></Query> <RowLimit>5000</RowLimit></View>";
        camlQuery.ListItemCollectionPosition = position;
        Microsoft.SharePoint.Client.ListItemCollection listItems = oList.GetItems(camlQuery);
        clientContextobj.Load(listItems);
        clientContextobj.ExecuteQuery();
        items.AddRange(listItems);
        position = listItems.ListItemCollectionPosition;

        page++;
    }
    while (position != null);

    //throw new Exception("My Exception!");
}
catch (Exception ex)
{
    //WSSLogger.Log("Loop Count:" + items.Count.ToString(), ex, RequestUrlAuthority);
}

try
{
    filteritems = items.Where(item => (item.FieldValues["Cust_Name"] == null ? string.Empty : item.FieldValues["Cust_Name"].ToString()) == strCustName
        && (item.FieldValues["Cust_Country"] == null ? string.Empty : item.FieldValues["Cust_Country"].ToString()) == strCountry &&
        (item.FieldValues["Status"] == null ? string.Empty : item.FieldValues["Status"].ToString()) == "Pending");

    //Pending,Active
}
catch (Exception ex)
{
    //WSSLogger.Log("-Custinfo-", ex, RequestUrlAuthority);
}