1

I'm trying to iterate recursively over all documents in a given site. This is what I'm using:

    using (System.IO.StreamWriter file = new System.IO.StreamWriter("docs.txt"))
    {
        foreach (var lib in docLibs)
        {
            CamlQuery query = new CamlQuery();
            query.ViewXml = "<View Scope = 'RecursiveAll'></View>";
            ListItemCollection items = lib.GetItems(query);
            clientContext.Load(items);
            clientContext.ExecuteQuery(); // Exception thrown here
            foreach (var item in items)
            {
                Microsoft.SharePoint.Client.File f = item.File;
                if (f.ServerObjectIsNull.HasValue && !(bool)f.ServerObjectIsNull)
                    file.WriteLine(f.Name);
            }
        }
    }

The error I'm getting is "The request uses too many resources" I'm guessing some List in my site has too many documents and therefore one of the requests is failing. Is there anyway to retrieve the documents chunk by chunk or some other approach to retrieve all documents?

0

Using CSOM you can add a min and max ID into your query to restrict your query when iterating through a large query.for example (where ID > 1 and ID < 5000).

When using the SharePoint Server api, you can use theContentIterator, to help with accessing more than 5,000 items in a large list without hitting a list throttling limit and receiving an SPQueryThrottleException.

ContentIterator implements a callback pattern for segmenting the query for processing a single item at a time. Consider using this capability if you need to process a large number of items that may exceed a throttling limit.

MSDN shows the following example

static int exceptions = 0;
static int items = 0;

protected void OnTestContentIterator(object sender, EventArgs args)
{
    items = 0;
    exceptions = 0;
    string query1 = @"<View>
        <Query>
            <Where>
                <And>
                    <BeginsWith>
                        <FieldRef Name='SKU' />
                        <Value Type='Text'>S</Value>
                    </BeginsWith>
                </And>
            </Where>
        </Query>
    </View>";

    ContentIterator iterator = new ContentIterator();
    SPQuery listQuery = new SPQuery();
    listQuery.Query = query1;
    SPList list = SPContext.Current.Web.Lists["Parts"];
    iterator.ProcessListItems(list,
        listQuery,
        ProcessItem,
        ProcessError
    );
}

public    bool ProcessError(SPListItem item, Exception e) 
{ 
    // process the error
    exceptions++; 
    return true; 
}
public void ProcessItem(SPListItem item)
{
    items++;
    //process the item.
}
  • for getting list items form the list it works fine but how would you add new list item, I am getting threshold exception. – Ali Mar 8 '16 at 6:08
0

Please try the below link ; Working with Large Lists

or Try optimizing the query by specifying only the fields you need to check and also try pagination when querying the list as shown in the below link .Pagination in Large Lists

  • 1
    Please do not provide only links as answers, if the link breaks your answer will be of very little help. Instead add the important parts from the link to your answer! – Robert Lindgren Dec 23 '13 at 11:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.