1

I've created some code that identifies all of the site collections on a farm and then loops through all of the sites and lists within those collections . It's not ideal, but I've been tasked with it, and need to do so.

Unfortunately the code is performing very slowly, and I believe it's due to the following code which finds all of the items within a list:

foreach (SPList list in web.Lists)
{
    query.RecordLists(list.ID.ToString(), list.Author.ToString(), list.Title,
                        list.DefaultViewUrl,
                        list.ParentWeb.Title, list.ParentWebUrl, list.ItemCount,
                        list.LastItemModifiedDate, list.LastItemDeletedDate);

    // check if files exist in the document library, if they don't break out to next level
    if (list.ItemCount <= 0) continue;

    for (int i = 0; i < list.Items.Count; i++)
    {
        try
        {
            SPListItem item = list.Items[i];

            if (!item["Created By"].ToString().Contains("System Account") &&
                !item.Url.Contains("_catalogs") &&
                !item.Url.Contains("Style Library") &&
                !item.Url.Contains("Cache Profiles") && 
                !item.ContentType.ToString().Contains("Theme Gallery") &&
                !item.ContentType.ToString().Contains("Converted Forms") &&
                !item.ContentType.ToString().Contains("Page Output Cache") &&
                !item.ContentType.ToString().Contains("Master Page"))
            {
                query.RecordItems(item.ID.ToString(), item.ContentType.Name,
                                    item.DisplayName, item.Name,
                                    "", item.Url, item["Created By"].ToString(),
                                    "",
                                    Convert.ToDateTime(item["Modified"]),
                                    item["Modified By"].ToString(),
                                    Convert.ToDateTime(item["Created"]),
                                    item["Created By"].ToString());
            }
        }
        catch (NullReferenceException)
        {
            SPListItem item = list.Items[i];

            Logger.Error(
                "[{0}] Filed moving on file {1} as not all content was present",
                item.Name);
        }

    }
}

Is there a faster way to do this? At its current rate this will take hours to execute.

  • 3
    My eyes started bleeding reading that code. I don't know what you do with the data, but I would deffinently advice that you use PowerShell to do that, or if you have to use it within a web part or timerjob, use CAMLquery as that works faster than what you have there. – user2536 Oct 2 '13 at 5:38
6

You can use SPQuery to retreive the items, and loop over the results instead of looping directly over list.Items which is a very slow operation.
To retrieve all the items of the list - do not put anything in SPQuery.Query.
In addition - set in SPQuery.ViewFields only the fields you want to retrieve.
Like this:

SPQuery query = new SPQuery();
query.ViewFields = string.Concat(
    "<FieldRef Name='Field1' />",
    "<FieldRef Name='Field2' />",
    "<FieldRef Name='Field3' />",
    "<FieldRef Name='Field4' />");
SPListItemCollection items = list.GetItems(query);

Then change:

for (int i = 0; i < list.Items.Count; i++)

to:

foreach(SPListItem item in items)

(foreach is faster than for in this case.)

Edit:
As Robert Lindgren said, you can use SPQuery.Query to include your filter, and thereby reduce the amount of the returned items.
This is a good reference for SPQuery.Query options.

  • 1
    Why not put all his checks also in the Query? :) – Robert Lindgren Oct 2 '13 at 10:28
  • @RobertLindgren - You're right, I did not notice it. – banana Oct 2 '13 at 10:45
1

can you clarify what your trying todo? I can give you part of the reason its taking long! your looping within a loop and looking in every list and every item (all data), this equates to huge huge ammounts! hence why it crashes or stops or just goes amazingly slow!

the key here is to know what data you want! if for example you only want to check on one specific colunm for all lists in a site collection than you need to look into camlquery and be more specific!

Do you need to iterate though every cell in a list?

also dont forget your going though each site, meaning a new connection so make sure your disposing of the site once your finished though each loop:

SPWebApplication webApplication = SPContext.Current.Site.WebApplication;
SPSiteCollection siteCollections = webApplication.Sites;

foreach (SPSite siteCollection in siteCollections)
{
    //do your code with list stuff

    //once complete with site close it before next loop      
    siteCollection.Close();
}

the above is one way of improving speeds

http://msdn.microsoft.com/en-us/library/ms462161(v=office.14).aspx

1

Hello two simple change would run you code like a charm: While iterating through the lists use a SPListCollection object to store the Web.lists. Second create SPListItemCollection object to store all the list items in to it. Then iterate each items from the SPListItemCollection object. User foreach in both the cases.

If your list would have too many entries then each time calling web.Lists and list.Items would make your performance slower.

If you are familiar with client object model then that would increase your performance to a greater extent.

-Vikram

  • Do you have an example of this you can post? – Michael A Oct 2 '13 at 22:38
  • 1
    SPListCollection listCol = web.Lists; foreach (SPList list in listCol) { SPListItemCollection itemCol = list.Items; foreach (SPListItem item in itemCol) { //Do your coding here } } – VIKS Oct 3 '13 at 2:36
  • Hope this helps – VIKS Oct 3 '13 at 2:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.