1

I have created a Timer Job that iterates through a list of 100,000 items and finds the ones that are older than 120 days and "moves" them to a new site collection for archival. I am using a ContentIterator and SPQuery with a RowLimit of 1000 to try to help with resources but the memory on the OWSTIMER.exe keep on increasing. After 1000 items processed it is using over 1GB in memory. I am using a using statement on my source and destination SPSite and SPWeb but it does not appear to help.

My code looks somewhat like the following:

ContentIterator iterator = new ContentIterator();
SPQuery query = new SPQuery();
query.Query = completedItemsQuery;
query.RowLimit = 1000;

using (sourceSite = new SPSite(sourceUrl))
{
    using (sourceWeb = sourceSite.OpenWeb())
    {
        sourceWeb.AllowUnsafeUpdates = true;
        sourceList = sourceWeb.Lists["Example List"];

        using (destinationSite = new SPSite(destinationUrl))
        {
            using (destinationWeb = destinationSite.OpenWeb())
            {
                iterator.ProcessListItems(sourceList, query, ProcessItem, ProcessError);
            }
        }
        sourceWeb.AllowUnsafeUpdates = false;
    }
}

My ProcessItem() code simply iterates through each SPListItem and its SPListItemVersion to create the items on the destinationWeb but since these are non-disposable objects I cannot use any using statements on them. I'm not sure if this is the issue?

Also, my SourceWeb, DestinationWeb, SourceSite, and DestinationSite are global variables. Could this be an issue?

Anyone have any ideas on this?

1

I've done some large list stuff as well - not 100,000 items worth, but definitely more than 1000 - and I haven't seen any dispose issues with SPLIstItem. I do know that SharePoint attempts to thread processes behind the scenes when it can; is the OWSTIMER.exe getting so large that it crashes your system, or are you just concerned that it might be getting unwieldy ahead of time? SharePoint can be a bit of a memory hog; I generally try to set aside at least 32GB of hard drive space for the swap file in addition to 16GB of RAM in 2010. One process exploding to 1GB is, well, not necessarily anything.

Otherwise, you're wrapping your calls to SPSite and SPWeb in using blocks like you're supposed to so unless your memory leak is elsewhere I don't see anything here that would necessarily be a problem.

  • So I left the archival job running over night and the OWSTIMER.exe got to 3.6gb (only 4gb on the server) and the sever was brought to it's knees although it was still running. At this point it had processed 9800 items. I'm wondering if the issue could be that the SPListItems's in the processItem called by the ContentIterator are not being disposed of until the SPSite gets disposed? – LukeP Dec 11 '13 at 16:37
  • My experience is that 4 gigs of RAM / swap drive space isn't enough for SharePoint, particularly with a large list. I would agree that it's likely that the list items you mention don't get fully disposed of until the SPWeb or SPSite item get disposed. The larger issue is that Microsoft/SharePoint are kind of aware of this and recommend you put, well, a lot more RAM than what you've got now in there. – NotVonKaiser Dec 11 '13 at 20:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.