Initiate long background process from SharePoint UI

Question

I have a web part that users use to initiate some processing which could take up to an hour to finish (it involves processing a lot of list items). If I start this processing on button click in standard server side code then obviously it times out. I am thinking of following options:

1. Create an item in a new list with details of what to process and trigger a workflow on that list that will run in background. Workflow can keep updating the progress in list item that I can show on web part if user refreshes the page. Here the question is Can a workflow keep running for an hour or so? Is there any time limit or something else that can make it to stop?
2. Trigger a timer job to do the same work. Again, any time limits for timer job
3. Write a web service and invoke it from client side. I will get all list item Ids to process first and then invoke from JavaScript in batches and report progress on UI

Looking for pointers on how best to handle this kind of scenario and pros/cons of various approaches. Goals are:

• Reliably complete the processing even if it goes on for hours
• Ability to track progress of background work
• Preferably a way to stop and restart the process from where it was stopped

Thanks

Answer 1

Sure workflows can run for days, but IMO any workflow is expected to reflect some business process, and should not be used just as a code execution environment. However if you choose to use workflow, I'd create a hidden list, setup a workflow triggered on item creation at that list, and create on item with all needed metadata when user clicks a button at webpart. Make a Progress column and update item's progress from the workflow, to just query a value in the webpart.

Timer jobs have schedules. And basically it's "1 job definition + 1 schedule = N timer jobs" logic. Repeating the same task over and over depending on time. And you say you want users to trigger it - does it mean it depends on user / date&time / any other external parameter? We can't pass parameters to timer jobs. I don't think you'll want to end up creating a new timer job definition each time just to run it once. However if you choose to use timer job, I'd define a timer job that runs every X minutes and reads new items in queue list (that are added from webpart) and processes them one by one. Like in the workflow scenario, but performance and control is much better.

But I'd stick to custom development here. Maximum control, you know.