Where to store data needed by a custom timer job?

Question

I have a custom ribbon button in each document library and when a user clicks it, I'd like to store some library-related data (URL, id ...) in a way that would make it available to a custom timer job. The timer job would then iterate the list of libraries and perform some operations on each of them. In other words, I'd like to give the user an option to enlist a document library for timer job processing. The timer job would need a table of data that looks something like this: What would be the best way to store this data for the timer job?

I won't need any complex queries. However, the amount of rows (libraries to process by the timer job) can go into several thousands (perhaps more). I'd like to allow enlisting of libraries to users who don't have admin rights. I don't want to be limited to a single web application data.

Here's what I've considered so far (based, among others, on this article):

• I know that one option is to store everything in a custom database table. But this seems a bit too complex given that I won't be needing any special queries and I'd have to complicate things further with database permission settings.
• I could create a custom list, but I've read bad things about its performance for even moderate-sized data. Also, I don't want users (sans perhaps the farm admin) to have the ability to view or edit the data - so I'd require some kind of hidden/read-only list which I could alter programmatically. Moreover, breaking the list into smaller parts might make sense (or would it?). Should I consider doing this?
• other options such as SPPersistedObjects, property bags, SPWebApplication properties etc. require admin rights for writing, which means a normal user can't enlist a library for timer job processing.
• (Just thinking aloud:) perhaps even deploying a custom data file (maybe even an embedded database) would make sense (not sure about the location though, maybe one of the mapped SP folders)? This would be easy to read/write using C#, but I'm not sure it it's a good idea.

What are my options?

Answer 1

SPWorkItemJobDefinition class is likely the best approach

Answer 2

You make a valid point that large list can be troublesome at times. Take a look at this link, it shows you how to handle large lists:
http://office.microsoft.com/en-za/windows-sharepoint-services-help/manage-lists-and-libraries-with-many-items-HA010173667.aspx
Also using CAML Query to access / refine your list is really fast.

As Arsalan Adam Khatri posted in a comment for you, you can easily hide a list.

To get you past your worry about the persisted objects, you can simply run your code in:

 SPSecurity.RunWithElevatedPrivileges(delegate()
 {
    using (SPSite site = new SPSite(web.Site.ID))
    {
// implementation code here
    }
});

I dont think you need to use a custom table for this. (Although it might be easier).