1

A requirement of my website is that I pull down data from external sources. To pull down the data, I have created a timer job (SPJobDefinition). In this job, I need to store the external data somewhere. First I was thinking I would just store it in cache, but then I was thinking I might want to have the data around longer (plus, I don't know how to get at the cache from my timer job). So, I was just thinking I would store my serialized object in a list. Since there is no xml column type in SharePoint, I was going to just store it in a plain text (multi-line) column. Is this an bad practice? Or is this an ok thing to do? The only other options I can think of are: to write a file out to disk; or to use a custom table on the db (but this makes managing and deploying a lot harder).

Any other options anyone can give me?

Oh, and are there any limits on the size for the multi-line plain text field? I will likely be storing a serialized list, so the xml might be pretty big.

Thoughts anyone?

6 Answers 6

2

I wouldn't say this is a "bad practice", but it does really depend on what you need to do with the data. I've had cases where storing xml in a list has been great, but sometimes its not so great.

One advantage of having xml in a list is that the data is easily searchable using search. However a disadvantage is that it is not as easily queryable. If you have a specific query, you need to loop through all list items, deserialize into your objects and then query your objects. This can become pretty inefficient once your list starts to grow.

From what you have said, it sounds like you only need to store your xml in a single list item, which will be regularly updated, rather than regularly adding new list items which will then need to be queried.

If that is the case, I think the convenience of a list is a sound idea!

1

I dont know how much data you are storing in the plain text field - I think it gets stored as a TEXT field in the database so should be ok for larger than 8k but it really might be best to store the XML either as an XML document or associate it somehow with an InfoPath form.

The only advantage of using the InfoPath form is that it allows quick and easy manipulation of the data without much effort on your part.

0

If you're using SharePoint 2010 you can use BCS to write your data to a list with an external content type which will be written to an external data source of your choice, and can be surfaced in SharePoint in any way you like.

SharePoint 2007's BDC is read-only so wouldn't be sufficient for your requirement.

1
  • Unfortunately, I'm still stuck with 2007. Thanks though.
    – TehOne
    Aug 26, 2010 at 16:58
0

Your idea sounds like a workable solution.

AFAIK the character limit for plain text is 2024, so I would recommend to try enhanced text fields instead.

0

I would look at creating a custom class that inherits from SPPersistedObject. This will allow you to serialize and deserialize your xml into the config database using the .update() method.

SPPersistedObject

1
  • 1
    I would not suggest storing application-specific data in the configuration database.
    – Russell
    Feb 5, 2013 at 23:09
0

I would suggest storing the XML in an application database, not inside SharePoint. This is a more scalable solution, and the extra work required to setup an application database would not be significant compared with updating/replacing list item contents.

The data becomes more easily queried, manipulated and maintained.

You could also replace your SP Timer Job with a SSIS package which does this data export and ETL for you, which would perform faster than a timer job.

You can always expose the data to SharePoint (if required) using External Lists, which would appear in search.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.