Uploading multiple files to custom lists through Web Service

Question

What is the best way to upload multiple files through web service without causing the timeout?

I am using the following code:

ListProxy.Lists newListProxy = new ListProxy.Lists();
                newListProxy.Url = "http://site/_vti_bin/Lists.asmx";

                // newListProxy.Credentials = System.Net.CredentialCache.DefaultCredentials;                
                newListProxy.Credentials=new System.Net.NetworkCredential("madhur", "password", "mydomain");

                FileStream fStream = File.OpenRead("F:\\TestWebService\\Test40.txt");
                byte[] contents = new byte[fStream.Length];
                fStream.Read(contents, 0, (int)fStream.Length);
                fStream.Close();
                newListProxy.Timeout = 60000;

                for (int i = 404; i <= 603; i++)
                {

                    newListProxy.AddAttachment("wstest", i.ToString(), "Test Attachment2", contents);

                }

In the above code, I am uploading 200 attachments to the custom lists. However the web service times out or connection gets closed after some items.

What is the best way to handle this ?

Answer 1

There is nothing really wrong with your code. It will time out only if you have extra large file that needs over 60 seconds to upload. Most likely your Sharepoint server is throwing time out exception.

You can check out this blog post: Upload a file to Sharepoint 2007 using webservices with a specific content type

And interesting part is:

Also we should estimate the maximum document size we plan to upload and network speed and set the maxRequestLength and executionTimeout values of the httpRuntime section in our web.config in accord to avoit any possible Timeout Exception.

I have verified web.config on my SP 2007

<httpRuntime maxRequestLength="51200" />

And this is known SP default limitation: File Name, Length, Size and Invalid Character Restrictions and Recommendations

The default max single file upload size is 50 MB by default for a web application.

executionTimeout is not defined and if you check MSDN httpRuntime Element you can see that default value is 1:10 minutes (I am really not sure if this applies on SP but it is still bigger value then 60 seconds you are using).

Also you need to check if you have enough disk space for your files. While testing this my content database filled up as well as transaction logs Truncate MS SQL transaction Log file for SharePoint 2007 DB (not really timeout exception).

There is slight possibility that one of your items already have attachment with same file name. Code will then throw SOAP exception (not timeout).

Finally you can use this code (or something similar):

FileStream fStream = File.OpenRead("F:\\TestWebService\\Test40.txt");
byte[] contents = new byte[fStream.Length];
fStream.Read(contents, 0, (int)fStream.Length);
fStream.Close();

NetworkCredential credential = new NetworkCredential("madhur", "password", "mydomain");
int serviceTimeOut = 60000;
string serviceUrl = "http://site/_vti_bin/Lists.asmx";

for (int i = 404; i <= 603; i++)
{
    using (ListProxy.Lists newListProxy = new ListProxy.Lists())
    {
        newListProxy.Url = serviceUrl;
        newListProxy.Credentials = credential;
        newListProxy.Timeout = serviceTimeOut;

        newListProxy.AddAttachment("wstest", i.ToString(), "Test Attachment2", contents);
    }
}

Since service proxy is very 'cheap' object you can use it per attachment upload and then dispose it (using statement).

I have tested this on 500 files (7MB each) without timeout or any other exception. When I used timeout value of 1000 (1s) I was getting timeout exception on first file.

Hope something I wrote will help you to solve your issue.