I've been using Client-Side Object Model in C# for Sharepoint2010 for a few weeks, and I've tried uploading files with File.SaveBinaryDirect and by adding the file to the list and then calling ExecuteQuery. Both methods are successful when I just upload just a few files with their size <= 3 MB.
However, let's say I try to upload 10,000 files of 10KB each using SaveBinaryDirect, it fails with a SocketException (Address already in use). It is called like 2000 times before it throws this exception, so it looks like it uses up all the ports before any Time_wait finishes waiting their default 240 seconds.
With the ExecuteQuery method, it works, but only because I upload them in batch, with the size of one batch always under 3 MB, and uploading never more than 20 files at once. So this method also has limitations: it can't upload files over 3 MB, and it also has the risk of throwing the SocketException. With 10,000 files, the method is only called 500 times. I'd guess that with a larger amount of files to upload, the method would be called over 2000 times too and throw the same error as the SaveBinaryDirect method.
Now, I don't know how to resolve the SocketException error in CSOM. Does anyone know how to resolve it?
Does anyone know a reliable way to upload any number of files with varying size (though they must always be under the default max upload size of 50 MB) to Sharepoint? Or which APIs for Sharepoint2010 could offer me this possibility?