I found here this QA: Connect client object model to https url which mentions this article how to do it in full .NET client object model.
That would be great if the class ServicePointManager
(MSDN) would be in Silverlight DLLs, but it is not.
Is there any other approach?