How do I connect with SharePoint Online Web Services in order to access list and taxonomy data?

Question

I am using the SharePoint SOAP web services to try collecting list and taxonomy data from a site on SharePoint Online.

The following code works nicely when connected to a standalone SharePoint farm. However, when connecting to SharePoint Online, the call to GetList fails with a SOAPException:

"Server was unable to process request. ---> Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))"

Code:

using (ListService.Lists service = new ListService.Lists())
{
    service.Credentials = new System.Net.NetworkCredential("username", "password");

    XmlNode list = service.GetList("Doclib");
}

using (TaxonomyService.Taxonomywebservice taxonomy = new TaxonomyService.Taxonomywebservice())
{
    // Set credentials
    taxonomy.Credentials = System.Net.CredentialCache.DefaultCredentials;

    for (int termSetCounter = 0; termSetCounter < allTermSetIds.Count; termSetCounter++)
    {
        int lcid = System.Globalization.CultureInfo.CurrentUICulture.LCID;

        string termStoreId = String.Format("<termSetIds><termStoreId>{0}</termStoreId></termSetIds>", allSharedServiceIds[termSetCounter]);
        string termSetId = String.Format("<termSetIds><termSetId>{0}</termSetId></termSetIds>", allTermSetIds[termSetCounter]);

        // Get one term set from term store using Web Service
        string resultXML = taxonomy.GetTermSets(termStoreId, termSetId, lcid, oldtimestamp, clientVersion, out timeStamp);

    }
}   

This blog post seems to indicate that the reason authenticating fails, is that Web Services against Office 365/SharePoint Online are not yet officially supported. There appears to be a workaround involving some shady gathering and pass-through of cookies from Internet Explorer, but this is a huge hack and I can't seem to get it working with a real-life application.

Microsoft does seem to have a list of Web Services available in SharePoint Online here, but this list does not seem to include the SOAP services mentioned above. I do not see any services that support taxonomy data either. Also, not even SharePoint Designer supports editing Taxonomy columns from the client-side. This does seem like a huge oversight, though.

Is there another way than Web Services to get access to list and taxonomy information from SharePoint Online sites? If not, is there any information available from Microsoft regarding when this will be supported?

Answer 1

You can connect to ShraPoint Online through the Client Object Model as show here: http://blogs.msdn.com/b/cjohnson/archive/2011/05/03/authentication-with-sharepoint-online-and-the-client-side-object-model.aspx

You can download the code sample and build on top of it. Get back to me if you need any help.

I agree that you think this is a "hack" but it works right now :)

Answer 2

I have now received an answer from a Microsoft employee (SharePoint Solution Architect at an office in Scandinavia) regarding this question. The problem is that SharePoint Online uses Claims-based authentication, but the built-in .NET API officially only supports Default (NTLM) and Forms-based authentication. Microsoft (probably) does not plan to make any major changes to the API until the next major version of SharePoint is released, probably around 2013.

Therefore, at the present time, the official line for performing Claims-based authentication from .NET code is to use the workaround described above. You will end up with a CookieCollection object representing the Session ID for the current Claims-based authentication session, which can be used to create an authenticated ClientContext object or to authenticate any of the SharePoint web services...including TaxonomyClientService. This will let you get the term set data necessary to use Managed Metadata programmatically.

The source code in the link is a good starting point, but you probably have to run it in a separate thread due to a COM issue.

Answer 3

The following example demonstrates how to authenticate SharePoint SOAP Services in SharePoint Online using SharePointOnlineCredentials class :

public static CookieContainer GetAuthCookies(Uri webUri, string userName, string password)
{
    var securePassword = new SecureString();
    foreach (char c in password) { securePassword.AppendChar(c); }
    var credentials = new SharePointOnlineCredentials(userName, securePassword);
    var authCookie = credentials.GetAuthenticationCookie(webUri);
    var cookieContainer = new CookieContainer();
    cookieContainer.SetCookies(webUri, authCookie);
    return cookieContainer;
}

Example

var webUri = new Uri("https://contoso.sharepoint.com/");
Lists list = new Lists();  //Lists SOAP Service 
list.CookieContainer = GetAuthCookies(webUri,userName,password);

//Invoke GetListCollection operation
XmlNode result = list.GetListCollection();

//Process result
var docResult = XDocument.Parse(result.OuterXml);
XNamespace s = "http://schemas.microsoft.com/sharepoint/soap/";
var listEntries = from e in docResult.Descendants(s + "List")
              select new
                 {
                     Title = e.Attribute("Title").Value
                 };