Retrieve all content from a web and it's sub webs using web services

Question

I am trying to retrieve the content of a SharePoint database, i.e. the subwebs and corresponding Pages lists, using the available SharePoint 2010 Web Services.

I can get a collection of all the subwebs using GetWebCollection and I can get the content of a Pages list using GetListItems, but I can't figure out how to tie them together. The web collection only contains the title and the URL of each web and I need the GUID of the Pages lists for each of them in order to retrieve the content of the Pages. How do I get the GUID of the Pages lists for each sub web?

Side note: I have also looked briefly at the following two migration tools: Sharegate and SharePoint Content Deployment Wizard. I was not able to export items from a web included sub webs using Sharegate, and the CodePlex tool was required to be installed on the machine where the content is exported from, and I do not have access to that.

Answer 1

Why do you need the list guid? The first parameter of GetListItems, listName, can be either the list's guid OR the list's name; just pass it "Pages".

Or, you can call GetList("Pages") to get the ID.

Or, you can call GetListCollection() to return all the lists in that web and search for the library yourself.

*Edit*:

Thank you for the suggestiong, but calling GetList("Pages") gives med a ServiceModel.FaultException. And I am not that surprised that it does not work, since there are many lists with name Pages. In the returned XML from GetListCollection() Name and ID contains the same GUID for all lists.

First, you need to get a better exception message from the FaultException. We can't assume we know what the error is because it could be a variety of things.

Try the following code to get fault exception detail:

try
{
    // your code...
}
catch (FaultException fe)
{
    MessageFault mf = fe.CreateMessageFault();
    if (mf.HasDetail)
    {
        XmlElement fexe = mf.GetDetail<XmlElement>();
        Console.WriteLine("\tError: " + fexe.OuterXml);
    }
}

Second, don't forget that your Service Reference has context to whatever web you gave it for an EndPoint address. When you added the service reference, it does this for you by setting some values in the app.config.

This means that if you want to get a Pages library in a different site you will need to create a new Webs or Lists client. For example, here is some code I ripped out of an app I have:

WebsSoapClient websClient = new WebsSoapClient();
websClient.Endpoint.Address = new System.ServiceModel.EndpointAddress(SITE_COLLECTION_URL + "/_vti_bin/Webs.asmx");
websClient.ClientCredentials.Windows.ClientCredential = System.Net.CredentialCache.DefaultNetworkCredentials;
websClient.ClientCredentials.Windows.AllowedImpersonationLevel = System.Security.Principal.TokenImpersonationLevel.Impersonation;
XElement webs = XElement.Parse(websClient.GetAllSubWebCollection().OuterXml);

foreach (XElement web in webs.Descendants())
{
    Console.WriteLine("Web url is {0}", web.Attribute("Url").Value);

    try
    {
        // ***** new client here with a context to the subweb ****
        ListsSoapClient listsClient = new ListsSoapClient();
        listsClient.Endpoint.Address = new System.ServiceModel.EndpointAddress(web.Attribute("Url").Value + "/_vti_bin/Lists.asmx");
        listsClient.ClientCredentials.Windows.ClientCredential = System.Net.CredentialCache.DefaultNetworkCredentials;
        listsClient.ClientCredentials.Windows.AllowedImpersonationLevel = System.Security.Principal.TokenImpersonationLevel.Impersonation;

        // do something with a list
        XElement contacts = XElement.Parse(listsClient.GetListItems(LIST_NAME, null, xmlQuery, xmlViewFields, ROW_LIMIT, xmlQueryOptions, null).OuterXml);
    }
    catch (FaultException fe)
    {
        MessageFault mf = fe.CreateMessageFault();
        if (mf.HasDetail)
        {
            XmlElement fexe = mf.GetDetail<XmlElement>();
            Console.WriteLine("\tError: " + fexe.OuterXml);
        }
    }
}

Note: This code won't iterate over every subweb; just the top level webs in the site collection.... but you probably get the idea.