4

I have some SP .NET Object library code that will allow me to query for the contents of a specific folder in a document library. It worked fine in a UAT environment where there were only a few dozen folders, but when I moved it to Production where there were 14K folders, it won't return the folders anymore.

I assume that I need a way to get directly to the folder, instead of retrieving the list of folders and then getting the one I need, but I don't know how to do it? I think i could do it with a CAML query, but I can't find a way to get directly to the folder and get the list of files?

Assumumptions:
Site Name = https://xyz.sharepoint.com/sites/CRMDocs/

Library Friendly Name = Case

Library Path = incident

Sample Folder = 0312380e

Full Path = https://xyz.sharepoint.com/sites/CRMDocs/incident/0312380e

Web web = clientContext.Web;
ListCollection lists = web.Lists;

var docLib = web.Lists.GetByTitle(entityFriendlyName);
clientContext.Load(docLib, d => d.Title, d => d.RootFolder.Name);
clientContext.ExecuteQuery();

String folderUrl = String.Format("/{0}/{1}", docLib.RootFolder.Name, folderName);

var folders = docLib.RootFolder.Folders;
clientContext.Load(folders, fldrs => fldrs.Include(fldr => fldr.ServerRelativeUrl));
clientContext.ExecuteQuery();

// When there are more than 5K folders, folders collection is NULL
var folder = folders.FirstOrDefault(f => f.ServerRelativeUrl.ToLower().EndsWith(folderUrl.ToLower()));
if (folder != null)
{
    var files = folder.Files;

    clientContext.Load(files);
    clientContext.ExecuteQuery();
    // Process Files
}
  • For getting folders try to use caml like here instead of using directly docLib.RootFolder.Folders – Sergei Sergeev Dec 20 '16 at 18:27
  • you can use content iterator class from sharepoint in your code. that may help – SharePointMan Dec 20 '16 at 18:30
  • I've tried the caml query, but I still get this: – Chris Harrington Dec 20 '16 at 20:56
  • "The attempted operation is prohibited because it exceeds the list view threshold enforced by the administrator." – Chris Harrington Dec 20 '16 at 20:57
5

In SharePoint Online, your only choice is to use REST. CSOM will not work. You call the REST endpoint and it will return a batch of 100 or 200 items, and if more exist, will return a paged link to access the next batch of data.

I've used this technique in the past to update 100k+ items in a document library with success (in PowerShell), CSOM would not work. Marc Anderson has it detailed here, http://sympmarc.com/2016/11/30/25696.

This is a sample from when I used it in PowerShell:

#https://www.itunity.com/article/working-lists-list-items-sharepoint-rest-service-windows-powershell-2077

function Get-Items{
    Param(
        [Parameter(Mandatory=$true,Position=0)][ValidateNotNull()]
        [string]$Url
    )
    $list = Invoke-SPORestMethod -Url "https://tenant.sharepoint.com/sites/site/_api/Web/Lists/getbytitle('Documents')" 
    $itemType = $list.ListItemEntityTypeFullName 
    $digest = (Invoke-SPORestMethod -Url "https://tenant.sharepoint.com/sites/site/_api/contextinfo" -Method "POST").GetContextWebInformation.FormDigestValue 
    $listItems = Invoke-SPORestMethod -Url $Url
    Write-Host "Processing batch starting at "$listItems.results[0].Id -ForegroundColor Yellow

    foreach($li in $listItems.results){
        $newTitle = $li.FileLeafRef.ToString()
        if($li.Title -ne $newTitle){
            $id = $li.Id
            $pgurl = "https://tenant.sharepoint.com/sites/site/_api/Web/Lists/getbytitle('Documents')/items($id)"
            Write-Host "Updating $pgurl" -ForegroundColor White
            $metadata = "{ '__metadata': { 'type': '$itemType' }, 'Title': '$newTitle'}"
            $item = Invoke-SPORestMethod -Url $pgurl -Method "POST" -XHTTPMethod "MERGE" -Metadata $metadata -RequestDigest $digest -ETag "*"
        }
    }
    Write-host "Batch complete" -ForegroundColor Green
    if($listItems.__next){
        Get-Items -Url $listItems.__next    
    }
}
cls
Get-Items -Url "https://tenant.sharepoint.com/sites/site/_api/Web/Lists/getbytitle('Documents')/items?`$select=FileLeafRef,Title,Id&`$orderby=Id%20desc&`$filter=(ID%20le%20313031)"
  • How do you A: pass in credentials? And B: overcome the CORS message? I am working on a .net web API proxy app that sits between Dynamics CRM online and SharePoint online – Chris Harrington Dec 21 '16 at 11:46
  • The invoke-sporestmethod in this case references a credential object username and password that is supplied previously when PowerShell is launched. There shouldn't be a CORS problem, I haven't seen one using REST against SPO. – Eric Alexander Dec 21 '16 at 13:02
  • Is there a C# / .NET equivalent to the invoke-sporestmethod? – Chris Harrington Dec 21 '16 at 13:05
  • I would imagine the WebRequest class – Eric Alexander Dec 21 '16 at 13:10
  • I found some code to connect to the REST API, but I am now getting 401 UNAUTHORIZED errors. I have good credentials, so I know that's not the problem. – Chris Harrington Dec 21 '16 at 15:02
0

Not a specific answer, but I did finally find a c# set of functions that will allow you to connect to SPOnline and pass in a REST call.

http://journeyintocrm.com/archives/1152

0

Actually, after working with this some more, I found an even better way that does not require a REST call.

Once you have the SharePoint clientcontext.Web, make a call to web.GetFolderByServerRelativeUrl(folderNameString);

Web web = clientContext.Web;
ListCollection lists = web.Lists;

var docLib = web.Lists.GetByTitle(entityFriendlyName);
clientContext.Load(docLib, d => d.Title, d => d.RootFolder.Name);
clientContext.ExecuteQuery();

String folderUrl = String.Format("/{0}/{1}", docLib.RootFolder.Name, folderName);

Folder folder = web.GetFolderByServerRelativeUrl(folderUrl);

if (folder != null)
{
    var files = folder.Files;

    clientContext.Load(files);
    clientContext.ExecuteQuery();
    // Process Files
}

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.