System.Net.WebException: The remote server returned an error: (429)

Question

I'm executing a CSOM Powershell script on my SPO Tenant.

This script exports all list items and documents which has unique permissions.

The script is working fine however I get the below errors suddenly and randomly andthe execution stops in the middle.This is not occurring on any particular site/list/library.

Error 1:

System.Net.WebException: The remote server returned an error: (429).
at System.Net.HttpWebRequest.GetResponse()
at Microsoft.SharePoint.Client.SPWebRequestExecutor.Execute()
at Microsoft.SharePoint.Client.ClientRequest.ExecuteQueryToServer(ChunkStringBuilder sb)
at CallSite.Target(Closure , CallSite , Object )

Error 2:

System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote hos
t. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
 --- End of inner exception stack trace ---
 at System.Net.GZipWrapperStream.Read(Byte[] buffer, Int32 offset, Int32 size)
 at System.IO.StreamReader.ReadBuffer(Char[] userBuffer, Int32 userOffset, Int32 desiredChars, Boolean& readToUserBuffer)
 at System.IO.StreamReader.Read(Char[] buffer, Int32 index, Int32 count)
 at Microsoft.SharePoint.Client.TextPeekReader.FillBuffer()
 at Microsoft.SharePoint.Client.TextPeekReader.Peek(Int32 position)
 at Microsoft.SharePoint.Client.JsonReader.SkipWhiteSpace()
 at Microsoft.SharePoint.Client.JsonReader..ctor(TextReader reader, ClientRuntimeContext runtimeContext)
 at Microsoft.SharePoint.Client.ClientRequest.ProcessResponseStream(Stream responseStream)
 at Microsoft.SharePoint.Client.ClientRequest.ProcessResponse()
 at CallSite.Target(Closure , CallSite , Object )

How to avoid this error and make my script run through?

Below is my full script:

$StartTime = Get-date
$ErrorActionPreference="SilentlyContinue"

#Load SharePoint CSOM Assemblies
Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.dll"
Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.Runtime.dll"
Add-Type -Path "C:\Program Files\SharePoint Client Components\Assemblies\Microsoft.Online.SharePoint.Client.Tenant.dll"
Clear-Content -path "C:\George\ps_scripts\uniquepermissions.csv"

Function Invoke-LoadMethod() {
 param([Microsoft.SharePoint.Client.ClientObject]$Object = $(throw "Please provide a Client Object"),[string]$PropertyName) 
 $ctx = $Object.Context
 $load = [Microsoft.SharePoint.Client.ClientContext].GetMethod("Load") 
 $type = $Object.GetType()
 $clientLoad = $load.MakeGenericMethod($type)  
 $Parameter = [System.Linq.Expressions.Expression]::Parameter(($type), $type.Name)
 $Expression = [System.Linq.Expressions.Expression]::Lambda([System.Linq.Expressions.Expression]::Convert([System.Linq.Expressions.Expression]::PropertyOrField($Parameter,$PropertyName),[System.Object] ), $($Parameter))
 $ExpressionArray = [System.Array]::CreateInstance($Expression.GetType(), 1)
 $ExpressionArray.SetValue($Expression, 0)
 $clientLoad.Invoke($ctx,@($Object,$ExpressionArray)) 
  }

  $UserName="[email protected]"
  $Password ="XXXXXX"

  #Setup Credentials to connect
  $Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($UserName,(ConvertTo-SecureString $Password -AsPlainText -Force))

#Function to Get all lists from the web
Function Get-SPOList($Web)
{
    #Get All Lists from the web
    $Lists = $Web.Lists
    $Context.Load($Lists)
    $Context.ExecuteQuery()

    #Get all lists from the web   
    ForEach($List in $Lists)
    {
        #Get the List Name
     #Get All Lists of the web
      $Listname=$List.Title

   #Setup the context
   $Ctx = New-Object Microsoft.SharePoint.Client.ClientContext($WebURL)
   $Ctx.Credentials = $Credentials

$List = $Ctx.Web.Lists.GetByTitle($ListName)
$Ctx.Load($List)
$Ctx.ExecuteQuery()

if($List.Title -ne "User Information List" -and $List.Title -ne "Style Library" -and $List.Title -ne "Master Page Gallery" -and $List.Title -ne "Preservation Hold Library" -and $List.Title -ne "Theme Gallery" -and $List.Title -ne "Web Part Gallery" -and $List.Title -ne "Workflow History" -and $List.Title -ne "Workflow Tasks")
   {
Write-host "Total List Items Found in "$List.Title "is : " $List.ItemCount

#Batch process list items - to mitigate list threshold issue on larger lists
Do {  
    #Get 50000 items from the list
    $Query = New-Object Microsoft.SharePoint.Client.CamlQuery
    $Query.ViewXml = "<view Scope='RecursiveAll'><rowlimit>50000</rowlimit></view>"
    $ListItems = $List.GetItems($Query)
    $Ctx.Load($ListItems)
    $Ctx.ExecuteQuery()
      
    $Query.ListItemCollectionPosition = $ListItems.ListItemCollectionPosition

    #Loop through each List item
    ForEach($ListItem in $ListItems)
    {
        Invoke-LoadMethod -Object $ListItem -PropertyName "HasUniqueRoleAssignments"
        Invoke-LoadMethod -Object $ListItem -PropertyName "RoleAssignments"
        
        $Ctx.ExecuteQuery()
        if ($ListItem.HasUniqueRoleAssignments -eq $true)
        {        
           $Roles= $listItem.RoleAssignments
           $Ctx.load($Roles) 
           $Ctx.ExecuteQuery() 
           $RolesCount = $Roles.Count
                       
            If($RolesCount -gt 0) 
        { 
         for ($i = $RolesCount -1; $i -ge 0 ; --$i)   
           {    
              $Link=$Roles[$i].Member 
              $Ctx.Load($Link) 
              $Ctx.ExecuteQuery() 
              $LinkTitle = $Link.LoginName
              
             If($LinkTitle -like "SharingLinks*") 
              { 
               $Groups=$Ctx.Web.SiteGroups
               $Ctx.Load($Groups)
               $Ctx.ExecuteQuery()
 #Get Each member from the Group
Foreach($Group in $Groups)
{
    if($Group.Title -eq $LinkTitle)
      {
    #Getting the members
    $SiteUsers=$Group.Users
    $Ctx.Load($SiteUsers)
    $Ctx.ExecuteQuery()
    Foreach($User in $SiteUsers)
    {
     Write-Host -f Green "List Item '$($ListItem["Title"])' with ID '$($ListItem.ID)' has Unique Permissions with count " $RolesCount "Link:" $LinkTitle                   "with members:" $($User.Email)
              
           $ListTitle = $List.Title
           $ItemName = $ListItem.Title
           $ItemID = $ListItem.ID
           $Itemdata = Add-Content -Path C:\George\ps_scripts\uniquepermissions.csv -Value "$($WebURL.ToString()), $($ListTitle.ToString()),$($ItemID.ToString()),$($LinkTitle.ToString()),$($User.Email)"                         
        
             }
             }
             }               
             } 
             $Link = $null 
             }   
             $Ctx.ExecuteQuery()            
             }       
             }
             }
             } While ($Query.ListItemCollectionPosition -ne $null)
             }
             }
             }

#Function to get all webs from given URL
Function Get-SPOWeb($WebURL) 
{
    #Set up the context
    $Context = New-Object Microsoft.SharePoint.Client.ClientContext($WebURL)
    $Context.Credentials = $Credentials

    $Web = $context.Web
    $Context.Load($web)
    #Get all immediate subsites of the site
    $Context.Load($web.Webs)  
    $Context.executeQuery()

    #Call the function to Get Lists of the web
    $WebURL=$Web.URL
    Write-host "Processing Web :"$Web.URL
    Get-SPOList $Web
    
    #Iterate through each subsite in the current web
    foreach ($Subweb in $web.Webs)
    {
        #Call the function recursively to process all subsites underneaththe current web
        Get-SPOWeb($SubWeb.URL)
    }
}

function Get-SPOTenantSiteCollections
{
 param ($sSiteUrl,$sUserName,$sPassword)
try
{    
   $Sites = Get-SPOSite -Limit ALL
 Foreach ($Site in $Sites)
 {
 $siteURL = $site.URL
 $SiteState = $site.LockState
 write-host "Site State:" $SiteState
 if($SiteState -ne "ReadOnly")
  {
    Get-SPOWeb $SiteURL      
  }  
  }
  }
  catch [System.Exception]
  {
    write-host -f red $_.Exception.ToString()   
    $EndTime = get-date
   }    
   }

   #Required Parameters
   $sSiteUrl = "https://tenant-admin.sharepoint.com/" 
   $sUserName = "[email protected]" 
   $sPassword = Read-Host -Prompt "Enter your password: " -AsSecureString       
   $sPassword= convertto-securestring "<User_Password>" -asplaintext -force
   $userCredential = Get-Credential -UserName $sUsername -Message "Type the password."
   Connect-SPOService -Url $sSiteURL -Credential $userCredential

   Get-SPOTenantSiteCollections -sSiteUrl $sSiteUrl -sUserName $sUserName -sPassword $sPassword

   $EndTime = get-date

   write-host "Start Time:" $StartTime
   write-host "End Time:" $EndTime

Answer 1

It's resource throttling.

Try to add UserAgent to request:

$Ctx.add_ExecutingWebRequest({
param($Source, $EventArgs)
$request = $EventArgs.WebRequestExecutor.WebRequest
$request.UserAgent = "NONISV|George|SomeApp/1.0"
})

more info: https://docs.microsoft.com/en-us/sharepoint/dev/general-development/how-to-avoid-getting-throttled-or-blocked-in-sharepoint-online

in your code add this after place where you create ClientContext object:

[...]
#Setup the context
$Ctx = New-Object Microsoft.SharePoint.Client.ClientContext($WebURL)
$Ctx.Credentials = $Credentials

$Ctx.add_ExecutingWebRequest({
param($Source, $EventArgs)
$request = $EventArgs.WebRequestExecutor.WebRequest
$request.UserAgent = "NONISV|George|SomeApp/1.0"
})
[...]