0

We recently upgraded from SharePoint 2013. We used a fresh install, new servers, and restored a backup from the old server. The current stack looks like this:

  • Application/Web Server - Windows 2019, SharePoint 2016

  • Database Server - Windows 2019, SQL Server 2017

  • Authentication via Kerberos in an enterprise environment

  • Max file size is set at 2GB in Central Admin

Symptoms

  1. Initially, file upload/download via both WebDav and Web GUI were limited to 5 MB.
  2. Subsite creation also failed.

Troubleshooting

  1. Recreated web application
  2. Ran health analyzer
  3. We explicitly set content and request length values in web.config and machine.config
  4. The 5 MB limit came up for the large-file-chunk-size. We made it larger by running the following:

stsadm.exe -o setproperty -pn large-file-chunk-size -pv 50000000

Current State

  • Files up to 50 MB work now.
  • Subsite creation works now.

Problem

We have a limited workaround. We have media files that exceed 200 MB, but I'm concerned about performance implications of increasing the chunk size that far. It seems clear that this behavior is out of the norm. Files should be allowed more than one chunk.

What else do I need to look at regarding this issue?

2
  • I wonder why the subsite creation would benefit from the increased large-file-chunk-size. Was there any explicit error given when creating the subsite? Can you include the original error message for that case or the relevant ULS log from that operation? – J. Tihon Sep 13 '20 at 22:47
  • To add, chunk size is irrelevant and has nothing to do with WebDAV or file upload. The OP needs to post a fiddler and/or network trace to get more details of what is going on. – Trevor Seward Sep 17 '20 at 5:30
1

The Maximum Upload file Size can be changed in CA-> Web Application general settings:

enter image description here

You could refer to this article: https://social.technet.microsoft.com/wiki/contents/articles/34128.sharepoint-2016-file-size-limit.aspx

In SharePoint 2016, Default File Size limit is set at 2GB and can be increased up to 10GB. 10GB is a boundary and you can't go beyond it.

1
  • I appreciate the response. The current limit was already set at 2GB. I'll update the question to reflect this. The only limit I can find is the chunk size. – dennythecoder Sep 11 '20 at 5:09
1

What limit is set in the WebDAV client (on your local computer)? You may also want to increase this if you use WebDAV to upload and download large files. See https://www.imageway.com/2017/email-hosting-blog/email-hosting-tutorials/increase-webdav-file-limit-in-windows Paul

4
  • Current limit is set at 181232896. WebDAV works with other servers. Additionally, this wouldn't explain the initial symptom of browser downloads & subsite creation failing. – dennythecoder Sep 11 '20 at 5:43
  • Good. We can exlude the client-side WebDAV settings as the cause. Networking: is there a proxy server between the clients and the server? i.e. can you establish a direct connection? Just double checking: is there a load balancer in the netwerk path? – Paul Sep 11 '20 at 7:58
  • No proxy or load balancing. – dennythecoder Sep 11 '20 at 13:27
  • Install Fiddler and capture the network traffic between the client and SharePoint. Check if the full file is being transferred to the server during an upload or that it breaks at an earlier stage. The other approach to use chunks to upload may offer a wrkaround but it does not solve the underlying configuration problem. – Paul Sep 17 '20 at 5:38
0

Here is my PowerShell solution to copy large files into small chunks

I know this may not suits you but it can help you out in case of copying large files via PowerShell as small chunks. So in our case we had 500MB pptx files too which we just uploaded to SP Online from On-Prem without any problem. Currently we used 10MB chunk size which can be altered as your wish.You can loop on whole library files too. If the file size is less then chunk size , file will be uploaded without chunking. Script uses Client Context

         Add-Type –Path "C:\Program Files\Common Files\microsoft shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.dll"
         Add-Type –Path "C:\Program Files\Common Files\microsoft shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.Runtime.dll"

        Function UploadFileInSlice ($DestinationCtx, $SourceCtx, $SourceFileUrl, $DestinationFolderUrl, $fileName, $fileChunkSizeInMB) {

# Each sliced upload requires a unique ID.
$UploadId = [GUID]::NewGuid()

# Get File by Server Relative URL
$File = $SourceCtx.Web.GetFileByServerRelativeUrl($SourceFileUrl)
$SourceCtx.Load($File)

# Get file Steam with OpenBinarySteam
$StreamToUpload = $File.OpenBinaryStream() 
$SourceCtx.ExecuteQuery()

# File size in bytes
$FileSize = ($File).length  

# Get Destination Folder by Server Relative URL
$DestinationFolder = $DestinationContext.Web.GetFolderByServerRelativeUrl($DestinationFolderUrl)
$DestinationCtx.Load($DestinationFolder)
$DestinationCtx.ExecuteQuery()

# Set Complete Destination URL with Destination Folder + FileName
$destUrl = $DestinationFolderUrl + "/" + $fileName
# File object.
[Microsoft.SharePoint.Client.File] $upload

# Calculate block size in bytes.
$BlockSize = $fileChunkSizeInMB * 1000 * 1000 
Write-Host "File Size is: $FileSize bytes and Chunking Size is:$BlockSize bytes"

if ($FileSize -le $BlockSize)
{
    # Use regular approach if file size less than BlockSize
    Write-Host "File uploading with out chunking"
    $upload =[Microsoft.SharePoint.Client.File]::SaveBinaryDirect($DestinationCtx, $destUrl, $StreamToUpload.Value, $true)
    return $upload
}
else
{
    # Use large file upload approach.
    $BytesUploaded = $null
    $Fs = $null
    
    Try {
        $br = New-Object System.IO.BinaryReader($StreamToUpload.Value)
        
        #$br = New-Object System.IO.BinaryReader($Fs)
        $buffer = New-Object System.Byte[]($BlockSize)
        $lastBuffer = $null
        $fileoffset = 0
        $totalBytesRead = 0
        $bytesRead
        $first = $true
        $last = $false
        # Read data from file system in blocks. 
        while(($bytesRead = $br.Read($buffer, 0, $buffer.Length)) -gt 0) {
            $totalBytesRead = $totalBytesRead + $bytesRead
            # You've reached the end of the file.
            if($totalBytesRead -eq $FileSize) {
                $last = $true
                # Copy to a new buffer that has the correct size.
                $lastBuffer = New-Object System.Byte[]($bytesRead)
                [array]::Copy($buffer, 0, $lastBuffer, 0, $bytesRead)
            }

            If($first)
            {
                $ContentStream = New-Object System.IO.MemoryStream
                # Add an empty file.
                $fileCreationInfo = New-Object Microsoft.SharePoint.Client.FileCreationInformation
                $fileCreationInfo.ContentStream = $ContentStream
                $fileCreationInfo.Url = $fileName
                $fileCreationInfo.Overwrite = $true
                #Add file to Destination Folder with file creation info
                $Upload = $DestinationFolder.Files.Add($fileCreationInfo)
                $DestinationCtx.Load($Upload)

                # Start upload by uploading the first slice.
                $s = New-Object System.IO.MemoryStream(,$Buffer)
                Write-Host "Uploading id is:"+$UploadId
                # Call the start upload method on the first slice.
                $BytesUploaded = $Upload.StartUpload($UploadId, $s)
                $DestinationCtx.ExecuteQuery()

                # fileoffset is the pointer where the next slice will be added.
                $fileoffset = $BytesUploaded.Value
                Write-Host "First patch of file with bytes"+ $fileoffset 
                # You can only start the upload once.
                $first = $false
            }
            Else
            {
                # Get a reference to your file.
                $Upload = $DestinationCtx.Web.GetFileByServerRelativeUrl($destUrl);
                If($last) {
                    # Is this the last slice of data?
                    $s = New-Object System.IO.MemoryStream(,$lastBuffer)

                    # End sliced upload by calling FinishUpload.
                    $Upload = $Upload.FinishUpload($UploadId, $fileoffset, $s)
                    $DestinationCtx.ExecuteQuery()

                    Write-Host "File Upload Completed Successfully!"
                    # Return the file object for the uploaded file.
                    return $Upload
                }
                else {
                    $s = New-Object System.IO.MemoryStream(,$buffer)
                    # Continue sliced upload.
                    $BytesUploaded = $Upload.ContinueUpload($UploadId, $fileoffset, $s)
                    $DestinationCtx.ExecuteQuery()

                    # Update fileoffset for the next slice.
                    $fileoffset = $BytesUploaded.Value
                    Write-Host "File uploading is in progress with bytes: "+ $fileoffset 
                }
            }

        }  #// while ((bytesRead = br.Read(buffer, 0, buffer.Length)) > 0)
    }
    Catch {
        Write-Host $_.Exception.Message -ForegroundColor Red
    }
    Finally {
        if ($Fs -ne $null)
        {
            $Fs.Dispose()
        }
    }
}
return $null
}

     #URL to Configure, in this case Destination is SP Online site URL
     #Adding up credentials hard-code, you can use Get-Credentails PS command too
    $DestnationSiteUrl = "https://your-domain.sharepoint.com/sites/xyz"
    $DestinationRelativeURL = "/sites/xyz/TestLibrary" #server relative URL here with library Name and Folder name
    $DestinationUserName = "xyz@your-domain.com"
    $DestinationPassword = Read-Host "Enter Password for Destination User: $DestinationUserName" -AsSecureString

   #URL to Configure, in this case Source is On-Prem site URL
   #Adding up credentials hard-code, you can use Get-Credentails PS command too
   $SourceSiteUrl = "http://intranet/sites/xyz"
   $SourceRelativeURL = "/sites/xyz/TestLibrary/myfile.pptx"          #server relative URL here with library Name and file name with extension
   $SourceUsername = "domain\xyz"
   $SourcePassword = Read-Host "Enter Password for Source User: $SourceUsername" -AsSecureString
   #Set a file name with extension
   $FileNameWithExt = "myfile.pptx"

   #Get Source Client Context with credentials 
   $SourceContext = New-Object Microsoft.SharePoint.Client.ClientContext($SourceSiteUrl) 
   #Using NetworkCredentials in case of On-Prem
   $SourceCtxcredentials = New-Object      System.Net.NetworkCredential($SourceUsername, $SourcePassword)
   $SourceContext.RequestTimeout =    [System.Threading.Timeout]::Infinite
   $SourceContext.ExecuteQuery();

  #Get Destination Client Context with credentials 
  $DestinationContext = New-Object   Microsoft.SharePoint.Client.ClientContext($DestnationSiteUrl) 
  #Using SharePointOnlineCredentials in case of SP-Online
  $DestinationContext.Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($DestinationUserName, $DestinationPassword)
  $DestinationContext.RequestTimeout = [System.Threading.Timeout]::Infinite
  $DestinationContext.ExecuteQuery();

   #All Set up, now just call the UploadFileInSlice with parameters
  $UpFile = UploadFileInSlice -DestinationCtx $DestinationContext -SourceCtx $SourceContext -DestinationFolderUrl        $DestinationRelativeURL -SourceFileUrl $SourceRelativeURL -fileName $FileNameWithExt -fileChunkSizeInMB 10

Here is the complete documentation for this one https://github.com/alimughalx/CopySharePointFileInChunks

1
  • I appreciate the script, but it does not address the problem we are experiencing. – dennythecoder Sep 16 '20 at 22:42
0

Try to disable Window Auto-Tuning feature on the clients. Run netsh int tcp set global autotuninglevel=disabled as an admin in a CMD. To enable it again run netsh int tcp set global autotuninglevel=normal. You can check the current settings usingnetsh interface tcp show global

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.