After support from Microsoft had a look, we found that the issue was caused by the fact that the Custom Script was not activated for the Site Collection. In case you are experiencing similar issue in the future, you can use the following steps to resolve it:
- Open PowerShell as Admin
- Type in Connect-SPOService "https://-admin.sharepoint.com" -credential "admin full email address"
- Type in Set-SPOsite "SiteURL" -DenyAddAndCustomizePages 0
Additionally this may resolve any issues you might have with connecting via SharePoint Designer, as this has the same culprit. After changing the value for DenyAddAndCustomizePages both issues go away.
If you want to check the configuration value for any site collection you can use the following command, after connecting with Connect-SPOService as described above:
- Get-SPOSite "SiteUrl" |fl