SharePoint 2013 Provider-Hosted App "Azure Access Control Service is unavailable"

Question

I am currently in the middle of Provider-Hosted App hell. I have developed several test apps for my QA environment following the dev center tutorials (Create: https://goo.gl/dDis9Q , Publish: https://goo.gl/ELCvdT). Debugging provides no issue whatsoever. However, when I proceed with publishing and deploying the application, I continue to receive the dreaded The Azure Access Control service is unavailable. We are completely on-premise and based on how the application is configured, ACS should not even be on option. I have read, from multiple sources, that this error can be a bit misleading and the real issue lies much deeper. Regardless, I have reconfigured, recreated, and reconfirmed until complete exhaustion.

Thus, my question is, has anyone else experienced this issue and what was the resolution? I am looking for any possible tip that may lead to something I overlooked.

Environment

• 1 SP WFE Server
  • Server 2012 R2 Standard
  • (2) Web Applications
    • portal.domain.com
    • my.domain.com
  • App Domain Configured
    • *.domainapps.com
  • (2) Certificates
    • Self-signed: portal.domain.com
    • Self-signed: my.domain.com
• 1 Db Server
  • Server 2012 R2 Standard
  • SQL Server 2014
• 1 Web Host Server
  • Server 2012 R2 Standard
  • (1) Certificate
    • Self-signed: portalapps.domain.com
  • (1) Web Application (RemoteApp) under 443 website bound to the certificate

Fiddler Logs:

• (2 Unauthorized followed by 1 OK)
  • 401 - /dev/DevSite/_layouts/15/appredirect.aspx?instance_id=bd8e792d%2D3f2a%2D409b%2D8744%2D2890f643b113 HTTP/1.1
  • 401 - /dev/DevSite/_layouts/15/appredirect.aspx?instance_id=bd8e792d%2D3f2a%2D409b%2D8744%2D2890f643b113 HTTP/1.1
  • 200 - /dev/DevSite/_layouts/15/appredirect.aspx?instance_id=bd8e792d%2D3f2a%2D409b%2D8744%2D2890f643b113 HTTP/1.1
• (1 Unauthorized followed by 1 OK)
  • 401 - /RemoteApp/Pages/Default.aspx?SPHostUrl=https%3A%2F%2Fportal%2Edomain%2Ecom%2Fdev%2FDevSite&SPLanguage=en%2DUS&SPClientTag=0&SPProductNumber=15%2E0%2E4867%2E1000 HTTP/1.1
  • 200 - /RemoteApp/Pages/Default.aspx?SPHostUrl=https%3A%2F%2Fportal%2Edomain%2Ecom%2Fdev%2FDevSite&SPLanguage=en%2DUS&SPClientTag=0&SPProductNumber=15%2E0%2E4867%2E1000 HTTP/1.1

Inspector

SPAppToken=&SPSiteUrl=https%3A%2F%2Fportal.domain.com%2Fdev%2DevSite&SPSiteTitle=Developer+SIte&SPSiteLogoUrl=&SPSiteLanguage=en-US&SPSiteCulture=en-US&SPRedirectMessage=EndpointAuthorityMatches&SPErrorCorrelationId=0b83f49d-ec7c-f04b-0171-93318614b234&SPErrorInfo=The+Azure+Access+Control+service+is+unavailable.

Correlation Logs:

Exception Message:The Azure Access Control service is unavailable. Stacktrace: at Microsoft.SharePoint.Administration.SPSecurityTokenServiceDiscoveryManager.DiscoverApplicationSecurityTokenService(SPServiceContext serviceContext) at Microsoft.SharePoint.ApplicationServices.SPApplicationContext.GetApplicationSecurityTokenServicesUri(SPServiceContext serviceContext)…

Application:

SharePointContextProvider.CheckRedirectionStatus(Context, out redirectUrl) consistently returns RedirectionStatus.CanNotRedirect

Answer 1

After a long series of reconfiguration…and reconfiguration…and reconfiguration, it would appear that the issue was directly related to the certificates in use. As mentioned in many other blog posts, the Azure Access Control Service is Unavailable error is often a red herring that requires one to step back through the entire add-in configuration process. I did this many times with no luck. I eventually added an extra layer of logging to the SharePointContext and TokenHelper classes, which allowed me to determine that I was targeting the correct certificate serial, but the following method was returning null:

storedCert = store.Certificates.Find(X509FindType.FindBySerialNumber,ClientSigningCertificateSerialNumber,true)

Specifically, it was returning null due to the validOnly flag. Once I removed the flag, the targeted cert was returned but I received a plethora of errors relating to the certificate chain, provider type, etc. Reconfiguring the self-signed certificates through a variety of methods (makecert.exe, OpenSSL, changing provider types, adding self-signed root, etc.) failed to resolve the errors. The only temporary resolution I was able to maintain involved deploying the remote web solutions on the same web server that hosts SharePoint (following the structure of many tutorials). Unless I have a fundamental misunderstanding of the High-Trust S2S process, this should not be necessary – even for a development environment.

Permanent Resolution:

The issue was eventually resolved by generating a certificate from our internal CA. We did, however, have to upgrade said CA to generate certs with the SHA256 hashing algorithm. Once installed and configured, all S2S handshake issues were resolved.

Takeaway:

Do not be thrown off by the generic ACS error, as I have continued to experience this from time to time in the web traffic logs (even when the issue has nothing to do with trust). Always retrace your configuration steps and walk through the process of establishing your trust. Also, self-signed certificates can be a bit of a drag outside of a contained, “Hello World’ proof-of-concept environment.