One of the Application server can not connect to DB server

Question

We have a SharePoint 2013 Farm with 4 Application serves, 4 Web Front End Servers and 5 DB servers. Everything was running fine. Few days back we identified that the App-server on which Search was configured was not able to connect to the database server. Following errors were found in the logs.

A database error occurred. Source: .Net SqlClient Data Provider Code: 0 occurred 0 time(s) Description: Error ordinal: 1 Message: The target principal name is incorrect. Cannot generate SSPI context., Class: 11, Number: 0, State: 0 at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action1 wrapCloseInAction) at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose) at System.Data.SqlClient.TdsParser.TdsLogin(SqlLogin rec) at System.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean ignoreSniOpenTimeout, TimeoutTimer timeout, Boolean withFailover) at System.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString connectionOptions, SqlCredential credential, TimeoutTimer timeout) at System.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(TimeoutTimer timeout, SqlConnectionString connectionOptions, SqlCredential credential, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance) at System.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, SqlCredential credential, Object providerInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString userConnectionOptions) at System.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions userOptions) at System.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnectionPool pool, DbConnectionOptions options, DbConnectionPoolKey poolKey, DbConnectionOptions userOptions) at System.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnectionOptions userOptions) at System.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnectionOptions userOptions) at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection) at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, TaskCompletionSource1 retry, DbConnectionOptions userOptions, DbConnectionInternal& connection) at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource1 retry, DbConnectionOptions userOptions, DbConnectionInternal& connection) at System.Data.ProviderBase.DbConnectionClosed.TryOpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource1 retry, DbConnectionOptions userOptions) at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry) at System.Data.SqlClient.SqlConnection.Open() at Microsoft.Office.Server.Data.SqlSession.OpenConnection() at Microsoft.Office.Server.Data.SqlSession.ExecuteNonQuery(SqlCommand command) at Microsoft.Office.Server.Search.ManagedSqlSession.ExecuteNonQuery()

and

WebHost failed to process a request. Sender Information: System.ServiceModel.ServiceHostingEnvironment+HostingManager/12036987 The farm is unavailable. at Microsoft.SharePoint.Administration.Claims.SPSecurityTokenServiceManager.get_Local() at Microsoft.SharePoint.IdentityModel.SPServiceIssuerNameRegistry..ctor() at Microsoft.SharePoint.SPServiceHostOperations.Configure(ServiceHostBase serviceHost, SPServiceAuthenticationMode authenticationMode) at Microsoft.Office.Server.Search.Administration.SearchServiceHostFactory.CreateServiceHost(String constructorString, Uri[] baseAddresses) at System.ServiceModel.ServiceHostingEnvironment.HostingManager.CreateService(String normalizedVirtualPath, EventTraceActivity eventTraceActivity) at System.ServiceModel.ServiceHostingEnvironment.HostingManager.ActivateService(ServiceActivationInfo serviceActivationInfo, EventTraceActivity eventTraceActivity) at System.ServiceModel.ServiceHostingEnvironment.HostingManager.EnsureServiceAvailable(String normalizedVirtualPath, EventTraceActivity eventTraceActivity) --- End of inner exception stack trace --- at System.ServiceModel.ServiceHostingEnvironment.HostingManager.EnsureServiceAvailable(String normalizedVirtualPath, EventTraceActivity eventTraceActivity) at System.ServiceModel.ServiceHostingEnvironment.EnsureServiceAvailableFast(String relativeVirtualPath, EventTraceActivity eventTraceActivity) Process Name: w3wp Process ID: 18820

NOTE :

• IIS services is running.
• We are able to ping the DB server from this server.

Answer 1

follow the below trouble shooting steps:

1. check whether there are any listening port or firewall restrictions such as 1433 or 1434 for default or named sql instance.

2. enable the verbose error log details to get more detailed errors.

3. see if there are any recent patch upgrades including system security patches also make sure you compare these with the other SharePoint servers in the farm.

4. as a final step, run the config wizard so it will correct if there are any problems or any corrupted files. (This step will let you know if the server is still in the farm or it may ask you to reconnect. Here you try to use either the fully qualified domain name of SQL or named instance details. Also, this will step will tell you if there is any problem with the service account)

5. when you say 5 DB servers then make sure how the fail-over settings were configured properly. check if there is any recent fail over.

6. from SQL end, try to inspect SQL logs w.r.to app server for root cause.