Take the 2-minute tour ×
SharePoint Stack Exchange is a question and answer site for SharePoint enthusiasts. It's 100% free, no registration required.

Yesterday, we suffered with the Global outage on our SharePoint 2010 farm as the "User Profile Service Application" was inaccessible during those hours.

Exception:
01/13/2012 08:41:20.16 OWSTIMER.EXE (0x2E44) 0x11DC SharePoint Portal Server User Profiles hyc6 High Exception occured while connecting to WCF endpoint: System.TimeoutException: The request channel timed out while waiting for a reply after 00:00:19.9843750. Increase the timeout value passed to the call to Request or increase the SendTimeout value on the Binding. The time allotted to this operation may have been a portion of a longer timeout. ---> System.TimeoutException: The HTTP request to 'http://mySAFserver:32843/7512455d976346808bf682dc83abe0f3/ProfilePropertyService.svc' has exceeded the allotted timeout of 00:00:20. The time allotted to this operation may have been a portion of a longer timeout. ---> System.Net.WebException: The operation has timed out at System.Net.HttpWebRequest.GetResponse() at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)

Solved:
Restarted the "User Profile Sync service" which solves the issue.

Wonder, If I could find the reason why the "User Profile Application" was inaccessible.

suspecting the below reasons:

  1. System might be suffered with the low resources (Ram/CPU/IO)
    Logs: 01/13/2012 09:00:01.68 OWSTIMER.EXE (0x2E44) 0x2610 SharePoint Foundation Health 2137 Critical The SharePoint Health Analyzer detected an error. Drives are running out of free space. Available drive space is less than twice the value of physical memory. This is dangerous because it does not provide enough room for a full memory dump with continued operation. It also could cause problems with the Virtual Memory swap file: SYSTEM (mySAFserver - C:).

however, the system has enough(15GB of freespace) space in "C:\" drive and 5GB of available RAM, so we may ignore this point.

  1. Application pool could have been stopped/crashed.
    Logs:Either the User Profile Application (UPA) service is offline and cannot service this request, or the current application pool identity does not have rights to connect. In certain configurations, you may need to publish the UPA from the Manage Service Applications page in Central Administration. \n The UPA says: "System.TimeoutException"

    Logs:
    UserProfileServiceUserStatisticsWebPart:LoadControl failed, Exception: Microsoft.Office.Server.UserProfiles.UserProfileApplicationNotAvailableException: This User Profile Application's connection is currently not available. The Application Pool or User Profile Service may not have been started. Please contact your administrator. at Microsoft.SharePoint.Portal.UserProfiles.AdminUI.ProfileAdminPage.get_CurrentApplicationProxy() at Microsoft.SharePoint.Portal.WebControls.UserProfileServiceStatisticsWebPartBase.LoadControl(Object sender, EventArgs e)

however, there was no clear indication against the application pool, so we may ignore this point as well.

  1. Database coud have not accepted the new connection request due to the huge amount of existing connections.
    Logs: ConnectionString: 'Data Source=MUVMP168;Initial Catalog="UPA Profile DB";Integrated Security=True;Enlist=False;Asynchronous Processing=False;Connect Timeout=15' ConnectionState: Closed ConnectionTimeout: 15

We have observed huge amount of SQL session was opened(3K) during that time however could not find enough details on the SQL front. Also, there is no direct log states an issue with the database connectivity.

  1. Network could have been down/unresponsible.
    No clue found on this.
share|improve this question
    
While I appreciate your desire to determine the root cause, I have a suspicion this issue will be hard to reproduce, it may have been a freak coincidence. I would suggest monitoring the farm for this to happen again, the good news is that you have a resolution for when it does. –  shufler Jan 19 '12 at 0:01
    
As part of the monitoring that @shufler suggests, I would also be sure to monitor the system's available RAM. 5GB of Ram is not enough to run a production system in nearly all cases. Lack of ram will lead to instability and process timeouts especially during peak times or while Search Crawl activites are running. –  Mike Oryszak Jan 22 '12 at 5:42
    
I've seen this on our dedicated Search Server where the User Profile service runs into this issue on occasion. Our production machines have the right amount of memory, but stopping and restarting services has not resolved it either –  MichaelF Apr 1 '13 at 15:35

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.