InsufficientMemoryException: Insufficient winsock resources available?

Question

I have a Sharepoint 2010 solution running on Windows Server 2008 R2. Sometimes users are unable to log in to SharePoint, and the ULS log provide an System.InsufficientMemoryException: Insufficient winsock resources available to complete socket connection initiation.

I'm leaning towards that this may be solved by installing a hotfix according to the article Kernel sockets leak on a multiprocessor computer that is running Windows Server 2008 R2 or Windows 7.

However I'm reluctant to install a hotfix without more evidence that this hotfix solves the problem. How can I solve the InsufficientMemoryException?

The complete exception:

01/16/2013 16:30:00.89  w3wp.exe (0x0898)   0x1B84  SharePoint Foundation   Claims Authentication   fsq7    High    
Request for security token failed with exception: System.InsufficientMemoryException: Insufficient winsock resources available to complete socket connection initiation. ---> 
System.Net.WebException: Unable to connect to the remote server ---> 
System.Net.Sockets.SocketException: An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full 127.0.0.1:32843     
at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)     
at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Int32 timeout, Exception& exception)     
--- End of inner exception stack trace ---     
at System.Net.HttpWebRequest.GetRequestStream(TransportContext& context)     
at System.Net.HttpWebRequest.GetRequestStream()     
at System.ServiceModel.Channels.HttpOutput.WebRequestHttpOutput.GetOutputStream()     
--- End of inner exception stack trace ---    
Server stack trace:      
at System.ServiceModel.Channels.HttpOutput.WebRequestHttpOutput.GetOutputStream()     
at System.ServiceModel.Channels.HttpOutput.Send(TimeSpan timeout)     
at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.SendRequest(Message message, TimeSpan timeout)     
at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)     
at System.ServiceModel.Channels.SecurityChannelFactory 1.SecurityRequestChannel.Request(Message message, TimeSpan timeout)     
at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout)     
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)     at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)     
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)    
Exception rethrown at [0]:      
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)     
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)     
at Microsoft.IdentityModel.Protocols.WSTrust.IWSTrustContract.Issue(Message message)     
at Microsoft.IdentityModel.Protocols.WSTrust.WSTrustChannel.Issue(RequestSecurityToken rst, RequestSecurityTokenResponse& rstr)     
at Microsoft.IdentityModel.Protocols.WSTrust.WSTrustChannel.Issue(RequestSecurityToken rst)     
at Microsoft.SharePoint.SPSecurityContext.SecurityTokenForContext(Uri context, Boolean bearerToken, SecurityToken onBehalfOf, SecurityToken actAs, SecurityToken delegateTo)    03717636-ca0c-4ab3-901d-8240d8ba1d93

Answer 1

This is related to a memory leak issue with the cachefactory object not disposing correctly and can be solved by using this:

http://cachescope.codeplex.com/

or it can be also that you ran out of buffer space for TCP.

Check the socket connections are being closed properly by typing this in the command prompt:

netstat -an -p TCP | findstr "CLOSE_WAIT"

A socket application has been terminated, but Netstat reports the socket in a CLOSE_WAIT state. This could indicate that the client properly closed the connection (FIN has been sent), but the server still has its socket open. This could be the result of one instance (among all threads or processes) of the socket not being closed.

NOTE: It is normal to have a socket in the TIME_WAIT state for a long period of time. The time is specified in RFC793 as twice the Maximum Segment Lifetime (MSL). MSL is specified to be 2 minutes. So, a socket could be in a TIME_WAIT state for as long as 4 minutes. Some systems implement different values (less than 2 minutes) for the MSL.

http://support.microsoft.com/kb/137984

and

CLOSE_WAIT is the state a TCP endpoint enters when it has received a FINished segment from the remote TCP, indicating the remote TCP will not be sending any more data. The local TCP sends an ACKnowledgement segment and when that arrives at the remote, the remote TCP endpoint will be in FIN_WAIT_2.

99 times out of 10, when a connection remains in CLOSE_WAIT for more than a fraction of a second, it means that the local application (the one where CLOSE_WAIT is found) has not looked for, or has ignored the "close indication" on the socket - eg the zero-byte return from a read() call.

http://h30499.www3.hp.com/t5/System-Administration/netstat-CLOSE-WAIT-status/td-p/5185282

The problem is your program running on the local machine is not closing the socket. It is not a TCP tuning issue. A connection can (and quite correctly) stay in CLOSE_WAIT forever while the program holds the connection open. Once the local program closes the socket, the OS can send the FIN to the remote end which transitions you to LAST_ACK while you wait for the ACK of the FIN. Once that is received, the connection is finished and drops from the connection table (if you're end is in CLOSE_WAIT

http://www.pchelpforum.com/xf/threads/a-question-about-close_wait-state.90482/

also refer to this article:

http://blogs.msdn.com/b/sql_protocols/archive/2009/03/09/understanding-the-error-an-operation-on-a-socket-could-not-be-performed-because-the-system-lacked-sufficient-buffer-space-or-because-a-queue-was-full.aspx

conclusion is you need to update your server to the latest version (service packs), update sharepoint as kb's are vital to patch known issues and bug/loopholes for attacks, this should solve your tcp/ip issue from the hosted appliation side. IF you still get an issue than you would need to take it to microsoft as it would be an os issue.

hope it helps :)