Avoid NTLM Authorization Handshake On Every Request to the REST API?

Question

I am accessing SharePoint 2010 REST API with a System.Net.Http.HttpClient. The server is set up with NTLM authentication, and I am accessing it over the Internet. Every time I send a request from my code, the service goes through NTLM authentication handshake, causing two 401s before showing the data. Here are two requests from my code:

When I do the same thing with the browser, only the first request goes through 401 handshake, with following requests getting an immediate 200 response:

Subsequent requests from the browser do not contain the Authorization header, and other than a few informative headers, are identical to what my code is sending:

GET /_vti_bin/ListData.svc/SharedDocuments HTTP/1.1
Host: contoso.com
Accept: */*
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.84 Safari/537.36
Cookie: WSS_KeepSessionAuthenticated={40c2b58d-f21f-4254-aec4-97a798dd17b5}

The server replies with 200 OK to the above request, though not always. After a while, it again requires the authentication handshake, and then again subsequent requests are served immediately without further authentication. I've tried this with Chrome on a Mac and with IE11 on Windows 7, and the behavior is the same.

My code sends the same WSS_KeepSessionAuthenticated cookie as the browser:

GET /_vti_bin/ListData.svc/SharedDocuments HTTP/1.1
Connection: keep-alive
Host: contoso.com
Cookie: WSS_KeepSessionAuthenticated={40c2b58d-f21f-4254-aec4-97a798dd17b5}

The server is always sending back the NTLM challenge to all my requests, forcing the handshake, even when two requests are sent from the same instance of HttpClient.

Here is the code I am using to contact the server:

let baseUri url = 
    let u = Uri url
    UriBuilder(u.Scheme, u.Host).Uri

let ntlmCred username password baseUri = 
    let cc = System.Net.CredentialCache()
    cc.Add(baseUri, "ntlm", NetworkCredential(username, (password:string)))
    cc

[<EntryPoint>]
let main argv =
    let username = argv.[0]
    let password = argv.[1] 
    let url = argv.[2]
    let baseAddr = baseUri url

    use w = new WebRequestHandler()
    w.PreAuthenticate <- true
    w.Credentials <- ntlmCred username password baseAddr
    w.UnsafeAuthenticatedConnectionSharing <- true

    w.CookieContainer.Add(new Cookie("WSS_KeepSessionAuthenticated", "{40c2b58d-f21f-4254-aec4-97a798dd17b5}", "/", baseAddr.Authority))

    use client = new HttpClient(w) 
    let r = client.GetAsync(url).Result
    // This second request from the same instance still causes the NTLM authorization handshake
    client.GetAsync(url).Result |> ignore
    r |> printfn "%A"
    0

My understanding was that providing a valid WSS_KeepSessionAuthenticated cookie should side-step the constant authorization on every request, but it does not seem to be the case, even though the cookie is required for rest calls (otherwise the server bombs with a 400 bad request).

Is there a way to avoid authentication handshake on every single call?

Answer 1

It looks like you're using C# but, FWIW, if you can use Java, I've found the Apache HttpClient (4.3.5) support for NTLM to be fantastic and I only see new challenges after the client context has be inactive for quite a while.

Edit BTW, here's one trick you might use if the challenge continues to happen on every REST call and you can't find a better library. As I've observed the challenge\response dance of NTLM, I've seen that it sends the entire request body back and forth each time (it appears to be 3 times based on my traces). So, any time you're going to be making a large payload call such as a large file upload, issue a small payload GET first. In my work, I've done this with a GET on the web root with ?$select=ServerRelativeUrl. I do this because, as I said, if my connection has gone unused for some period of time, the challenge may occur again under the covers. So, if I issue a small payload GET first, the negotiation can happen back and forth with a small payload.