Why is my client certificate not being included in the TLS handshake?

Question

I wrote a Windows service a few years back that processed data from several sources, one of which was a third party SOAP Web Service. It authenticated with this Web Service using a client certificate provided by the third party. This worked fine at the time, and is still working fine on the original machine it was deployed to, but now they are migrating to a new machine where the connection attempt is throwing an exception with the message:

The HTTP request was forbidden with client authentication scheme 'Anonymous'

The same thing is now also occurring on my development machine. This is the code that configures the connection to the web service:

Friend Shared Function GetProperlyConfiguredConnection() As SEMOService.MIWebServiceClient
    ' Ensure that the settings are valid
    ValidateSettings()

    Dim destAddress As New System.ServiceModel.EndpointAddress(Url)
    ' The service uses SOAP 1.1 which is what BasicHttpBinding uses. WSHttpBinding uses SOAP 1.2.
    'Dim destBinding As New System.ServiceModel.WSHttpBinding
    Dim destBinding As New System.ServiceModel.BasicHttpBinding
    With destBinding
        .CloseTimeout = New TimeSpan(0, 1, 0)
        .OpenTimeout = New TimeSpan(0, 1, 0)
        .ReceiveTimeout = New TimeSpan(0, 10, 0)
        .SendTimeout = New TimeSpan(0, 1, 0)
        .BypassProxyOnLocal = False
        .HostNameComparisonMode = ServiceModel.HostNameComparisonMode.StrongWildcard
        .MaxBufferPoolSize = 524288
        .MaxReceivedMessageSize = 2147483647
        .MessageEncoding = ServiceModel.WSMessageEncoding.Text
        .TextEncoding = System.Text.Encoding.UTF8
        .UseDefaultWebProxy = True
        .AllowCookies = False

        With .ReaderQuotas
            .MaxDepth = 32
            .MaxStringContentLength = 2147483647
            .MaxArrayLength = 50000000
            .MaxBytesPerRead = 4096
            .MaxNameTableCharCount = 16384
        End With

        .Security.Mode = ServiceModel.BasicHttpSecurityMode.Transport
        .Security.Transport.ClientCredentialType = ServiceModel.HttpClientCredentialType.Certificate
    End With

    Dim wcfService As New SEMOService.MIWebServiceClient(destBinding, destAddress)

    ' Load the certificate from the specified file.
    Dim keyData As Byte()
    Dim keyFileInfo As New IO.FileInfo(SEMOServiceSettings.KeyFilePath)
    Dim keyFileReader As New IO.BinaryReader(New IO.FileStream(SEMOServiceSettings.KeyFilePath, IO.FileMode.Open, IO.FileAccess.Read))
    keyData = keyFileReader.ReadBytes(keyFileInfo.Length)
    keyFileReader.Close()

    Dim cert As New X509Certificates.X509Certificate2
    cert = New X509Certificates.X509Certificate2(keyData, SEMOServiceSettings.KeyFilePassword)

    wcfService.ClientCredentials.ClientCertificate.Certificate = cert

    Return wcfService
End Function

It is not actually a WCF web service, but Visual Studio didn't seem to mind when auto-generating the client code from the WSDL. The client certificate is loaded up from a file rather than from the certificate store. The same certificate file is in use on all the machines, and appears to be loaded up fine when stepping through the code.

I compared the requests being made from the machine that works to one from my dev machine using WireShark, and it appears that the client certificate is not being included in the handshake when it should be:

No client certificates

This is the corresponding packet from a request on the machine that works:

Full certificate chain

There is much about WCF, SOAP and cryptography that I don't understand, so I am at a bit of a loss as to what could be different about the environments to result in this behaviour, and what needs to change to correct it.

This question seems to be related, but I can't seem to access any RequireClientCertificate property through code, and am not using an app.config to configure the binding.

It is possible to add a callback to the ServicePointManager class for performing custom validation of the server certificate. Does the base client class or the binding perform some validation of the client certificate before sending it to the server? If so, is there a way I can intervene in that process in the same way that I can for the server certificate validation, so that I can see what is going on?

Answer 1

I found the solution, though I'm not sure I understand it fully. The problem was that the client certificate (or perhaps just some of the other certificates in the chain) needed to be present in the Personal certificate store of the user making the request. I logged on as the user that the Windows Service runs as, imported the certificate into its store, and everything is working now.

This suggests that client certificates are validated in some way even if they are loaded from files rather than referenced in the store. This was actually evident in the output of ProcMon, if I'd been paying better attention I would have realised that it was searching the certificate store and coming up with NOT FOUND results.

It would be nice if Microsoft's WCF client code threw an exception if it has a problem with a certificate, rather than just trying to carry on without it. Ah well...

Answer 2

Seems like this may be a deployment problem with their new server. I would only say this because of the fact that "it worked before". I would have them check the configuration, specifically the app-pool and the application/user account that is supposed to be running it. There is a good chance that they installed the service under the System or Local Service that doesn't have rights to access something. In many cases I have seen a deployement where they copied everything, to include the service/user account, but never created the account! This would give you that error, and if the service doesn't have permissions to access the machine certs or run the chain, it will never request for the client to send the certificate!

Check out "The HTTP request was forbidden with client authentication scheme 'Anonymous'?

EDIT: To answer your other questions, which I didn't before... The client does not do any type of check of the client certificate before sending, with the exception that it uses the certificate to sign messages in message credential mode. Your client does validate the service, if you are using service credentials, in particular, if you are using a certificate with your service (even SSL through IIS), your WCF client will ensure that certificate is trusted and in the store before allowing any communications.

I have never seen any way to create a custom client-side service validator, however your best bet is to turn on Diagnostic Tracing and Message Logging client side, and analyze from there.

EDIT2: Ok, so I didn't read into your question quite as well. I would assume that your original service was running as a service account, probably network service (since that makes the most sense right?). Well, as it turns out, in newer versions of Windows some of the built in service accounts have been hamstrung, to help prevent security issues, such as bots and such that would do something similar to what you are trying to do. On top of that you are utilizing a proxy, according to your setup, which I would venture to guess might need windows authentication to work. I would suggest following the "proper" way of doing things and create a specific service user for that service. I would be willing to bet if you set the service to run as "Local System" it might work (though not sure about the proxy).

Answer 3

I ran accross this same issue in consuming a third party web service that required cert authentication at the transport level on a windows 7 server. We had to set up tracing at the socket level to find the root cause which was our service consumer running under a custom application id did not have access to read the private key on our client cert. We saw the cert was being found but since the private key could not been read the AcquireCredentailsHandle would fail with an error 0X8009030D.

socket trace:

System.Net Information: 0 : [5708] SecureChannel#15271547 - Certificate is of type     X509Certificate2 and contains the private key.
ProcessId=6572
ThreadId=1
DateTime=2013-12-18T03:01:54.0189534Z
Timestamp=38085697406996
System.Net Information: 0 : [5708] AcquireCredentialsHandle(package = Microsoft Unified Security Protocol Provider, intent  = Outbound, scc     = System.Net.SecureCredential)
ProcessId=6572
ThreadId=1
DateTime=2013-12-18T03:01:54.0194535Z
Timestamp=38085697408545
System.Net Error: 0 : [5708] AcquireCredentialsHandle() failed with error 0X8009030D.
ProcessId=6572
ThreadId=1
DateTime=2013-12-18T03:01:54.0724641Z
Timestamp=38085698170444

once we grant our application id access to the priavte key it was able to complete the tls hand shake and our certificate was sent to the service. Granting access to the private key is much easier in windows 7 server that it used to on xp machines. Just have open the cert store in mmc find your client cert then rightclick alltask-->manage private keys