Message Authentication Failure : TimeStamp & Clock Skew Issue (Part 1)

Recently I came across an interesting issue with service binding and message security in WCF. The service was hosted in IIS and used message level security with UserName Credential and https for transport security. The two exposed bindings are as follows:

<wshttpbinding>
  <binding name="wsHttpWithMessageSecurity">
    <security mode ="TransportWithMessageCredential">
      <message clientCredentialType="UserName" negotiateServiceCredential="true"/>
    </security>
  </binding>
</wshttpbinding>

<basichttpbinding>
  <binding name="MyBasicHttpBinding">
    <security mode="TransportWithMessageCredential">
<transport clientCredentialType="None" proxyCredentialType="None" realm =""/>
      <message clientCredentialType ="UserName"/>
    </security>
  </binding>
</basichttpbinding>

Everything worked fine when I testing my client and service hosted on the same machine i.e. local IIS but when I deployed the service to a remote server (IIS6/Win2K3 server) the service started throwing “Error verifying security of the message” exceptions/faults. I made sure there where no server/client certificate or SSL issues but that did not seem to be the issue.

Here is the stack trace of the exception caught on the client side:

System.ServiceModel.Security.MessageSecurityException was caught
Message=”An unsecured or incorrectly secured fault was received from the other party. See the inner FaultException for the fault code and detail.”
Source=”mscorlib”
StackTrace:
Server stack trace:
at System.ServiceModel.Channels.SecurityChannelFactory`1.SecurityRequestChannel.ProcessReply(Message reply, SecurityProtocolCorrelationState correlationState, TimeSpan timeout)
at System.ServiceModel.Channels.SecurityChannelFactory`1.SecurityRequestChannel.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)
Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)

InnerException: System.ServiceModel.FaultException
Message=”An error occurred when verifying security for the message.”

I enabled Security Auditing on the service for MessageAuthenticationAuditLevel =”Failure” and retested the client and checked the auditing log on the server. The log had the following entry:

Message authentication failed.

Service: https://mytestserver.com/service.svc/endpoint01

Action: http://schemas.xmlsoap.org/ws/2005/02/trust/RST/SCT

ClientIdentity:

ActivityId: 00000000-0000-0000-8f01-0060000000a2

MessageSecurityException: The security timestamp is stale because its expiration time (‘2008-08-04T20:29:58.924Z’) is in the past. Current time is ‘2008-08-04T20:45:18.828Z’ and allowed clock skew is ‘00:05:00

Basically the  client message failed message authentication since it is outside the acceptable time range when processed by the service. The main reason for this being the server & client clocks out of sync or mismatched.

In the next post (i.e Part 2) I will explain the reason and the work around for this issue.

Security Event Logging & Auditing

Few days ago I was trying to debug a WCF service hosted on a remote server which was giving a very generic boilerplate security exceptions/faults. After scratching my head for sometime I decided to figure out how to enable logging these security exceptions on the server. I think for debugging purposes this is a great tool and should be the first step when hunting down an elusive bug.

Its actually pretty straight forward to enable Security event auditing in WCF. You simply need to enable this in the configuration of the service.

<configuration>
    <system .serviceModel>
      <behaviors>
        <behavior>
          <servicesecurityaudit auditLogLocation="Application" serviceAuthorizationAuditLevel="Failure" messageAuthenticationAuditLevel="Failure" suppressAuditFailure="true" />
        </behavior>
      </behaviors>
    </system>
  </configuration>

‘auditLogLocation‘ specifies which log to write to with possible values Default, Application and Security.

‘messageAuthenticationAuditLevel‘  & ‘serviceAuthorizationAuditLevel‘ indicated when to log the event i.e. on None ,Failure ,Success and SuccessOrFailure for message and service authorization respectively.

‘suppressAuditFailure‘ basically defines who (i.e. the service or the system) handles a AuditFailure event (like unable to write to log or log is full, etc). In case of true Audit Failures are ignored and request is processed normally.

In the Event Viewer note the source is listed as ServiceModel and message related information is logged in an un-encrypted form.

Generating Service Client Proxy

To consume a service you generally need a client side proxy of the service being consumed to talk to your application. Most of the time the simpler approach is to add a reference to the service in VisualStudio and let it work its magic. Eventually there will be situations where this approach is not sufficient and need more manual intervention. SvcUtil.exe (ServiceModel Metadata Utility) is a command line tool which comes as a part of the .NET Framework and does this job. In fact VisualStudio itself use this utility behind the scenes with all the right switches.

There are usually two steps involved with creating the client proxy, namely Get the Service metadata and Generate code from the metadata.

There might be two scenarios you might encounter when dealing with external Services, i.e.

1.      Hosted or Running Services. (should expose a MEX endpoint)

2.      Service assembly or library.

To generate client proxy and configuration from a compiled service assembly required two steps:

1.      Extract the service metadata as:

SvcUtil.exe /t:metadata serviceassemblyname.dll

This step would produce the WSDL and the XSDs from the assembly.

2.      Generate proxy from the metadata as:

SvcUtil.exe /t:code [path]*.wsdl [path]*.xsd /out:[path]Proxy.cs /config:[path]App.config

Now in case of a hosted/running service both steps 1 & 2 are usually combined as one as below:

SvcUtil.exe /t:code [Service URL] /out:[path]Proxy.cs /config:[path]App.config

I ran into an interesting problem when I tried to execute the above command against an IIS hosted service which exposes the MEX binding over https. ScvUtil failed to pull the service metadata and threw a “WS-Metadata Exchange Error. Could not establish trust relationship for SSL/TLS secure channel. The remote certificate is invalid according to the validation procedure.”

Now I know this is cause of the fact that the server certificate is not signed by a TrustedRootCA or installed in the trusted store. This is not unusual for development and test environments.

One option is to install the server certificate in the trusted store but I am trying to find if there is an option to override the validation procedure that the underlying protocols use to establish a secure channel. I know VisualStudio gives a warning in this case but you can still proceed if you choose to. I don’t know if there is way around with just using SvcUtil directly from command line, however for your Client side there is a work around explained below.

When a client applications trys to connect to a URI, it requests a System.Net.ServicePoint class instance through System.Net.ServicePointManager class. Each ServicePoint maintains its connection to the URI for specific time and then recycles. The ServicePointManager class has callback property (which is our hook-in point): public static RemoteCertificateValidationCallback ServerCertificateValidationCallback { get; set; } which basically verifys the remote SSL certificate used for authentication.

The sample below demonstrates how.

public class CustomCertificateValidation
    {
        CustomCertificateValidation()
        {
            System.Net.ServicePointManager.ServerCertificateValidationCallback +=
            new System.Net.Security.RemoteCertificateValidationCallback(CertValidate);
        }
        bool CertValidate(object sender,
        System.Security.Cryptography.X509Certificates.X509Certificate cert,
        System.Security.Cryptography.X509Certificates.X509Certificate.X509Chain chain,
        System.Net.Security.SslPolicyErrors error)
        {
            //Put your validation logic here
        }
    }