Jul 9, 2010

IIS 7 Kernel Mode Authentication, Kerberos and DNS Alias (CNAME)

The main idea of IIS 7 Kernel Mode Authentication is to allow HTTP.sys to handle authentication, which means the computer account running HTTP.sys (not application pool account)  will be used during authentication negotiation (see this post for negotiation details).  If site URL is constructed from server name, kerberos will work automatically regardless of app pool account, since Host/servername is registered when servers join domain. If site URL is not constructed from server name, negotiation will choose NTLM unless SPN is registered correctly with the server computer account. If SPN is register by mistatke with any other account, the negotiation chooses Kerberos, but authentication can never go through.


This is a huge improvment over IIS 6.0 with its infamous 401.1 deny issue when site URL is constructed from NetBIOS and IIS pool account is a domain account. 

CNAME is generally considered not good for kerberos (particulary for IIS 6), since the kerberos client is  not to ask SPN with CNAME. For instance, if intranet.company.com is an alias to server1.company.com, the client is going to ask KDC for http/server1.company.com, not http/intranet.company.com. This doesn't necessarily mean CNAME will break kerberos: in case of kernel mode, CNAME actually make kerberos work without the need to registering any SPN (in contrast, if it is an "A" record in DNS, SPN, http/intranet.company.com, will have  to be registered to the server machine account)

Does this mean Kernel Mode is a silver bullet for kerberos? No, kernel mode won;t work for Load Balance situation with multiple WFEs: it is impossible to register the same SPN for more than 1 server machine account.

So for system designed to be multiple server farm such as SharePoint, Kernel Model has to be unchecked in order to configure Kerberos.

As best practice, for multiple server system, turning off Kernel Mode and then handling authentication the same way as IIS 6. For single server, don't truning Kernel Mode off (instead registering SPN correctly when needed to configure Kerberos). If site URL is server name, truning off Kernel Mode will result in authenticatin fail. See this http://support.microsoft.com/kb/871179 for details.

Note: even though Authentication works, it doesn't mean delegation work as well. computer account is NOT granted delegation right by default.


Even though the kernel mode in IIS as well as CNAME record in DNS make debugging kerberos very difficult, but the basic rules still stand:
  • if kerberos client won't be able to get the SPN either because SPN is not registered or is duplicated, the negotiation falls back to NTLM.
  • if Kerberos client can get the SPN, negotiation ends with Kerberos (see negotiation process here)
    • If the SPN is registered to a wrong account, A SPN must be registered to a correct account: either server computer account (in case of kernel mode) or application pool domain account. If it is registered to a different account, the kerberos authentication will fail, normally you get prompted indefinitely and can nerve get in.
    • If the SPN is registered to the right account, either server computer account (in case of kernel mode) or application pool domain account, authentication succeed then.