Monday, May 20, 2013

Demystifying Certificate Requirements in Mutual TLS

Understanding Certificates and SSL/TLS long ago became an IT fundamental.

Somehow, the industry seems to have not noticed.  In my quest to take fewer calls on this stuff, here is my attempt to help demystify all the certificates involved in Client SSL/Mutual TLS.  I seem to be spending 2+ hours a day on the phone talking web and server admins through this stuff.

The reason I am doing this is because Google failed me.  There are a lot of docs that focus on all the other aspects of the SSL/TLS negotiation, but none of them focuses on the certificates and what they do for us. Microsoft has one of the better papers on SSL/TLS, but there is a lot lacking around the certificates and the SSL/TLS handshakes are overly summarized.

Here is the handshake that MS shows us.


While this may be a four message handshake, there are a few different conversations taking place, simultaneously, in the handshake.  The conversations and their certificate requirements are more easily understood when they are pulled out of the stick context of the message format.

I will say that I am doing a bit of summarizing.  SSL/TLS has four major components: authentication, message integrity, key negotiation and encryption.  This post focuses heavily on the authentication aspect.  I am lumping SSL and TLS together as the basic functions of the negotiations are the same, only the implementation details vary a bit.  I am also leaving out a bit about session key generation. 

I break the handshake down into 4 different conversations.

A.      Hellos

a.       The client tells the server that it wants to go secure and offers the suite of ciphers it supports.

b.      The server selects a cipher from the list.

c.       No certificates are involved here.

B.      Server Certificate presentations

a.       The server sends over the SSL certificate AND the chain

b.      The server must have the private key for the SSL certificate.

c.       If the chain, (CA hierarchy) is not sent, negotiation MUST fail.

d.      The client SHOULD perform full cert and chain validation against these certs, making sure they chain to a Root CA that is trusted explicitly by the client.

C.      Key Exchanges

a.       The client and server exchange data that allows the negotiation of session encryption keys.

b.      If EDH is used, then the RSA/DSS key is used to block MITM of the key negotiation.

c.       If RSA key negotiation is used, the client encrypts its initial key material with the public key on the server SSL certificates, assuming cert and chain validation succeeded.

D.      Client Certificate  presentations

a.       The server issues a Certificate Request.  This lists the key types allowed on the certificate and a list of Distinguished Names (DNs).  These DNs are used as a hint to help the client determine which of its many possible certificates is the “right” one.

b.      The client selects a certificate, for which it MUST have the private key, and sends the certificate to the server.

c.       The client sends data that is signed using its private key, proving it holds the certificate in question.

d.      The server checks the signature.  If the signature passes, the server may be configured to pass/fail the client based on data in the certificate.  Often the certificate serial number or Subject DN must match a static list, or the certificate must be one that is on file.  The decision to pass/fail the client cert is not specified in any RFC.




More below

Certificates Needed




A - Hellos



B - Server Certificate

Root CA Cert for Trust

SSL Cert and Chain to send

C - Key Exchange

Public key from Server Cert

Private key for Server Cert

D - Client Auth

Client Cert and Private key

Client cert for explicit trust or CA cert to implicit trust.  Cert values MAY be checked.


Client and Server Perspective

One often forgotten part of this is that when there are two parties, the role of client and server may change.   For instance, let's say A pushes a full DB copy every night to B, and then B pulls deltas hourly during the day. In this scheme, at night, B is the server, but during the day, B is the client.  This means that both parties need a cert and a server cert with chain, and they need to know how to implicitly or explicitly trust other clients, and they need Root CA certs so they can trust the server cert when they are the client.


Take the time to understand how your connections are established and it is fairly easy to figure out which certificates are used when. When it comes to figuring out what is wrong, you can usually figure out the issue by where the handshake fails.  The handshake failures usually can only be seen via packet captures.


Friday, March 29, 2013

Hello IT Person, Welcome to the Security Organization

This is a post that is long overdue.   The IT industry went through a revolution and most people in IT missed it and are still missing it. 

If you are in any form of IT related job, you are in the information security field.

You may say, “No, I’m just an IT Project Manager (analyst, whatever), security is another team”.  You are wrong and your career is heading towards a cliff.

It only takes a tiny bit of Googling to realize that everyone is getting hacked.  Even the biggies, RSA, Microsoft, Facebook, Symantec, and Apple aren’t immune. There are too many actors with too many motivations.  If it is connected to the web, there is either money in hacking it, or it can be used as a foothold to hack for money.  If it’s connected to the web, there is probably someone with a social or political agenda that makes it a target and if not, it is a platform for the hacktivists to leverage.  On top of the myriad of highly skilled and motivated attackers, there are thousands off wannabe hackers simply looking for low hanging fruit to test their skills or to get a thrill.  Even if you don’t have a penny to your name, your computing power is a commodity if it can be added to a botnet. Hopefully, you already know that it’s a given that everyone is a target.  If you still need persuading on this point--don’t worry, there are plenty of Wal-Marts that need greeters.  Get your app in early.  (That’s “app” as in “application”, which is a paper form you will fill out with a pen. You won’t need LinkedIn for this.—The Editor)

So, why is security your job?  Every day (week if you are a slacker?) you make decisions that impact security.  It doesn’t matter if you specialize in a niche, like UI or UX; or something broad, like program or project management.  If you are working with data in any way, that data has value to your organization.  There will certainly be negative impact if the data is compromised or corrupted.  Even if you run or maintain a static website in which the content is public and can easily be restored if lost, you don’t want your system to be a foothold into your important systems or take part in a DDoS attack.

Your Role

Backup Operators – Don’t lose the data and don’t lose those unencrypted backup files.  This role should be a no brainer.  This role has Confidentiality, Integrity, and Availability components.

Systems Administrators – Once again, this is a no brainer.  SAs have all the access themselves and they configure access to ALL of your data.  Remember, all of your applications sit on a host that an SA “owns”.

DBAs and DBOs – Please, for the love of all that is good, you guys MUST know you have to protect that data. Do you??

Project and Program Managers – I know lumping you guys onto one line will get me all sorts of hate mail.  You guys decide things like what use cases exist, for what users, accessing what data. You decide whether to engage formal security teams for assistance. You decide to cut product or project scope, and everyone knows security is the first to get cut.      

Developers – You guys are the worst.  Yeah, I’m saying it. Inventing your own “cryptography”, passwords in log files, backdoors, assuming users will use your apps the way you want them to, and on and on.  Seriously devs, get your acts together.    

Network Engineers – I think this is the only role that most folks actually think of as a security role.  In fact, this role is the least interesting in terms of security roles. Read those manuals and change those default passwords.  And stop using self-signed certs.  Can you really not remember how a MITM attack works long enough to say to a manager: “Wait, we need a real cert on that appliance admin portal page”?  You know what, if you can’t explain to a manager what a MITM attack is and why your choice of cert matters, you’re part of the problem.


Your day to day job may be focused on something else, but it is only a matter of time until IT folks start getting fired for massively bad security lapses.  If it is on the OWASP Top 10 or the SANS Top 25 and you don’t know a bit about it, you may want to pick up that Wal-Mart job app.  No one is asking you to figure everything out about security, but you need to at least understand some basics and keep your eyes out.  If you do run a system, you’d better become a security expert in the context of that system.



Friday, February 8, 2013

Senators Marco Rubio, Orrin Hatch, Amy Klobuchar hate America

I posted a snarky blog on H1B Visas last year, but it appears that based on current events, it is time to post again.  I doubt I will say anything that hasn’t been said before, but this needs to be said, over and over. 

H1Bs are a tool of “The Man” to keep America’s record breaking income disparity on track for even bigger numbers.   Yeah!  Maybe we can break a new record next year!

Proponents of the H1B, or increasing the allowed number of them, will tell you heartwarming patriotic tales of the U.S. needing these so we can keep companies on shore, in business, and competing against the world.  Capitalism is the best thing ever, but when you run out of local resources, you have to import them to keep your business from failing.  It’s just supply and demand. We have demand. If these companies can’t get smart employees, they will go out of business and then America loses.  Limit the H1Bs and Microsoft will have to fold up shop, then the Seattle area dies with it, then Silicon Valley and LA aren’t far behind.  Plus, hey, we are forced to pay all these folks prevailing wage.

All of this is horseshit. 

(Sorry, parents who are reading my blog to your kids before bed.  I’ll try not to use that language again…)

Why is it horseshit?  (Okay. I’m not trying very hard.)

Here’s why:

The H1B is simply access to a pool of lesser-paid employees.  In my previous post, I referred to them as indentured servants.  (Work at Microsoft for a few months, and you will see it. Comments welcome.)

Further, when the onshore talent is forced to compete against the indentured servants, they have to put in the same kind of hours the indentured do.  When it comes to supply and demand, the prevailing wage rule doesn’t let the market come into play.  The extra supply of labor hours drives down demand.  It’s just that simple.

H1Bs are anti-American and chip away at the American Dream every day.  One could make an argument that the visa is just opening a bigger market, thus very capitalist. But seriously, I kind of want America to be around and to have jobs when my kid grows up. 

Here’s how the H1Bs are ruining America.  If we stop or cut the H1B numbers, the demand for smart people goes up.  With demand up, wages go up.  Wages don’t just go up for the top tier professionals; they go up for every other tier as well.  Crap, there goes our chance at another Guinness Book Income Disparity Record.  If the wages go up, the jobs get much more attractive to America’s youth.  If demand for those jobs goes up, we need more training institutes for whatever technical jobs are in more demand.  Kids will go to college and get smart if there is a path to riches.  Right now, average wages for many of these jobs are comfortably nice, but they don’t scream, “Learn me, you can vacation in Park City every year and drive an Escalade!”

The H1B keeps kids from seeing any American Dream, so the ones lucky enough to afford college just get yet another Art History Degree and fill out that McDonald’s job application.

I have met a few folks who have left tech for trade jobs, because the pay is similar and the stress is lower.

One might make a claim that there is a short term need for the smart folk, as American kids aren’t up to speed yet.  I concede the possibility, but how do we fix the cycle?  On our current path, it seems that we don’t, we just continue to import cheap labor.

The market only works for us if you let it.

If that were the end of it, one might be able to let go.  As it turns out, and I won’t pretend to know everything about these companies, many of the big consumers of H1Bs are foreign owned companies who rarely hire Americans and specialize in filling headcount for companies who can’t get H1Bs or have more than their fair share. These companies are funneling their revenue right out of America.  These aren’t even American companies in need of talent, hoping they can compete!

Don’t get me wrong, I don’t hate foreigners; I worked with some great ones at MS and many great ones at T-Mobile.  What I want is to protect America, its future, and its market for high paying jobs.  Keeping wages low in this manner hurts America.  We need to fix this for our children.

H1B visas are nothing more than a way to keep money out of the hands of workers and in the hands of execs and larger shareholders.

Taking a look at some of the top H1B sponsors, I can’t help but notice three distinct categories.  We have offshore consultancies, household name tech companies, and banks/wall street firms. None of these companies are lacking in cash to bump up the pay of their employees or to fund training and education for the lesser employees with potential.  This is pure greed.  

Here a look at H1B data for 2001 – 2012

Household Names




Microsoft Corporation



IBM Corporation



Oracle Corporation



Intel Corporation



Qualcomm Inc



Accenture Ltd



Cisco Systems, Inc



Google Inc



Hewlett-Packard Company



CVS Rx Services, Inc. dba CVS Pharmacy



Rite Aid Corporation




Wall Street







JPMorgan Chase & Co



Goldman, Sachs & CO



Bank of America



Merrill Lynch & Co., Inc



Credit Suisse First Boston






Capital One Financial Corp




Offshore Consultancies




Satyam Computer Services, Ltd



Tata Consultancy Services



Infosys Limited



Larsen & Toubro Infotech Limited



Wipro Limited



Infosys Technologies Limited



Cognizant Technology Solutions



Patni Americas, Inc





Well, at least Steve Ballmer gets to keep his fleet of Ferraris.

…And Larry Ellison gets to keep tropical islands.

…and our children will get to keep…what?

Tuesday, January 15, 2013

NTLM hasn’t been relevant for like 12 years... and other lies.

A surprising number of foolish Slashdotters have pointed out that my latest work, breaking the NTLM and LM handshakes and phishing for users’ NT hashes, is totally irrelevant and has been for 12ish years.  

As a fan of debate, I’ll start with points that are interesting but have no real bearing on the topic.
  • Slashdotters are clearly not qualified to make this assessment.  Their Appeal to Authority fails.
  • Microsoft wouldn’t issue an advisory and Fix-It if it weren’t relevant.  My Appeal to Authority is better than theirs.  ;-)
  • The existence of a newer protocol, Kerberos, does not make NTLM simply disappear.
  • Wikipedia actually doesn’t say NTLM is long dead.  Wikipedia as an appeal to authority is a joke.  I link to it regularly, not for its completeness, but because it is written for a layman audience.  It is a great place to start if you don’t know something.
  • I’m not a Linux fanboy looking to disgrace MS.  I’m a long time MCSE and even gave MS some props in my post.  
  • Finally, my work is what it is, probably the last nail, of hundreds, in the coffin.  I make no claim to it being inspired by God.

Now to the relevant details...  When MS introduced Active Directory in Windows 2000, they implemented Kerberos 5 as the default authentication protocol FOR DOMAIN ACCOUNTS. This is a pretty important requirement. If a machine is not domain joined, or the account is not a domain account, Kerberos is not an option. The upside here is that when machines are in workgroups, it is much less likely that the accounts will have any sort of value off of the host.  However, this does not stop the SSPI from trying to authenticate using your cached account credentials when accessing resources that are not on the host.  This means a workgroup host could still be vulnerable to my “Send the Hash” attack if not properly configured.

Even for machines that are domain joined, while Kerberos is the default, NTLM is used in several situations:
  • If the service is not Kerberos enabled (Kerberized).  Maybe it runs on an NT server?
  • The service/server does not have a Service Principal Name (SPN) registered.
  • The service/server has duplicate SPNs registered.   
  • When accessing the system by IP rather than name.
  • Improperly built clusters.
  • 3rd party system implemented incorrectly.
  • When accessing data across forests, using an older domain type trust.
  • When the client can’t access a KDC/DC, such as when it is outside the firewall.
  • When the KDC/DC is behind NAT.

Before getting in depth with a couple of these cases, I’ll make a generalization, “Kerberos is very hard to get right, except under simple conditions.”

Outside the Firewall or Behind NAT
MS’s implementation of Kerberos requires that the clients, servers, and KDC/DCs all be on the same routed network with AD integrated DNS or DDNS that allows the DCs to register SRV records. The clients must be able to find and access the KDCs to get Kerberos tickets.  I am not going to cover all the details of Kerberos, but this is a key difference.  With NTLM, the server you want to access does the job of finding a DC and getting the DC to validate the challenge/response after your client has done its handshake.  The resource server passes the challenge and response to the DC over RPC using packet privacy and gets back a pass/fail and a list of group memberships which it uses to build the user’s access token. This is super simple and easy when the client is outside your firewall.  You only need to open one port, the application's port.  

If you intend to make Windows Kerberos work across NAT or behind a firewall, prepare for pain.  Each Windows client has a component called the dcLocator.  Its exact operations vary slightly from version to version of Windows.  You might think you just need to open up TCP88 to a KDC and you’re set.  You might get a pony in the mail too.  

I’ll blog on the exact details at some point, but the dcLocator first needs to find the KDC DNS SRV records in the zone.  Right off the bat, this means that you need split DNS, as the answers inside your firewall will not be the same IP as outside. Once you have your external DNS zone and main DC records, the client will ping all the DCs and select the fastest to respond. If ICMP is blocked, nothing proceeds. The client sends a CLDAP query to the fastest DC. This is connectionless LDAP over 389 UDP.  This query is to ask which AD site the client is a member of.  This query is not answered in a traditional way, based on the filter.  Instead, AD uses the source IP to map the IP to an AD subnet which maps to an AD site, which the LDAP search response will contain.  If your client is behind NAT, then the source IP will likely be a SNAT address.  From this response, the dcLocator then does a second DNS SRV query to get the DCs that are in the AD site returned from the CLDAP query.  The dcLocator then pings each of those DCs and the first to respond is queried and if the response is satisfactory, then this becomes the default DC for a period of time.  This time can vary by OS version.  Now we are ready to do Kerberos. Some versions of Windows try UDP88 first and then when they get back the “response too large” they try TCP88 route. If UDP is blocked, these versions may not try TCP88 even if it is open. I will not be swearing to this in court as it has been over a year since I configured this type of scenario and I am writing this without a net.  

This means, that for Kerberos to work outside the firewall or behind NAT, you need to:
  • Setup Split DNS
  • Create at least one domain level SRV pointing to the external IP address
  • Create a site DNS SRV record for EVERY DC in the default site, pointing to the external IP address.
  • Open port 389 UDP
  • Open port 88 UDP
  • Open Port 88 TCP
  • Open ICMP
Kerberos Diagram

NTLM is a lot easier to use in both NAT and outside the firewall scenarios.

NTLM Diagram

Messed up SPN Scenarios
A Service Principal Name (SPN) maps an instance of a service running on a server to the account that it runs under.  Kerberos uses shared secrets, between each party and the KDC/DC, to allow for authentication and key exchange.  When a client wants to use a service, it asks the KDC for a ticket to the specific service/server combination.  The ticket is encrypted by the KDC so that only the service can open it.  If there is no SPN registered for the service, a ticket can’t be granted.  If two accounts have the SPN registered, the ticket cannot be granted.  If the service is running under the context of a different account, the ticket cannot be decrypted.  

Many web admins just allow their appPools to run as local system or network service.  This means they are running under the machine account, which is a domain account and has an SPN or at least can have one.  In order to make your cluster work with Kerberos, all appPools must run under the same domain account and usually an SPN must be manually created by an admin. MS has made some changes to IIS to make this easier, but in many cases it is out of the frying pan...

SQL server tries to register its SPN whenever it comes online.  If you are following best practices, and using a domain account to run SQL, then SQL will try to register the SPN attribute on that account.  This is a proper configuration, however most accounts so not have the SELF write SPN permission (ACE), and fail.  This means that Kerberos won't work and NTLM is negotiated.

Connection by IP Address

This fails as there is not an SPN set that uses the IP, such as HTTP/ Rather the SPN is HTTP/  This can be overcome by registering the IP SPN, but does not scale well.  I’ve thought about building a simple tool to do this, but my list of to-dos is long.

In the end, the threat and attack models are very different for the enterprise and stand alone users.

Enterprise Exposure

The enterprise is likely to block many outgoing ports, making it less likely that “send the hash” attacks will succeed.  This however doesn’t mean there is no risk from NTLMv1.  With any type of foothold, the attack is VERY effective.  All it takes is one compromised host, one badge not checked, a network port activated outside of your secure areas, etc.
Enterprise users spend time in hotels, airports, hospitals, etc. These types of places are ripe for the picking. 

Workgroup Exposure

For workgroup members, the threats vary considerably.  The user may spend much of their time behind NAT, which may make it hard for an attacker who steals a hash to use it.  The user may spend time in coffee shops, etc, making them ripe for attack.


While Kerberos is more secure than NTLMv2, it is not really fair to say it is better.  They both have pros and cons.  NTLM is VERY commonly used today both by design and due to it being a fallback for failed Kerberos.  Depending on your systems’ settings, you may be sending LM and NTLMv1.  ALL VERSIONS of Windows still accept LM and NTLM by default, so you may still be allowing the issue. You may not be initiating dirty handshakes, but you will accept them if offered.
Finally, the fix is very simple. There is almost no downside to making the change. If due to some crazy twist of fate you have a system that only works with NTLMv1 or LM, that system will break.


Tuesday, January 8, 2013

NTLM Challenge Response is 100% Broken (Yes, this is still relevant)

NTLM Challenge Response is 100% Broken (Yes, this is still relevant)
Updated again: CloudCracker has dropped off the web. A new successor has arisen. Meet
Updated as it seems a lot of folks don't read. The reason this attack is interesting is that I steal or phish the hash in the attack WITHOUT physical or administrative access. Yes, I am aware of the 30 amazing tools that "already did that" using local admin access. Also here for why this is still matters.
First off, I can’t say I broke the NTLM handshake; the march of time did it.  Apparently, I am just the one who bothered to put it together.
There have been numerous whitepapershacker conference sessions, and blog posts dedicated to the weaknesses of NTLM (and LM) authentication.  However, the weaknesses described in previously published works were theoretical, or required stealing hashes using admin rights. This means the host was already compromised, thus the exploits themselves are a bit boring and redundant.  One couldn’t just phish for the hashes or MITM the hashes; due to the challenge response mechanism.   The best case for getting to the hash or password from outside the host was to do a MITM attack (or a phish) and substitute a chosen challenge.  This only worked if the victim was willing to negotiate NTLM without the Session Security Flag. This would then allow an attacker to build rainbow tables to get the hash or password. Rather, the attacker probably already had tables built for the chosen challenge.  This scenario is a pretty high bar to reach.
To frame the conversation, there are actually 4 handshakes in the NTLM suite that move up in security and complexity. They are LM, NTLM, NTLM with session security, and NTLMv2.  As of now, only NTLMv2 stands as secure.  There is no way, other than encrypting your link with say IPSec, to secure the 3 weak handshakes.  The good news is, all Microsoft OSs already have a registry key that can control the handshakes options.  MS will release a KB and Advisory on 1/8 on this and the information can be found below.
Now, thanks to Moxie Marlinspike’s Cloudcracker, an attacker can skip the pre-chosen challenge and brute force the challenge response to get the NTLM hash.  If the victim is running XP, the situation is even worse, as Cloudcracker will return the LM hash which can always be broken overnight to derive the user’s password .
Why Does this Matter Now?
Before I get to the exploit details, let me clarify how relevant these technologies still are in today’s world. You might think that with all the papers and presentations, no one would be using NTLM...or, God forbid, LM. NTLMv2 has been around for quite some time. Surely, everyone is using it. Right?
According to the last data from the W3 Schools, 21% of computers are running XP, while NetMarketShare claims it is 39%.  Unless someone has hardened these machines (no MS patches do this), these machines are sending LM and NTLM responses!  While these lists leave out server OSs, 2003 Server still sends NTLM responses by default.  Yes, every MS OS since NT 4.0 SP4 has supported NTLMv2, but NTLM and LM were not excluded by default until Vista.  
But wait, there’s more! It is also very common for companies that have heterogeneous environments to use Active Directory Group Policy to keep the settings weak, usually out of fear of breaking Samba connectivity.  Sure, Samba has supported NTLMv2 for a long time, but most IT folks tend to think “Why beef up security if you might break something? No one is claiming to have broken NTLM.”
Well, here it is: I’VE BROKEN NTLM.
Now, get to fixin’.
More on fixin’ later.  (I can’t take credit for breaking NTLM. I’m no math whiz. I just happen to specialize in applied crypto, and I looked in the right place at the right time.)
The Attack
When I read a summary of Moxie’s MS-CHAPv2 crack, I saw that the big deal was not that the implementation had some crazy flaw, it was that Moxie had affordably built a system that can brute force the DES keys that make up the heart of the challenge response mechanism.  In less than 24 hours, given a known 64 bit plaintext (challenge) and a ciphertext (response), Cloudcracker can return the key to you.  
This made me wonder what else was broken, given affordable DES brute forcing now exists.  
I didn’t dig a lot deeper into the attack at the time, as I was researching NTLM so I could write a blog post on Pass the Hash Attacks.  I know this is well covered territory, but I never found a paper that covered all my questions, so I figured I’d do it myself.  Much of my research was done by reading the protocol details on Eric Glass’s exhaustive page on the topic. I’ve found no better source for understanding the protocols. 
That’s until I stumbled across this: (Bold added for emphasis.)
The NTLM response is calculated as follows:
The MD4 message-digest algorithm (described in RFC 1320) is applied to the Unicode mixed-case password. This results in a 16-byte value - the NTLM hash.
The 16-byte NTLM hash is null-padded to 21 bytes.
This value is split into three 7-byte thirds.
These values are used to create three DES keys (one from each 7-byte third).
Each of these keys is used to DES-encrypt the challenge from the Type 2 message (resulting in three 8-byte ciphertext values).
These three ciphertext values are concatenated to form a 24-byte value. This is the NTLM response.
Note that only the calculation of the hash value differs from the LM scheme; the response calculation is the same.

That’s right, to reverse the challenge response, I just need to brute force two DES iterations that have 56 bit keys and one that only uses 2 bytes of the 56 bit key space for the last crack.  Moxie shows it like this, for those who are more visual.

I immediately went to Moxie’s posts on the topic to figure out how to use Cloudcracker to brute force out my hashes.  That’s when I discovered that MS-CHAPv2 uses the EXACT same math as the LM and NTLM challenge response.  Moxie was kind enough to point me to the line in his code that demonstrates how one can submit the challenge and response to Cloudcracker to get back the LM or NTLM hash.

print "CloudCracker Submission = $99$%s" % base64.b64encode("%s%s%s%s" % (plaintext, c1, c2, k3[0:2]))

Just concatenate your challenge, and the first two thirds of your response, and k3 as base64 and Bob’s your uncle.  The only thing that threw me was, why is k3 only 2 bytes?  To save on processing power, I assume, you must brute force the final key before sending it to Cloudcraker, which just appends it to the first two recovered keys and sends it back.  As the last key is the one that has 5 bytes that are always all 00, this is easy.
Using Eric Glasses example, I sent in $99$ASNFZ4mrze8lqYwcMegYR0ZrKbLfRoDz2Ag=  to Cloudcracker.  This is the challenge 0123456789ABCDEF, low response 25A98C1C31E81847, mid response 466B29B2DF4680F3, and D808, the first two bytes of the final third of the hash, which I brute forced locally.  If you decide to follow along with me, make sure to look at Glass’s parity adjusting of the keys.  DES keys are really 64 bit, not 56, but as the right bit of each byte is a parity bit, it can’t count toward the total entropy or key space.  Or, you can just use his provided Java code.
Then I just had to wait until I got an email with this:

CloudCracker has successfully completed its attack against your CHAPv2 handshake. The NT hash for the handshake is included below, and can be plugged back into the 'chapcrack' tool to decrypt a packet capture, or to authenticate to the server:
This run took 68799 seconds. Thank you for using, this concludes your job.

For those who missed it, I just got the NT hash from the challenge and response, which are easily observed on the wire. I just got the hash WITHOUT compromising the host first.  To use this in an attack I just need the right MITM foothold or a phishing email.

To: All Employees
From: HR Communications
Subject: Updates to the Employee Handbook
Body: Human Resources has completed a significant rewrite and update to the Employee Handbook.  While some of the changes are minor, it is worth a look for all employees.  Employees with aging parents will likely be excited to see the increase in paid time off for emergency care of elder dependents.  The guidelines for company events where alcoholic beverages are provided have also been updated. 
Finally, with the passing of Washington Initiative 502, we are publishing the new guidelines for Marijuana in the work place.     
The handbook can be found here:                                                                          
Best Regards,
Human Resources

I call this attack “Request the Hash”.  Sorry for the pun, but who wouldn’t immediately click the link to find out their companies new pot policy???
The Good News
First of all, I commend MS for changing the default to NLTMv2, as of Vista, and for leaving us the option if we just can’t live without LM or NTLM.  At this point though, I think it is incumbent on MS to push the issue at the local security policy level, letting domain administrators override common sense as needed.
There might be some testing and some interoperability failures for enterprises, but the good news is the settings for LM, NTLM, and NTLMv2 have been around a long time. The reg key HKLM\SYSTEM\CurrentControlSet\Control\Lsa\LmCompatibilityLevel allows you to choose the level of fallback your system will allow.  Set this to 3 or greater for workstations and you are probably set. This situation is a bit more complex as almost all windows computers are not just clients, but also servers, offering you the admin$ share and c$, etc.  The same setting has slightly different bearing on which versions the machine will accept when acting as a server, say when doing remote administration of your desktops. This great article, by Jesper Johansson, covers both the client and server aspect. 

Personally, I am setting all my systems to 5.  If you allow systems to be set below 5, you may be masking the fact that clients are still performing NTLM or LM handshakes. 
Group Policy Name
Prohibits Sending
Send LM and NTLM Responses
NTLMv2 Session Security is negotiated
Session Security (on Windows 2000 below SRP1, Windows NT 4.0, and Windows 9x)
Send LM and NTLM—use NTLMv2 session security if negotiated
NTLMv2 Session Security is negotiated
Send NTLM response only
NTLMv2 Session Security is negotiated
LM and NTLMv2
Send NTLMv2 response only
Session Security is always used
Send NTLMv2 response only/refuse LM
NTLMv2 Session Security
Send NTLMv2 response only/refuse LM and NTLM
Session Security

You may ask, “Who cares if my server accepts LM or NTLM?”  If not all your clients are managed, your server could be unwittingly used to compromise an account used by a client with weak settings.
I did a fresh Ubuntu install and verified that its SMB/CIFS client only sent NTLMv2 response, even when I attempted to downgrade using Cain.  I have not yet had a chance to test web browsers that support NTLM auth via SPNEGO.   That’s right; most web browsers perform NTLM auth in HTTP Headers when a trusted site requests it. 
A Reminder About the Downside of Doing Nothing
While web browsers only perform the NTLM auth for trusted sites by default, generally, Windows OSs seem to make no distinction between trusted and un-trusted zones for other protocols, such as CIFS/SMB, SQL/TDS, and RPC.  This means that a phishing attack with a file://server//share link could yield great results for an attacker.  Based on the rapid succession of tries and retries, it is safe to assume that the first 2 – 5 attempts by a machine are the OS, via SSPI, trying to auth with the victim’s cached creds.  This means the attacker does not need to see a successful auth to trust that the credentials are valid. Maybe you took my advice on Pass the Hash, and blocked all the protocols that use NTLM, at your perimeter.  This doesn’t stop one of your clients, say a laptop at home outside your perimeter from trying to do something that won’t try an NTLM handshake. There are just too many ways to fail, if you allow LM or NTLM in any context. OK, so the attacker has the hash, but they are outside the firewall… You’re safe, right?  No!  At this point, the attacker has a valid hash and is ready to use the account to get your data.  Now the attacker has two options:
·         If your system is sending LM responses, getting that hash makes for super light work in cracking the password.  If you are sending the better NTLM hash, you are still in hot water.  Cloudcracker also has a huge set of rainbow tables.  For another $20 I can submit the hash and likely get the password of an average user. There are surely plenty of ways inside your perimeter with just username and password. Moxie, if you are reading this, how about a one step operation (from the customer perspective) that pipes the recovered hash into the rainbow tables after it’s been cracked. Maybe a price a tad under $40 for the two cracks? 
·         BUT WAIT!!  Attackers don’t need your stinking password! If I have the hash, it is password equivalent! The first step of every auth involving NTLM and NTLMv2 is to convert the password to the NT hash.  Your weak settings just saved the attacker a step.  Once the hash has been stolen, the only ways to render it useless are to change the password or remove all of the channels by which it can be passed. (The latter feels infeasible). 
Protecting Yourself and Your Company
If you are a lone user, make sure all of your systems are set to LmCompatibilityLevel 3 or higher via local security policygroup policy, or via the registry key.
If you are a corporate type, make sure the GPO is set to force 3 or higher.  You may notice that levels 4 and 5 talk about domain controllers refusing different levels.  This is not a great protection and may even mask a problem.  While the DC blocking the LM auth passed by a client won’t let the user access the system, the damage is already done: The LM response was sent, and the attacker may have it.  Sure, it is nice for the attacker to see the successful auth so they know the crack is worth the time, but the rapid fails and retries will tell the attacker that the client OS had the creds cached and the hash that was used is almost certainly good, at a minimum on that client machine, if not the domain.
If you are running a Samba client, enter this line ‘client ntlmv2 auth’ into your smb.conf file.
Final Paranoid Thought
If you have any of the weak settings, CHANGE YOUR PASSWORD after fixing them!
I made this discovery when I realized DES was brute-forcible on a budget.  I immediately started thinking of a list of all of the processes that broke along with DES.  I’m sure this is exactly the list that every nations’ spy services started making when they acquired the computing power to brute force DES.  I assume that it has been affordable for most of them for 5 – 10 years now. 
Sorry, spies, if this closes a door for you.  I read a book that says another one will open.  ;-)
Appendix A: Proof of Concept Code
In order to validate my work, I built a simple tool proof the crypto and create the CloudCracker token.  First off, I am not a great or professional coder, so don’t laugh when you look at the code.  Second, Eric Glass’s code was a great help.  His is in Java, while mine is c#.  Last, I used Bouncy Castle’s c# crypto library, for two reasons.  The .NET libraries do not support MD4, because for hashing it is VERY weak and the .NET libraries block the use of weak DES keys.  NTLM does not account for weak keys.
My code is a fair bit redundant, checking and rechecking itself.  This is because each CloudCracker summation costs $20 and I didn’t want to have to send/spend several failed submissions during debugging.
The code can be found here.
The challenge and response data should be in hex, while the password should be ASCII. 

Inputting falsified referrals to this site violates the terms of service of this site and is considered unauthorized access (hacking).