What are Credit Ratings Good For?

On the front page of today’s Wall Street Journal (well, the on-line version anyway), there’s an article about how the major credit rating agencies ‘failed to see defaults coming.’  You can read some of the article on their public site, but you’ll need an on-line subscription for the entire text.  The point of the article is that a given country’s or company’s official credit rating usually severely underpredicts the real probability of default.  This is not news in the industry.

A few years ago as the credit crunch was getting underway, several of our business partners asked if we could model corporate bond credit risk using the official ‘big three’ credit ratings as inputs.  While this initially sounded pretty straightforward, my crack team of financial engineers showed me why it really doesn’t work.  Basically, there is supposed to be a relationship between the credit rating and the chance of default.  S&P claims that a rating of ‘single B’ means there is a 2% chance of default within 1 year.  But if you look at 100 different companies rated ‘B’, far more than 2 of them defaulted in the coming year.  Given how poorly the ratings predict what they are supposed to, my quants emphatically refused to build a system that used credit ratings as the basis of default.  But at the same time, many of our competitors were selling exactly that type of system – one that estimated how much money a fund could lose based on the credit ratings of their investments.  And these systems were selling very well.  The only problem was that they, just like the big credit ratings, severely underpredicted the real risk.

Just to be sure we were on solid ground in our reluctance to build such a system, my quants and I visited a few industry experts and select academics.  The unanimous advice was that we could make a lot of money selling systems that relied on credit ratings like everybody else, but that in the end none of those risk estimates really hold any water.  Quite a predicament.  Right around the same time, I happened to be speaking at an industry conference on a beautiful lake in Ontario where I was introduced to one of the industry’s “celebrity” authors on credit risk management.  On the bow of a 60 foot boat, sailing around a scenic lake, he told me that in his personal experience not one of the industry standard credit risk systems ever really worked.  My competitors underpredicted credit risk by 2 to 5 to 10 times.  His expert conclusion: credit ratings are simply useless for estimated real risk.  But then the pressure really turned on – the US Government put out a draft regulation that required certain funds to estimate their risk using these credit ratings.  Why would the government do that?  My best guess is that they wanted to use a well-known standard, and credit ratings are just that – a standard.  I can only wonder if they ever questioned their accuracy.

So instead of capitulating and joining the crowd of firms that built systems based on this flawed method, my firm decided to challenge the draft regulation’s requirement of using credit ratings.  I went to several conferences and meetings with senior executives explaining why we would not build such a system.  This is actually a much tougher position to defend than it sounds.  It’s not that we could not.  It’s that we would not.  Yes, I told them, it is industry standard to use the credit rating.  Yes, such data is available for a reasonable fee.  Yes, my quants know how to build this type of system and yes, clients would buy it.  Worst of all: Yes, the US Government is contemplating requiring that it be used.  “If it’s industry standard  – meaning that you can’t be singled out for blame if it doesn’t work – and the federal government wants to use it and you know how to  build it, and clients will pay for it, then why on earth would you not build this?”  Gulp.  But we stuck to our guns.

The past three years have vindicated us: it’s now universally accepted that any estimate of credit risk based on the credit rating are, at best, not to be relied on.  Today, my firm is actively working on building a much higher quality credit risk product – one based on market prices of Credit Default Swaps.  This approach has a much higher predictive value than credit ratings produced by agencies that are hired by the very firms they’re paid to evaluate.  Oh, I forget to mention that part – the credit rating agencies are paid by the same firms that are rated.  So instead of using that data, Investor Analytics is going to use the daily prices of CDS’s, which can be thought of as insurance against default.  The buyer of a CDS gets paid only if the firm (or government) defaults on a payment.  So the market prices of these securities give an indication of what real market participants – unbiased participants – think about the probability of default.  And when you test this default probability prediction, it works much better than those biased credit ratings.  I only hope the rest of the industry decides to use a tool that actually works.

As for the answer to the title of this post, I supposed that credit ratings are good for one thing: making the credit ratings agencies a lot of money.