Risk at the Limit

You have a problem.  A big problem.  And it’s unfortunately a common problem for a Risk Officer.  What do you do when your risk measures are all in the red zone except for one of them, which happens to be your boss’s favorite?  And he just happens to be the Head Trader of your fund.  It’s like an engineer telling the captain of the ship that the engines can’t go any faster – “Captain (in good Scottish brogue), I’m givin’ it all she’s got.”  To which the Captain replies “No, Scotty, look at this other gauge – it says we’re only at half capacity.”  Which gauge is right – the captain’s or the engineer’s?  Fortunately, there is a way to tell.  Unfortunately, captains don’t like to accept it when they’re wrong.

My favorite way of testing a risk gauges is called backtesting, and I like it so much because it forces you to be honest.  This is especially important if you were personally involved in building or designing the guage.  Since we all have some level of “pride in authorship,” it’s really important to try and remove as much of this bias as possible.  Backtesting works like this – if you have a tool or method that predicts something will happen, all you have to do is collect a number of predictions and compare to what actually happens over that same time period.  It’s obvious in theory but can be difficult in practice.  Let’s try it out:

  • Magic 8-Ball: that oversized liquid filled 8-ball might have magical powers of prediction, especially when used at teen-age sleepover parties.  Let’s see.  You ask it a bunch of questions, gather the various “ask again later”, “most likely” and “don’t count on it” answers and compare to what actually happens.  Aaawww, shucks, this thing doesn’t actually work.  Well, at least it’s still fun in between rounds of spin-the-bottle.
  • Astrology: the predictions in your horoscope often sound like they’re…you know something, on second thought, astrology is just too ridiculous to discuss in a serious blog.  Let’s move on.
  • Risk Measures: day by day you collect the risk system’s predictions and you compare to your actual daily profits and losses.  If the risk system predicts that the chances of being down more than 5% are only “one in a hundred” then you should not see such big losses more than 3 days a year.  But you should also not see them less than that often, either.  Collecting the predictions and actual P&L is the key to seeing if a particular risk measure does its job.  One of the best backtests I’ve seen compares predictions of different quartiles and quintiles of returns and examines if the measure works for large and small, up-side and down-side returns.  Nice.

So the Risk Officer backtests his risk gauges that are all currently in the red zone and he backtests the Head Trader’s risk gauge that is currently in the green zone.  Afterward, he schedules a meeting to tell his boss that the only risk gauges that pass the test are in the red.  In fact, the backtesting shows that his boss’s gauge really has no predictive power.  The two of them get into a heated tit-for-tat about “gut feel” and “mathematical accuracy.”  The Head Trader talks about how his gauge has been reliable for years and years.  The Risk Officer shows graphs of the back-testing results with cold hard evidence that it doesn’t actually work.  At some point, when the Head Trader realizes that he’s not going to win, he starts attacking the back-testing methods.  “How did you do this test?  What makes you think this is right?”  The Head Trader is convinced, deep down inside, that his gauge really works.  It’s what he’s been using his whole career and that means he probably takes it personally.  They start talking about raising the fund’s risk limits because the Head Trader is convinced that they are squandering a money making opportunity.

This is exactly where Risk Management matters most. What the Head Trader and the Risk Officer decide in the next few minutes can have serious impact on the future of the fund.  Raise the risk limits and you could potentially lose a lot of money if the Risk Officer’s numbers are right.  Stick to the established limits, the Head Trader argues, and the fund will probably not make as much as it could.  Would the Head Trader accept testing his ideas in a small account to minimize the downside?  Probably not – the whole point is that the entire FUND is missing an opportunity, and trying things out on a small account is not going to seize the opportunity.  And if it does work on a small account, the Head Trader will be infuriated that they didn’t do it on the whole portfolio.  Worse still, he may be emblazoned to overrule the Risk Officer in the future.  Not a good situation.

The best approach is to avoid getting into the situation in the first place by contemplating this scenario before it starts and agreeing on how to handle it when it materializes.  Before someone’s taking it personally, before there is a potential squandering of a money making opportunity, and before the entire purpose of the Risk Officer’s role is put into question – that’s when things like this should be decided.  Once the dialogue is heated, once one side or the other is emotionally involved, it’s a bit too late.  The Risk Officer’s entire function is questioned and could very well be undermined if he’s overruled.  The Head Trader will be frustrated and upset about missing the opportunity.  That’s when mistakes are most easily made.

This situation comes up more often that either a Risk Officer or Head Trader would like to think.  The best practice is to establish, beforehand, what tests or circumstances can warrant a change in risk limits.  It’s also important to consider how investors are likely to react to such a change.  What constitutes a valid reason to change the risk limits?  If the firm uses many risk limits, how many of them need to be in agreement to overrule a rogue measure – can just one measure in the green overrule five others that are in the red?  What about the other way around – can just one measure in the red force traders to reduce their risk?  All these sorts of questions should be answered in the fund’s risk policies and procedures.  And those policies and procedures should be updated every time an issue happens that wasn’t covered by them beforehand, so that at least next time you can avoid the heated discussion and concentrate on making risk managed profits.

One Response to Risk at the Limit

  1. this is directly in response to “….should the Head Trader accept testing his ideas in a small account to minimize the downside…..?”. My answer is yes. As an engineer, i advocate a little lab to test both the strategy and the measurand. Nothing is designed (e.g., building type, airplane, rocket engine, Dam) without extensive testing (theoretical and and physical) , before its rolled out to the field. Why should financial engineering be any different?

    Buyside entities are onto this. For instance, CalSTRS has setup an office of Innovation and Risk, with the goal of testing new ideas/strategies in a real but limited environment, and only after a period of testing/observation, to roll it into the “real” portfolio.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s