NY Times ran an article Risk Mismanagement:
VaR uses this normal distribution curve to plot the riskiness of a portfolio. But it makes certain assumptions. VaR is often measured daily and rarely extends beyond a few weeks, and because it is a very short-term measure, it assumes that tomorrow will be more or less like today. Even what's called "historical VaR" -- a variation of standard VaR that measures potential portfolio risk a year or two out, only uses the previous few years as its benchmark.
Nonsense. VaR is an innocent and useful mathematical construct, completely independent of the distribution or the model used. It can be as simple as subtracting variance from the mean to penalize for the risk of a distribution. Don't throw the baby (VaR) out with the bathwater (dubious VaR practices).
The real failure of risk management was in the bad short-tailed models (that underestimated the probability of a default) that they fit in a bad way (overfitting to a small amount of one-sided historic data, without using priors that would include the possibility of a disaster).
But even once these problems are fixed, economy will still be a feedback system, not something pretty, simple and linear. Let's hope for a marriage of statistical modeling with systems and complexity theory. In the meantime, I'll hope for using more common sense based on substance and less mathematics based on arbitrary metrics. This would help prevent disasters in the first place.
In that entire lengthy article -- I don't see any mention that these models were actually inappropriately fitted -- or that they were more extrapolations. Lending practices during the sub-crime fiasco changed to such an extent, that using historical models of default rates would be useless (see calculatedrisk)
I'm not sure how you could come up with reasonable priors for this scenario -- or whatever priors people would have used would have mattered. The change in lending practices would have been a gigantic qualitative change, not a minor tweak to a coefficient.
If its so easy to game a metric to reward yourself, while endangering your company -- maybe it is time to throw out the baby. If you can game it by just buying a put on the same stock you just bought -- that sounds like a joke. Sometimes no information is better than misleading information?
Nz, a simple prior is that there is a major crash every 15-20 years or so. While short-term speculation could continue to work under such conditions, long-term strategies shouldn't.
When it comes to ignorance: a fun paper is Grunwald and Halpern's When Ignorance is Bliss. Still, with a good prior, more true data is always better.
The problem with all these models is ignoring the level of debt used to finance speculative investment. Real debt (ie corrected for GDP) has doubled over the last decade, with a comparable increase in the rate of new debt, which finances the interest on existing debt. So historical data is irrelevant because it has nothing to do with the current system.
Any engineer will tell you that this is an increasingly unstable system but it seems lost on most economists and statisticians. The change in the flow of money as greed turns to fear eventually can't be counteracted by the fiddling of reserve banks and the economy collapses.
Se http://online.wsj.com/article/SB118736585456901047.html for an article on Minsky who developed a sensible model.
At the moment VaR went from being a short term management metric to something people were compensated on, the system was bound to be corrupted unless there were very tight controls over the operational definitions.
Whatever you compensate people on, they will game. That's the nature of any system -- and no group would be more aware of it than Wall Street guys.
They didn't care because once you pocketed this year's bonus, they weren't going to go back 5 years later and take it back from you.
Aleks,
I don't know anything about VaR, but I did see grad students at Columbia studying statistics and mathematical finance who knew all about the difference between the American and European options (whatever that means) but knew nothing about experimental design and not much about statistical data analysis. So I certainly think there was a problem in the way that some very small set of concepts were given too central a role in the world of finance.
On my facebook blog, I'm walking my readers through the construction of a simple derivative that has a 50% probability of complete loss within 2009 but a VaR metric of $0 (riskfree). This is possible by carefully selecting a combinatorial mix of investments that individually have a history of fat tails and skewed loss probability distributions.
IMHO, the problem is VaR because its modeling and methodology practices don't have adequate safeguards to protect against risk modeling for my example case. Other risk assessment methods do. So one way to fix VaR is to simply use one of the methods that sheds light on this sort of nonsense. However, doing so is expensive for the investment bank. It assumes that the risk management division/group will be beefed up with more people to actually understand the products and design appropriate risk analytic measures. Using a normal distribution instead of well constructed priors saves risk management money.
This problem is very similar to hiring white hat hackers to find bugs and systemic problems in Internet Explorer. They can easily find gaping holes but it requires spending time in the architecture of the system to find the really bad systemic holes.
In the case of Internet Explorer, the engineers might have the incentive to "ship" while the white hat hackers have the incentive to find defects. In the case of financial engineers, they have short term pay incentives to hide information from the risk managers (and the risk managers unfortunately have pay incentives to wink and nod). VaR's reliance on normal distributions was a selling point to spend less money per instrument on risk management.
There is no free lunch in risk management. The NYTimes article doesn't get to the heart of the problem with VaR because the author doesn't understand the debate. It can be summarized as: "VaR metrics were created during a time when financial engineering was being born. The risks associated with financial engineering are high, so a cost effective method of scaling the creation and use of the products was needed. VaR's methodology fit that role as long as you cared more about selling the products than actually managing risk. But the 'no free lunch doctrine' eventually won. VaR's promise of dramatically reduced risk management cost for complicated products was imaginary."
I completely agree that once VaR became a statistic reported publicly to quantify a fund's "health", and thereby a basis for compensation, the pressures to mold it's use to the needs of the managers rather than as a diagnostic tool for the risk managers became overwhelming. Even if the VaR model is wrong (and channeling Box, they all are), it can still be a useful signal in a process-control sense that the underlying assumptions of the model are breaking down, provided it's consistently applied and measured. But it often wasn't. I wrote about this yesterday too, and there are a couple of examples here:
http://blog.revolution-computing.com/2009/01/did-statistical-analysis-cause-the-global-credit-crisis.html
Why is it that insurance companies calculate ruin probabilities, while finance ones (a much riskier business) does not?
I have to agree with Taleb. VaR is a flawed indicator of risk. Goldman Sacks did what I do on a daily basis in IT - I use what little quantitative data I have and add a qualitative approach to make risk decisions. I strongly believe risk is a function of both and using only one will result in a flawed (and possibly catastrophic)result.
Aleks,
I think the real failure of risk management was unchecked fraud by people and firms selling collateralized debt obligations and other types of repackaged mortgage securities. This article from the NYT is both informative and more than a little shocking (link is courtesy of juniorprof. Given the contents, I don't think the misuse of VaR had such a great effect in this current collapse.
Due to firewalls in cities, efficent firefighters and sanitary systems, good understanding of occupational risk (a new aspestosis saga is unlikely) there are now far fewer systematic risks for fire-, accident and life insurancecompagnies. Climate change and pandemics might change this.
ZBicyclist: "They didn't care because once you pocketed this year's bonus, they weren't going to go back 5 years later and take it back from you."
To me, this is the core - people were compensated short term (and extremely highly!) on 'projected' profits from longer-term deals. This included (in fact, was concentrated in) the people near the top of the companies, who had the power to alter the information flows and discipline that otherwise would have kept a lid on this.
Under such circumstances, one shouldn't be surprised that when the guys at the top had the chance to loot the system, they did, right up until it collapsed.