One thing I noted during a look through all of the major banks’ APS 330 disclosures is that the view of risk they took was completely one sided. For example, in the market risk disclosures VaR exceptions were reported as if the only exceptions were the downside ones – if the bank made money on the day it was not an exception even if it was outlide the VaR limits.
One thing that the episode with the NAB in 2004 should have shown is that you need to investigate the good stories as well as the bad ones – a point driven home by Bernie Madoff recently. The money quote from the Forbes article is correct –
It’s virtually impossible to have returns like Madoff reported, and it should have been a major warning signal.
It should have been, but it was not as few people cared to look at the signal, as a one sided view of risk is common.
Looking at APS 330, though, I think the view they have taken of the disclosures is actually not following the Standard and is likely to encourage risk taking activity. Not, of course, that I think that any of the banks here are running any rackets like Bernie’s.
For example, if you look at the Westpac disclosures under APS 330 you see the same sort of view of risk1.
Westpac VaR October 2007 - September 2008
See that red line going down at an angle? This implies that it is only a VaR breach if there is a loss. There are, however, at least four (and maybe a further one) more sitting above where a line would have been if a line were drawn above the x axis to mirror the red one that is there. See them? There are two where the daily VaR was AUD20m, with returns of around AUD30m and a further two where the daily VaR was AUD30m but the returns were about AUD45m and AUD 55m. The AUD55m one is in fact the biggest VaR breach the bank had. The further one is (I think) at the daily VaR of AUD10m, with a return that looks suspiciously like AUD15m.
If those are not VaR breaches then I know little about what VaR is. But that’s OK, isn’t it? The bank made a profit.
Umm, no. APS 330 is clear if you look at page 19. Table 11(d) calls for:
…a comparison of VaR estimates with actual gains/losses experienced by the ADI, with analysis of important ‘outliers’ identified in back-test results.
Nowhere in there does it say that the outliers are only to be the downside outliers. Table 11(b) may allow the regulator to approve this interpretation and it may be that APRA have allowed them to ignore the upside outliers, but if this is so then APRA is endorsing this.
I am not sure which is worse – the Bank ignoring VaR breaches of the possibility it is under instruction from the regulator to do so. I just hope they are not reported internally like this.
In any case, may I cordially suggest that this not continue. A VaR breach is a VaR breach and should be treated as such, positive or negative. Risk is double sided – high returns normally come with high risk. A failure to investigate a high return encourages traders to take more risks and there are many trading departments where a disasterous loss has been heralded by an impressive gain.
Just ask Nick Leeson, Gianni Gray, David Bullen, Vincent Ficarra and Luke Duffy, amongst many others.
1. I am picking on them as the data on this for them is at least public. I have no idea what it is for the others.
6 comments
29 December, 2008 at 06:55
johan steunenberg
Hi Andrew,
as far as I know this is what VaR is, though I agree with you that this is another flaw in the VaR methodology.
but: a VaR breach is a VaR breach on the downside, not on the upside, AFAIK.
Have a good start of the next year, and I wish you lots of interesting posts again.
29 December, 2008 at 19:14
Andrew
Not what I was taught – I always prefer to see risk as double sided. Have a good new year.
13 January, 2009 at 16:39
AUSQUANT
Andrew,
At some places I have worked (in australia) we certainly tracked both upper and lower VAR breeches. However from memory the only ones APRA were really interested in were lower VAR breeches. Certainly that is the criteria (4 lower VAR backtesting breeches in a single year) before they ask for explaination.
In fact the impression I had they more or less regarded our presentation of the two sided analysis as something suspicious, as though we were trying to hide the downside moves. I remember thinking that this was ridiculous, in most markets its as easy to generate a short as long position so either is a test of the model.
13 January, 2009 at 17:10
AUSQUANT
By the way I am also quite skeptical aobut the comparison with Actual P&L. VAR is on the same basis as hypothetical P&L. My experience is that there was frequently a low correlation between the two. You would get negative hypothetical moves on a day where the room would get substantial positive moves as they changed some major positions intraday – or the reverse would also be true.
I don’t see that it tells you much about your model integrity.
13 January, 2009 at 18:52
Andrew
Ausquant,
Every place I have worked (Aus and UK) always tracked both – and required explanations and testing of the model when either was breached. I am not a quant (more on the reg side), but to me, as you say, a breach either way is a test of the model.
I am not entirely surprised that APRA asks for this, but I am disappointed.
13 January, 2009 at 21:27
AusQuant
Yeah everywhere I have been they have tracked both as well. However I think they show you the most flattering one. From what I understand the hypothetical results at all Aus banks would not be really flattering. This is to be expected since we have had steeply rising volatility, but I imagine they don’t want to scare the punters by saying we expected 2.5 but got 25.