You are currently browsing the category archive for the ‘Stress Testing’ category.
Slightly wonky topic today, but one that is very important. Most of you out there doing financial modelling should already know this (but could do with some reminding) and those looking into this sort of modelling (and bank management who get fed these reports) should know.
For most situations the normal distribution is not the one you should be using for financial modelling.
I’ll say it again in a different way – if you are going to use the normal distribution first prove that it is the correct one to use.
I know that the Basel II Accord mandates its use (even helpfully giving the Excel function “NORMSDIST” in the footnotes) as do many other regulations but there is a mountain of analysis in the academic sphere showing that the normal distribution is not the most correct – understating the probability of the “long tail” or “black swan” events.
Lévy distributions are much more correct – there are particular examples that give much more weight to the long tail than a typical normal distribution does. Work done by Mandelbrot and Taylor (yes, Mandelbrot of the pretty fractal pictures) in the 1960s first showed this and there has been considerable work since then. Just have a wander through Google Scholar for some fascinating reading.
Why then do we keep using the normal distribution? I think that it is just a habit and that we get it drummed into us at university. There has been an enormous amount of work done on the normal distributions, so we also look to leverage off this.
My message to management, then, is this – if a risk assessment comes to you having used the normal distribution the first question you should ask is “Why have you used the normal distribution?” If the answer is something along the lines of “We always use it.” then you should ask them to go back and justify its use. If the answer is “We have been told to do so by the regulators.” then you should tell them “Fine – use it for regulatory numbers. Just give me the real numbers as well.” If the answer is “We have modelled the market and the normal distribution is the best fit after considering the other possible distributions.” then your answer should be “Fine – thanks for that.”
It may well be time to pull out that pandemic plan and at least give it a good look over, ensure key staff are aware of what they should do etc. etc. etc. Also take a good look at how your systems may cope with something like 50 to 75% (or more) of your people working from home – which should be a part of any such plan.
The swine influenza outbreak may, as most of these do, turn out to be a tragic event with some loss of life but not one on the scale of 1918. All the same it would be better to have everything in order.
Points to look at:
- Website readiness – make sure it can be updated with the latest information on a regular basis. As BWA found out last time they had a major problem it was the lack of information that really annoyed people.
- Media plan – ditto. Make sure you have the numbers for relevant journalists handy.
- Staff planning – get as many of them vaccinated as possible and have some masks on hand at the branches. They may not be needed, but what is a $5 mask compared to your staff being sick? Do the staff that are not immediately needed know that they are the ones who will not be immediately needed and may stay at home?
- Systems – with most admin staff at home, will your systems cope? Are there enough laptops ready for all the ones you really need?
You did have all this ready a year or so ago, so it should not take much to make sure it is all there. You did have it ready, didn’t you?
Jennifer (see the previous post) pointed me to this presentation to the IAA’s annual conference by Dr. Mark Lawrence. While it breaks many of the rules I normally set for a group of slides it is one of the presentations I would like to have been at.
While I do not agree with all of it (he thinks that fair value accounting is an issue, for example), it makes a number of very good points – for example:
Regulators should support the private sector’s efforts to improve transparency, with particular reference to harmonization of disclosure requirements among different jurisdictions. The official sector should work closely with industry and market participants to improve market understanding of Pillar 3 disclosure content. Disclosure requirements should be based on a risk-and principles-based approach to qualitative as well as quantitative information
…
In the view of many, culture is the single most important determinant of risk management effectiveness
…
Risk Managers concerns [are] often pushed aside [ask HBOS about this one]…
…
Capital models were originally created for important business purposes, and won’t go away …
Therefore, users (and supervisors) must understand very well the weaknesses and limitations of the these models:
- exactly what is measured, and what is not
- exactly what the models can and cannot be relied upon for
- when they work (i.e., under what conditions) & when they don’t
It is worth at least a read. The point about understanding your capital models is crucial. They are not and will never be the be-all and end-all of risk management. Like any model they are only as good as the common sense put behind them.
This is a guest post by Jennifer Lang. It provides the background behind a presentation on economic capital which is being made to the Institute of Actuaries of Australia’s Biennial Convention
How and why should we measure it?
Economic capital means different things to different people. But for this presentation, the purpose of economic capital is to assist companies in appropriately measuring the rate of return a company is getting in proportion to the risk it is taking.
Economic capital is not:
- Regulatory capital – regulatory capital is the amount of capital a regulator has determined an institution needs to hold, but is generally not as specific to the institution as economic capital would be. The capital for particular risks would be calculated more broadly, and the definition of risk would be a systemic one, rather than an institutional one.
- Value of the organisation – the value of an organisation (in the long term) should be the discounted value of future distributable profits. There is no reason for economic capital (which is a measure of extreme risk) and discounted value of future profits to have any defined relationship. Companies should, however, be earning adequate return on the capital they hold – the purpose of economic capital is to help them work out what that return should be.
- A capital resource – economic capital measures a capital requirement, not how much capital a company might have available. Net assets (below) are a capital resource
In working out what to do with economic capital, the capital
resources available to the company can include net assets, future value
of profits, and some other assets which may not be recognised for
accounting purposes. On the flip side, some accounting assets (such as
the goodwill paid for a recent acquisition) may be valueless in an
economic capital scenario.
Read the rest of this entry »
The new paper from the BIS on stress testing looks to be an important one. I have not had a chance to go over it in detail yet, but if you are in this area it is one you should read urgently. I would also add that much of this is likely to be incorporated in the next version of the Accord.
I will put together some comments on it over the next week as time allows.
One thing I noted during a look through all of the major banks’ APS 330 disclosures is that the view of risk they took was completely one sided. For example, in the market risk disclosures VaR exceptions were reported as if the only exceptions were the downside ones – if the bank made money on the day it was not an exception even if it was outlide the VaR limits.
One thing that the episode with the NAB in 2004 should have shown is that you need to investigate the good stories as well as the bad ones – a point driven home by Bernie Madoff recently. The money quote from the Forbes article is correct –
It’s virtually impossible to have returns like Madoff reported, and it should have been a major warning signal.
It should have been, but it was not as few people cared to look at the signal, as a one sided view of risk is common.
Looking at APS 330, though, I think the view they have taken of the disclosures is actually not following the Standard and is likely to encourage risk taking activity. Not, of course, that I think that any of the banks here are running any rackets like Bernie’s.
For example, if you look at the Westpac disclosures under APS 330 you see the same sort of view of risk1.
Westpac VaR October 2007 - September 2008
See that red line going down at an angle? This implies that it is only a VaR breach if there is a loss. There are, however, at least four (and maybe a further one) more sitting above where a line would have been if a line were drawn above the x axis to mirror the red one that is there. See them? There are two where the daily VaR was AUD20m, with returns of around AUD30m and a further two where the daily VaR was AUD30m but the returns were about AUD45m and AUD 55m. The AUD55m one is in fact the biggest VaR breach the bank had. The further one is (I think) at the daily VaR of AUD10m, with a return that looks suspiciously like AUD15m.
If those are not VaR breaches then I know little about what VaR is. But that’s OK, isn’t it? The bank made a profit.
Umm, no. APS 330 is clear if you look at page 19. Table 11(d) calls for:
…a comparison of VaR estimates with actual gains/losses experienced by the ADI, with analysis of important ‘outliers’ identified in back-test results.
Nowhere in there does it say that the outliers are only to be the downside outliers. Table 11(b) may allow the regulator to approve this interpretation and it may be that APRA have allowed them to ignore the upside outliers, but if this is so then APRA is endorsing this.
I am not sure which is worse – the Bank ignoring VaR breaches of the possibility it is under instruction from the regulator to do so. I just hope they are not reported internally like this.
In any case, may I cordially suggest that this not continue. A VaR breach is a VaR breach and should be treated as such, positive or negative. Risk is double sided – high returns normally come with high risk. A failure to investigate a high return encourages traders to take more risks and there are many trading departments where a disasterous loss has been heralded by an impressive gain.
Just ask Nick Leeson, Gianni Gray, David Bullen, Vincent Ficarra and Luke Duffy, amongst many others.
1. I am picking on them as the data on this for them is at least public. I have no idea what it is for the others.
Market risk is the oldest part of Basel II, being substantially
unchanged from the 1996 (1998 implemented) market risk amendment to the
original Basel I Accord. As such, the “Big 4” have all been using these
methodologies for about a decade. All that really changed with Basel II
was the addition of Pillar 3 – these disclosures. As such, you would
expect that these should be the clearest disclosures as the underlying
systems and processes are very well understood.
Over the last year, though, this area has probably undergone the most
change, both globally and in Australia due to the market turmoil. The
VaR numbers have probably moved enormously, particularly over the last
few months as some of the extreme movements have come through to the
modelling.
That makes the disclosures of great interest – the only real problem is
that they are effectively two months out of date when they come out –
except in the case of the Commonwealth, where they were five months out
of date.
Qualitiative Disclosures
These range from the almost insultingly short (that’s you, Westpac) to the sorts of disclosures that probably come close to best practice (thanks – CBA again). The CBA diagrams could be used as slides in a presentation and the text used to read out at that presentation. They are that good.
In reverse qualitative order, then:
Westpac
Westpac has kept them as short as is humanly possible while still maintaining compliance with APS 330, provided you use a generous interpretation of what “describe” means. For example, I do not see this desciption of backtesting as in the spirit of comprehensive disclosure:
Daily backtesting of VaR results is performed to ensure that model integrity is maintained. A review of both the potential profit and loss outcomes is also undertaken to monitor any skew created by the historical data.
That’s it – the entire description of their backtesting regime. The descriptions of stress testing and everything else are equally short. If you are looking for which disclosures to use as an example of best practice under Pillar 3, well, you would not use these unless you are attempting to work out how minimal your compliance needs to be. If you want to know how Westpac actually goes about managing these risks then using these disclosures will not be an appropriate place to start.
Score – 1 out of 5 as it does comply with the standards. Sort of.
ANZ
The ANZ is slightly better than Westpac – but still on the marginal side of good – with just a brief description of internal processes and a little bit better description of things like backtesting. At least they are using some hard numbers for what a backtesting exception is, even if they give no examples. Westpac at least beats them here – but only with a diagram that really tells you very little. Again, do not use these to find out what the ANZ is up to or how it really manages its risks.
2 out of 5.
NAB
The NAB is appreciably better in all areas than either the ANZ or Westpac, with some real disclosures of how they manage the market risk processes. The chart of backtesting results, with actual dates on it, represents a form of disclosure that gives me some confidence that they are actually managing the risks. Given the problems that the NAB have experienced in this area over the last few years, I am happy to see this. Their stress testing description, for example, while not exactly adding to the published literature is actually usable.
3 out of 5.
Commonwealth
The guernsey for the best disclosures go to the Commonwealth (again). The diagram on the first page of those disclosures, by itself, beats anything that either Westpac or the ANZ has put out. Read on and it just gets better. For example, not only have they put in a table giving each and every day over the period where backtesting exceptions occurred, but they have added the hypo loss and VaR numbers for each of those days. They have also named the functions responsible for each of the areas of risk and have added their reporting lines and some of the delegations.
4.5 out of 5 – but only because I like to leave some room for improvement.
Quantitative Disclosures
The important numbers here are the VaR numbers – but these cannot be directly compared as each of the banks has their own, differing, portions of their portfolio covered by the Standardised method – i.e. they are not included in the published VaR numbers, even if they are modelled internally using VaR. The other problem is that CBA’s numbers are as at 30 June and the others are as at 31 September. Given the increased volatility over those three months, the CBA numbers would have increased.
With those caveats, though, the capital numbers1 in both alphabetical and size order are:
- ANZ: $175m Standardised and $34m IMA
- CBA: $225m Standardised and $135m IMA
- NAB: $274m Standardised and $133m IMA
- WBC: $291m Standardised and $234m IMA
While fairly small in the context of each of the banks, these numbers are intersting in themselves, with WBC being by far the largest and the ANZ the smallest. Given that, it is doubly disappointing that WBC chooses to say the least.
1. They are buried in differing ways in each of the disclosures, but this is my calculation from the published numbers
Following on from the discussion in the previous post on whether bank1 deposits are money the question arises as to what happens when bank depositors try to convert their bank deposits into money – make a withdrawal, write out a cheque, pay a bill or uses any of the other methods to get at the funds. Will the bank be able to meet the demand for cold, hard, cash?
In short, how do banks manage liquidity?
The Problem
For banks, the problem is actually a fairly simple one to state. Long term, banks typically make money by borrowing short and lending long. As yield curves are typically upward sloping this works well – borrowing borrowing from people who want to deposit short and are prepared to receive between 0 and 4 or 5% to do so and then lending this to people who want to borrow to build homes and pay from 6 to 10%, run credit card balances at around 12% (or more) is a good business. With modern banking practice this even is profitable at a net interest margin of less than 2%.
Given that bank makes the most money by transforming short-dated liabilities into long dated loans the way to make the most money in the long term is to lend it all out and for as long as possible. Great strategy – with only one flaw. Some depositors are inconsiderate enough to want to be able to actually ask the bank to do what the bank has promised to do – pay their deposit at call.
The trick to making the most money, then, is to make sure that you only have enough liquid assets on hand to meet all your depositors calls on the funds and as little as possible more. This is because liquid assets pay little interest, with the most liquid, cash, paying none at all.
Getting this right is the responsibility of the ALM (Asset / Liability Management) function, usually headed by the (gloriously named) ALCO (Asset Liability Committee).
Get it wrong and, no matter how solvent your bank, if it cannot pay depositors calls you will very shortly not be a functioning institution. Read the rest of this entry »
It is not that far away to the date of Basel II implementation. Most banks would be on their final hurdle, well, at least for Pillar 1.
Up until now, firms who were planned to be beyond standardized, have spent millions. With lot of projects start winding up. One question has floated to the surface, which somehow, did not have a clear answer back then.
For Basel modellers, a common problem is a relatively short period of data. In most firms, they might not even able to get up to 5 years worth of data. Hence, the PD models would normally reflecting a Point in time (PIT) estimate or a Short run cycle (SRC) estimate.
Now, its nothing wrong having a PIT estimate. It is really up to the Bank’s senior management to decide if they want: –> You can read more here
Most popular