ozrisk.net

Harking back to the issue of time granularity, and anticipating some default analytic calculations yet to come, let’s get back to small details and look inside the smallest data unit i.e. the time grain.

For typical retail products this granularity would be a month, which is the example carried forward in this post.

Monthly data warehoused for analysis purposes would typically be on a calendar month basis. An alternative (for CC?) might be data on a monthly payment cycle basis.

Even though a grain is ’small’ there is still latitude for vagueness because data recorded against a month may relate in several ways to the time axis within that month:

• point in time at the beginning of the month
• point in time in the middle of the month
• point in time at the end of the month
• the whole time window comprising that month

For most cross-sectional studies, the time axis is calendar date and the ’status’ variables like account balance would usually relate to the end of the month, as that would be their most up-to-date value. Other variables that summarise or count transactions (for example) would relate to the whole time window. Certain calculated values (like hazards AWML) may relate to the mid-point of the month.

In cross-sectional studies there is no difficulty in finding the point-in-time variables as at the beginning of a month, because these will be the (end-of-month) values from the previous month’s record - i.e. closing balance for Feb = opening balance for March etc.

If numeric date values are used as the key on a data table, they would most logically perhaps be set equal to the last day of each month, which is unfortunately a bit messy and harder (for a human) to remember than the obvious choice of the 1st of each month.

A non-numeric-date month key like “200805″ avoids specifying any particular part of the month, and leaves it up to the user to figure the time relationships from the metadata. A slight disadvantage of such a key is that date arithmetic (figuring out the difference between two dates) becomes non-trivial.

Longitudinal studies would typically rely on performance data for each individual account that is stored cross-sectionally i.e. by calendar month. This introduces a slight wrinkle because the account opening date can be anywhere within a month, whereas the performance data is only available at month ends. So the first performance measurement point an account reaches may come up in only 1-2 days (if the account opened on the 29-30th of a month) or alternatively may represent up to 30 days of exposure-to-risk. Longitudinal studies have MOB rather than calendar date as their time axis, and this means that the MOB=1 analysis really represents on average about 0.5 months of exposure, and likewise all subsequent MOB points really represent on average half a month less. (This example assumes your MOB counting convention starts at 1 rather than from 0.) But in any case, it would be most representative to start at 0.5 and count upwards as 1.5, 2.5, etc.

The above may sound picky, but it can quite easily come about that one analyst’s 12-month OW is another analyst’s 13-month OW due to choices at this level, and this could make a significant change to risk measures.

Further intra-grain issues will be met when calculating hazards. This basically means dividing the number of defaults (at a certain MOB) by the number of accounts that were exposed-to-risk of default. In a longitudinal study the number of accounts exposed-to-risk will always be declining, as accounts close good or go into default. Good practice would therefore be to find the average number (month_start + month_end)/2 of exposed-to-risk accounts during that month for use in the denominator of the hazard.

Actuaries are good at these deliberations because of the care and expertise put into estimation of mortality statistics. If you can’t find a tame actuary, the recommended approach is large diagrams on whiteboards and a bottle of headache tablets.  

In the context of retail applicants, application scorecards etc., is there a well defined meaning for “risk appetite”?

My feeling is that it could be referring to either the marginal or the average situation, depending which hat is worn.

The risk management function acts as the gatekeeper, drawing the line on acceptable levels of risk - risk appetite - by setting cut-offs. This is wearing a marginal hat: the cutoff is the margin. For standard retail products it may conveniently be set in terms of PD, although more completely it would be an expected loss calculation. To caricature this risk manager, he doesn’t mind how profitable the applicants are, as long as they are just above the cut-off.

OTOH the business will be looking at the overall profitability of the product/campaign/portfolio whatever. The business manager is more likely to phrase his risk appetite targets in terms of average PD. To caricature the business manager, he doesn’t mind if a number of poor decisions are made around the margins as long as the venture as a whole makes a good return.

So is the setting of risk appetite about trying to decline applicants below a certain marginal PD, or is it about trying to achieve a certain average PD for the accepts? 

Complicating the discussion is the role played by volume. Higher cut-offs naturally mean lower volumes of successful applicants and this frustrates the assumptions on the business case.

The previous post discussed setting of scorecard cut-off by the criterion of marginal profit, which translates to marginal PD. But what about average profitability (or average PD) as a measure for a portfolio or as a criterion for arriving at a cut-off?

The general issue of marginal vs average cost, revenue & profitability is a familiar one in business economics and won’t be revisited here.

However, the particular feature of the debate that is relevant to setting a scorecard cut-off is the set of assumptions made about the TTD (through-the-door) population.

A cut-off is by its nature a marginal issue. If the cut-off is at (say) odds of 15:1 then we know that the only accepts will be those with PD of 1/16 or better. But what will the average PD of all the accepts be? That will depend on the distribution of the TTD - proportionately how many applicants there are in each risk band. Obviously the average PD of the accepts will be better than 1/16, but how much better will depend on a calculation based on the shape of the TTD distribution. A typical assumption is that future TTD will be like past TTD for similar products, but this assumption can sometimes turn out to be quite wrong. The drivers of TTD are a complex mix of marketing, the competitiveness of the product, actions taken by competitors, and the economic climate. In plain English, you might have an excellent scorecard, but if lousy applicants walk through the door, it will be hard to do good business.

Nevertheless, it is natural for the business to ask questions about average profitability (equivalently, average PD) because that characterises the overall returns on the portfolio. Also note that besides the shape of the TTD distribution, a big parameter assumption is the volume of TTD. Volumes are important for diluting the ‘fixed cost’ aspects of the business costings.

So the business will probably want to target a certain average PD. The cut-off decision, though, is strictly a marginal one, which has only an indirect effect on the average, mediated by the TTD assumptions. Modellers should communicate these levels of uncertainty to the business as it is all too easy for a computer printout to look infallible.

Odds provide a useful frame for considering that important business question: where to set the score cut-off.

The basic business logic is to set the cut-off at the score where marginal profitability equals zero - i.e. if you moved the cut-off any lower you would be losing money on each additional applicant so approved, whereas if you set the cut-off higher you would be leaving money on the table. Easy to say, but not so easy to do, because the concept of marginal costs is a movable feast depending on accounting treatments and assumptions about fixed and variable costs, as well as the context within the current business strategy. 

But anyway, the odds allow one to frame the question in an easy-to-grasp way: how many goods does it take to offset one bad? If the answer is 15, it means that your tipping point is at 15:1 odds, which can be converted to the score as per previous post. This would then be the cut-off. This post assumes a simple automatic accept/decline score, ignoring ‘refer’ bands and contested decisions and overrides etc. 

To arrive at “15″ would involve a full revenue/cost modelling through the product cycle (lifetime customer value?), for 15 goods versus 1 bad. Naturally the “cost” that dominates here is the credit loss of principal (LGD) for the default.

Don’t pay any attention to the example value “15″ used above - it’s going to make a lot of difference what product is involved, secured vs unsecured, limits, etc.  

The credit risk world likes to work with ‘odds’ and related quantities so these are covered today.

You could just do everything in terms of probability, i.e. PD, which is unambiguous. PD lies in [0,1] and a small number (like 0.002) is a better customer than a bigger number (like 0.013). In typical modelling situations (in Australia, in the good times..), a lot of PDs would have one or two or even three leading zeroes and these numbers are not handy for transcription or to quickly convey which zones they lie in.

It goes without saying that it often more palatable to format a PD as a percentage, e.g. PD = 0.013 as PD = 1.3%.

‘Odds’ have a special status because they are intimately linked with logistic regression, the main PD-modelling statistical tool. Odds can be worked out from the PD, and vice versa, as follows:

• odds = 1/PD - 1
• PD    = 1/(1 + odds)

For example, odds = 8 means exactly the same thing as PD = 1/9 = 0.1111.. 

Odds are generally taken to be the Good:Bad odds; thus a bigger number for odds is a better situation. I have seen analysts using Odds the other way up i.e. the Bad:Good odds. You can come out alive but it will confuse your colleagues; +/- changes of sign will cascade through and graphs will tilt the opposite way.

One step closer to the logistic zone is to transform to “log_odds”.

• log_odds = ln(odds)
• odds        = exp(log_odds)

‘ln’ means natural logs, i.e. to the base ‘e’. Actually, mathematicians always mean natural logs when they say log and as a matter of pride would never mention the base, or contemplate a base other than ‘e’ unless it was a neat way to summarise a problem that had structure particular to integral bases. Ambiguity can arise: computer systems that are tech-oriented, like SAS or MATLAB, assume ‘log’ means ln, whereas those that are business-oriented, like MS/Excel, assume that ‘log’ means log_to_base_10. It also doesn’t help that ‘ln’ is not comfortable in speech.

By ‘log’ I always mean natural log, and I use log10 or log2 to mean logs to base 10 or 2. For the meantime, the terminology ‘log_odds’ will be used, which is easy in speech, but if anyone can suggest better nomenclature they are welcome to put it forward.

If we’ve taken the right choices so far, a bigger number for log_odds is a better situation. Note that log_odds can be negative (when odds < 1 which is when PD > 0.5).

To make the numbers more convenient to handle, it is common practice to convert the log_odds to a ’score’ on a user-friendly scale that wouldn’t involve negatives or decimal places. For the first time in this chain of transformation, arbitrary scaling constants are involved in this choice: one for location and one for scale (spread). A typical approach is illustrated below:

• for location: bang a stake in the ground at the point that will represent odds of 1 (== log_odds of zero == PD of 0.5): so, for example, choose a score of 500 to represent this point (which BTW would be a lousy customer)
• for scale: this is normally done by specifying how many points it takes to double the odds (PDO). A comfortable choice would be PDO=20, which says that a score of 520 <=> odds=2, 540 <=> odds=4, 560 <=> odds=8 etc.

Because log_odds is a logarithmic scale, the above choices work out and amount to a linear transformation of log_odds to score. The two scaling parameters, and hence the transformations from log_odds to score and back, will depend on these fairly arbitrary choices.

PDO=20 gives a nice granularity to the scores, which will mostly land in the 500-800 zone and you won’t feel the need to use decimal points i.e. whole-number scores suffice. As long as PDO is chosen to be positive, it will still be the case that a bigger score is a better situation.   

All the above transformations are absolute arithmetic ones that always apply, irrespective of context such as outcome window, default definition, calibration, closed goods in/out, etc. If you find you disagree with someone via these calcs, it means you started from different contexts and therein lies the entire explanation for your disagreement.

Following the comments on indeterminate, it is timely to introduce some shorthand notation that will help in discussions that follow.

The issue is the point-in-time default definition. Your default definition should in the first instance produce a decision at every point in time as to whether the account is in default or not. The set of possible points in time is determined by the time granularity of your data systems. A typical situation would be monthly data for CC with default flagged for >=90DPD assuming the outstanding balance exceeds some materiality parameter(s).

But why point-in-time default definition? Because this is not the final default story; the re-ageing logic still needs to be superimposed. Re-ageing involves an extension of the point-in-time default definition to the concept of a default episode, which has temporal extent i.e. it is a time window having a start date and an end date. Today’s post, however, covers only the point-in-time default issue, and the qualifier “point-in-time” will be left out to avoid clutter.

Default history for any particular account can be summarised by the string of consecutive default statuses: for example GGGGGGIIIBBG shows the account was ’good’ for the first 6 months, ‘indeterminate’ for the following three months and then ’bad’ for two months but then ’good’ again in the 12th month. These 12 months could be the first 12 months since the account opened, if you are doing longitudinal analysis, or it could be the 12 months of a cross sectional analysis, in which case it might represent something like MOB 33-44.

The definition details of ‘bad’ and ‘good’ will be particular to each institution and product, but status codes that I have found useful include:

• B = Bad, i.e. point-in-time in default
• I = Indeterminate. Optional status, not all situations require that one should need to distinguish these from G and B i.e. ask the question: how does ‘I’ differ from ‘G’?
• R = in recoveries
• C = in collections
• W = has been written off
• G = Good, i.e. not bad nor any other status with a higher precedence
• U = Undrawn. This can apply to loan accounts that have been set-up and are open on the books, but where the capital has not been drawn down yet. For HLs there can be a few months delay if there are hold-ups in transfer. In the meantime, they appear as accounts with zero balance outstanding. This only applies when this situation happens at the beginning of an account’s history, i.e. not for zero balance accounts that can occur later. Undrawn does not apply to some products such as CC because they are only activated when the first transaction is made.
• D = Dormant. It may be useful to identify accounts that appear to be dormant, i.e. have returned to a zero balance and there is no customer initiated activity for a long time. Because of Basel treatment, but also for commercial reasons, the bank may want to identify these and do something about them. 

I am going to break my (self-imposed) silence and make a post on this. Following on from an earlier post on the situation in the UK it looks like the legal situation is the same here - or in Victoria at least. Today’s Crikey has a story on how one of their contributors took Citibank to the small claims tribunal over a $40 fee - and won, with costs, after Citi simply failed to turn up. Even better, Citi had paid out the claim even before it hit the tribunal.

The Crikey piece notes that this does not set a binding precendent, but

the fact that a full-time VCAT member provided a judgment noting that the bank-fee charged was unenforceable and amounted to an unfair term in the contract is an indictment on the conduct of a financial institution. While Citigroup did not defend the matter, the VCAT member would have been within his rights to dismiss the application if he was of the opinion that it was without merit.

So, this one is just waiting for a test case. We have an interesting possibility here. If you believe the fees you are paying are excessive then - claim them all back. All of them. Just find a lawyer willing to take on your bank.

Surprisingingly, Citi’s newsroom says nothing on the topic of this court loss.

Thanks to The Sheet for pointing me at this.

In the PD model building world, “indeterminate” seems to have more than one meaning. If any readers feel they could give a balanced view of common usage in Australia (or elsewhere), please do so and this blog will record it and adopt it.

Meaning #1: a status of an account at a point in time which is not “in default” but is some way down the track to being considered “in default”. For example, if the CC default definition requires 90DPD, accounts might be called “indeterminate” if they are 60DPD - or whatever other “not completely good” your default definition might permit.

Meaning #1.1: a meaning derived from #1 can then be evolved for the status of an account across a time window such as an OW for modelling purposes. “Indeterminate” might now mean “ever went indeterminate during the OW without ever going intodefault during the OW”. However, one also sees composite definitions of Indeterminate across an OW such as “ever went 60DPD or went 30DPD on two occasions”.

The idea behind the above definitions of Indeterminate is that the account, whilst known not to be “Bad”, is also known not to be completely “Good”. IIUC these above meanings are the most common in the banking industry but your corrections will be tallied and recorded here.

It will also be handy to adopt the likewise common terminology “Bad” for “in default”, along with “Good” and “Indeterminate” and their abbreviations “B,G,I”. These are used in textbooks for technical formulas like odds ratios and information values AWML. In today’s post this usage remains casual and by “G” might be meant “not B” or perhaps in another context “not I nor B”.

Meaning #1.1.1: A special situation related to #1.1 deserves noting. As it stands, #1.1 means that the account was known not to have gone bad during the OW. It isn’t a situation of doubt as to the outcome (contrary to what the English word “indeterminate” connotes). However, one particular case does involve reasonable doubt: when an account has reached the penultimate stage at the end of the OW - for example, a CC has gone 60DPD in the last month of the OW where the default definition is 90DPD. Unlike other Indeterminates that may have gone 60DPD and then rehabilitated, with this ”horizoned” account one doesn’t know which of the categories G, I or B it should really belong to. OK, it belongs to “I”, but not in the same sense as an account that reached 60DPD and then rehabilitated during the OW.    

Meaning #2: More like the natural English usage, this meaning covers situations where one isn’t sure about assigning “G” or “B”. For example, consider application modelling with an OW of 24 months:

• An account closes good after only 2 MOB. Is this a “Good” account? Not in the same sense as one that was exposed to risk for the full 24 months. One might call it indeterminate in the sense that one doesn’t know whether it would have been G or B if it had hung around for 24 months. I prefer the more specific term “closed good” for this situation. 
• (Similar to above) Only the first few MOB are known because the account opened recently. I prefer the more specific terms “out of sample” or “out of time” for these situations.
• For whatever other reasons, such as incomplete data, one doesn’t know the exact outcome of some account at some point in time or across some time window.

This post is only about the nomenclature, and is not even definitive on that point! As to what you do or don’t use “indeterminate” for in the modelling world, that subject is too long for this week.

“Churn” is used here to refer to accounts closing ahead of schedule for reasons not related to default. Perhaps this is not ideal terminology - I tend to use it because it is short and specific - but other suggestions for common usage would be welcome.

One variation encountered is “closed good”, which will be used later in discussions of “closed goods in” versus “closed goods out” as bases of analysis. This nomenclature is more comfortable than “churneds in/out” would be.

Meaning varies amongst products. For CC, there is no fixed product schedule and churn would normally have the marketing meaning of customers taking their business elsewhere - e.g. “balance transfer” to another CC issuer. This has been a particular concern with aggressive marketing by competitors offering low or zero interest for an introductory period.

For term loans with a fixed principal & interest amortisation schedule, churn could come about from re-financing of a HL or PL with another lender. A similar issue is the early paying down of the loan balance on products that allow this. “Churn” is not a descriptive word for this behaviour - the account may remain open and active but have a much lower loan balance than the bank was expecting. Lower funds at risk means lower earnings for the bank, affecting the profitability model for the product cycle. What would be of particular concern, and likely in practice, would be the correlation between early payment and low PD, i.e. the lowest risk customers reducing in proportion of funds at risk.

As regards default analytics and PD models, churn is a countervailing force to default. If a portfolio has high churn, it will make the default experience look better (if analysed on a “closed goods in” basis AWML). To make a clear analysis of a portfolio it is better to analyse the effects of churn and default separately from each other. For profitability studies, each plays a role.

This post is as close as I get to a “rant”.

Some parts of Basel formulas have unnecessary complexity, which involves not just inefficiency but also potential pitfalls.

The specific example is the formula for asset correlation which appears in Basel paragraph [283] and which includes a term 0.12 x (1-EXP(-50 x PD)) / (1-EXP(-50)). There are similar terms elsewhere in Basel formulas, but for focus let’s look at just this case. Surely, this term should be given as simply 0.12 x (1-EXP(-50 x PD)).

Presumably the casters of the formula felt a need to normalise the term to handle PD its full range of [0,1]. This may satisfy academic neatness but, I maintain below, at significant risk of causing error or wasted resource.

The materiality of the normalising denominator is as close to nil as any banker could imagine. The term EXP(-50) evaluates as 2 x 10**-22 which means 0.0000000000000000000002. When one subtracts this from 1 it makes no difference and you still end up with 1. In the old days, this used to cause a computer error known as underflow, whereby the floating point arithmetic processors rearrangeing numbers for calculation would discover that during this process one of the quantities had disappeared, which although not automatically a fatal error, would probably be something you wanted to know about. In the case of the above Basel term it’s not an error but it is frivolous formulaic complexity and in practical terms the denominator equals 1 and the term should be simplified to 0.12 x (1-EXP(-50 x PD)) .

OTOH to make a pedantic point if Basel wants the answer to come out exactly the same, they could change the multiplier from 0.12 to 0.1200000000000000000000024  . Or, add a sentence in the doc saying that, whilst a normalising denominator was academically desirable, it was omitted on materiality grounds.

Whatever, the materiality of the denominator term is less than a thousandth of a cent even when multiplied against a capital figure of $100billion.

What makes this a non-trivial rant is that there is significant cost to extra complexity. In my experience, only the most adept of the technical team would be able to transcribe such a formula without error. Others not directly familiar with the context, such as managers or computer programmers, are prone to transcription errors. But, most perversely, if I were to see such a formula presented by an intermediary - say, for example, as part of a computer program - I would assume strongly that a transcription error had been made because the logic of the formula fails the “sanity test”.

Much as I admire maths, a Basel implementation is fraught with thousands of small hurdles (OK and big ones), and we owe it to the business community to adopt pragmatic standards.