Stress Testing is Not Suited to Treasury Capital Policy

There is a lot of discussion of stress testing lately, with no real explanation of what stress testing is. I thought, therefore, that the subject could use a bit of explanation for those who have not been around Wall Street for ten years or so. At that time, stress testing was routinely used in rating bonds, ensuring that AAA-rated asset-backed securities and other constructs could withstand virtually any business environment. Sadly, stress testing has fallen into disuse as of late.

Those who bother to look beyond value at risk (VaR) – the much maligned risk measurement technique – know that the main weakness of VaR is that it only answers the question, “how much can I expect to lose with X% probability in N business days?” As described in recent press, therefore, VaR fails to analyze the question, “how bad can things get?”[1][1] Hence, it is strongly recommended (in properly taught risk management courses) that stress testing be used as a compliment to VaR.

But just like VaR, a number of practical issues complicate the application of stress testing. When imposing a stress test, the practitioner first selects a benchmark scenario. Back in the old days (1996), when Fitch used a Great Depression scenario as the benchmark for a AAA rated asset-backed security. According to Fitch, “Under the most severe depression scenarios, properly structured ‘AAA’ credit card asset-backed securities (ABS) should repay investors 100% of their original investment plus interest.” (Fitch, “ABCs of Credit card ABS,” April 1996 at 6.) Lesser ratings equated with weaker stress scenarios, i.e., an A rating looked to a 1980s recession scenario.

The benchmark scenario then has to be translated into some sort of portfolio-relevant benchmarks. Fitch’s AAA rating scenario for credit cards assumed that chargeoffs rose five times their average level, portfolio yield declined by thirty-five percent, and the monthly payment rate (a measure of the rate at which credit card borrowers pay extra above their minimum payment) declined by fifty percent – all at the same time. Only if a credit card asset-backed security could withstand that kind of stress could it be rated AAA (okay, those were the old days). An A rating required a three times chargeoff increase, twenty five percent drop in portfolio yield, and thirty-five percent lower payment rate.

The scenario, however, is also routinely tailored to the issuer or portfolio of interest. Fitch strengthened or weakened the benchmark depending on issuer and portfolio performance. For instance, the Fitch article cites the Household Affinity Master Trust as a deal in which the benchmarks were raised for various reasons of credit quality and contract risk, while Sears Credit Card Master Trust was held to lower benchmarks because of its seasoning and stability. Already, then, there is a great deal of subjectivity in stress testing, making it a poor choice for a policy instrument.[2][2]

But there’s more. Like VaR, stress testing seems simple when applied to a single security or small portfolio of securities. But when applied across an entire bank or a portfolio of complex securities, the many assumptions required to implement the analysis can substantially increase the confidence interval around the final result. Remember this is statistics, so there is no one “right” number at the end of the road but a probabilistic range of estimates that can each be correct with varying degrees of probability.

The same technique used to estimate market values of non-traded securities in VaR – risk mapping – will have to be used in bank stress tests just to carry out the exercise. Of course, the parameterization of the risk mapping will determine the validity of the result – garbage in, garbage out. The idea goes like this. Suppose you have known prices of actively traded six-month and one-year bonds. You wish instead, however, to estimate the value of a nine-month bond. Without going into excruciating detail (but read any good risk management textbook, if you wish), we can interpolate the risk of the nine-month bond from the six-month and one-year bonds and use that risk as the bond’s sensitivity for the stress test. Now imaging doing this millions of times in a bank portfolio with a large number of illiquid and untraded instruments (even in a sanguine economic environment) and it is easy to understand that what comes out at the end is less than the degree of accuracy than you might desire for any policymaking that makes a real difference.

But wait, did Treasury ever intend to make a difference with the suggested stress testing application? Treasury only said that they would run a stress test and then… what? It appears that the intent is to allocate capital to every bank failing (by whatever benchmark) the stress test. Hence, solvent banks are not rewarded for their good behavior and there is no minimum level of soundness required for capital – all failed banks qualify. Ostensibly, the worse off the bank is, the more capital they qualify for! The use of the stress test, therefore, seems to be merely a standardized (but not really standardized, as discussed above) justification for allocating TARP II capital almost the same way that TARP I capital was allocated before. Almost, except for the rational and desirable allocation of capital to sound banks to absorb the assets of the large insolvent banks that should be closed after failing to meet a minimum, rather than just a maximum, stress test standard.

Of course, the discussion above assumes we do not already have risk measures that should be used for the purpose at hand. Don’t we already have Basel I and II capital ratios, prompt corrective action cutoffs, CAMELS ratings, and more? Do we really need another regulatory measure? Why? If the ones we already have are not usable for this purpose, what is wrong with them? If you have not already read it, the Office of Inspector General, in their Material Loss Review of ANB Financial, National Association (November 25, 2008) already showed lax examination standards allowed ANB to aggregate substantial additional risky assets by the time they failed, leading to greater losses to the FDIC than would otherwise have occurred. The OIG recommended that OCC “Re-emphasize to examiners that examiners must closely investigate an institution’s circumstances and alter its supervisory plan if certain conditions exist as specified in OCC’s Examiner’s Guide to Problem Bank Identification, Rehabilitation, and Resolution.” (p. 27) and “Re-emphasize to examiners that formal enforcement action is presumed warranted when certain circumstances specified in OCC’s Enforcement Action Policy (PPM 5310-3) exist.” (p. 27) I am looking forward to many more OIG reports, but I cannot help but wonder whether the pie in the sky stress test idea is really just a way to perpetuate the folly in the meantime.


[1][1] Those of you who stay up on these things also know that VaR exceedances – the times that losses go beyond VaR – regularly go beyond the X% probability level, but that is a topic for another day.
[2][2] Not to mention the obvious difference in application: Fitch uses stress tests a priori to set capital structure limits, while Treasury is proposing to use such tests ex post to – well, nobody really knows the purpose.

2 Responses to "Stress Testing is Not Suited to Treasury Capital Policy"

  1. DW   February 17, 2009 at 2:43 pm

    The issue of ANB is an interesting one given that every exam, prior to the issuing of the formal agreement from federal regulators, was an A+. At what point is the FDIC culpable and if at all to what extent when a bank like this fails? Shame on the bank officers for their poor management, BUT shame on the FDIC for allowing and then eventually condemning this type of gross mismanagement without ever looking at themselves to notice that they in fact were complicit.

  2. M.G. in Progress   February 20, 2009 at 3:43 am

    It is clear that stress test in a market clearly made of lemon banks is just a nice we to buy time, wait and see. Exactley what markets do not need.See at http://mgiannini.blogspot.com/2009/02/lemon-brothers.htmland http://mgiannini.blogspot.com/2009/02/stress-test-and-testing-new-losw.html