In more than twenty years of bank risk management, I have never been hurt by a model that failed loudly. Loud failures are merciful: the number is obviously wrong, everyone stops, there is a post-mortem, someone is reassigned. The models that did the damage failed quietly — returning smooth, plausible numbers, month after month, while the world underneath them had already changed. [author assertion]
A quiet failure has a particular cruelty: it trains trust. A model that is slightly wrong every day does not trigger scrutiny; it accumulates a track record. Committees learn to wave its output through. New staff learn that the number is simply what the number is. By the time the model is badly wrong, the institution has organised itself around believing it — limits, bonuses, capital plans, all downstream of a figure nobody has genuinely questioned in years. [author assertion]
Validation, as usually practised, does not catch this. Validators test a model against history, and history is precisely the wrong witness: the risk lives in the regime change that the sample does not contain. A model can pass every back-test ever devised and still be a machine for being surprised, because back-tests measure how well it describes the world that produced the data — the one world we know it will not meet again. [author assertion]
Correlation is the classic quiet failure, and I have watched it from inside. Portfolios that are diversified on paper — different names, different sectors, different regions — concentrate, in the one state of the world where diversification is actually needed, into a single exposure. The spreadsheet says the risks are independent. The spreadsheet was calibrated in fair weather, and independence is fair weather's most generous assumption. [author assertion]
Underneath all of this is a governance fact that risk managers are reluctant to say aloud: a model is an opinion with an org chart. Challenging the model means challenging the people whose budgets, headcount and status the model justifies. So the challenges that do occur are technical — a parameter here, a window there — because technical challenges are safe. The challenge that matters, whether the whole framing still fits the world, has no natural owner. [author assertion]
I no longer ask whether a model is right. I have never once seen that question answered in a way that survived contact with the following year. I ask instead: what would we see, and how soon would we see it, if this model stopped being right? A risk function should be built around tripwires rather than verdicts — observable signs, agreed in advance, that the quiet failure may have begun. [author assertion]
Model risk, in the end, is not a modelling problem. It is an attention problem. The number on the page is the least dangerous part of any model; the exposure is the confidence arranged around it. Institutions that understand this hold their numbers the way sailors hold a forecast — gratefully, and with a hand on the rail. [author assertion]
