Adapting to data limited uncertainty - an alternative approach to Mortgage and Consumer Lending credit loss and performance forecasting
There are times when historical data is not sufficiently predictive. Certainly, the pandemic is an example of a sudden change causing historic credit data to lack predictability. Also, future trends like Environmental, Social, and Governance ("ESG") are expected to have significant impact on lending volumes, credit risk, and profitability. Yet no one knows exactly how and when ESG will impact Mortgage and Consumer Lending businesses. Changes like these are reverberating through our economy with many unknown effects. Change is only accelerating. This uncertainty tests those responsible for forecasting the future. The uncertainty environment challenges those responsible for credit loss estimation and providing accurate inputs to the Allowance for Credit Loss (ACL), loan pricing, loan decisions, and a wide variety of financial services analytical needs.
This is Part 2 of a 3 part series:
(Please click the boxes for the other articles.)
So, what is to be done about it? If the behavioral predictiveness of our data is suspect and a new phase state is in its infancy or unknown, how does one predict credit losses? That is, how are credit losses predicted on a book of loans created in the old risk-known world but unexpectedly transitioned into a new uncertainty-affected world? Similarly, how is performance and volume predicted on our forward book of business?
The following suggests an approach to adjust thinking and methods:
Transition the thinking – stop pretending like the old models work. If the words “On Top Adjustment” are being used to adjust for uncertainty, they should be discouraged. This is hard for us credit quants, we have invested years of work to interpret, engineer, accumulate, and cleanse data to do our job. It feels like a good friend just died. There will be natural reluctance like a traumatic change. The imaginative thinking needed to manage uncertainty is an adjustment for those grounded in risk data.
Consider the old, risk world models for what they are – while the new uncertainty environment may be dramatically different, core human behavior resulting from many millennia of evolutionary biology is relatively constant. People are incredibly consistent, resulting from: a) our cognitive biases, like our emotional influences related to fear and greed, b) how our brains flood our synapses with neurotransmitters, and c) our unchanging logical fallacies. Thus, people are still people. Behavioral models may no longer provide accurate credit loss point estimates. However, in some cases, they still rank order risk, especially in models where independent variables specific to human behavior are utilized. (Credit Bureau data is a good example.)
Make bets on the new world – Now that the thinking is transitioning and the existing models are appropriately considered, it is time to implement a new forecasting method appropriate for the uncertainty world. This is uncomfortable for some that are accustomed to using deep data sets to build a unique perspective. These bets are structured with the assistance of Monte Carlo simulations (“sims”). That is, a new simulated world is modeled. The sim worlds are built on judgments of how people will respond in the new world, such as 1) what their buying behaviors are, 2) how unemployment impacts their ability to service current debt, 3) how the “reverse” wealth effect, as related to housing and security portfolio values, impact willingness to invest, 4) how long employment reallocation lasts and how it impacts different subsegments, 5) the length and degree of government support provided, etc. In effect, the simulation describes how the new world impacts the borrower's ability to pay their existing “pre change” originated loan. It also predicts how loan origination demand will adapt to the new world. It is possible some of the old models could be appropriately utilized as inputs to the sims.
Make multiple independent bets – In any situation with limited information, one would rarely make a single bet. The same is true in the new credit loss uncertainty world. Recognize, there are multiple sim views as directly related to the degree of uncertainty. To that end, multiple independent simulations should be created. Individual participants or small teams of credit-capable members should be assigned to the task. Please note: Having a single person or team performing the simulation is better than nothing. However, the accuracy of the forecast increases substantially as more credit-capable independent participants are engaged. Participant independence is important to drive a unique understanding of how the new world will play out. Also, multiple aggregated sample paths should be considered for each sim. That is, vary key assumptions based on outcome likelihoods. Each result should be considered, challenged, and ultimately utilized for a consensus. Frequent updating is important as the new world unfolds. As we learn more about the new world, expect the independent simulations to converge. (8)
Simulation approach for an uncertain world: Using environmental information and expert judgment:
In support of the simulation approach, there is a rich literature base suggesting “improper linear models” can be just as good as linear regression models, and certainly better than models suffering from overfit or a lack of accuracy resulting from uncertainty. Per Robyn Dawes and other researchers, a set of independent variables, likely correlated to the dependent variable, with judgmentally set coefficients will likely perform quite well.
"The immediate implication of Dawes’s work deserves to be widely known: you can make valid statistical predictions without prior data about the outcome that you are trying to predict. All you need is a collection of predictors that you can trust to be correlated with the outcome." (9)
The next section presents a simulation solution framework grounded in traditional lending decision techniques, known as the "Five Cs of Credit." These judgmental techniques were regularly utilized prior to the heavy use of credit decision modeling today. The framework will assist the simulation participants to develop, as Dawes' suggests, "a collection of predictors you can trust to be correlated with the outcome."
Please follow the links at the top of the article for additional sections.
For more information, please contact Definitive Business Solutions, Inc.:
John Sammarco, President | JSammarco@definitiveinc.com
Jeff Hulett, Executive Vice President | JHulett@definitiveinc.com
Notes:
(1) Roberta Wohlstetter, Pearl Harbor: Warning and Decision, June, 1962
(2) The 9/11 Commission Report at 9-11commission.gov, July 2004
(3) Wall Street and the Financial Crisis: Anatomy of a Financial Collapse, United States Senate Permanent Subcommittee on Investigations, April 2011
(4) Stephen Kinzer, The coronavirus pandemic is a failure of imagination, The Boston Globe, March 2020
(5) Bill Gates, Who Has Warned About Pandemics For Years, On The U.S. Response So Far, NPR, Heidi Glenn, April 2020.
(6) In 2005, Dr. Rajan said the financial system was at risk “of a catastrophic meltdown.”
(7) The subtle difference between risk and uncertainty is manifest when considering OTAs. Generally, making an OTA relates to potential qualitative events that may impact the risk-based expected loss, specific to its standard deviation. Uncertainty, on the other hand, is focused on tail risk, or kurtosis. Thicker tails may cause wildly different loss results, as per the Financial Crisis. Please see our article The numbers don't lie - anticipating risk, managing uncertainty, and making quality decisions for more information.
(8) In general, using independent participants grounded approach to creating simulations is known as the "common task method." Independent participants, utilizing common data sources compete to develop accurate outcomes. While this is not as common in credit loss estimation, it is common in computer science in the development of machine learning algorithms for standard tests or photograph interpretations. Other examples include (a) In 1980, Robert Axelrod, professor of political science at the University of Michigan, held a tournament of various strategies for the prisoner's dilemma. He invited a number of well-known game theorists to submit strategies to be run by computers. (spoiler alert: a very simple "Tit For Tat" strategy won). b) In 2020, after a long-term, longitudinal study about "Fragile Families" was completed, a competition of over 160 teams was held to find the most predictive algorithms to answer key questions concerning teenage children at risk. Interestingly enough, consistent with the game theory example, the simple models did quite well, almost as good as the more complex Machine Learning models. Given the danger of unrevealed overfit, one can argue simple is always better, especially when different algorithms have similar explanatory power. (To wit: Occam’s Razor)
(9) This quote is from the book Noise: A Flaw in Human Judgment by Daniel Kahneman, Olivier Sibony, Cass R. Sunstein.
(10) Troy Segal, The Five C's Of Credit, March, 2020
(11) Thomas L. Saaty (1982) Decision Making for Leaders: The Analytic Hierarchy Process for Decisions in a Complex World
(12) Definitive Business Solutions, Inc. designed and integrated a bespoke solver engine, utilizing a number of solver programing code sets and data transmission capabilities, uniquely needed for the AHP and cloud environment.
Комментарии