It is no secret that AI improves accuracy of credit decision models, however lenders looking to switch to AI from traditional scorecard development or human decided rule-based credit decision modelling often bump up against the tough questions:
How can I trust an AI credit decision model?
And how do I know why I am getting the results I am getting?
It is a great question since a lot of AI is a black-box decision making, and it is impossible to understand how all variables interacts and correlates with each other. For many applications, this is not a concern, as for example if you are doing ad campaigns or understanding if it is a cat or not in a picture. In these applications, you just need to know that it works but not how it works.
However, for regulated use cases such as credit decision, the government requires explanation of certain decisions, such as the risk your organization takes and ensuring that no bias against people for reasons such as ethnicity or gender exist. And regardless of regulatory requirements, it is dangerous to operate a lending business if you don´t know how decisions are being made. That is why Evispot developed real explainability into or AI-credit decision platform so lenders can operate AI models safely and profitably.
What does real transparency mean?
Being able to understand exactly how each variable that goes into the model operates and interact with each other, regardless if the model uses thousands or dozens of variables. Real explainability is extremely important during model development to figure out, which variables that are most important and how each variable influence the prediction.
The table below was generated by our own software Evispot Traits and shows the impact of an applicant’s traditional credit score (a higher score means a lower likelihood of default). The interesting column is called Evispot score and varies between -0.5 to +0.5 and represent the impact of the variable interval. A dot skewed towards the left – a negative value – increases the probability of default. While a dot skewed to the right – a positive value – decreases the probability of default.
From the above table, a credit analyst can see the true impact of the risk score variable. For example, risk score above 6 means that the variable has a positive impact of the final score.
However, from the last row in column Evispot score (see column Evispot score and column interval is 7-10) it can be read that risk score above 7 has very little improvement of the final score, since regardless if an applicant has 7 or 10 in risk score he or she will receive a positive value close to ~0.2. This type of insight is something we very often see when we do modelling.
Another interesting insight can be read from the second row of Evispot score (risk score interval between 4-5). The length of the line tells us that only using this variable will be hard to understand if the applicant is a good or bad borrower. Instead, the model has learnt to use other variables to differentiate the good and bad applicant´s with risk score 4 or 5. A lender that uses traditional model would probably lose a lot of potential customer if treating all 4:s and 5:s the same.
Risk Management process
Another reason lenders need total transparency into their AI models? The world changes constantly and so do market forces and customer habits. Risk factors that today may be important might not be important in a few months. To ensure that a credit model performs as expected in live production, a lender must have a reliable monitoring tool.
The graph below was generated by our Evispot Trait software and here we can see that since October 2017 a lot of people above 38 started to apply for a loan. This could have a huge impact if not noticed, leading to rejecting good loans and accepting bad loans. Being able to catch these changes are critical, especially if you use AI models which, uses hundreds or even thousands of variables these small changes could have huge impact. By using Evispot Traits we help you catch these changes real-time, helping you to always have the best possible credit decision model that exist.
If you’re ready to capture the benefits AI can give your credit risk models, let’s have a chat! Contact firstname.lastname@example.org to learn more.