LAS VEGAS–Even with one company sharing how its app uses data found on cellphones to make credit decisions, there is still the question of whether ChatGPT, which can write, can also underwrite?
That was question at the center of a session during the Money 20/20 meeting here, where panelists agreed the answer is no—at least for now.
The session, which began with an introduction written by ChatGPT that was then announced to the audience using a synthesized male voice, focused on what AI can and cannot do well, where investments are being made, and challenges that are causing insomnia.
Sharing their expertise during the discussion were:
- MacKenzie Sigalos, tech correspondent with CNBC, who moderated
- Maik Taro Wehmeyer, CEO and co-founder of Taktile
- Matt Mollison, chief data scientist with Branch International
- Seema Amble, partner, Andreessen Horwitz
- Sarah Hinkfuss, partner, Bain Capital Ventures
Here’s what they had to say:
Sigalos: What does ChatGPT offer that’s new vs. the machine learning that’s been around?
Wehmeyer: ChatGPT is the product from OpenAI. It allows you to create net new content. But AI and ChatGPT are not the same thing. ChatGPT is a subcategory of AI. AI has been used in underwriting for many years.
Sigalos: You don’t believe ChatGPT is made for underwriting?
Mollison: I am very skeptical. People think it’s going to solve problems. I don’t believe ChapGPT is made for real underwriting decisions.
Hinkfuss: There is traditional AI, which is often predictive, and there is generative AI. When people talk about ChatGPT that is just one form of generative AI. When it comes to generative AI, there is still relevance but it is not at the core of the underwriting itself. It is critical for the other pieces that need to be done.
There are four stages to underwriting…If you think about where generative AI is used, can’t be used in the decision itself, but it can be critical in the assessment itself.
Sigalos: How has historical writing traditionally been done?
Mollison: At Branch, we operate in emerging markets where there are no credit bureaus to interact with, so we have to find our own data. We have an app you can download, and by using some data from their phone…it allows us to make a decisions. We look for data to paint a picture of financial life. In some cases we connect to a bank account; other data sources tend to be a little more manual, like income verification or bill payments. We use alternative data services to try to make a responsible decision around riskiness.
Augmenting the Data
Sigalos: How can you augment data sets and how can ChatGPT enhance that?
Mollison: At Branch we are using ChatGPT to help improve our underwriting. But as an overview, ChatGPT is trained on lots and lots of unstructured data. It may not be the most appropriate. It’s biased. But it can analyze tons of unstructured data and help us to label data.
I tell our engineers to try to automate themselves out of a job so as to free themselves up to do other things. If we can label millions and millions of SMS messages, we can build our own kind of bespoke data. These SMS messages are sending all the time. It’s hard to be that agile.
Where Investments are Being Made
Sigalos: Can you share examples of the kinds of companies in which you are making investments?
Amble: A lot of what we’re talking about is around the underwriting process, but not the decision itself. Much of what we are seeing is around data labeling.
The customer interface is so important. (AI) can answer questions around why decisions were made. It’s not making a judgement call, it’s reading the data on a decision that has already been made.
Hinkfuss: Here are some other examples. In insurance and the underwriting of policies themselves, there is a lot of assessment of data. One company says 40% of its underwriters’ time is invested there. They want those people to instead spend time measuring risk. In the securities space, a company is able to leverage more interactive text-based forms to ask about public data. The underwriter can better understand less commonly studied attributes to understand how a stock price may trend over time. None of them is using ChatGPT to hit the go button,
Loss of Control
Amble: Why is that you can’t use ChatGPT for underwriting decisions?
Mollison: When you think about a language model and the kind of data it is trained on, there are two main things that push me away from ChatGPT: We’re not in control of the data it’s been trained on, and we’re not in control of the model itself. With these black box models, we’re not quite sure what is going on behind the scenes.
Amble: ChatGPT is really trained on giving you the sixth word after it has the first five. Loan approvals are thumbs up, thumbs down. This oversimplifies it, but the model is really trained to do that.
Hinkfuss: We see across a lot of our portfolio companies it’s not just the case today where a company is using just one of these models. The best companies are culling the different APIs for different steps in the process, including a human in the loop component. It doesn’t have to be OpenAI to solve everything; there are other models as well.
You can find other models for fine-tuning data in your ecosystems. We are seeing the evolution of domain specific models. Some of the largest FIs are starting to develop their own models and thinking about licensing it out as Data as a Service.
One is BloombergGPT, which leverages 40 years of their data. Another example is Intuit, which has the largest store of data on personal financial decisions. They are building an internal IntuitGPT.
Mollison: It’s amazing how we can fine tune. But you do have to remember at the end of the day they have learned from human data. When you have these credit risk decisions and insurance decisions, they are really important to people, and we have to be careful.
The models we are using at Branch are not yes or no, they are predicting the probability you are going to pay.
Amble: That probability is very different from what ChatGPT is.
What About Regulations?
Sigalos: What about regulations?
Amble: Black box underwriting has been a point of contention predating AI. I worked at the CFPB. All the CFPB’s regulations are set up to one, ensure the borrower has transparency over a decision, and two, an audit should reveal the (model) behind a decision. The CFPB and other regulatory bodies are very intent that consumers are protected as we start unrolling AI use-cases in lending.
The Insomnia Creators
Sigalos: What keeps you up at night? What are the headwinds?
Hinkfuss: Cost is one. It’s not cheap to use these models, particularly if you are just using OpenAI. Another is data security. When we talk about fine-tuning models, the reason that is valuable is you are leveraging proprietary data. (But a third party) may not guarantee security. A third is talent and resources. This is a whole new field.
Amble: Change management. I know it sounds like a big corporate word, but if you are a bank or established (institution) with established processes, it takes time to change all that out. There is inertia.
We talk to a lot of Fortune 500 companies and have never seen this much interest in something as how they can apply AI to their businesses. But it does require a cultural change to happen.
Just Because It’s Autumn Doesn’t Mean Falling for Schemes to Overcharge You for CU Industry News. Instead, Try This Crisp Offer
The biggest, best and freshest news reporting in credit unions remains free! Each morning CUToday.info delivers its daily Fresh Today news update offering the latest headlines and breaking news right to your email, with the easy-to-read headlines format allowing you to click on the stories that interest you most in order to learn more. So stop paying those bank-fee-like subscription prices from other so-called “news”” publications!
If you haven’t yet signed up for the new email solution on which CUToday.info has partnered with ResponseGenius, you can do so here. Signing up requires less than one minute of your time—and it’s free!
Please note that after signing up you may need to go to your Spam/Junk folder and mark the morning headlines email as safe. CUToday.info does not provide its list of readers and emails to outside parties, and we will not be contacting you to sell you an extended warranty or sending you any links so you may cash in on an inheritance you didn’t know was coming.
And did we mention it’s free?