BOSTON–While many credit unions—and credit union workers—have either experimented with or are actually using AI such as ChatGPT to improve performance, a new study suggests it can actually hurt job performance.
According to new research from Boston Consulting Group, a group of researchers from BCG, Harvard, Wharton, and MIT conducted an experiment to see how access to AI impacts white-collar workers' productivity and quality of work.
“To test this, researchers randomly assigned 758 consultants at BCG across one of three groups: one with no access to AI; one with access to ChatGPT powered by GPT-4; and one with access to ChatGPT, as well as instructional videos and documents on prompt engineering strategies,” Business Insider reported. “After establishing performance baselines, consultants in each group were assigned one of two categories of tasks.”
18 Tasks & The Results
According to the report, one category included 18 tasks that exist "inside the frontier" of what the AI can do, like brainstorming innovative beverage concepts or coming up with a thorough business plan for a new footwear concept.
“The other category contained more open-ended tasks that exist ‘outside the frontier’ of AI's capabilities,” Business Insider reported. “While ‘consultants would excel’ at these tasks, ‘AI would struggle without extensive guidance,’ the study said. For example, consultants assigned this set of tasks were asked to offer recommendations to the CEO of a hypothetical company by using internal financial data and interviews with company insiders — information the AI didn't have access to.”
Business Insider reported researchers found stark differences in the results of the three groups, depending on their access to ChatGPT.
The Findings
The researcher found:
- For tasks "inside the frontier," consultants using AI were "significantly more productive" and "produced significantly higher quality results" than those who weren't using the chatbot.
- Consultants using AI to complete tasks "outside the frontier" were "19 percentage points less likely to produce correct solutions compared to those without AI." That's because the consultants with AI were found to indiscriminately listen to its output — even if the answers were wrong.
Business Insider noted the findings demonstrate AI's "uneven" capabilities.
Exercise Caution
“While the study's findings show that AI is ‘exceedingly good’ at helping humans with some tasks, humans should exercise caution when using the technology to avoid errors,” Saren Rajendran, one of the researchers involved in the study, told Business Insider in an email.
Just Because It’s Autumn Doesn’t Mean Falling for Schemes to Overcharge You for CU Industry News. Instead, Try This Crisp Offer
The biggest, best and freshest news reporting in credit unions remains free! Each morning CUToday.info delivers its daily Fresh Today news update offering the latest headlines and breaking news right to your email, with the easy-to-read headlines format allowing you to click on the stories that interest you most in order to learn more. So stop paying those bank-fee-like subscription prices from other so-called “news”” publications!
If you haven’t yet signed up for the new email solution on which CUToday.info has partnered with ResponseGenius, you can do so here. Signing up requires less than one minute of your time—and it’s free!
Please note that after signing up you may need to go to your Spam/Junk folder and mark the morning headlines email as safe. CUToday.info does not provide its list of readers and emails to outside parties, and we will not be contacting you to sell you an extended warranty or sending you any links so you may cash in on an inheritance you didn’t know was coming.
And did we mention it’s free?