Research Showed How Often AI Makes Mistakes in Financial Advice

Published: September 20, 2026 – 09:36 UTC

Saturn tested ChatGPT, Claude, Copilot, Grok, and Gemini on financial questions.
For complex queries, 88% of the answers were incorrect.
Chatbots often give incorrect answers about taxes and pensions.

Research by the tech company Saturn found that popular chatbots often give incorrect answers to financial questions, FT reports. In tests of the AI models ChatGPT, Claude, Copilot, Grok, and Gemini, they were wrong an average of 57% of the time, and for more complex queries involving multiple calculations — 88%.

The findings highlight the risks of using generative AI for tax, pension, and investment decisions, where a mistake can lead to real financial losses.

AI Was Wrong in 88% of Complex Financial Queries

Researchers tested more than 100 different money-related questions using free and paid versions of the models. In total, 18 models received more than 10,000 prompts, and each question could be repeated up to five times.

Issues identified included:

Errors in financial calculations
Ignoring future changes to tax legislation
Made-up rules
Incorrect answers to complex queries involving multiple calculations

For some models, the error rate on complex questions reached 99%. The best result came from Claude Opus 5 in “reasoning” mode, but even that model was wrong in 39% of its answers.

Amal Jolly, Saturn CEO, said that the scale of AI use for financial advice creates additional risks.

“Millions of people are trusting the AI models for money advice, but they are getting wrong answers that can lose them money,” he said.

AI Mistakes Could Cost Thousands of Pounds

One example was a query to the free Claude Haiku 4.5 model about pension taxation. According to the study, its mistake could have resulted in a HMRC charge of £17,500 (about $23,430) for a pension saver.

In another case, Claude “made up a rule” about student loans and claimed that a graduate can stop making payments if they move abroad.

Paid models were generally more accurate than free ones, and newer versions delivered better results. At the same time, Sarah Coles, head of personal finance at investment platform AJ Bell, noted that AI can still be useful for budgeting and finding information.

“It’s a good example of why people who need support need proper advice rather than trusting AI,” she said.

The results also align with a recent study by the UK Financial Conduct Authority (FCA), which found that one in five adults in the UK is willing to let AI make financial decisions on their behalf. The strongest interest is seen in complex areas, including debt, pensions, and investments.

According to Jolly, AI financial advice is currently unregulated, so users do not have the same level of protection and compensation as they would when turning to a human professional.

As a reminder, OpenAI recently launched ChatGPT for the financial sector.

Сообщение Research Showed How Often AI Makes Mistakes in Financial Advice появились сначала на INCRYPTED.

Source: Incrypted