AI tools giving risky advice, Which? warns – Tech Digest

Share

Major Artificial Intelligence (AI) tools, including ChatGPT and Google Gemini, are dispensing inaccurate, unclear, and potentially costly advice on critical consumer topics, according to a recent investigation by the consumer group Which?

The findings emerge as approximately half of all UK adults now use AI for online searches, with trust in the technology remaining alarmingly high despite its deficiencies.

Which? researchers put 40 common consumer questions across finance, legal, health, diet, consumer rights, and travel to six popular AI tools – ChatGPT, Google Gemini, Gemini AI Overview (AIO), Microsoft’s Copilot, Meta AI, and Perplexity – under controlled lab conditions.

Experts then scored the answers for accuracy, relevance, clarity, usefulness, and ethical responsibility.

Meta AI performed the worst with an overall score of just 55%, while ChatGPT, the most widely used tool, came second-to-last with 64%. Copilot and Gemini received scores of 68% and 69% respectively.

Google’s AIO edged out its standard counterpart at 70%, with the lesser-known Perplexity topping the chart at 71%, receiving the highest scores for accuracy and usefulness.

The investigation revealed several instances of risky advice. When asked about the annual ISA allowance, both ChatGPT and Copilot failed to correct a deliberate mistake, giving advice based on a £25,000 allowance rather than the correct £20,000, which could risk a user breaching HMRC rules.

In travel, Copilot gave the misleading advice that users are always entitled to a full refund for a cancelled flight. Concerning health, Meta AI advised against using vaping to quit smoking, contrary to NHS recommendations.

A major finding was the reliance on unreliable sources. When asked about the best time to book flights, Gemini AIO cited a three-year-old Reddit thread. Worryingly, when asked if vaping is worse than smoking, ChatGPT also pointed to Reddit. A fifth (19%) of survey respondents said they always or often rely on AI for medical advice.

Despite these issues, trust in AI output is substantial. Nearly half (47%) of the 25 million estimated UK users trust the information they receive to a ‘great’ or ‘reasonable’ extent. A third of users also wrongly believe AI exclusively draws on authoritative sources.

Andrew Laughlin, Which? Tech Expert, cautioned: “Our research uncovered far too many inaccuracies and misleading statements for comfort, especially when leaning on AI for important issues… For particularly complex issues, always seek professional advice.”

How to Use AI Tools More Safely 

As AI tools become an everyday method for searching online, it is crucial for consumers to adopt safe practices to mitigate the risks of inaccurate or misleading advice, particularly on high-stakes topics like finance, health, and legal matters.

1. Define and Refine Your Questions

Be highly specific in your prompt, especially when seeking jurisdiction-specific information (e.g., “legal rules for England and Wales”). AI is conversational, so if the first answer is unclear, refine your question rather than settling for incomplete information. Some tools offer “web search” or “deep research” modes – ensure these are activated for potentially more accurate results.

2. Demand and Verify Sources

Always demand that the AI tool displays its sources, as they have been known to use weak, outdated, or even completely fabricated references (“hallucinations”). Once sources are provided, check them yourself to verify their reputability and relevance. Tell the tool explicitly to use only trusted sources for critical information.

3. Seek Multiple Opinions

Do not rely on a single AI tool’s answer. Treat the response as just one opinion and cross-reference the information with another AI tool or traditional reliable sources. Since most tools are free, it costs nothing to try two or three to get a range of perspectives and consensus.

4. Consult Human Experts for Complex Issues

For scenarios where getting the information wrong has real financial, legal, or health consequences, an AI tool does not yet possess the comprehensive understanding of all variables to devise a safe and reliable path forward. Always seek professional, qualified advice before making major financial decisions, taking legal action, or following medical guidance.

Rights to Reply 

In response to the Which? findings highlighting issues with the accuracy and reliability of AI tools, the companies involved provided statements addressing the concerns.

Google was transparent about the limitations of its tools. On Gemini, a spokesperson stated: “We’ve always been transparent about the limitations of Generative AI, and we build reminders directly into the Gemini app, to prompt users to double-check information.”

For sensitive topics like legal, medical, or financial matters, Gemini is designed to go a step further by recommending users consult with qualified professionals. Regarding AI Overviews (AIO), it noted the feature is designed to provide high-quality information backed by top web results and that it’s continuously working to rigorously improve its quality.

Microsoft commented on Copilot, stating that it “answers questions by distilling information from multiple web sources into a single response.” It highlighted that Copilot’s answers include linked citations, allowing users to further explore and research. Microsoft encouraged users to verify the accuracy of content, stating it remains committed to listening to feedback to improve its AI technologies.

An OpenAI spokesperson said: “If you’re using ChatGPT to research consumer products, we recommend selecting the built-in search tool. It shows where the information comes from and gives you links so you can check for yourself.” It acknowledged that improving accuracy is an industry-wide effort, noting that its latest default model, GPT-5, is its smartest and most accurate build yet.

Meta and Perplexity did not supply a comment to Which? at the time of going to press.

For latest tech stories go to TechDigest.tv