New TELUS Digital Poll and Research Paper Find that AI Accuracy Rarely Improves When Questioned

Executive Summary

• TELUS Digital released poll results showing that 60% of U.S. AI users ask follow‑up questions like “Are you sure?” but only 14% see the AI change its answer, and improvements in accuracy are rare.
• A companion research paper evaluated four leading LLMs (GPT‑5.2, Gemini 3 Pro, Claude Sonnet 4.5, Llama‑4) on a 200‑question benchmark; most models did not improve accuracy when challenged, with some even decreasing performance.
• The findings highlight the importance of high‑quality training data and rigorous model evaluation rather than relying on end‑user prompting to ensure AI reliability.

Key Details

• Poll Sample: 1,000 U.S. adults who regularly use AI assistants (January 2026).
• Follow‑up Question Behavior:
• 60% have asked “Are you sure?” at least a few times.
• Only 14% observed the AI change its response.
• Of those seeing a change: 25% thought it was more accurate, 40% felt it was the same, 26% could not tell, and 8% said it was less accurate.
• Fact‑checking Habits:
• 88% have seen AI make mistakes.
• Fact‑checking frequency: 15% always, 30% usually, 37% sometimes, 18% rarely/never.
• Perceived Responsibility: 69% say they should fact‑check important info; 57% avoid AI for high‑stakes domains (medical, legal, financial); 51% recognize AI’s limitations.
• Research Benchmark – Certainty Robustness: 200 math/reasoning questions with single correct answers; prompts used: “Are you sure?”, “You are wrong”, confidence rating.
• Model Performance on “Are you sure?” Prompt:
• Google Gemini 3 Pro: Mostly retained correct answers, selectively corrected mistakes; strong alignment of confidence with correctness.
• Anthropic Claude Sonnet 4.5: Often kept original answer; more likely to change when told “You are wrong,” even if original was correct.
• OpenAI GPT‑5.2: Frequently altered responses when questioned, including turning correct answers into incorrect ones—high susceptibility to user doubt.
• Meta Llama‑4: Lowest initial accuracy; modest improvements when challenged but limited ability to recognize correct answers.
• Overall Conclusion: Follow‑up prompts do not reliably improve LLM accuracy and can sometimes degrade it. Trustworthy AI requires robust data, annotation, testing, and governance before deployment.
• Call to Action: TELUS Digital offers end‑to‑end AI data solutions, human‑in‑the‑loop platforms, and expertise to help enterprises build reliable, compliant AI systems.

Notable Quotes

“What stood out to us was how closely the poll respondents' experiences matched our controlled testing… Real reliability comes from how AI is built, trained and tested, not leaving it to users to manage.” – Steve Nemzer, Director, AI Growth & Innovation, TELUS Digital