A new study prepared by Columbia Journalism Review has indicated that AI search engines are wrong most of the time, and they give incorrect information and answers to their users, when asked about correct citation of news articles.
The study by Columbia Journalism Review was done on paid and free versions of the AI chatbots, such as ChatGPT search, Google’s Gemini, China’s DeepSeek search, Perplexity AI, Perplexity Pro, Grok, Grok-2 search, Grok-3 search, and Microsoft Copilot.
The study has highlighted how often the AI tools gave answers and how often they were correct or incorrect. The researchers, Klaudia Jazwinska and Aisvarya Chandrasekar, randomly chose 200 excerpts from different publications, and the researchers assured that each story they selected was within the top three results in a Google search.
Thereafter, they checked each AI search tool and graded the tools’ accuracy based on whether the AI tools had correctly cited the article, the news organization, and the URL of the source.
The study highlighted that, “Overall, the chatbots often failed to retrieve the correct articles. Collectively, they provided incorrect answers to more than 60 percent of queries. Across different platforms, the level of inaccuracy varied, with Perplexity answering 37 percent of the queries. incorrectly, while Grok 3 had a much higher error rate, answering 94 percent of the queries incorrectly.”
Further, the study also highlighted that paid versions of the AI tools are AI models like Perplexity Pro ($20/month) or Grok 3 ($40/month), providing the most incorrect answers compared to their free versions. It demonstrates that the belief that paid AI models provide accurate responses due to their higher cost and perceived superior computing capabilities is completely false.
And it also indicated that the generative AI tools that the researchers tested have cited wrong answers. Even if they were able to identify the correct articles, they failed to link the source.
But the chatbots were able to identify the content and give accurate responses to queries related to their partner publisher. For example, ChatGPT and Perplexity AI have a tie-up with the Times, and they provided 100% accurate responses to the queries related to the Times.
Also Read: Manus AI vs. DeepSeek: Comparing China’s Top AI Contenders
