Four major artificial intelligence (AI) chatbots are inaccurately summarizing news stories, according to research carried out by the BBC. The BBC gave OpenAI’s ChatGPT, Microsoft’s Copilot, Google’s Gemini and Perplexity AI content from the BBC website and then asked them questions about the news. The resulting answers contained “significant inaccuracies” and distortions.
The BBC gave the chatbots 100 news stories and asked them to summarize each one. Journalists who were experts in the subject of the article then rated the quality of the AI’s answers. The results showed that 51% of the AI’s answers had significant issues, and 19% contained factual errors, such as incorrect factual statements, numbers, and dates.
Some examples of inaccuracies included Gemini incorrectly saying the NHS did not recommend vaping as an aid to quit smoking, ChatGPT and Copilot saying Rishi Sunak and Nicola Sturgeon were still in office even after they had left, and Perplexity misquoting BBC News in a story about the Middle East.
The study suggested that the chatbots struggled to differentiate between opinion and fact, editorialized, and often failed to include essential context. The BBC is calling on the tech companies to “pull back” their AI news summaries, and to show how they process news and the scale and scope of errors and inaccuracies they produce.
Source link
