Technology Reporter
Getty ImagesAccording to a study conducted by the BBC, four major artificial intelligence (AI) chatbots inaccurately summarise news articles.
The BBC asked the BBC website Openai’s ChatGpt, Microsoft’s Copilot, Google’s Gemini and Prperxity AI content about the news.
He said the results contained “significant inaccuracy” and distortions.
On the blog, Deborah Turnness, CEO of BBC News and Current Affairs, said that AI has brought “endless opportunities,” but the companies developing the tools “played with Fire.”
“We live in a troubled time. How long does it take for an AI-set headline to cause major real-world harm?” she asked.
Tech companies that own chatbots are approached for comment.
“Look back”
In this study, the BBC summarized 100 news articles to ChatGpt, Copilot, Gemini and Prperxity, and evaluated each of the answers.
To assess the quality of responses from AI assistants, there were journalists who were experts related to the subject of the article.
It turns out that 51% of all AI responses to news questions were determined to have some form of significant problem.
Additionally, 19% of AI responses cited BBC content introduced de facto errors, such as false factual statements, numbers, and dates.
On her blog, Turnness said that the BBC is looking to “open new conversations with AI technology providers,” and therefore “can work together in partnerships to find solutions.”
She called on tech companies to “pull back” AI news summary, as Apple did after a complaint from the BBC that Apple Intelligence misrepresents news stories.
Some examples of inaccuracies discovered by the BBC include:
Gemini incorrectly states that the NHS does not recommend vaping as aid to stop smoking, and Copilot after Rishi Snack and Nicola Sturgeon misquoted false quotes in stories about the Middle East But even after Iran said it had shown “control” he still remains in office. Described Israel’s actions as “aggressive”
Generally, Microsoft’s Copilot and Google’s Gemini had issues that were more important than Openai’s ChatGpt and confusion, counting Jeff Bezos as one of the investors.
Normally, the BBC blocks content from AI chatbots, but in December 2024 we opened a website over the period of testing.
In addition to including factual inaccuracy, the report stated that the chatbot “had a hard time distinguishing between edited opinions and facts, and often failed to include essential context.” .
Pete Archer, program director at the BBC at Generative AI, said publishers need to control whether or not the content is used, and AI companies can make news by using it. “We need to show whether we’re going to handle it and generate a scale and range of errors and inaccuracies.”
