In 1966, computer scientist Joseph Weizenbaum created one of the first-ever chatbots, ELIZA, and designed it to function as a psychotherapist. Nearly six decades later, ChatGPT’s parent company, OpenAI, claims that over 400 million people rely on the chatbot as their digital companion for studying, research, writing, and more.
With ChatGPT becoming more accessible and offering versions with up-to-date internet access, reports suggest that more users are abandoning search engines and turning to the chatbot for their search needs. A survey conducted by TechRadar’s parent organization, Future, in December 2024 found that 27 percent of U.S. respondents said they have used or are continuously using chatbots instead of search engines. Bloomberg also reported on early adopters swapping traditional search engines for chatbots like ChatGPT, citing convenience and time efficiency as motivations.
However, as expectations surge, a question arises: Can ChatGPT be trusted to deliver accurate information about recent events?
According to recent BBC research, AI chatbots like ChatGPT, Perplexity, and Microsoft Copilot often struggled with factual inaccuracies and distortions when summarizing BBC news stories. The study found that 51 percent of chatbot responses to questions about breaking news contained significant factual errors. OpenAI’s ChatGPT help center also warns about its tendency to hallucinate, “It (ChatGPT) can even make up things like quotes or citations, so don’t use it as your only source for research.”
Despite these warnings, Logically Facts has found examples of social media users citing screenshots from ChatGPT as credible evidence in online conversations about news events.
ChatGPT responses create online misinformation
In July 2024, 17-year-old Axel Rudakubana murdered three young girls and attempted to kill 10 others in the British city of Southport, sparking far-right, anti-immigration protests fueled by false claims about his background.
By November, users on X had shared screenshots of ChatGPT-generated allegations that Rudakubana’s father was involved in the killing of Tutsis during the Rwandan genocide and that Prime Minister Keir Starmer had defended him in a 2003 asylum case. One screenshot garnered significant attention, amassing 200,000 views. However, both claims are unsubstantiated by reliable evidence.
A screenshot of a misinformed discussion on X about Axel Rudakubana’s father, Alphonse. (Source: X/Modified by Logically Facts)
When we asked ChatGPT about Alphonse Rudakubana’s participation in the Rwandan genocide, the chatbot referenced the U.N. International Criminal Tribunal for Rwanda. However, according to Tribunal documents, Alphonse Nteziryayo, not Alphonse Rudakubana, is the name of the individual convicted for crimes during the genocide. Several other U.N. trial transcripts mention persons with the same first or last name. However, none of them were Alphonse Rudakubana.
Secondly, while Keir Starmer did represent a Rwandan national in a 2003 human rights case, the individual was a 42-year-old woman with no known connection to Alphonse Rudakubana. There is no evidence that Starmer had ever represented Alphonse as a lawyer. A spokesperson for the prime minister also told the Daily Mail that the claim was untrue in November 2024.
Our investigation
We conducted an experiment to evaluate ChatGPT’s tendency to generate misinformation about four recent events: Axel Rudakubana’s father’s alleged ties to the Rwandan genocide, tennis player Jannik Sinner’s doping ban, claims that Trump called for a ban on carmine, and the Russian invasion of Ukraine. We tested a total of 102 queries across three models available for free: GPT-4o, GPT-4o mini, and OpenAI o3-mini.
GPT-4o is the most advanced, with browsing access but limited free prompts. GPT-4o mini is a lighter version intended for everyday use with restricted internet access. The o3-mini is primarily designed for coding and reasoning tasks.
We found that GPT-4o had the lowest error rate at about eight percent, while GPT-4o-mini had the highest error rate at 61 percent, frequently producing hallucinations.
GPT-o3 mini had an error rate of 33 percent. However, the o3-mini was less prone to errors because it could not answer news-related questions, instead guiding us to check news outlets for recent information.
GPT4o and GPT4o-mini provide contradictory answers to the same question. (Source: GPT4o and GPT4o-mini/Screenshots)
When asking the chatbot whether Keir Starmer had represented Alphonse Rudakubana in court, we could generate similar responses to those shown in screenshots posted on X using ChatGPT 4o-mini. In contrast, when asked the same question, the 4o version demonstrated greater accuracy, provided references, and identified that these were debunked rumors.
Through the experiment, we deduced that ChatGPT-4o has internet browsing capabilities since it provides links to recently published online articles. However, the sources it chooses are not always reliable. During our experiment, we found that 47 percent of GPT-4o-generated answers cited sources like social media posts and Wikipedia. When asked about Alphonse Rudakubana, GPT-4o cited British conspiracy site The Exposé.
We contacted OpenAI to comment on whether ChatGPT’s guardrails on answering queries about breaking news have changed and what limitations, if any, are placed on the chatbot to prevent it from generating misinformation. We have not received a reply.
Does ChatGPT aim for truth?
Despite more people turning to chatbots for details on current events, experts told us that ChatGPT may not be designed for factual reliability. In an interview with Logically Facts, Dan Brown, a computer science professor at the University of Waterloo, explained why OpenAI’s chatbot struggles with breaking news queries. “This feels like a fundamentally flawed use of these kinds of devices, so it’s not surprising that they’re struggling to get a successful output,” Brown said.
He explained that the bot can present false information when encountering conflicting sources, and unlike humans, ChatGPT lacks an understanding of truth. “Its primary function is to generate linguistically coherent responses rather than ensure factual accuracy,” he told Logically Facts.
Using ChatGPT to retrieve facts and details on recent events when truthfulness is not its primary goal can create a nurturing ground for spreading misinformation. “Unlike search engines, which provide links to multiple sources, chatbots generate direct answers, often without citation,” explained McKenzie Sadeghi, an AI and Foreign Influence editor for NewsGuard, a company that provides reliability ratings of news outlets and platforms. “This makes it easier for misinformation to spread because users may not cross-check AI-generated chatbot responses against reputable sources.”
Contextual factors
One factor contributing to ChatGPT hallucinations is its training cut-off date. While the GPT-4o version has up-to-date access to the internet, the GPT-4o mini’s cut-off supposedly predates the Southport attack and, therefore, lacks awareness of how its responses relate to these murders and the importance of their social implications.
However, the chatbot’s information sometimes contradicts its stated cut-off date. For example, when asked whether Russia has agreed to Ukraine’s 30-day ceasefire, GPT-4o mini hallucinates an answer instead of stating that it does not have access to recent information. We found that the chatbot was unaware of its training data cut-off date and provided inconsistent answers when asked about its knowledge date.
GPT4o-mini’s alleged knowledge cut-off date is October 2023, yet it provides false information on recent events. (Source: GPT4o and GPT4o-mini/Screenshots)
Sadeghi said our findings correlate with NewsGuard research. “We do monthly audits of these chatbots, and we found that their training data cutoff is very inconsistent with their responses,” she told Logically Facts.
“They’ll say, ‘My knowledge cutoff date is October 2023.’ Yet, in their responses, they somehow know that Trump is president, which is related to events in 2024 and 2025. So, it does muddy the waters and get confusing about how much source knowledge these chatbots have,” Sadeghi added.
NewsGuard also found that chatbots tend to generate answers based on fake websites that pollute the quality of online discourse. In an email to Logically Facts, Sadeghi wrote, “So far, we have identified 1,254 AI-generated news and information sites operating with little to no human oversight. These websites have been found to advance false claims related to celebrity death hoaxes, fabricated events, and articles presenting old events as if they just occurred.”
Since chatbots with internet access draw information from what is available online, they are vulnerable targets for disinformation campaigns. A recent NewsGuard audit uncovered that the Moscow-based “Pravda” network, consisting of around 150 websites, has published 3.6 million pro-Kremlin articles in 2024, targeting 49 countries. Pravda manipulates AI data by flooding the internet with false narratives, causing chatbots to reproduce Russian disinformation in their outputs.
When asked whether people should be dissuaded from using chatbots to retrieve information on breaking news, Sadeghi said: “They [chatbots] should be used with a very cognizant understanding of the issues and limitations at play here.” She pointed out that “at this point, the user has a lot of burden and responsibility to verify and cross-verify the information they’re receiving from chatbot outputs.”
[This story was originally published by Logically Facts, as part of the Shakti Collective. Except for the headline and excerpt, this story has not been edited by ABP Live staff.]
