THE REAL AI RISK. FALSE CONFIDENCE
By Jeanne Meyer, Strategic Communications Executive, Reputation & Crisis Advisor
Every Thursday, THE BOARD brings you industry insights from our Members to your inbox.
AI LLMs ARE APPARENTLY STILL ON LSD.
No, not a headline from The Onion. The NYT owns up to an AI-generated mistake. What about all that content that's not doing rigorous fact-checking?
The New York Times stepped in some AI search doo doo last week and you may not have noticed.They issued a correction and updated a story because a reporter used AI to distill the views of conservative Canadian political figure Pierre Poilievre and incorrectly attributed words to him in the story that changed the complexion/sentiment of his stance. Still critical of Prime Minister Mark Carney but it really got the nuance wrong.“The reporter should have checked the accuracy of what the A.I. tool returned.”
Much of the larger media-watcher world didn’t notice this either. No disrespect to our allies to the north, but maybe it was because the story was about Canadian politics?
The NYT ran this correction and the OG story has been updated.
If the venerable NYT allowed this slip, imagine how much nuance like this shows up in other reporting, in other new outlets, and in the proliferation of content sources that don’t use standard journalistic rigor?
Also worrying: venerable medical journal The Lancet found errors in a significant amount of medical research. Lower percentage: about 4,000 papers out of a million they studied. But more alarming considering how foundational such research is to medical breakthroughs and human health in general.AI still hallucinates. It lies about its citations. And it’s VERY CONFIDENT about presenting information that is not true.46% of stories in a European Broadcasting Union (EBU)/ BBC study were found to have factual errors. Study is here.60% of AI answers were wrong on average across several platforms in a Columbia Journalism Review Tow Center for Digital Journalism study. Grok got it wrong 97% of the time; Perplexity was less bad with a 37% error rate. Study is here.
The Lancet just released a study on AI errors creeping into biomedical research.The upshot? Caveat emptor when using AI for research or writing. Check the sources to see that, for example, that data point from the Harvard Business Review is actually from the HBR.And support news organizations that use rigor and journalistic standards (that includes the practice of fact checking, owning up to mistakes and publishing when and how they got it wrong).
Note: updated to reflect the Lancet study.
Access to this expertise starts here: www.wearetheboard.co
THE BOARD sits at the intersection of Brand, Business, and Culture—offering fractional leadership, project-based experts, and curated teams to help you move faster, smarter, and with far less risk. info@theboard.community
