If you’re finding it harder to get your content past the compliance department and trade media editors, it’s not your imagination. These eagle-eyed reviewers are increasingly under pressure to verify facts and matters of attribution before going live. They know content shapers are increasingly relying on AI to assist them. They know AI tools often rely on outdated sources or misidentify sources or sometimes simply make them up. (More on than in a minute) So, they’re scrutinizing sources like never before. Don’t make them doubt you.
Example
Here’s a paragraph from a book manuscript a prospect sent to us for review:
“If you write something down, there’s a 42% chance you’re going to accomplish what you said you’re going to do. You must always know what you’re going to do today, tomorrow, next week and at the end of the month. It’s not just about committing to what you’re going to do today. For instance: ‘I must deliver three tax returns by the end of the week. Period.’
Only 16% of people write down their goals. Those who write down the goals are 42% more likely to achieve them.”
The stats he cited sounded very reasonable. But when we asked where he got them, he hesitated before saying: “I forget. I read it somewhere.” That’s not good enough. Then he admitted he used AI to help him draft the manuscript “and it always gives me sources,” he assured us. So when we asked to see the sources AI came up with he gave us a link to this blog post on LinkedIn. Yes, the post mentioned the 42% statistic, but it didn’t cite where it came from.
Two wrongs don’t make a right.
A week later, the author somewhat sheepishly came back to us and said the missing source was CNBC because he heard it on a news report they did. To their credit the CNBC correctly cited Dr. Gail Matthews, a psychology professor at Dominican University of California, as the source of the 42% statistic. But alas the CNBC link was broken and it pointed to a general bio page for Dr. Matthews, not the source of her research containing the 42% statistic. We run into second-hand bias like this often from clients who say they read something in The Wall Street Journal or Financial Times or Bloomberg. But a news report in well-respected mainstream media is typically not the source — it’s just a conduit.
If you’re curious about Dr. Matthews’ research, the 42% statistic came from a paper she presented at the 9th Annual International Conference of the Psychology Research Unit of Athens Institute for Education and Research (ATINER), Athens, Greece. That’s your source.
Be wary of citing sources from AI
No one likes to have their hard work and research being ripped off without attribution. Academic research conducted by Middle Tennessee State University, Arizona State University and Duke University among others have found that ChatGPT and other AI tools will cite articles by an author who usually writes about your topic. AI might even identify a journal that published on your topic, but the title, pages numbers, and dates are completely fictional, according to the aforementioned academic studies. This is because ChatGPT and other AI tools are not connected to web search, so they have no way of identifying actual sources.
Since ChatGPT is based on a Large Language Model researchers found it does not have the ability to match relevant sources to any given topic. “It may do OK with some topics or sources, but it may also fabricate sources that don’t exist,” noted the Duke study. “Depending on the topic and availability of data it has on that topic, it may summarize the wrong source or provide inaccurate summaries of specific articles — sometimes making up details and conclusions,” Duke added.
Conclusion
We’re all super-busy these days. That doesn’t give us the right to be lazy when it comes to citing sources for your stats, facts and attribution. So do the work or hire someone to do it the right way for you. You’re an Elite Professional; Don’t Act Like a Jamoke.
What are you and your colleagues doing to improve your fact checking and research capabilities? I’d love to hear from you and why.
#businesscommunication, #practicemanagement, #factcheck
