The increasing prevalence of AI-generated content is making it harder to maintain the integrity of scientific publishing and informational resources. While the concept of AI "hallucinations" emerged with models like GPT-3, even more advanced systems today, such as Claude and Gemini, require verification of the information they produce. Benjamin Skuse, a doctor of applied mathematics, demonstrated this by simply instructing an AI to conduct research and write a paper, revealing that such effortless content could bypass the defenses of publishers like IEEE.
Currently, the human review processes and research-integrity checker programs of publishers like IEEE are keeping pace in a cat-and-mouse game against content generated by free AI tools. However, more advanced large language models are no longer making the simple mistakes seen in free versions. This situation locks publishers into an arms race between increasingly plausible AI-generated content and the systems designed to detect it.
This challenge affects not only commercial publishers but also Wikipedia, a global repository of human knowledge. According to Ellia Lebleu, a founding member of the volunteer effort WikiProject AI Cleanup, Wikipedia is facing the same struggle as commercial publishers but is equipped with fewer people and tools to detect AI material.
The proliferation of false content can create uncertainty about the accuracy of scientific records or basic facts, ultimately undermining trust in institutions. Experts note that in a future where AI conducts science and writes papers for other AI systems, humans could find themselves completely left out of the loop.