98.7% confidence: One detector flags a 2019 book as AI-written

A single passage from a 2019 book about artificial intelligence recently triggered a 98.7% confidence score on an AI detection tool, despite being written entirely by a human author. The incident highlights a growing problem for writers who face automated screening on their work.

Sally McGraw, a nonfiction writer with more than 20 years of experience, recently had a writing sample flagged as machine-generated by multiple tools. She told NPR she was shocked because she had written the text herself without using AI. Although the client said the detection results were not the only reason for rejecting her, McGraw believes the flags cost her the opportunity to write a full book proposal. She was paid for her sample but considers the job a loss.

Editors are using these tools to manage a surge in submissions. Neil Clarke, publisher of the science fiction magazine Clarkesworld, said monthly submissions doubled from roughly 1,100 in 2021 to more than 2,000 by February 2023. The increase forced the magazine to temporarily close its submission portal. Clarke estimates his own workload has grown by about 30% and notes that the tools introduce mistrust into the editing process.

However, the accuracy of these tools is inconsistent. A study published in June in the International Journal for Educational Integrity tested 160 academic papers using four different detectors. The papers included human-written, AI-written, and mixed texts. The study found that different tools often reached different conclusions about the same piece of writing. Pangram was the most consistent detector in the study, correctly identifying AI use in 37 of 40 combined-text papers and 37 of 40 human-rewritten AI papers.

Max Spero, CEO of Pangram, acknowledged that the detector can make mistakes, particularly when evaluating short pieces of writing. Chris Callison-Burch, a computer science professor at the University of Pennsylvania, explained that detectors often measure how predictable a passage is. He noted that language models frequently identify text from their training data as unusually predictable, which can cause false positives on previously published work.

Janelle Shane, a science blogger, tested excerpts from her 2019 book You Look Like a Thing and I Love You. One detector reported the 98.7% confidence score on her text. Shane described the result as a glaring false positive. Callison-Burch said that even unpublished drafts can be flagged based on specific writing patterns, such as the use of em dashes, lists of three items, or opening sentences with adverbs like notably or crucially.

The Authors Guild warned in May that AI detectors create an environment reminiscent of witch hunts. The group stated that a false positive can cost an author their contract and reputation, while a false negative can cause readers to lose trust in the industry. Clarke said he does not personally use detectors to screen manuscripts but believes they can help manage volume if editors verify the findings. He emphasized that submissions should end in rejection, not accusation.

McGraw said the fear of future accusations has changed how she writes. She now avoids certain words and sentence structures and sometimes omits commas to prove her work is human. She miss using em dashes, a habit she learned in journalism school and used for decades. She described the pressure as trying to fool a machine into thinking she is not a machine.