Substack’s AI detector mostly works—but misses some text

Substack is adding an AI detector to posts, notes, comments and replies, giving readers a way to estimate how much of a piece was written or assisted by AI. The publishing platform is partnering with Pangram, whose scanning tool is available to all Substack subscribers for content published on or after Tuesday and longer than 100 words.

Creators can also scan drafts before publishing and add a statement explaining how they made the work. The feature is available now on the web and iOS; Android support will arrive later.

Substack co-founder and CEO Chris Best said the feature is intended to reduce what the company calls “Claudefishing”: readers consuming AI-generated content without realizing it.

“When readers have to wonder if what they’re reading is real, it undermines trust in authorship and threatens the livelihood of writers — including those who use AI tools thoughtfully to produce work they believe in,”

Pangram estimates that about 40% of text on some social media platforms is AI-generated, including roughly two-thirds of the content on LinkedIn. AI growth agency Graphite has also said AI now writes as many online articles as humans. A Gartner survey found that 68% of people question whether the content they see is real, while half of consumers prefer brands that do not use AI in marketing.

Recommended reading

Aronofsky’s AI studio raises $11 million

Testing Substack’s Pangram scanner

I tested the tool by publishing several posts. First, I pasted an article I had written without AI assistance into a new Substack post. After opening the live post, I selected “Scan for AI text” from the three-dot menu. Pangram correctly labeled the article “Fully Human-Written” and displayed the estimated shares of AI-generated, AI-assisted and human-written text.

×

Next, I inserted a long paragraph generated by ChatGPT into the same article. Pangram reported that 10% of the post was AI-written, which was roughly accurate. A fully AI-generated post was correctly identified as 100% AI-generated.

The detector struggled with a shorter mixed-content test, however. When I added two AI-generated sentences to an otherwise human-written post, it labeled the entire piece 100% human-generated. The results suggest the system can work well in broad cases but may miss small amounts of generated text.

×

How to use Pangram to detect AI content. Substack

False positives remain a concern

Best acknowledged that the detector is “not perfect,” while citing research that found Pangram 97.5% accurate at detecting fully generated AI text. The University of Maryland found it 99.3% accurate at identifying “humanized AI-generated text,” referring to AI content that a person has rewritten or paraphrased.

That still leaves a serious concern for writers who work without AI: being incorrectly labeled as AI-assisted. Substack said creators can scan drafts before publication, report incorrect results and have false scans removed.

Publishers can also add a “How I make this” statement to disclose how much AI, if any, they used. Best said Pangram cannot determine whether AI helped gather information or served as a source for a written piece.

“We’re not against people using AI to assist their work, and we think people should be free to choose which tools they use to express themselves. But people should know what they’re getting.”