How to Audit Content You Didn't Write

The Hanover Institute for Public Policy published more than a hundred research reports, complete with footnotes, tables of contents, and the flat neutral register that policy writing has.

It does not exist.

Responsible Statecraft traced it to Piro, Inc., and Politico found the Department of Justice filing showing $900,000 of Israeli government funding behind it. GPTZero flagged eleven of twelve sampled articles as machine-written.

The interesting part is who the reports were written for. Piro’s founder said it on LinkedIn: “When someone asks ChatGPT, Gemini, or Perplexity about your category, an answer comes back in one confident paragraph… we spent months reverse-engineering it.”

Not readers. The machine that answers readers.

Your blog runs on the same economics #

Nobody spent $900,000 on your content.

That is the point. They did not have to, and neither did whoever produced yours, because manufacturing text that reads like expertise stopped being expensive somewhere around the middle of 2023.

Ask what your blog archive actually is. An agency wrote some of it, on a monthly retainer measured in posts, and a freelancer wrote more.

Then a tool started drafting, and the person approving its output was not equipped to check the technical claims inside it. Nobody in that chain was lying. Each link did the job it was paid for, and the job was volume.

Nobody can tell you how much of the web this is #

You would think there is a number. There are several and they disagree.

Graphite sampled 43,000 CommonCrawl URLs published between January 2020 and May 2025, ran them through Surfer’s detector, and put the crossover - more machine-written articles than human ones - in November 2024.

Pew ran roughly 490,000 English-language pages from the same archive through Open Pangram this month and found 10% of all sampled pages showing significant signs of AI authorship - rising to over a third when you look only at pages published after ChatGPT shipped.

Ten percent, a third, or half. The numbers are not contradicting each other so much as answering different questions, on different samples, with different detectors.

Both teams publish their error bars, which is the part worth copying. Graphite reports a 4.2% false-positive rate against GPT-4o content and no measurement at all of AI-assisted-then-human-edited writing. Pew says plainly that “AI detection models aren’t perfect” on individual documents and only hold up in aggregate.

One more number from Pew is the one that should interest you: around one in ten .com pages carry those signals - roughly double the .org rate and ten times what .edu and .gov show. Commercial sites are where this concentrates, and yours is a commercial site.

So the honest position is that the web’s average is unknown and the detectors that estimate it are themselves approximate. Which is fine, because the average was never the thing you needed. You need to know about your property, and your property is countable.

Four checks, none of which require you to read code #

We ran these on our own archive. They are ordered by what they cost you to skip.

1. Rank by who reads it, not by how bad it looks.

Start with your search console, sort pages by impressions, and work down.

We got this wrong first. The worst-sounding claim we found sat on a page flagged featured in the site config, which felt urgent, and it turned out to have four impressions in ninety days while the page that actually mattered had thousands.

A false claim on a page nobody opens is a liability. On a page that ranks, it is the first thing a prospect reads.

2. Search for the shape of invented work, not the words.

A fabricated case study is written in ordinary vocabulary, so no word list catches it.

Its structure gives it away: a heading saying “Case Study”, followed by a company that is never named. “A mid-sized content platform.” “An anonymous HR tech SaaS with 15,000 customers.” Precise numbers attached to a subject nobody can look up.

Real client work names the client or does not get published. That is the whole test, and you can apply it without understanding a word of the subject matter.

3. Ask whether a claim can be checked at all.

This one surprised us.

Roughly two in five of our own substantial posts cited nothing external whatsoever - no link to a framework’s documentation, a study, a release note, anything at all. Those posts are not necessarily wrong.

They are unverifiable, which means nobody could have checked them, including the person who wrote them. Uncheckable is where wrong survives. A post making technical claims with zero citations is not a red flag about that post’s accuracy so much as a flag that accuracy was never tested.

4. Check whether the advice has expired.

Any post with a version number in the title has a shelf life its author never wrote down.

We found a migration guide sending real traffic to a framework release whose security support had ended five months earlier. Nothing in it was invented.

It was true when written and became harmful without changing a word. Two minutes on the vendor’s support-policy page settles it.

What a check like this cannot do #

It cannot tell you whether a claim is true.

We tried to build that and failed honestly. Two candidate patterns for detecting wrong technical claims got measured against the archive before we trusted either: one matched ten passages, mostly legitimate; the other matched a hundred and eighty-five, almost all ordinary prose. Both would have cried wolf until people stopped reading the output.

A wrong explanation is a well-formed sentence using correct vocabulary, pointing roughly the right direction. There is no pattern for it. What catches it is someone who knows the subject, reading with permission to disagree - which is a different investment, and the one worth making after the four checks above have narrowed where to look.

Where to start this week #

Open your search console. Take your top ten pages by impressions. For each one, ask: does it name a client, does it cite a source, and does it mention a version number.

That is an afternoon, and it tells you whether you have a problem worth spending more on.

If the answer is yes on several, the fix is not a rewrite of everything. It is the same order as the checks: highest-traffic first, fabrications before staleness, and a rule that whatever replaces a bad claim needs its own source. That last one matters more than it sounds - a correction is a new claim, and it tends to get less scrutiny than the thing it replaced.

Sources #

• Nick Cleveland-Stout, “Israel creates fake think tank in likely attempt to dupe AI chatbots”, Responsible Statecraft, 2026-08-17 - the Hanover Institute, the Piro, Inc. attribution, the $900,000 DOJ filing, and the GPTZero result
• Graphite, “More articles are now created by AI than humans” - 43,000 CommonCrawl URLs, January 2020 to May 2025, Surfer detector, with the authors’ own 4.2% false-positive rate
• Pew Research Center, “How Much of the Internet Is Written With AI?”, 2026-08-20 - ~490,000 English-language Common Crawl pages, January 2021 to July 2026, Open Pangram, including the per-domain breakdown and the authors’ caveat on per-document accuracy