How to Audit Content You Didn't Write

There is no Hanover Institute for Public Policy. There is a website carrying more than a hundred reports under that name - footnotes, tables of contents, the flat neutral register that policy writing has - and behind it a marketing firm working on a government contract.

NewsGuard analyst Alice Lee connected it to Piro, Inc., reported by Responsible Statecraft; Politico first reported the Department of Justice filing, under which Piro has received $900,000 from the Israeli government for its work. GPTZero flagged all twelve sampled articles as AI-written - eleven with high confidence, one moderate.

The interesting part is who the reports were written for. Piro’s founder said it on LinkedIn: “When someone asks ChatGPT, Gemini, or Perplexity about your category, an answer comes back in one confident paragraph… we spent months reverse-engineering it.”

Those reports were never aimed at readers. Their audience was the machine that answers readers, and the reports were shaped to be the thing it repeats.

Your blog runs on the same economics #

Nobody put a government contract behind your blog.

That is the point. They did not have to, and neither did whoever produced yours, because manufacturing text that reads like expertise stopped being expensive somewhere around the middle of 2023.

Ask what your blog archive actually is. An agency wrote some of it, on a monthly retainer measured in posts, and a freelancer wrote more.

Then a tool started drafting, and the person approving its output was not equipped to check the technical claims inside it. Nobody in that chain was lying. Each link did the job it was paid for, and the job was volume.

Nobody can tell you how much of the web this is #

You would think there is a number. There are several and they disagree - ten percent, a third, or half, depending on whose sample and whose detector.

Graphite put the crossover, more machine-written articles than human ones, in November 2024. Pew ran ~490,000 Common Crawl pages through Open Pangram this month and found 10% carrying AI-authorship signals, rising to over a third among pages published after ChatGPT shipped. Graphite states its own 4.2% false-positive rate and notes on its own page that this finding has since been superseded by a study averaging three detectors. Pew publishes no error bars at all - only the caveat that detectors “sometimes misclassify individual documents” and hold up in aggregate. Graphite tells you how wrong it might be; Pew tells you it might be wrong.

Pew also split it by domain, and that is where you come in:

Commercial sites are where this concentrates, and yours is a commercial site.

So the honest position is that the web’s average is unknown and the detectors that estimate it are themselves approximate. Which is fine, because the average was never the thing you needed. You need to know about your property, and your property is countable.

Four checks, and the commands that run them #

Run them in this order. I have ranked them by what each one costs you to skip, not by how quick it is.

The commands assume a Hugo or Jekyll-shaped repo where posts are markdown files. Adapt the paths; the shapes they look for are the same everywhere. If you do not have repo access, these are exactly the four things to ask whoever does.

1. Rank by who reads it, not by how bad it looks.

Export your top pages from Search Console, then work down that list and nothing else.

The instinct is to start with whatever reads worst. Resist it - the page that makes you wince and the page that gets read are rarely the same one, and a featured flag in your site config tells you what someone decided once, not what anyone has done since.

Impressions are what readers actually did.

# Export "Pages" from Search Console as CSV, then rank what you actually have:
sort -t, -k2 -rn pages.csv | head -20

A false claim on a page nobody opens is a liability. On a page that ranks, it is the first thing a prospect reads.

2. Search for the shape of invented work, not the words.

A fabricated case study is written in ordinary vocabulary, so no word list catches it.

Its structure gives it away: a heading saying “Case Study”, followed by a company that is never named. “A mid-sized content platform.” “An anonymous HR tech SaaS with 15,000 customers.” Precise numbers attached to a subject nobody can look up.

That shape is greppable:

# every case-study heading in the archive
grep -rniE '^#{2,4} .*case stud' content/ 

Run it and read every hit. Real client work names the client or does not get published, and you can apply that test without understanding a word of the subject matter.

Do not try to automate the second half. We tried: an anonymous-subject pattern (a mid-sized, a large) returned ten matches that were nearly all legitimate, because “a large number of” and “a large team” are ordinary English. The heading is the cheap signal; the judgement stays human.

3. Ask whether a claim can be checked at all.

Count how many of your long posts link to nothing outside your own domain.

A post with no external citation is not necessarily wrong. It is unverifiable, which means nobody could have checked it - including whoever wrote it, at the moment they wrote it.

Uncheckable is where wrong survives, and the number usually comes back higher than anyone guesses.

Count yours:

# posts over 400 words carrying zero outbound links to anywhere but your own domain
find content/blog -name '*.md' | while read -r f; do
  words=$(wc -w < "$f")
  links=$(grep -oE '\]\(https?://[^)]+\)' "$f" | grep -vc 'yourdomain.com')
  [ "$words" -gt 400 ] && [ "$links" -eq 0 ] && echo "$words words, 0 sources: $f"
done

Use find, not content/blog/**/*.md - bash does not expand ** recursively unless shopt -s globstar is set, so the glob version silently matches nothing and reports success. That is the exact defect this post is about, and it was in my first draft of this command.

A post making technical claims with zero citations is not a red flag about that post’s accuracy so much as a flag that accuracy was never tested.

4. Check whether the advice has expired.

Any post with a version number in the title has a shelf life its author never wrote down.

Every framework you write about has a support table, and every one of your version-numbered posts is silently betting that the version it recommends is still on it. When that stops being true, the post does not change and nothing in it becomes false - it just starts pointing readers at an unpatched release.

That is the whole failure. No invention required.

# every post whose title names a version - each one has an expiry date
grep -rlE '^title:.*[0-9]+(\.[0-9]+)?' content/blog/ | head -30

Then check each against the vendor’s own support table. Laravel, Rails and Node all publish one; it takes two minutes and it is the only way this class of defect surfaces.

What a check like this cannot do #

It cannot tell you whether a claim is true.

We tried to build that and failed. Two candidate patterns for detecting wrong technical claims got measured before either was trusted: one matched ten passages, mostly legitimate, the other a hundred and eighty-five, almost all ordinary prose. Both would have cried wolf until people stopped reading the output.

A wrong explanation is a well-formed sentence using correct vocabulary, pointing roughly the right direction. What catches it is someone who knows the subject reading with permission to disagree - a different investment, and the one worth making after these four have narrowed where to look.

Where to start this week #

Open your search console. Take your top ten pages by impressions. For each one, ask: does it name a client, does it cite a source, and does it mention a version number.

That is an afternoon, and it tells you whether you have a problem worth spending more on.

If the answer is yes on several, the fix is not a rewrite of everything. It is the same order as the checks: highest-traffic first, fabrications before staleness, and a rule that whatever replaces a bad claim needs its own source. That last one matters more than it sounds - a correction is a new claim, and it tends to get less scrutiny than the thing it replaced.

Sources #

• Nick Cleveland-Stout, “Israel creates fake think tank in likely attempt to dupe AI chatbots”, Responsible Statecraft, 2026-08-17 - the Hanover Institute, the Piro, Inc. attribution, the $900,000 DOJ filing, and the GPTZero result
• Graphite, “More articles are now created by AI than humans” - 43,000 CommonCrawl URLs, January 2020 to May 2025, Surfer detector, with the authors’ own 4.2% false-positive rate
• Pew Research Center, “How Much of the Internet Is Written With AI?”, 2026-08-20 - ~490,000 English-language Common Crawl pages, January 2021 to July 2026, Open Pangram, including the per-domain breakdown and the authors’ caveat on per-document accuracy