Every agency in the world spent this year being asked the same question by clients: can we just use AI for the blog. So we ran the experiment properly. We took forty real client briefs and three of the popular writing tools. We gave the same briefs to a human writer. Two editors then reviewed every draft blind, without knowing which was which.
What the tools did well
Structure was excellent. Given a keyword and an outline, the tools returned drafts that were organized, on topic and grammatically clean. That took about four minutes, against roughly three hours for a human first draft. Some briefs are genuinely commodity content: a definition page, a straightforward how to, a glossary entry. There the gap in quality was small and the gap in cost was enormous.
They were also good at the boring surrounding work. That means meta descriptions, alternative headlines, summaries, and turning one article into a social thread.
Where AI writing tools broke
Three failures repeated across almost every brief. First, invented specifics. We saw statistics with no source, quotes from people who never said them, and case study numbers that looked exactly like real ones. Our editors caught eleven fabricated figures across forty pieces. None of them looked wrong at a glance. At scale, that is a reputational accident waiting to happen.
The fix is procedural, not technical. Treat every number in a draft as unverified until a human attaches a source. Delete any figure that nobody can source, however plausible it sounds. A claim you cannot trace is a liability, and your client is the one who signs under it.
Second, no point of view. The drafts summarized the consensus of everything already published, which by definition means they said nothing new. We scored pieces on whether a reader would learn something not already on the first page of results. The human drafts won thirty eight times out of forty.
Third, and most costly for our clients, no first hand detail. A model writing about paid campaigns in Egypt has never seen a client invoice. It has never argued with a platform support agent. It does not know that cash on delivery breaks ad measurement and changes the whole funnel. Those details are exactly what makes content rank and convert.
How we use them now
We use them as a research assistant and a first draft engine for commodity pages, never as a publisher. Every piece goes to a human who adds the numbers, the opinion and the local reality. An editor then checks every claim. Net time saved is real but modest, around 30 percent, and quality holds. We would rather save less time and publish nothing we cannot defend.
If you want to repeat the test in house, keep it honest. Use briefs you have already published, so you know what good looks like. Remove the tool’s name from the drafts before anyone reviews them. Ask the reviewer to check every number against a source, not just to read for flow. Fluent prose is exactly what makes a fabricated figure easy to miss. Finally, score the result on usefulness to the reader, not on how quickly the draft arrived.
Google has been fairly consistent about this. It rewards helpful content regardless of how someone produced it, and it demotes content produced mainly for rankings. We saw the same logic in the March 2023 core update. That standard has not changed, which is why our SEO services still start with a strategist rather than a prompt.
