Say you test three headlines on your best-selling product page. Six weeks, 41,000 product page sessions.
Add to cart comes back at 5.1%, 5.3% and 5.2%.
The read is nearly always the same. The headline doesn't move anything, so keep the original and go and work on something that does.
There's usually nothing wrong with how the test was run. Even split, six weeks, and nobody called it early.
But at roughly 13,700 sessions per variant, the smallest difference that test could reliably detect was about three quarters of a point between any two variants. Closer to nine tenths once you account for testing three of them against each other. The three finished two tenths apart.
So the test was clean. The conclusion drawn from it was wrong.
Because the three variants in that kind of test almost always look like this.
“Elevate your everyday.” “Designed for how you actually live.” “Considered essentials, made to last.”
You can't tell what any of those sell. Three sentences that say nothing give a test nothing to measure. A result like that isn't a verdict on headlines. It isn't a verdict on those three either. It's a test that could never have returned anything else.
Why Did Three Different Headlines Finish In A Dead Heat?
Claude Hopkins answered this in 1923, in a chapter called Being Specific, and he wasn't being gentle.
“Platitudes and generalities roll off the human understanding like water from a duck. They leave no impression whatever.”
Vague claims don't just fail to land. “They lead readers to discount all the statements that you make.” The empty line at the top of the page taxes everything underneath it.
“But a man who makes a specific claim is either telling the truth or a lie. People do not expect an advertiser to lie.” Which is why, he says, a definite statement usually gets accepted.
A specific claim is believable because it could be checked, and because you'd be caught if it were false. A vague one carries no risk for you, so it carries no information for the customer. If a sentence can't be wrong, it can't be convincing.
Hopkins even gives you the test case. “Say that a tungsten lamp gives more light than a carbon lamp, and you leave some doubt. Say it gives three and one-third times the light and people realize that you have made tests and comparisons.” The number does two jobs at once. It tells the customer what they get and that you measured it.
You get the same idea from a much less romantic source. The FTC says out loud that it generally will not pursue cases over obviously exaggerated claims, the ones ordinary consumers do not take seriously. That is the regulator conceding nobody is expected to believe them. But the moment a claim becomes objective and measurable, you need what the FTC calls a reasonable basis for it, in hand, before the ad runs. That's US law, and other regimes handle puffery differently, and it isn't a free pass either way: the Commission is equally clear that some exaggerated claims, however, may be taken seriously by consumers and are actionable.
The line the FTC draws sits almost exactly where Hopkins drew it. The claims it treats as beneath enforcement are the ones it assumes nobody takes seriously. The claims it polices are the ones people act on.
There's measurement behind it too, though you should hold it loosely. In 1997 John Morkes and Jakob Nielsen at Nielsen Norman Group ran 51 users across five versions of the same site, between-subjects. The control was written in what they called marketese, which they defined as “exaggeration, subjective claims, and boasting, rather than just simple facts”. Stripping that out scored 27% better than the control, and the version that was concise, scannable and objective at once came in 124% better. Be precise about what that number is, because it gets quoted badly: it's an improvement on a usability score built from task time, errors, memory, time to draw a sitemap and satisfaction, on one tourism site, in 1997. Nobody measured sales.
Where Empty Copy Actually Sits On A Product Page
Once you're looking for claims that could never be wrong, they turn up in the same five places on nearly every product page I open.
The line above the fold names a feeling rather than the product, so the first thing she reads gives her nothing she can use to decide whether to keep scrolling.
The description opens with the founding story, which answers a question she hasn't asked yet and pushes the specifications she came for below the fold.
Superlatives stand in for measurements. Premium, luxurious, expertly crafted. Every competitor open in her other tab is claiming exactly the same thing.
Benefit lines arrive with no mechanism under them. Sleep cooler says nothing about how, and she has to take that on trust from a brand she met ninety seconds ago.
A block of prose sits where the specifics should be. Baymard's benchmark found only 22% of the top-grossing US and European ecommerce sites they test structure descriptions by highlights, even on their own best selling products, and baymard describes what users face instead as a wall of text.
Every one of those is the same failure wearing different clothes. Not one of those lines could be proven wrong, so not one of them is doing any work. Your competitor could paste any of them onto their page tonight and nothing would break.
Ask What The Receipt Is
The reason this survives is that most headline reviews ask the wrong question. Is it on brand. Does it sound like us. Is it too plain. Those questions have no settleable answer, which is why the meeting goes around twice and the founder picks the one he likes on the day.
Here's the question that does have an answer. The Receipt Test: Go down your product page line by line and ask what the receipt is for each claim in it. If you can't name the evidence you already hold, that claim is decoration.
Receipt means evidence you hold today, not evidence that could exist. One line can carry three claims, one receipt and two blanks.
Run it on the three variants from the top of this email and none of them survive the first pass. There's no evidence you could produce for “elevate your everyday”, because there's no claim in it to evidence. Run it on “a merino tee that holds its shape through 60 washes” and the receipt is obvious, as long as somebody actually washed it sixty times.
It grades rather than passing or failing outright, which is what makes it usable. “Premium Italian leather, made to last a lifetime” is three claims in a coat. Italian leather has a receipt sitting in your supplier paperwork. Premium has none. Made to last a lifetime has one only if you offer a lifetime warranty.
One boundary, so you don't over apply this. "Over 40,000 orders shipped" passes the Receipt Test and is still the wrong headline. That's a popularity receipt, not a product receipt. The test tells you whether a line is a real claim. It doesn't tell you whether it's the right claim.
The question changes what the meeting is about. You stop arguing about taste and start asking what you can actually prove, and that's a question your team can go and answer.
Run This On Your Five Best Sellers This Week
You don't write your way out of this at a desk. The specific language already exists, in your reviews and your inbox, and your job is to go and find it. Pull 40 reviews across your five best sellers, plus the last 60 days of pre-purchase messages, roughly 20 of them.
Joanna Wiebe, who founded Copyhackers in 2011, puts it about as bluntly as it can be put:
“Review mining data should write your copy, you shouldn't write copy.”
On which reviews to read, her guidance is that “You can look at five star and one star, but what you're mostly going to find is in two, three, and four stars.” I go narrower and start at three and four star. That's my own narrowing of her advice, not hers. Five star is usually praise, and one star is usually a courier problem rather than the product.
Reviews live in Judge.me, Okendo, Loox, Yotpo. Messages live in Gorgias, Zendesk, Shopify Inbox, and in your Instagram DMs.
Competitor Amazon listings are worth a look for category language, where the listing still shows a customer questions block. Amazon has replaced it with an AI summary on a lot of them now.
Then tag. Every sentence that contains a number, a body measurement, a time period, a named alternative, a named failure, or a named use case. Those six markers are the raw material. Everything else is sentiment, and sentiment is what you already have.
Two things to watch for while you read. The detail no copywriter would have invented, like the reviewer who writes that she's 5 foot 4 and the medium sits just above her knee. And the question that arrives four times a week in four different phrasings, which is the one your page has been failing to answer for months.
If every review reads love it, great quality, fast shipping, you're in the wrong window. Those are post-delivery reviews prompted by an automated email, and they're about the parcel. Go to the pre-purchase questions instead, because doubt is what you're mining for and doubt happens before the order.
Then rebuild the line with whichever formula your tagged sentences can actually pay for. If the reviews keep supplying a measurement, you've got the measured claim: “built to last” becomes “A merino tee that holds its shape through 60 washes.”
If they keep telling you who the product is wrong for, you've got the disqualifier, which is the one most founders flinch at: “A five layer mattress for side sleepers over 200lb. Too soft if you sleep on your front.” Naming who shouldn't buy is the hardest thing to fake, which is why it reads as proof that you meant the rest of it. All five formulas are in this week's deck.
One rule before you paste anything live. You have to be able to stand behind the claim before it goes on the page. A reviewer saying it isn't the same as you being able to prove it. If a reviewer says it held up through two winters, that's her sentence and it belongs in a review block. If you want it as your headline, you need to be able to stand behind it as a claim about the product.
One honest note on where this lives. The description block is editable from the Shopify admin on any plan. The line above the fold is usually a theme setting or a metafield, so changing that one is a theme edit rather than a content edit. Worth knowing before you promise it by Friday.
Clever is what you write when you have nothing to prove. Go and find something to prove.
This Week's Free Resource
This week's newsletter comes with a free eight-slide deck, The Receipt Test, with the one-minute check for a headline that proves nothing and the five formulas that replace it. Every before-and-after in it is a constructed illustration of the shape, not a line off a real client page.
Thanks for reading!
Conal Brady
Ecom Luxx
P.S. Learn more about Ecom Luxx and our GrowthOps™ Partnership below:
