How to prove AEO results: control questions and day-21 reads

Every team doing AEO eventually gets the question: "did that work?" The honest answer usually isn't a yes or a no but "here's how much the questions we aimed at moved, against the ones we didn't". This guide is how to get to that answer without fooling yourself.

Why "it went up after we published" isn't proof

Answers drift for reasons that have nothing to do with you. An assistant gets a model update. A rival publishes a comparison page. A roundup that cites you is re-crawled. Buyers' questions change with the season. Google says its AI features may issue "multiple related searches across subtopics and data sources" for one question, so any change to any of those sources can move the answer.

That means a before-and-after chart for one set of questions can't separate your work from the weather. You need a comparison group, the same way a drug trial needs a placebo.

The method in five steps

1. Name the targets before you ship

Write down the exact tracked questions the change is meant to move: the three buying questions a new comparison page answers, say, or the fact questions a corrected pricing page should fix. Do it before the page goes live. Choosing targets after you've seen the results is the most common way people prove what they hoped.

2. Keep controls untouched

Control questions are tracked questions the change doesn't aim at, ideally in the same tier and product line as the targets. They feel every market-wide force the targets feel, minus your page. Don't work on them during the evaluation window, or you lose the comparison.

3. Record the live date

The clock starts when the change is public and crawlable, not when it was drafted or approved. If you can, ask for re-crawling (Search Console's URL inspection for Google, IndexNow for engines that support it) and note the date.

4. Wait long enough

Assistants pick up changes as their sources are re-crawled and re-ranked, which takes weeks, not days. Keats reads a first verdict at day 21 and a second at day 35. Reading at day 3 mostly measures noise; reading only at day 90 leaves too much time for other things to change.

5. Compare the gap with the noise

At each read, compute how much the targets moved and how much the controls moved over the same weeks, then look at the difference. That difference has to clear the noise band to count, because both numbers come from samples of answers.

A worked example

Prepco, our made-up exam-prep brand, publishes a page comparing its course with two rivals for working adults. Its targets are three buying questions about prep for people with full-time jobs. Its controls are the other twelve buying questions in the set.

At day 21, the targets are up 11 points in the share of answers naming Prepco, and the controls are up 3. The gap, 8 points, is wider than the noise band of 5 points for that many answers, so the read says "likely worked". At day 35 the gap holds at 9. That's a result you can report with a straight face, along with the counts behind it.

In Keats, the same comparison is drawn for every finished card. Here it is on Prepco from our demo workspace:

Expand

Mistakes that fake a result

• Moving the goalposts. Adding the questions that rose to the target list after the fact.
• Working the controls. Publishing a second page that touches the controls during the window.
• Changing the wording. A reworded question is a new question with a new baseline.
• Reading one assistant. If the change moved ChatGPT and not Gemini, say that, rather than averaging it into a smaller, vaguer win.
• Ignoring flat results. A change that did nothing is useful to know. Write it down, revise once, and move on.

Reporting it

For each change: what shipped, the live date, the targets and controls, the two moves and the gap, the noise band, and the verdict. That's six lines, and it's the part of an AEO report that clients and bosses trust, because it can come out "no". Our guide for agencies reporting AEO to clients shows how to fit it into a monthly report, and measuring AI visibility covers the weekly numbers the verdicts are built from.