This is a sandbox. We built a small simulated web for one product category — desk lamps with built-in air purifiers — with five fictional brands and 27 invented pages (brand sites, review roundups, a forum thread, video transcripts). None of it was ever published. Below, the new entrant is called Brand V and the category leader Brand L. Retrieval was simulated the way answer engines work (query fan-out, ranking, reading the top pages); the answers were written by a real language model acting as a shopping assistant. Publishing invented products to the real web to sway AI answers would be data poisoning — so we don't.
The short answer
Your own pages get you mentioned and described correctly. Independent third-party verification gets you recommended. A self-ranked “best of” list does nothing.
Design
8 buyer questions × 5 runs × 4 conditions = 160 answers. Each run saw 6 of the top 8 retrieved pages, shuffled.
A Baseline: Brand V has a spec-free marketing homepage; one forum comment mentions it with the wrong price and “no reviews yet”.
B Own pages: + a fact page, a factual spec comparison and an FAQ on Brand V's site.
C Own pages + third-party coverage: + an existing review roundup adds Brand V after testing it, an independent measured review, and a creator video with a disclosed free unit.
D Own pages + self-ranked list: + a post on Brand V's own blog ranking itself #1.
Whether an answer recommends Brand V was coded blind by two independent coders (condition hidden; 98% exact agreement, weighted kappa 0.99); counted only when both agreed.
Results
Baseline
Own pages
Own pages + third-party coverage
Own pages + self-ranked “best of” list
Mentions Brand V
27.5%[7.5%–47.5%]n=40
55.0%[32.5%–77.5%]n=40
85.0%[65.0%–100.0%]n=40
72.5%[45.0%–95.0%]n=40
Recommends Brand V
0.0%[0.0%–32.4%]n=40
7.5%[0.0%–25.0%]n=40
55.0%[25.0%–82.5%]n=40
10.0%[0.0%–27.5%]n=40
Brand V is the top pick
0.0%[0.0%–32.4%]
5.0%[0.0%–17.5%]
27.5%[0.0%–62.5%]
5.0%[0.0%–17.5%]
Tells people to skip or wait on Brand V
15.0%[0.0%–35.0%]
5.0%[0.0%–17.5%]
2.5%[0.0%–10.0%]
5.0%[0.0%–17.5%]
Questions the source of Brand V claims
25.0%[7.5%–45.0%]
55.0%[32.5%–77.5%]
65.0%[40.0%–87.5%]
67.5%[40.0%–90.0%]
Cites a third-party page about Brand V
0.0%[0.0%–32.4%]
0.0%[0.0%–32.4%]
65.0%[35.0%–92.5%]
0.0%[0.0%–32.4%]
Paired differences (same questions)
Comparison
Recommends
Mentions
Own pages vs baseline
+7.5pp[+0.0, +25.0]
+27.5pp[-10.0, +62.5]
+ third-party vs own pages only
+47.5pp[+20.0, +75.0]
+30.0pp[+5.0, +57.5]
+ self-ranked list vs own pages only
+2.5pp[-7.5, +15.0]
+17.5pp[-10.0, +45.0]
third-party vs self-ranked list
+45.0pp[+17.5, +72.5]
+12.5pp[-7.5, +35.0]
What we learned
Own pages: mentioned, not recommended. The model treated the brand's own site as a claim, not evidence — over half the answers added caveats like “these are the maker's own numbers” or “the lab report is only available on request”. Publish the evidence itself.
Independent measurement is what flips it. Recommendations rested on “the reviewer measured 24 dB”, not on the spec sheet.
Self-ranked lists don't work. Every answer that saw the self-ranked post called it out as the brand ranking itself.
Mention rate can count bad news as visibility. In the baseline, Brand V was mentioned in over a quarter of answers — all prompted by one forum comment or its spec-free homepage. None recommended it; several told people to wait for reviews. Stance has to be measured separately.
Errors persist where your pages don't reach. The wrong forum price kept being repeated on a budget question for which Brand V's fact page was never retrieved. Fix errors at the source.
Recommendations were added, not swapped. The category leader was mentioned in every answer in every condition.
Test the measuring tool too. Our pre-registered text rule for “recommended” missed every prose recommendation, and a fact-checking rule misfired on competitor numbers in the same sentence. Both are now fixed or backed by human review.
Limitations
Simulated retrieval over a corpus we wrote; the experiment shows what happens once coverage exists, not how hard it is to earn.
One model family wrote the answers and coded them; 8 questions × 5 runs gives wide intervals.
Pre-registered hypotheses that failed are reported as failed; the blind stance coding was added after the recommendation rule broke, and is labeled as a deviation.