A/B testing usually means a platform, a script, and a dashboard. But the most common marketing experiments — which landing page, which link copy, which send time — only need two links and honest measurement. This guide covers the experimental discipline: what to test, how to split, how big the sample needs to be, and how to read the numbers.
The three tests that need only links
1. Destination tests. Two landing pages, two links, same channel slot:
https://yas.sh/signup-a → landing page A (short form)
https://yas.sh/signup-b → landing page B (long form)
Measure clicks and destination-side conversions (conversion tracking); the click-through tells you which link wins attention, the conversion tells you which wins business.
2. Copy/alias tests. The same destination, two aliases — this tests the label, which matters in SMS and print where the URL is read first:
https://yas.sh/get-started vs https://yas.sh/free-trial
3. Timing tests. Two sends (Wednesday vs Thursday, 9am vs 4pm) with identical content — the daily series in analytics makes the comparison.
The setup pattern
The discipline that keeps tests honest:
1. One question per test (no double-testing)
2. Two variants only (A/B, not A/B/C/D)
3. Parallel timing (same day/week — seasonality is real)
4. One difference between variants
5. Pre-declared sample size
Splitting traffic happens upstream (email tool, social scheduler, SMS platform) — each variant gets its own link; the links do the measuring.
Sample sizes: the math that prevents self-deception
The shortcut table for binary outcomes (clicked vs not):
| Expected difference | Clicks per variant needed |
|---|---|
| 10% → 11% | ~16,000 |
| 10% → 15% | ~1,300 |
| 20% → 25% | ~1,600 |
| 50% → 60% | ~400 |
Rule of thumb: 600–1,000 clicks per variant detects a meaningful (≥5-point) difference at 80% power. Under ~200 clicks per variant, "wins" are indistinguishable from noise — label the result a pilot and re-test at volume.
Reading the results
- Compare click-through rates, not totals. Total clicks scale with list size; the rate is the decision.
- Check the series, not just the total. A variant that wins on day one and dies by day three is a spike, not a signal.
- One metric per test. If clicks were the question, conversions are a follow-up test, not a bonus finding.
- Document the cutoff date. Results after the decision date don't retroactively change it — and cherry-picking the "right" window is how teams fool themselves.
Variant A: 512 clicks / 2,000 sent = 25.6%
Variant B: 428 clicks / 2,000 sent = 21.4%
Δ = +4.2 points — above the noise floor at this sample size → declare A
When not to A/B test
- Sample size impossible (B2B lists under ~500) — run a pilot and iterate qualitatively instead.
- One-off moments (launch day) — split attention and you underpower both variants; run a control/rollout instead.
- Infrastructure questions — whether redirects are fast or links are secure is not a popularity contest; those are measured differently.
Conclusion
Two links, one question, a pre-declared sample size, and a series — that's an honest A/B test. The dashboard gives each variant its own analytics; the UTM guide keeps the variants comparable; conversion tracking closes the loop on what the winner actually earned.