A/B testing a paywall with a free trial: when can you decide?
Not when the traffic stops. Wait for the longest free trial among the plans people buy to end, then about a week for refunds. Until then, a trial start isn't a payment and the test can't tell you which version earns more.
When the decision day falls
A 4-week paywall test that starts on Monday, November 9:
| Free trial | Traffic ends | Decide on |
|---|---|---|
| None | Dec 6 | Mon, Dec 14 |
| 3 days | Dec 6 | Thu, Dec 17 |
| 7 days | Dec 6 | Mon, Dec 21 |
| 14 days | Dec 6 | Mon, Dec 28 |
The week after the last trials bill is for refunds: a version that wins on first payments can still lose once refunds land.
Why waiting matters
- Trials aren't payments. A version can win trial starts by making the price harder to see and then lose on paid. Read the result on paid conversion or money per view.
- The last trials bill late. Someone who starts a 7-day trial on the last day of the test pays a week later.
- Refunds come after. Watch refunds as a guardrail: if B's refund share is clearly higher, its win may not hold.
- Testing the trial itself? Adding a trial, changing its length or putting it on the annual plan changes who pays and when; judge it on money per paywall view after the longest trial has billed.