Pricing experiments

What experimentation on pricing is

The experiments here change how plans are presented, not what they cost: the default billing period, the plan selected first, how the saving is shown, how many plans are on the page and which one sits in the middle, and how the comparison reads. Price changes are not experiments in this sense. They change the contract, they cannot be reverted for the people who bought, and they sit outside the sandbox this product works in.

The metrics

The upgrade rate from the pricing page is primary: the share of eligible visitors who start a paid plan in a window of weeks. It stays primary even when the question is about annual plans, because a framing that only shifts the mix from monthly to annual, with the same number of people upgrading, has changed cash timing and nothing else. Annual share of new upgrades and revenue per eligible visitor are the secondary metrics that say what kind of upgrades they were.

The bottlenecks

A default that is not the plan most people should be on. An annual saving buried in a toggle. Three plans where the middle one is not the obvious choice. A comparison table that lists features rather than answering the visitor's question. A pricing page reached by people who are not signed in and cannot act on it.

The guardrails

Refund requests in the first month and cancellation before the first renewal, because a presentation that reads as a trick is paid for later. Support conversations about billing. The upgrade rate itself, held as a guardrail on any test whose primary question is the mix.

When a test here is feasible

Fewer visitors reach the pricing page than the signup screen, and upgrade rates are low, so tests on this stage take the longest to reach reliable evidence and many cannot at all. A test the arithmetic says will take a quarter should not run. The change should be decided by judgment, shipped, and watched with the guardrails instead. Size it before it starts, with the calculator and the sample size guide, and read the feasibility guide on when not to test.

The patterns

Selecting annual by default with the monthly-equivalent price beside it is the example written out in full below. The middle plan and the order of monthly and annual are the same lever at other positions on the page.