Checkout experiments

What experimentation on checkout is

Checkout is the step between deciding to buy and having bought: the basket, the delivery and address details, the payment form and the confirmation. Experiments here change what that flow asks for, in what order, and what it shows about the price. They do not change the price. A discount, a delivery charge or a tax rule is a commercial decision, it cannot be reverted for the people who already paid under it, and it sits outside the sandbox this product works in. What can be tested is the order of the steps, the fields each one requires, when the total and its parts become visible, which payment methods are offered and where, and whether an account is demanded before the order can be placed.

The metrics

Completed orders per checkout start is primary: the share of sessions that begin checkout and end in a paid order, in a window of days. It stays primary even when the change sits at one step, because a change that moves a drop-off from the last screen to an earlier one flatters the completion rate of every step after it while selling nothing more. Revenue per checkout start says what kind of orders they were, because a flow can turn more sessions into smaller baskets. Step-level completion and time in checkout explain a result. They should not decide it.

The bottlenecks

Costs that appear late: delivery, tax and fees added after the shopper has entered an address. A total that cannot be seen until the payment step. An account demanded before the order can be placed. A form asking for a company name, a second address line and a phone number that nothing in the order needs. Too few payment methods, or the one the shopper uses buried under the card form. An error that clears the form. Each one asks for effort or for trust at the point where the shopper has the least reason to give either.

The guardrails

Revenue above all: revenue per checkout start, held against an easier flow that sells less. Refunds, returns and chargebacks in the first month, because a checkout that hurries a decision is paid for after it. The payment failure rate, because a change near the payment form can break more than it moves. Support conversations about the delivery cost or the total. On this stage the guardrails are not a formality. They are the reason a checkout test is safe to run at all.

When a test here is feasible

Far fewer sessions reach checkout than reach a product page, and this is the stage with the most money attached to each one, so it puts a small population next to a change that must not go wrong. That is the worst pair of conditions for a fast answer. Size the test before it starts, with the calculator and the sample size guide, check it against the feasibility checker and the feasibility guide, and read the guardrail guide before deciding what the test is allowed to cost. A change the arithmetic says will take a quarter should be decided by judgment, shipped, and watched on the guardrails instead.

The patterns

Showing the delivery cost and the order total from the first step is the example written out in full below. Offering the order without an account, moving the express payment methods above the card form, and cutting the address form to the fields a delivery actually needs are the same instinct at other points in the flow: take a demand off the screen, or answer a question before it is asked.