Experiment feasibility checker

The answer is a yes or a no about one test in one window, and the two numbers under it say why. The days needed is how long the effect you stated takes to reach a sample that can tell it from noise. The smallest effect your window can detect is the other side of the same arithmetic: at this traffic, over these days, anything smaller than that figure will finish the run indistinguishable from nothing, whatever the chart looked like on day four.

A no is not a verdict on the idea. It says this test, at this step, in this window, cannot produce evidence, and the three ways out are a larger change, a step with more traffic, or a longer window. A test run anyway is not a cheap test. It costs the same engineering, occupies the same surface for the same weeks, and returns a number nobody should act on, which is worse than not running it, because a team acts on it.

When no effect at all fits, the window and the traffic between them cannot detect even a very large change. That is a fact about the step, not about the idea, and it usually means the test belongs somewhere with more traffic.

Visitors are independent and each is assigned once. The metric is binary: a visitor converts or does not. The effect is stated as a percent of the baseline. The test runs to a fixed sample and is read once at the end. The significance level is five percent two-sided, the power eighty percent, and the split even, which are the industry defaults and the settings the sample size calculator opens on; it has the others. The traffic you give is eligible traffic reaching the step, not visits to the site.

This decides feasibility and nothing else. It does not say the change is a good idea, does not rank it against other ideas, and cannot know whether the effect you expect is plausible; that number is your estimate and the answer is only as good as it is. It models one metric, two arms and a fixed horizon. Sequential designs, variance reduction such as CUPED, and multiple variants all change the arithmetic and none is modelled here.

The smallest detectable effect is found by searching the sample size formula rather than by inverting it, so the two tools agree exactly on any case you put through both. The search stops at 400 percent of the baseline, and at whatever effect would take the variant rate to 100 percent, because past either the arithmetic stops describing a funnel step.