target population accounted for

But why? Because we tend to ignore base rates. While this example simplifies the concept and translates it to web traffic, the principle stands: We focus on the specific information we’re given and lose sight of the bigger picture. The big picture in this case is the proportion of people who are actually personal trainers. There are 267,000 personal trainers in the United States, which is a lot until compared to the 49,933,000 gym memberships (discounting personal trainers) in circulation.



For the sake of the example, if we assume that the only two populations  Buy TG Database inhabiting the planet are gym-goers and trainers (and that they both spend equal amounts of time online), for every one trainer browsing the web, there are 187 gym-goers browsing the web. The odds of a random visitor being a trainer? Low. The same rule generally applies to your target audience. Why visitor makeup matters Which metrics do you consider in your tests? Most people will look at sample size, conversion/click-through rate and confidence interval.



A handful may look at statistical power, length of run and check for errors across various browsers and devices. What is often overlooked is the number of marketing qualified leads generated (or lead-to-MQL rate). In other words, you fail to segment different populations in your A/B test. You focus on aggregate conversion rate (CVR) as a go-to metric because it’s easier. We take mental shortcuts all of the time – and this is one of them. The metric you need to uncover is the conversion rate of your target population and what percentage of visitors your .