A/B testing may seem straightforward, but the truth is that there are a lot of "little-known" factors that can shift how your tests perform.You might have heard about sample sizes, statistical significance, and time periods required for proper testing of landing pages...But what about the History Effect? The Novelty Effect? The Instrumentation Effect? or the Selection Effect?Watch to find out why the above threats could invalidate your A/B tests (and how to avoid skewing your data).
The History, Instrumentation, Selection and Novelty effects are 4 validity threats that could invalidate your A/B test data, giving you the illusion that one variation won when in reality, it lost.Keep them all in mind when analyzing your test data, and don't forget to analyze your results in Google Analytics and/or Mixpanel to see the what truly lies behind averages, and spot the signs of flawed tests.