The Voice of Online Marketing | MEDIA KIT | NEWS TIPS
The latest practical news and developments at the intersection of search, email,
social media, mobile marketing, web analytics, online advertising, ecommerce and more.
Marketing News on Twitter Interactive marketing RSS newsfeed

How to: Validate your A|B test results for websites | And email marketing

How to Validate your A|B test results for a Website ….

Of those marketers that perform A|B and multivariate testing, 40% don’t validate their test results, according to the Get Elastic blog. "Clearly, validating your test results should be a key part of the conversion testing process, or you’re going to be acting on bad data and losing cash," writes Amanda Dhalla. There are four reasons why the test results might be inaccurate, she says in her detailed post, which includes tactics for combating them. Briefly, these are:

Too small a sample size.

An external event that changes visitor behavior.

A change in your technical environment or measurement tools.

A change in incoming traffic sources or traffic mix.

….And Email Marketing

Jay Miller at Exact Target took the ball and ran with it for email testing. A test for a new subject line could point to a 10% improvement - which is great but only if it is not due to random chance, he says in his blog post. He suggests focusing on:

How big is the effect you're trying to measure?

How large should the sample size be?

How confident do you want to be in your result?

Miller runs through the scenarios for each question in his excellent post.


Related Topics


Subscribe to MarketingVOX|News

Latest interactive marketing news Latest media planning news & facts Latest marketing data & research