Janine Popick, president and CEO and founder of Vertical Response, tells of the surprising test results stemming from a test the company recently tried on an email marketing campaign. The bottom line to the story, she concludes, is that even when one assumes he or she knows what a particular result will be, it is still better to test - because, in fact, you do not know for sure.

Case in point: an email that the company sends each month to customers that have signed up for its free trial, but haven’t activated their account. Two different creatives were sent a month apart, in April and in May. One encouraged customers to activate their account, and then offered a discount if they "liked" Vertical Response on Facebook. The second version didn't offer an incentive and focused solely on encouraging customers to activate their account.

It was that version, the one that focused solely on prompting customers to activate their account, that received 32% more opens and clicks. Specifically there was a bow around a finger image, "activate your account" links in the first three lines of the email, and a call-to-action button - all of which garnered the majority of clicks, according to Popick.

More elaborate testing, such as multivariate and AB - which Vertical Response didn't do in this case but often does - however, require more thought in crafting the test.

Bigger Tests, Bigger Considerations

When deciding between full factorial parametric designs and fractional factorial designs, writes Tim Ash at ClickZ, there are some broad brush considerations. For example, full factorial parametric designs do not scale very well, but get more complete information about the exact relationship, he writes. Fractional factorial designs can scale to larger search spaces, but make assumptions about the underlying process that may actually lead you astray.

For companies that don’t have the time or resources to try out such full-blown options there are free web tools available.