What is Multivariate Testing?

3 May
2010

If you're new here, you may want to subscribe to my RSS feed. Thanks for visiting!

Code washing
Creative Commons License photo credit: Ezu

This is a guest post from Craig Smith of Trinity Insight.

Any internet marketer knows the importance of “optimization”. Simply stated, optimization is the effort to improve previous performance.

On the web, we do this via online testing and the monitoring of key metrics that associate to business goals. In the past, businesses were confined to conducting “split tests”, otherwise testing version “A” versus version “B”. These tests, albeit effective, were difficult to scale in measuring multiple elements on a page.

Enter multivariate testing; a new form of testing based upon technologies that allow for the measurement of multiple aspects of a webpage.

Multivariate testing is the process of creating varied “recipes” of web content, that are served to your visitors in order to decipher intelligence into which mix of messaging and imagery works best. In the past, this type of solution was only available to the very large enterprises who had substantial dollars to spend on additions to their content management systems. As eCommerce and internet marketing has progressed however, so has the availability of tools that execute these conversion improvement functions.

Simply put, there is no reason why your organization or business should not be improving your conversion rates through online optimization.

Why you need to execute multivariate testing

Testing is a fundamental aspect of direct marketing. Ask any cataloger over the last 100 years. They have been testing covers, layouts, imagery within the print medium as long as the market has been active.

For a print business, the goal is clear. Test different variations of catalogs in order to isolate the best performing version for upcoming campaigns. Seems pretty basic and simple right? Well, it is, but 90% of businesses that I talk to on a daily basis are not involved in testing in the online medium, even within the available tools today.

Think of your website as a brochure that has 2 seconds to peak someone’s interest. That is the exact environment that exists today. With homepage and key landing page bounce rates exceeding 40% many times, the lost opportunity with not testing is substantial.

Be it your unique value proposition on your homepage, messaging that relates to your key services or products, or just the shape and size of a button that you want to test, multivariate testing will help answer questions about your users and web content with hard data not gut feelings.

How it works

Most multivariate testing solutions are based upon Javascript technology and the “injection” and “swapping” of content within a page. The most widely used multivariate testing platform is Google Website Optimizer and this tool is also based upon Javascript.

Basically, each area of your webpage that you want to test is encapsulated with Javascript within the test set up process. Alternatives to each location are defined within the test set up and you set goal pages to accurately track which version performed best.

As users come to your site, they get served different versions. The version that they received is populated within their cookies, this way on subsequent visits they receive the same version.

A variety of metrics can be utilized to gauge performance with the most popular being conversion. Other metrics that can be leveraged are time on site, pages per visit, or the aforementioned bounce rate metric.

Challenges with multivariate testing

As in any technology solution, there are always challenges. With multivariate testing, challenges come into play with content management systems and eCommerce platforms. Because these systems are based upon templates and the use of databases to render content, some additional work needs to be done to ensure that tests function properly.

Also serving as a hurdle can be the “tagging” process that needs to be done regarding the site elements on a page. In situations which software is licensed and access to the source code is not readily available, the tagging process needs to be done by a software vendor and can equal headaches and expensive per hour fees for remedial steps.

Regardless of these challenges, multivariate testing is something that is a must if you operate a web-based or eCommerce business. To not test different types of layouts, promotions, and calls to action is flying blind and leaving dollars on the table. Complacency is the enemy of efficiency and don’t be complacent with the conversion rate of your website. Start multivariate testing today.

About the author

Craig Smith is the CEO of Trinity Insight – a leading eCommerce consulting company that specializes in multivariate testing.

Bookmark and Share

Comment Form

top