photo credit: Ezu

This is a guest post from Craig Smith of Trinity Insight.

Any internet marketer knows the importance of “optimization”. Simply stated, optimization is the effort to improve previous performance.

On the web, we do this via online testing and the monitoring of key metrics that associate to business goals. In the past, businesses were confined to conducting “split tests”, otherwise testing version “A” versus version “B”. These tests, albeit effective, were difficult to scale in measuring multiple elements on a page.

Enter multivariate testing; a new form of testing based upon technologies that allow for the measurement of multiple aspects of a webpage.

Multivariate testing is the process of creating varied “recipes” of web content, that are served to your visitors in order to decipher intelligence into which mix of messaging and imagery works best. In the past, this type of solution was only available to the very large enterprises who had substantial dollars to spend on additions to their content management systems. As eCommerce and internet marketing has progressed however, so has the availability of tools that execute these conversion improvement functions.

Simply put, there is no reason why your organization or business should not be improving your conversion rates through online optimization.

Why you need to execute multivariate testing

Testing is a fundamental aspect of direct marketing. Ask any cataloger over the last 100 years. They have been testing covers, layouts, imagery within the print medium as long as the market has been active.

For a print business, the goal is clear. Test different variations of catalogs in order to isolate the best performing version for upcoming campaigns. Seems pretty basic and simple right? Well, it is, but 90% of businesses that I talk to on a daily basis are not involved in testing in the online medium, even within the available tools today.

Think of your website as a brochure that has 2 seconds to peak someone’s interest. That is the exact environment that exists today. With homepage and key landing page bounce rates exceeding 40% many times, the lost opportunity with not testing is substantial.

Be it your unique value proposition on your homepage, messaging that relates to your key services or products, or just the shape and size of a button that you want to test, multivariate testing will help answer questions about your users and web content with hard data not gut feelings.

How it works

Most multivariate testing solutions are based upon Javascript technology and the “injection” and “swapping” of content within a page. The most widely used multivariate testing platform is Google Website Optimizer and this tool is also based upon Javascript.

Basically, each area of your webpage that you want to test is encapsulated with Javascript within the test set up process. Alternatives to each location are defined within the test set up and you set goal pages to accurately track which version performed best.

As users come to your site, they get served different versions. The version that they received is populated within their cookies, this way on subsequent visits they receive the same version.

A variety of metrics can be utilized to gauge performance with the most popular being conversion. Other metrics that can be leveraged are time on site, pages per visit, or the aforementioned bounce rate metric.

Challenges with multivariate testing

As in any technology solution, there are always challenges. With multivariate testing, challenges come into play with content management systems and eCommerce platforms. Because these systems are based upon templates and the use of databases to render content, some additional work needs to be done to ensure that tests function properly.

Also serving as a hurdle can be the “tagging” process that needs to be done regarding the site elements on a page. In situations which software is licensed and access to the source code is not readily available, the tagging process needs to be done by a software vendor and can equal headaches and expensive per hour fees for remedial steps.

Regardless of these challenges, multivariate testing is something that is a must if you operate a web-based or eCommerce business. To not test different types of layouts, promotions, and calls to action is flying blind and leaving dollars on the table. Complacency is the enemy of efficiency and don’t be complacent with the conversion rate of your website. Start multivariate testing today.

About the author

Craig Smith is the CEO of Trinity Insight – a leading eCommerce consulting company that specializes in multivariate testing.

photo credit: gui.tavares

When I first heard about Magento eCommerce I got excited. Very excited.

For years I had been using a modified Zen Cart codebase which worked well, was easily hackable but constantly frustrated me with bad system design, poor templating, low standards compliance and a bickering development community. It was stuck in the past but I made it work with my own modifications over time.

Magento appeared on the scene at about the time everyone was quibbling over what Web 2.0 means. It was a time of shiny new applications, short & brandable urls, gradients and buttons. At first it seemed too good to be true; an MVC approach (using Zend framework), latest technology, well thought out features, upgrading platform, company backed and developers not afraid to use the latest versions of PHP and MySQL.

I installed an early Beta, played with it a little, loved the features but I didn’t give the codebase much thought. It was strange. A lot of files, a lot of directories, a completely OOP approach (while I find OOP useful, I had never seen it used so strictly and extensively) and an unusual use for XML. I accepted this as a learning curve and decided to use it on a commercial project when the opportunity arose.

I must stop here and make clear that I don’t intend to cruelly pull apart any particular methodology, application design or any other development decisions. My beliefs are that these kinds of decisions depend on a variety of factors including:

• The skill set of developers working on and with the system
• Technological constraints (server technology etc.)
• Intended use of the system (private, open source, licensed etc.)
• Lifespan of the system
• Type of application

Therefore I am saying it is not a case of “which methodology/programming language/framework etc. is best” but more a case of “what is this project and what technology will suit it best”. As I will explore below I believe Magento made decisions based on what would be cutting edge, fresh and idealist instead of looking at what would suit a new open source eCommerce system.

So back to my story and I had found an opportunity to use this new system. A project requiring product filtering, high numbers of orders, one page checkout (which Magento does beautifully) and many addons that would require a lot of hacking in Zen Cart. I took the plunge and went with Magento.

Not long after getting started on the project I realised that the application design didn’t “fit” with me. I have worked with a lot of custom & open source web applications, database systems, retail software, warehouse management software and I always managed to find an understanding with them. No matter if the code resembled long, thin, cylindrical pasta of Italian origin or if it was written in a foreign language I managed to get my head around it and work with it comfortably.

With Magento this just did not happen. I could not understand what decision process resulted in thousands of directories and tens of thousands of files (a lot of which contain empty class declarations). I couldn’t figure out the pattern of where code went and what the abstract folders called “Convert”, “Entity”, “Layer”, “Resource” etc. are for. The naming convention for folder names, model names etc. didn’t sink in – sometimes you needed lower case, sometimes camel case and sometimes one capital letter.

Ultimately, I look at the results. It took me 5 times longer to perform any task in the code (template changes, adding features, debugging etc.) and even magento specific consultants agree with this. Documentation was sketchy at the time but I don’t believe this matters. Apart from quick references I have never read a manual, watched a video or anything else when trying to figure out a new system. I follow the code, look at the file structure and experiment.

Due to the Zend/OOP/MVC influence on Magento it is impossible to follow the code. Classes are referenced dynamically, various aspects are contained in XML files and there is no clear flow that you can just debug through. The sheer volume of files and folders makes finding something unbelievably tedious.

Even the database is a minefield. In every other system I have used finding data is easy. In Magento, the use of EAV means that data is split amongst hundreds of abstract tables. Again, it doesn’t flow and it doesn’t make sense without a great deal of time developing a solid understanding of what they have done.

All of this and more ate away at me until I began to resent working on the project. I concede that this might be isolated to a minority like me, but I do know I am not the only one to have problems. Different freelancers I employed to do custom work didn’t follow conventions and hacked into the core (causing upgrade problems, which will be discussed later). I regularly see forum threads discussing the same issues and blog posts expressing their concerns.

The common argument I see are along the lines of “the web is evolving, applications are getting more complex” and  “for more features we need more structure and this was necessary with Magento”. I even saw one elitist claim that a system like Magento is “proper PHP development” whereas systems like WordPress are for “PHP Hackers”.

Essentially I disagree.

Yes, Magento is an advanced system and I am sure once the rules are learned and understood these application design decisions make sense. Unfortunately I do not believe the time needed to understand (and continue to understand) is worth the potential advantages.

As I touched on before, application design depends on the facts. Here is my take on the facts applying to Magento and the community that will use the system (or any modern open source eCommerce system):

1. Potential developers will have a wide variety of different skills
2. Potential developers will have varying ability and experience
3. Potential developers will have different opinions on coding standards, application design, templating etc.
4. Profitability (e.g. support, hosting, addons) requires a good uptake and community
5. Hosting technology will vary massively
6. Open source developers will want to contribute code and participate in a community
7. Due to the free nature, many people will want to quickly try the application
8. Free software does not benefit from the same attention span as paid software
9. Maintain reputation of security and progression

On looking at the factors involved, I believe these are good aims for the project as a whole:

1. Code set must be easily picked up by developers from varying backgrounds
2. Code base must be intuitive for “Newbies”
3. Coding standards must not be elitist or provoke time wasting debates and indecision
4. Must be focused on good uptake (easy to install, get started with etc.) to allow commercial opportunities
5. Must be forgiving to different hosting configurations
6. Easily allow sharing and developing addons. Addon system must be logical and easy to understand.
7. Must allow quick install and trying out of the application
8. Must not require excessive reading of guides to get started
9. Allow easy updates to ensure security and new features are rolled out. Discourage “hacking” the code.

From my experience most of these points are affected in some way by the need to use this complicated approach to code. When the rules are too abstract hacking occurs. When the code is hacked upgrades are a big job. When upgrades are difficult, upgrades do not happen and new features/security fixes aren’t implemented.

Hosting is also a notorious problem for Magento; fussy configuration means many users complain of speed issues. The only hosts running Magento well are the ones with good knowledge of the system – this is an unusual scenario for a hosting company.

From my experience, working with Magento requires a lot of learning and re-learning. This just isn’t practical for your average web developer simply because of time & budget constraints. It has nothing to do with skill level, ability to learn or unwillingness to try new methods. It is about pragmatism over perfectionism.

By being pragmatic I mean finding an optimal solution to the problem without doing the unnecessary. I have been down the perfectionist path and it did not work for me. I now work to goals instead, to much more success. Ultimately what is better; achieving your goals within budget or using the best technology with the latest programming techniques and methodologies?

What would have been a good approach to Magento? In my opinion, the development hours put in to a solution like Magento could have been better spent finding a good structure that is easily understood by the majority of developers. The structure might not be perfect, it might not follow academic methodologies and it will probably draw criticisms from purists.

But ultimately if it achieves your goals, it is a success.

A great example of this is WordPress. I have often thought the code base is dodgy by professional standards, but it works. I have grown to love it because I know if there is a problem I can find the section of code easily. The structure makes sense to me and I never feel the need to hack the core. The amount of plugins avaialable is a testament to the success of the system, as is the number of websites using WordPress and the successful commercial arm.

WordPress is achieving its goals.

A final Cynical Note… Many people claim that the complexity of Magento is somewhat intentional. The profitability of Magento relys on consulting, technical support and installations. Making the codebase complex could mean that many developers will start out, get stuck and pay for help. If this is the intention of Varien then perhaps they have been very goal focused, although I doubt this plan would have long term appeal. Personally I believe that Magento is simply the result of focusing too much on cutting edge methodologies and not enough on what they actually want to achieve with the system.

Comments, thoughts, opinions or complaints? Leave a comment below…