Test, Test and Test some more: The importance of A/B & Multivariate Testing
Some may not like it, or know how to do it, that is why you have Solter Consulting, but Testing is one of the most important ways to know what is working well or not, and how to improve.
“By leveraging the power of experimentation and testing tools, you can run experiments on your site and let your customers tell you what works best.” (Kaushik, 2010, 8)
Quantitative-capable of being measured or expressed in numerical terms.
- Comparison: A/B Testing vs. Multivariate Testing (MVT)
A/B Testing is basically using two versions of the same page and assessing which version receives a better reaction by either drawing in more unique visitors or getting these visitors to engage in the desired outcome. The pros of A/B Testing are that it is fairly easy to do and extremely cost effective. It is a great starting point for flushing out the best version of your website pages.
The downside or cons to A/B Testing include a bit of difficulty deciphering exactly what change to the page layout made the difference. No doubt, you may make several changes and there is no way to determine what elements made the most difference. Kaushik cautions us that A/B Testing could mean you are making changes too slowly (Kaushik, 2010, 198). Life on the web is ever changing. It is fast and furious!
- Approach: One of the first things I will want to do on my website is an A/B Test of the top landing page with the highest bounce rate. I will then create an alternative page and see how it fairs. I will be sticking with A/B Testing initially, so it may be worthwhile to test my search, email and comment pages as Kaushik advises. I am trying to get more feedback, engagement and loyalty from visitors, so my subscription page and comment pages may also need good analysis/testing to optimize.
Qualitative-quality or character of something.
- Comparison: Usability Studies vs. Online Surveys
Usability Studies help us understand the customer and the why of their actions.
Pros of the usability study are that there are web-based versions that are fairly inexpensive to conduct and can be completed quickly. These tests are golden in that you are face to face with your customer and that is rare for almost any web-based business. Some of the cons are that lab usability tests measure a user’s ability to complete tasks and are a much more involved process and costly. You need a room, subjects and a test administrator. Once you have gathered information, you still need to analyze the data. Kaushik cautions that you will only have a small sampling of the population you serve and thus may skew the data. Ultimately, the effectiveness of this testing lies in eliminating the worst ideas of those presented.
Online surveys-customer/visitor self-administered questionnaires help us to understand the why and give the customer a voice.
There are many pros of the online survey. They are quick, easy and affordable.
The downside or con of surveys is that you will only get a certain type of visitor to answer the survey; therefore, you will not have the complete picture. “Page-level surveys are not good for collecting feedback about intent or complete site experience.” (Kaushik, 2010, 181)
I think it would be fun and easy to create a survey and pop it on my website to see what brought visitors to my site. I would like to hear their opinion about what they enjoyed or did not enjoy. I might place a page-level survey after a blog post to see if the content was relevant and of interest, so I can adjust my topics accordingly.
I also like the site-level survey approach to collecting complete data about customer/visitor motivation across the site. Kaushik suggests a pop-up window with 25-28 questions. I, like many, am annoyed by pop-ups but, it is a necessary evil in order to glean quality insights for the future direction of your website.
Give us a call, we will get you where you need to go! Solter Consulting