Websites

The Real Cost of Do-It-Yourself Testing and Optimization

Posted by Mark Simpson on December 5th, 2012 at 5:48 am

Do-It-Yourself anything is hot, hot, hot right now. Just look at Pinterest—there’s more pins of arts and crafts ideas than anyone has time (or creativity) for. It’s no different in the technology world either—where better, faster, easier and cheaper is king. And thanks to the plethora of free technology platforms out there, people and companies are becoming instant online professionals — tackling everything from publishing, website programming and mobile development to SEO, behavioral analysis and content production.

But what gets lost in translation somewhere between identifying a business need and the free technology that promises to satisfy it is the fact that technology is only one part of the equation when it comes to doing things right—and cost effectively.

The pros and cons of free technology solutions are different, and the benefits of using them will ultimately depend on the quality of the platform itself and/or the skillset of the people on your team. But no matter which type of free (or cheap) solution you’re considering, there are likely a number of hidden costs you should prepare for—if your goal is to achieve professional-level results.

Since I reside in the world of multivariate and A/B testing, I’m going to take you on a tour of the hidden costs of “free” and “cheap” as viewed through the lens of DIY solutions in my industry. These things are no doubt easily applicable to free technology solutions across a number of industries, however.

You Get What You Pay For?

To give you the lay of the land, there are three basic grades of testing solutions (in terms of cost): Free ($), Mid-Level (US$ 100 to $US 1000/month), and Enterprise Level (US $5,000 to US$ 50,000/month).

All of the testing solutions out there — no matter what their price — promise similar results: optimize your website, convert more visitors into buyers and increase overall revenue as a result. It sort of makes you wonder: if they all achieve the same things, why not base your decision on price? Because the upfront price tag leaves a lot to the imagination…

Going Beyond the Price Tag

No matter how low-cost the tools you’re using, there are a number of variables necessary to achieving the results you want — things like strategy, expertise and general technological know-how. To determine the actual costs of the technology you’re using, don’t just look at the sticker price; consider the money you will save — or waste — using this formula:

TECHNOLOGY PLATFORM
+ TIME
+ DIRECT COSTS
+ OPPORTUNITY COSTS
+/- RESULTS
= COST OF DOING THINGS RIGHT

For our purposes, here’s how each of these things factors into the real price of testing, although all easily apply to determining the cost of doing whatever it is you’re doing.

1. Testing Technology
(See “You Get What You Pay For” above.) In many cases, free testing platforms can provide quite a bit of value, but they’re only as good as the people using them. So if you decide to use a free or cheap solution, remember that its price tag only applies to the technology itself, not the other mandatory aspects of testing—such as coding, design, strategy, QA, implementation—you name it.

2. Time
Deciding to conduct your testing internally is really making a commitment to spend your and your employees’ time on becoming experts in an area that’s not otherwise valuable to your core operations. It also means that getting the program up and running is going to take away from time you could be testing.

Unless you have a team of trained testing specialists on staff, you should not plan to rely on your internal team members to handle the bulk of your testing activities. Why? Because it’s a waste of their time and yours. Plus, who’s going to do the training? You? And finally, who’s going to do their jobs if they’re busy testing? Hmm?

3. Direct Costs
Many testing solutions have built-in fees that account for the time and expertise necessary to get the testing started, as well as maintaining it over time. These ongoing fees afford you and your team the ability to be hands off and focus on what you do best.

We’ve found that companies that use external consulting services as part of their enterprise-grade solutions have experienced 6 times more uplift in conversions than those running DIY testing programs.

4. Opportunity Costs
Because testing ultimately leads to conversions, and conversions lead to revenue, missing out on testing opportunities means missing out on money.

More thorough testing produces more thorough insights, which an experienced testing expert will translate into a site optimized to meet your business goals. Most remedial solutions, on the other hand, provide only basic testing tools.

Also, if you’re relying heavily on internal team members to carry out your testing initiatives, you should assume that they’re not only dropping the ball on their core responsibilities, but also missing out on the opportunities that emerge as a result of their testing work — most likely because they don’t recognize them or understand what they should be doing next.

Which elements might contribute to missed opportunities in the DIY testing arena?
• Lack of dynamic updating abilities
• Inability to conduct multiple tests simultaneously
• Inability to keep up with updates that result from testing insights
• Overall less thorough testing
• Overlooking opportunities to test and optimize as a result of inexperience

• Etc.

5. Results
Some testing solutions talk a big game, but when it comes to achieving what they’re supposed to achieve, well, just remember that sometimes an expensive price tag can come with big results and a cheap price tag can come with big costs.

Don’t get me wrong, free technology platforms are often terrific tools for the people who know how to use them. But those people often don’t work for the companies that choose to rely on them. And even when they do, their energy is probably better spent on activities directly related to the core competencies of their businesses.

The money you stand to save on “free” and “cheap” is usually a lot less than the amount you stand to lose in opportunity costs and the expenses associated with using them effectively.

One Response to “The Real Cost of Do-It-Yourself Testing and Optimization”

  1. David says:

    You make some good points. Decreases expenses versus increases in revenue and insights. It is important to make sure you partner with the right team that you can communicate effectively with.

Leave a comment