Finding Patterns, not Needles, in Haystacks

Posted by Kevin Lyons on December 3rd, 2013 at 8:55 am

Many correlate data analytics with “finding a needle in a haystack.” I’ve never been a subscriber to the analogy. In fact, I strongly disagree with it, as it seems to bestow analytics with the ability to pull meaning out of sheer randomness.

To me, analytics is not about magically finding a random small object in a big pile of hay. It’s about finding and understanding patterns and relationships between many small items.  It’s about understanding how the needle is interconnected with all of the individual straws of hay and how all of the straws of hay are interconnected to one another in a structured and systematic way.

And, at its best, predictive analytics finds robust, predictable patterns where none at first sight seemed to exist at all.

What can predictive analytics accomplish?

Consider a brand of soda that falls into the consideration set of 2% of the US internet population.  A random run-of-network (RON) campaign directed at 1 million unique users will therefore correctly target 20k people and incorrectly target 980k, being “wrong” 98% of the time.  If each of these 20k individuals spends $5, then the campaign will generate $100k.

Now, let’s say that predictive analytics can produce a targeting campaign that is wrong “only” 96% of the time, an improvement in accuracy of only 2% and still “wrong” most of the time. Yet, the company’s revenue will grow 200%, as now 40k individuals will spend $5 for a total of $200k. That additional $100k, based upon a small improvement in analytics, represents a staggering 100% improvement in campaign results.

Done right, smart marketers can use predictive analytics to solve a wide array of problems around understanding customers, prospects and the competition. You can learn to decrease risk and fraud, deliver more one-to-one experiences and make better marketing decisions.

How is performance measured?

Brand or Action Lift is the most common way to measure advertising analytics. It’s a measure for comparing the performance of the analytically targeted group against current targeting methods.  In its simplest form, it’s very easy to calculate. For example, if current methods produce a 0.002% conversion rate and the analytically targeted group converts at 0.008%, then the lift is 0.008% divided by 0.002% or 4x.

Lift vs. Scale and Accuracy

Lift and accuracy are related, but they are not the same thing. In the above example, predictive analytics produced a very healthy 4x lift, and yet the model was still “wrong” (or inaccurate) over 99% of the time, having a response well below 1%.

And whether to stress lift or accuracy is driven by business needs. For years, companies have grown and prospered with direct mail acquisition response rates hovering somewhere around 1%.  In other situations, however, a decent lift may not sufficiently fulfill the business requirements at hand; this is generally true when modeling out demographics for brand advertising campaigns, for example.

In addition, when targeting an audience for a marketing campaign, there is always a pull from two opposing directions: lift and scale. In order to maximize revenue, you need to juggle between casting a wide net and minimizing the number of misidentifications.  It would be easy to create an audience with immense lift by limiting oneself to highly-targeted remarketing campaigns. Unfortunately, in doing so, a vast portion of the market would be excluded.  On the other hand, it’s equally easy to create an overly loose selection criterion and target a very broad audience that would hardly ever respond to your campaign.

Defining Overall “Success”

There are many ways to measure success, depending upon the business goal. But, it is often defined as achieving a lift that pays for the total cost of developing and implementing an analytics project, plus some premium.  Return on Investment (ROI), which is calculated as (gains from marketing – cost of marketing)/cost of marketing, is often used to judge success.

Avoiding failure

Unfortunately, many analytics exercises fail to prove their value because someone forgot that prediction is just one part of a process that begins with business acumen and ends with implementing the findings. Unless you fully understand the business, collect the best available data and fully operationalize the results of your analyses, you’ll never achieve the success you demand.

Leave a comment