How standard email marketing metrics fall short

When you’re trying to optimize the profits of your business, most web metrics are unhelpful at least, and deceptive at worst. But what about the world of email marketing ? Kevin Hillstrom, in his excellent Mine That Data blog, gave this example to illustrate how conventional email metrics look at the wrong things:

Say you have a list of 500,000 e-mail addresses. You send your standard campaign on a Monday. Later in the week, you tabulate your results:

  • 500,000 recipients
  • 20% open rate = 100,000
  • Of the opens, 20% click through to the website = 20,000 visit website
  • Of the clicks, 5% convert and buy something = 1,000 orders
  • Average Order Value = $100
  • Total Demand = 1,000 * $100 = $100,000
  • Demand per Recipient = $100,000 / 500,000 = $0.20 [per customer]

He compares these finding with what you’d get if you did something called a mail/holdhout test. You compare a control group that does not get the email with a test group that does. For instance, he suggests this breakout:

  • Mailed Group = 400,000 Recipients, $300,000 spent = $0.75 per customer
  • Holdout Group = 100,000 Held Out, $45,000 spent = $0.45 per customer
  • Incremental Lift = $0.75 – $0.45 = $0.30 per customer

Much more insightful!

This is why I’m not a fan of open/click/conversion. A mail/holdout test proves the actual value of an e-mail marketing campaign. In this case, we observe $0.30 lift, whereas open/click/conversion yields $0.20 lift. E-mail marketers, why would you not want to know that your campaigns are working 50% better than when measured via opens/click/conversion?

Kevin goes on to provide other interesting observations from his years of doing this sort of testing. You can’t go wrong by following his blog, and trying his approach to data-driven online marketing.

Can you guess the winner in this A/B test?

I’ve been following the site Anne Holland’s* “Which Test Won?” for a while, and was particularly intrigued at this test, for several reasons.

As you’ll read if you take the test, one achieved a better email open rate and click-through-rate, while the other was more effective at inducing purchase.

Can you guess which is which, and why?

I guessed correctly. One would hope, given my background. You can read my rationale in the comments section, which becomes available once you vote. Don’t be disappointed; at least when I took the test, a clear majority guessed incorrectly.

Full disclosure: Sony Creative Software — and Kevin St. Angel, their director of ecommerce extraordinaire — was a client of mine in a “past life.” My policy of never talking about client work in this blog doesn’t come into play here.

It’s not that they are no longer a client. That wouldn’t matter. But since Sony has made this test public I feel free to discuss it and encourage further discussion on Anne’s site.

Good luck taking the one-question test!

* If Anne Holland’s name sounds familiar, it’s because, among her other accomplishments, she is the founder of Marketing Sherpa.

What it takes to go viral

The very thought of an online effort going viral seems like the ultimate triumph: The ROI on a successful viral campaign is huge. Take a fast, meteoric ride on the exponential pass-along curve and you’re looking at the marketing equivalent of striking it rich on a single lottery ticket. So why doesn’t it happen more often?

I see at least two factors standing in the way of an existing brand actually lighting the fuse: Relinquishing control and a sincere willingness to lighten up.


It takes a huge leap to create something and set it free. In a nutshell, that’s the dilemma of using social networks to further a brand — the potential for chaos. Once in the hands of the social graph, a concept can take on a life of its own. And with independent life comes the chance of betrayal.

No one wants to see their viral effort hijacked and distorted into mockery. Unfortunately, it’s this very tension that make viral efforts so fun to watch.


E.B. White wrote the following:

Humor can be dissected as a frog can, but the thing dies in the process and the innards are discouraging to any but the pure scientific mind.

It’s a pity, then, that something as ephemeral as humor is at the heart of a successful viral effort. But it’s true, and in the hands of a committee-driven brand team, a funny concept all too easily becomes so many frog innards. Humor is that fragile.

It was actor and director Sir Donald Wolfit who was reputed to have said on his deathbed, as a parting reassurance, “Dying is easy. Comedy is hard!”

So are concepts that succeed in going viral.

This post was inspired by one that I received on Friday that is an unqualified success. I received a get-out-the-vote email from a friend in the form of a personalized video. It got the point across and also elicited several heartly laughs. Try it yourself on a friend (of the right political inclinations — it’s from

I’d also like to challenge readers: Do you know of any viral efforts that do not, to some extent, involve humor?

Why don’t newspapers tout their power to sell across channels?

One thing that separates humans from other creatures is our ability to use the same tool in different ways. The ultimate example is the computer, which has hundreds of uses. But even a doorstop can make a pretty impressive paperweight when push comes to shove. So why is it so tough to sell a print ad to serve a new strategy? I’m thinking of its use as a cross-channel tactic.

Is it that the typical ad rep isn’t attuned to this medium’s use? Or is it that the typical ad buyer wouldn’t warm to the new tactic even if it could help turn a mediocre print campaign into something extraordinary? As usual, it appears the marketers on both sides of the desk are clueless, and the consumers are the only ones arriving at the party on time.

Research done using Google Print Ads activity, and conducted by Clark, Martire & Bartolomeo, found that consumers definitely do not look at newspaper ads in a vacuum. They often use the web to evaluate and purchase. This research focused on a segment of consumer who tends to research products and services seen in newspapers. My guess is this could be a consumer looking for any considered purchase, where the resources risked by a bad decision are significant.

Not surprisingly, two-thirds use the web in their research. What was noteworthy was that of this group, 70 percent say they went on to make a purchase following the research. Although this is self-reported, it shows the pathway that many multi-channel purchasers take. (Which explains why Kevin Hillstrom is smiling broadly in the picture on his blog!)

I see three take-aways:

  • Any newspaper advertiser that doesn’t have a strong web presence is wasting money
  • Any web site that isn’t fully optimized for organic search should be considered a defective site, since researchers may not use a URL printed in an ad to do the research
  • Ad reps should be pushing harder on selling the off-line / on-line tactic, whether through unique URLs printed in ads or more innovative tactics (think mobile research), such as ShopText.

I just came out of a lunch meeting with two ad reps for a national weekly newspaper. No matter which way I probed, it was clear that they weren’t selling — and ad buyers weren’t buying — multi-channel ad strategies.

Here’s a press release on the study on the Newspaper Association of America web site. Let’s hope the ad sellers — and buyers — read the study and take heed. Consumers are waiting to google the next item they see in print.

Neurotic? Narcissistic? We knew from your email address

A recent study of 100 college students showed that strangers can accurately guess traits like openness, agreeability, neuroticism, conscientiousness and narcissism by looking at a person’s email address. The study was published in The Journal of Research in Personality, as reported today in NewScience (subscription required).

The panels’ guesses agreed most with a personality survey completed when it came to qualities like openness, conscientiousness and narcissism, and diverged most on the trait of extroversion. Addresses that gave away personality often contained full stops, numbers or a name that was obviously not genuine.

It’s interesting how much we can reveal in a few characters and numbers. However, the study did look at the email addresses of teens. Perhaps we get better at hiding our personalities as we mature. Don’t you think? Let me know, at