TV Really Is Easier to Measure than Social Media
July 28, 2011 5 Comments
KMart’s CMO made waves recently by saying TV, and other traditional channels, are easier to measure than social media.
Social media proponents responded, defending social media’s measurability and highlight the data social media provides. My favorite was this tweet from Jay Baer (who I have a tremendous amount of respect for and I don’t disagree with lightly).
http://twitter.com/#!/jaybaer/status/93742009034084352
The problem is, most social media measurement is myopic. It focuses on direct interaction and ignores the fact it is just one part of a larger communications mix. My verdict: KMart’s CMO is right.
Using media mix models, companies like KMart assess incremental business impact, accounting for the fact that TV is one of many channels. Unfortunately, this approach doesn’t work with social media today. Here’s why:
Scale
TV has scale at KMart today and larger changes are easier to measure. From this perspective, if TV isn’t easier to measure, KMart has a major problem.
The solution, with a hint of sarcasm, is simple: make an 8-digit investment in social media. But spending $40 million on social media, starting with today’s first baby steps, would be a major challenge. Doing it effectively, without history to learn from, would be a daunting initiative for almost any CMO.
Unit of Measurement
A model requires a consistent input. For TV, the input might be GRPs by target audience and DMA. Social media does not have a similar measurement today. In fact, social media can’t get close. This is a gap social media measurement companies need to help close.
The hub and spoke social media model, with an owned media property at the center, can capture granular audience data from the hub isn’t more broadly available in social media.
Stability (see the next section) will be a challenge with hub activity, but if I were a CMO at a large company, I would at least consider testing data from a hub as a proxy for overall social media reach.
History
A model requires history to provide meaningful results. Two years is usually the minimum, five is far better. It will be another year before most companies can realistically attempt this.
Models also requires stability in the data used. Social media is changing rapidly. Even with a consistent unit of measurement, would it be consistent over time? Would a 2008 social media impression be comparable to a 2011 impression? Early attempts to include social media in a marketing mix model, even once history is available, will initially struggle because of this.
In the meantime, correlations and interactions between channels can begin to be assessed. But when looking at overall business impact, these results are the color commentary, not the definitive answers.
The Challenge
Clearly, these changes will not happen overnight. Organizations that embraced social media two years ago, on faith or based on direct measurement, have a huge head start. Many are seeing results, in anecdotes and direct activity, and making the case to continue investing.
Social media measurement continues to develop, but the continued reliance on clickstream and social monitoring generates mountains of data without getting to real incremental business impact. This approach is plagued by two systemic measurement problems:
- Undermeasurement: It ignores influence communications have beyond an individual. If someone emails a link or tells a friend, it is unaccounted for.
- Overmeasurement: It disregards the influence of any other communication activity. It doesn’t consider the possibility that someone saw a TV commercial and responded by visiting your Facebook page or saw a recruitment billboard and looked up your company on LinkedIn.
Human behavior isn’t cut and dry. Using clickstream data implicitly assumes it is.
Your Turn
What will it take for major corporations to assess the impact of social media on top line and bottom line business results? Is there an alternative to marketing mix models? Is there a way to control social media distribution in order to test within paired markets?
Or are the current tools sufficient, and as Jay’s tweet implies, I’m just not trying hard enough? Share your thoughts in the comments below or with me on Twitter.
About Eric Wittlake
I am a digital and B2B marketer with a background in online media and analytics. I work with B2B clients on media and integrated marketing programs. You can connect with me on Twitter at @wittlake or in the comments here on my Digital B2B Marketing blog.
I am a digital and B2B marketer with a background in analytics and online media. Based in Portland, Oregon, I run the media group at 
Digital B2B Marketing 
Oh Eric, my man. I think this is the one of the toughest issues in B2B Digital marketing. So strap in (I had 3 cups of coffee this am.) I think I’m game to try and respond…
I tried to address it with my post on Marketing Attribution: Who Gets Credit for ROI (http://www.b2bmarketinginsider.com/strategy/marketing-attribution-who-gets-marketing-roi-credit) where I suggest we need to look at all companies that were touched by marketing campaigns that ultimately became sales-opportunities. Then I suggest looking at all the contacts from those companies and all the marketing touches made on those contacts. Finally, you need to assign a relative (and imperfect) value to the activities that correlated to pipeline dollars. This will help to get to a better view of Marketing ROI for Demand Gen activities.
The problem with my suggested approach is that it only calculates ROI for the marketing touches that can be assigned to a contact. So in your example, TV and Social Media would both miss out on getting any credit. And I know that lots of investment in social or TV with an effective approach will yield results and influence buyers.
An alternative approach I would strongly suggest is to conduct detailed customer media consumption research and ensure that your marketing mix models line up at least in relative priority to the media our target consumers are spending the most time with. Now this doesn’t consider effectiveness directly so you will need other models like pre / post unaided awareness studies and probably some regression and multi-variate hocus-pocus to get a better view on what really works.
Bottom line is that this is all very hard, especially when you look at the bigger marketing mix question. I agree with you that KMart is right that it is easier to measure TV (and direct mail and email and paid search) but that doesn’t mean that we should only do the things that are easy to track.
In the end, for me, it comes down to this: our customers use social and we need to be there. If we are there and can figure out how to be effective, we can influence buying behavior. If we’re not there, we’ll never have the chance and will miss out on a huge opportunity.
So I just wrote a book report. Apologies but hope this helps to clarify my view.
Best, Michael
@brennermichael
Eric,
First of all, when are you going to move over to WordPress.org?!?!
OK, on a serious note, I think there are two things going on here (and sorry in advance if this comment doesn’t brush over some of the issues you brought up):
1) Agreed that TV is easier to track than social media. Just ask the Nielsen folks. Of course, you can only track it if you have a call-to-action (phone campaigns work best) or if you only do it for a period of time. Social is everywhere all the time – have you ever seen a report on your tweet’s market share of tweet time? Of course not.
2) As for Jay’s note, I think it is a knee-jerk reaction to the fact that social media _is_ also measurable in terms of the tracking that can be done across digital properties.
The big difference I see is that TV is one to many at the same place and time whereas social is spread out all over the place in terms of timing and place. The calculations of a TV campaign are much more crisp to define unless you have an extremely viral social campaign.
Michael made a good point that, at the end of the day, we need to be where our customer is, which i totally agree.
So it’s not about TV vs. social media. They are both important, and neither is going away. Marketers need to understand where their target audience is, what makes them tick, and how to measure their effectiveness of marketing budgets. Traditional media has decades of infrastructure and experience to allow marketers to do this. With social media, it is coming. Either way, when dealing with new media, it takes a very holistic approach mixed with a lot of creativity as well as some experimentation to gauge the effectiveness of things. Marketers will need to become less digital and more analog. Might I suggest that marketers need to become artists?
Keep up the good thought-provoking blog posts 😉
@NealSchaffer
Wow, Michael and Neal, thanks for taking the time to share your thoughts on this topic, I really appreciate it.
It is safe to say, we all believe in social media. We see the research on the audience, and more importantly, we have participated and experienced first hand the opportunity to establish valuable mutually beneficial relationships.
As Michael points out, it is a real challenge to get the type of business ROI that both of you regularly make a case for. Even with attribution modeling or ethnographic research, you haven’t closed the loop to business impact.
Hopefully though, improvements are on the horizon. As investments scale, hopefully we will find a stable unit of measurement. I believe with a unit of measurement, some of the sophisticated media mix practitioners will find ways to assess the bottom line impact of social media that will stand up to CFO and Board scrutiny at large companies.
Thank you again for taking the time to share such great comments, I really appreciate the feedback (and the wordpress.com dig!)
What gets me about arguments/discussions like these is why does someone have to be in one camp or the other? Who cares which one is easier to measure? Does it mean the other is obsolete? Of course not. Each is not mutually exclusive.
I don’t care which one is easier to measure. I use them both for my clients and I use them for different reasons. Get over this XXX is better than YYY stuff.
Jay, glad you took the time at least to read the article. I’m with you, I don’t care about X versus Y, but I do care about how different disciplines learn from each other, and how we assess the potential bottom line impact of emerging tactics like social media.
You mention using social media for different reasons that TV. If you are willing to share, what reasons do you see social media most appropriate for? How do you measure the overall impact, or are intermediate measurements sufficient? Would be interested if you are willing to share.