Not Everything In Marketing Can Be Measured

In a recent article for martech.org, Christopher Penn wrote, "Everything in marketing is measurable, from top to bottom, from brand to customer satisfaction to purchase - you can measure 100% of marketing." He argued that when people say some aspect of marketing can't be measured, they mean that, ". . . they don't have the budget and resources to measure what they care about." 

Christopher Penn is the Chief Data Scientist of Trust Insights and the author of AI For Marketers:  An Introduction and Primer, which I recently reviewed. He's a recognized authority on artificial intelligence and data science, so I'm reluctant to disagree with him on anything relating to marketing analytics. But, I have a different view on this issue.

Every marketing channel, tactic, or program can be measured in some ways, but not everything about every marketing activity is measurable.

Despite what you've heard and read, you can't actually measure the financial impact of most individual marketing channels, tactics, and programs. The financial performance of these aspects of marketing can be modeled if you have the right data, tools, and skills. However, the information produced by a model is qualitatively different from the information obtained through measurement.

Measurements vs. Models

Measurement can be defined as the process of quantifying some property of an object or event. In marketing, we can measure the number of visits to a website, the number of e-mail opens and click-throughs, and the number of times a particular content resource is downloaded.

We can also measure the duration of events, such as how long someone spends on a web page. The common denominator in these and all other measurements is that they are based on observations of actual objects or events.

A model, on the other hand, is a simplified representation of a process or system created using statistical principles and mathematical equations. The usual purpose of a model is to describe the current or past behavior of a system or process or to predict its future behavior.

Because models rely on statistical rules and mathematical calculations rather than observations of actual things or events, model outputs are inevitably approximations or estimates of what exists or occurs in the real world.

A model that is properly designed and trained using a sufficient amount of the right data can produce reasonably reliable outputs, but there is always a "margin of error." In short, a statistical model, despite the appearance of mathematical precision, will always be more like an impressionist painting than a photograph.

Why Financial Impact Can't Be Measured

The financial impact of most individual marketing channels, tactics, or programs can't be measured because it can't be observed. To determine the financial impact of these aspects of marketing, you would first need to determine how much of customers' buying decisions were due to the channels, tactics, or programs you're trying to value. And there's just no reliable way to observe that causal effect.

It's tempting to think you could use a survey to ask customers about the impact of various marketing channels, tactics, and programs on their buying decisions. After all, customers should be able to tell you what caused them to make a particular purchase decision. To understand why this approach doesn't work, consider this thought experiment.

Below are the ingredients used to make Nestle's famous "Toll House" chocolate chip cookies:

• 2 1/4 cups all-purpose flour
• 1 tsp baking soda
• 1 tsp salt
• 2 sticks butter
• 3/4 cup granulated sugar
• 3/4 cup packed brown sugar
• 1 tsp vanilla extract
• 2 eggs
• 2 cups Nestle Toll House Semi-Sweet Chocolate Morsels
• 1 cup chopped nuts

Suppose that these are your favorite cookies - you really love them. If I gave you this recipe and asked you to tell me how much of your enjoyment each ingredient is responsible for, how would you answer? What percentage of your enjoyment is due to the flour? The salt? The brown sugar?

I suspect you would say my question is impossible to answer, and you would be right. That's because your enjoyment - the "value" of the cookies - arises only when all of the recipe's ingredients are combined, and there's no accurate way to attribute a percentage of the enjoyment/value to individual ingredients.

The same is true of buying decisions. A customer's decision to buy results from the combined effect of all the interactions and experiences that occurred during the customer's buying cycle, and there's no realistic way to observe the mental impact of any individual interaction or experience. And, since those impacts can't be observed, they can't be measured.

The financial impact of individual marketing channels, tactics, and programs can be modeled using marketing mix modeling and multi-touch attribution modeling, but marketers need to remember that the outputs of such models are only approximations of reality and treat them accordingly when making decisions.

Image courtesy of Pat Pilon via Flickr (CC).