In a recent article for martech.org, Christopher Penn wrote, "Everything in marketing is measurable, from top to bottom, from brand to customer satisfaction to purchase - you can measure 100% of marketing." He argued that when people say some aspect of marketing can't be measured, they mean that, ". . . they don't have the budget and resources to measure what they care about."
Christopher Penn is the Chief Data Scientist of Trust Insights and the author of AI For Marketers: An Introduction and Primer, which I recently reviewed. He's a recognized authority on artificial intelligence and data science, so I'm reluctant to disagree with him on anything relating to marketing analytics. But, I have a different view on this issue.
Every marketing channel, tactic, or program can be measured in some ways, but not everything about every marketing activity is measurable.
Despite what you've heard and read, you can't actually measure the financial impact of most individual marketing channels, tactics, and programs. The financial performance of these aspects of marketing can be modeled if you have the right data, tools, and skills. However, the information produced by a model is qualitatively different from the information obtained through measurement.
Measurements vs. Models
Measurement can be defined as the process of quantifying some property of an object or event. In marketing, we can measure the number of visits to a website, the number of e-mail opens and click-throughs, and the number of times a particular content resource is downloaded.
We can also measure the duration of events, such as how long someone spends on a web page. The common denominator in these and all other measurements is that they are based on observations of actual objects or events.
A model, on the other hand, is a simplified representation of a process or system created using statistical principles and mathematical equations. The usual purpose of a model is to describe the current or past behavior of a system or process or to predict its future behavior.
Because models rely on statistical rules and mathematical calculations rather than observations of actual things or events, model outputs are inevitably approximations or estimates of what exists or occurs in the real world.
A model that is properly designed and trained using a sufficient amount of the right data can produce reasonably reliable outputs, but there is always a "margin of error." In short, a statistical model, despite the appearance of mathematical precision, will always be more like an impressionist painting than a photograph.
Why Financial Impact Can't Be Measured
The financial impact of most individual marketing channels, tactics, or programs can't be measured because it can't be observed. To determine the financial impact of these aspects of marketing, you would first need to determine how much of customers' buying decisions were due to the channels, tactics, or programs you're trying to value. And there's just no reliable way to observe that causal effect.
It's tempting to think you could use a survey to ask customers about the impact of various marketing channels, tactics, and programs on their buying decisions. After all, customers should be able to tell you what caused them to make a particular purchase decision. To understand why this approach doesn't work, consider this thought experiment.
Below are the ingredients used to make Nestle's famous "Toll House" chocolate chip cookies:
- 2 1/4 cups all-purpose flour
- 1 tsp baking soda
- 1 tsp salt
- 2 sticks butter
- 3/4 cup granulated sugar
- 3/4 cup packed brown sugar
- 1 tsp vanilla extract
- 2 eggs
- 2 cups Nestle Toll House Semi-Sweet Chocolate Morsels
- 1 cup chopped nuts
Image courtesy of Pat Pilon via Flickr (CC).