[Book Review] "Escape from Model Land" by Erica Thompson

Source:  Basic Books

Predictive mathematical models touch our lives virtually every day. Every weather forecast we watch, hear, or read is formulated based on multiple atmospheric models. And that's just one example.

Predictive models have also become an integral part of modern marketing. For example, marketers use mathematical models to determine the optimal mix of marketing programs (marketing mix models), identify the attributes of their best prospects, and personalize marketing communications and other forms of marketing content.

The primary function of most mathematical models in marketing is to identify patterns in existing data and then apply those patterns to predict the likely future outcomes or results of marketing decisions or programs.

The use of predictive models in marketing is poised to increase significantly because of continuing advances in artificial intelligence. If you need proof of this growth, just look at the explosion of generative AI applications since the public release of OpenAI's ChatGPT in November 2022.

All this makes it vital that marketers have a basic understanding of how mathematical models are constructed, how they work, and why they don't always produce accurate forecasts. This makes Escape from Model Land:  How Mathematical Models Can Lead Us Astray and What We Can Do About It (Basic Books, 2022) a book all marketers should read.

Escape from Model Land was written by Erica Thompson, an associate professor at University College London (UCL) and a Fellow at the London Mathematical Laboratory. Previously, she was a senior policy fellow at the Data Science Institute at The London School of Economics and Political Science. Thompson holds a PhD in physics from Imperial College.

What's In the Book

Escape from Model Land contains ten chapters. In the first six chapters, Thompson focuses on the attributes and limitations of mathematical models. She observes that people who design and build models work in a wonderful place she dubs "Model Land." In Model Land, she writes, all the assumptions that underlie a model are "literally true," and all the uncertainties are quantifiable.

The problem is that these conditions don't exist in the real world. Thompson writes, "Deep or radical uncertainty enters the scene in the form of unquantifiable unknowns:  things we left out of the calculation that we simply could not have anticipated . . . In that case, your carefully defined statistical range of projected outcomes would turn out to be completely inadequate."

Escape from Model Land discusses several other limitations of models. For example, Thompson observes that all models are oversimplifications of the real world, which means they provide an incomplete picture of reality. She writes, " We might think of models as being caricatures . . . Inevitably, they emphasize the importance of certain kinds of features . . . and ignore others completely."

Thompson also points out that a model builder makes numerous choices when developing a model - what to put in, what to leave out, what scientific and mathematical approach to take, etc. Therefore, a model will reflect the values, education, and culture of the model builder, which means that it only presents one perspective of a given situation when, in fact, several perspectives are possible.

Throughout the book, Thompson exposes the limitations and "blind spots" of predictive models, but she does not argue they should be relegated to the junk pile. Near the end of Chapter 1, Thompson includes a passage that describes the challenge she hopes the book addresses. She writes:

"I have tried to find a balanced way to proceed in between what I think are two unacceptable alternatives. Taking models literally and failing to account for the gap between Model Land and the real world is a recipe for underestimating risk and suffering the consequences of hubris. Yet throwing models away completely would lose us a lot of clearly valuable information."

Thompson uses the final chapter of Escape from Model Land to offer five suggestions for addressing this challenge.

• Define the Purpose - "As a starting point for creating models, we need to decide what purpose(s) they are supposed to be put . . . Most models are not adequate for the purpose of making any decision, although they may be adequate for the purpose of informing the decision-maker about some parts of the decision."

• Don't Say "I Don't Know" - "If we can give up on the prospect of perfect knowledge and let go of the hope of probabilistic predictions . . . there are alternative narratives in each model which in themselves contain useful insights . . . We know nothing for certain, but we do not know nothing."

• Make Value Judgements - "All models require value judgements . . . When you understand the value judgements you have made, write them down . . . Allow for representations of alternative judgements without demonising those that are different from your own."

• Write About the Real World - "When you're explaining your results to somebody else, get out of Model Land and own the results . . . in what ways is this model inadequate or misinformative? What important processes does it fail to capture?"

• Use Many Models - ". . . gathering insights from as diverse a range of perspectives as possible will help us to be maximally informed about the prospects and possibilities of the future."

My Take

Escape from Model Land is well-written, accessible, and engaging. Erica Thompson does an excellent job of making the complex, technical aspects of mathematical models easy for those of us who aren't trained data scientists to understand.

This book is not specifically about marketing, but it contains a message that is important and timely for marketers. Over the past several years, marketers have increasingly relied on data to inform their decisions, and recent advances in artificial intelligence will likely increase this reliance.

There's no doubt that data analytics and AI can help marketers make more evidence-based decisions, but these tools also have limitations that often go unrecognized - or at least underappreciated.

The apparent precision of numbers and the halo of scientific validity surrounding AI can easily create an illusion of certainty that gives us a false sense of confidence in the outputs these tools produce.

Escape from Model Land reminds us that marketing should always be "data-informed," but never totally "data-driven."