Have you noticed that whenever you buy something online they show you a bunch of other stuff under a heading like “you might also enjoy?” Obviously, the retailer is trying to get you to spend more money on their site. But how do they know which items might tempt you?

These suggestions come from recommender systems which are a specialized type of mathematical model. You might not realize it, but mathematical models are everywhere and they have more influence on you than you might think.

Escape from Model Land: How Mathematical Models Can Lead Us Astray and What We Can Do About It, by Erica Thompson, is an incredibly important look at how mathematical models are constructed and used, how they reflect and increase the power of certain groups in society, and how they can be used more responsibly.

It’s admittedly a technical and even nerdy subject but models play a such critical role in our lives – more on this in a moment. It’s also a subject I know something about. I’ve spent the better part of the last twenty years of my career helping to build models using machine learning to detect email spam, credit card fraud, account hacking and other Internet threats. So, I was excited to read this book. It did not disappoint.

Escape from Model Land is mainly intended for people who build mathematical models, and decision-makers who use them. In this review I’ll try to explain in a non-technical way why this is such an important subject even though this might not be a book you ever read. After all, you and I are decision-makers too. Equally important, mathematical models are used by powerful people in business and government to make decisions that have direct impact on our lives, not always to our benefit.

Erica Thompson is a Senior Policy Fellow at the London School of Economics’ Data Science Institute where she focuses on the ethics of modeling and simulation.

Escape from Model Land:
How Mathematical Models Can Lead Us Astray and
What We Can Do About It
By Erica Thompson
Basic Books, New York, 2022

What is a mathematical model?

Before we go any further, what exactly is a mathematical model?

Thompson uses the term “model” very broadly. She says models are frameworks, representations or metaphors that help us understand some part of our world. She’s particularly focused on mathematical models that help us build relationships between data, often with some degree of uncertainty or probability.

A mathematical model is usually composed of one or more equations. A very simple model might be this equation which defines the relationship between the volume (V) of a box and its height (H), length (L) and width (W):

V = L × H × W

Large complex models, like climate models, might use dozens or hundreds of related equations.

Since it’s impossible to reproduce the entire world inside a model, model builders must simplify. They make assumptions and choices that shape a model’s representation of the real world. As a result, models emphasize some details and downplay others. The box model above doesn’t take into account the thickness of the box material, for example.

By making these assumptions and choices, modelers leave the real world and enter what Thompson calls Model Land.

Model Land is a wondrous place where all your assumptions are true. In Model Land, the model is reality.

Models everywhere

Mathematical models are everywhere, even if you might not be aware of them.

They predict tomorrow’s weather, identify email scams, determine your credit rating, recognize your face in photos, and set the price of your Uber ride.

Have you noticed your computer has started suggesting the next word or phrase when you’re writing a document or an email? This feature is called predictive text and it’s made possible by language models. A new generation of large language models, like ChatGPT from OpenAI, can answer questions in full paragraphs, write short essays and even poems.

On a larger scale, models help predict the winner of the next presidential election, whether the economy is heading for a recession, the likely impact of climate change, and the number of people who could die in the next pandemic.

Mathematical models are so important because they are used by decision-makers – from you and me to presidents and prime ministers – to help us decide what actions to take and what policies to implement. As Thompson says, they help us predict “how we can take more effective actions in future in support of some overall goal.” [p. 3]

All Models Are Wrong …

Thompson quotes the statistician George Box who said, “All models are wrong, but some are useful.” 

Large parts of Escape from Model Land concern the many ways in which models can be wrong. I’ll just roll them up into two groups:

First, mathematical models are wrong because they don’t perfectly represent the real world. They can’t. Instead, they’re approximations, shaped by modelers’ choices and assumptions like deciding what problem they’re building a model for, what data to include and what to exclude, and which parameters are important and which can be ignored.

Second, models reflect their builders. As Thompson points out, all those choices and assumptions inevitably reflect the culture, background, values and biases of the model builders. In other words, modelers inject something of themselves into Model Land. That means model building isn’t just a mathematical exercise. It’s an economic, social and political one too. 

Therefore, the results — the predictions — made by models will inevitably be inaccurate to some degree.

… But Some Are Useful

How do we know if a mathematical model is one of the useful ones?

“Useful” in this context usually means accurate, but not always, as we’ll see in a moment.

OK, how do we verify a model’s accuracy?

In many cases, model verification is possible either because the model is about something simple like cardboard boxes, or because there is readily available data that can be used to find and correct errors in the model. To check the accuracy of a weather model, for example, we just wait until tomorrow to see if it rains as predicted. Well, it’s more complicated than that, but the point is that it’s possible to improve these types of models with more data.

Climate models, on the other hand, are very different, Thompson explains. It’s difficult if not impossible to get data to verify the predictions of climate models. How will we know if today’s models predicting sea level rise in the year 2100 are correct? We certainly can’t wait until 2100 before acting. Economic and pandemic models are in this category too.

That leads to another crucial point Thompson makes: often we want to use models to help us take actions that will prevent the model’s forecast from ever happening.

Verifying these large complex models and deciding how their results can be applied to real world problems means exiting Model Land. It’s a qualitative exercise that takes expertise and judgement. The rest of us have to trust the modelers.

I’ve tried to illustrate all this in the following diagram – itself a model.

Use Responsibly

So, we have imperfect models, many of which are impossible to verify.

At this point you might think we should throw up our hands and toss out all the models. But Thompson argues against this because they are so powerful and beneficial when used responsibly. She makes a set of proposals for how to carefully use mathematical models so they don’t lead us astray.

First of all, Thompson says, a model must be adequate for purpose. Remember that the purpose of models is to help us make better decisions relative to some goal. That means we should not use them for unrelated goals. This sounds obvious; you wouldn’t expect a weather model to tell you how to pick stocks. But models may still perform poorly if we ask them questions they weren’t specifically designed to answer even in related areas. A spam detection model won’t do a good job identifying COVID misinformation, for example.

Thompson says it’s important not to be over-confident about the predictions given by models. She calls for a big dose of humility especially with large complex models. It’s important for modelers to recognize the context, imperfections and limitations of their models, and to openly talk about those things whenever they present their results. She credits the Intergovernmental Panel on Climate Change (IPCC) with doing this especially well in their climate reports. They provide detailed analyses of their results along with disclosures about the limitations of their models and the confidence levels of their predictions.

To build better models, and to build more public trust in models, Thompson calls for greater diversity in approaches to modeling, and more diversity in the model builders themselves. She notes that most of the people building models these days are well-educated white men from rich western industrialized democracies.

Different types of models, or models based on different perspectives using different data can help us by corroborating each other and giving us greater confidence in their results. And if they disagree, that helps to identify problems that need further investigation.  In other words, she wants modelers to seek out and triangulate views from different parts of Model Land.

When it comes to climate, she says we need more imaginative models. The IPCC has to be so conservative with its predictions that even though their reports sound increasingly dire, they actually don’t present a clear enough picture of the dangerous possibilities that lie ahead if we don’t take radical action. 

“… there are no non-radical futures anymore, since we will either make radical voluntary changes to decarbonize our systems or else be forced to accept the radical change that will occur due to the influence of climate and ecosystem changes on economies and geopolitics.” [p. 176]

Climate models, she says, are “just a mathematical version of near-future climate fiction” like Kim Stanley Robinson’s book The Ministry for the Future. They’re just not very compelling. This reminds me of a quote from Richard Powers‘ novel The Overstory:

“The best arguments in the world won’t change a person’s mind. The only thing that can do that is a good story.” [p. 488]

I think Thompson would agree with this. One of the most important messages of Escape from Model Land is that models don’t have to be accurate to be useful. She calls models “boundary objects” that help transmit and translate information from one community to another: from scientists and mathematicians to the general public and to leaders and decision makers. Models can be hooks for narratives and discussion, providing a framework for thinking about issues and possibilities, and for evaluating alternative courses of action.

“The future is unknowable, but it is not ungraspable, and the models that we create to manage the uncertainty of the future can play a big role in helping to construct that future. As such, we should take the creative element of metaphor and narrative in mathematical models at least as seriously as their predictive accuracy.” [pp. 222-3]

Unsolicited Feedback

I really got a lot out of Escape from Model Land. That’s partly because my professional work with machine-learned models gives me direct experience with many of the issues Erica Thompson writes about. I’ve seen first-hand how models can do important, beneficial things that are far more important than just getting you to buy more stuff.

I like the idea of Model Land as a separate space, a separate universe, distinct from the real world. It’s a useful construct to help modelers distinguish when they’re talking about an idealized world and when they’re dealing with the messy real world.

But I also like that the book isn’t just about the technical aspects of modeling. It’s largely about our values and how they get reflected in what we choose to model, how we build models, and how we use the results.

I like that it covers how models can help us tell stories and communicate with each other. Used well, models can facilitate discussion and democratic debate as well as decision-making, even when they aren’t completely accurate.

Some areas of the book are weaker than others. Her call for greater diversity among modelers, for example, makes sense but she doesn’t have any strong suggestions for how to achieve it.

I also think there’s some tension between her desire to see more imaginative models, and the necessity to be careful about recognizing their limitations and imperfections.

One thing to note is that Escape from Model Land is about modeling rather than artificial intelligence even though AI is built on models. The book doesn’t deeply address growing concerns about the dangers and responsible uses of AI even though many of Thompson’s arguments apply quite well to AI too. That’s either a gap in this book, or the subject of another.  

This book won’t appeal to everyone. It’s targeted mainly at model builders and the people who pay for and use them so it doesn’t offer guidance for most of us consumers or voters. It’s not like Calling Bullshit which gives you a toolkit you can use to detect nonsense. However, even if you don’t have a background in model building or AI, Escape from Model Land will give you important insights into a one of the most powerful tools of our time.

Thanks for reading.

Related Links

On the abuse (and proper use) of climate models
Interview with Erica Thompson on the Volts podcast, Jan. 27, 2023