Prediction and Causation Are Different Questions
Most organizations that ask for "a model" are actually asking one of two very different questions:
- What is likely to happen? — a prediction question.
- What happens if we change something? — a causal question.
A well-tuned gradient-boosted tree can answer the first question extremely well while giving a misleading answer to the second. This isn't a minor technical footnote — it's the single most common source of expensive decisions made on good-looking dashboards.
A concrete example
Suppose a lending model finds that a customer's credit-score-checking frequency is one of the strongest predictors of default. It's tempting to conclude: "if we discourage customers from checking their credit score, default rates will fall." That's very likely wrong. Checking frequency is probably a marker of financial stress (people under stress check their accounts more), not a cause of default. Intervening on the marker does nothing to the underlying cause.
The model playground on this site includes an interactive version of exactly this failure mode — you can turn confounding strength up and down and watch a predictive model stay accurate while a naïve treatment-effect estimate becomes increasingly biased.
Why this keeps happening
Predictive models are optimized for one thing: minimizing error on held-out data drawn from the same distribution and the same set of causal relationships as the training data. They have no mechanism for distinguishing a variable that causes the outcome from one that merely travels with it. Nothing about a low validation loss tells you which of the two you're looking at.
Causal identification — a DAG, a set of assumptions, a design like difference-in-differences or an instrumental variable — is what supplies that missing piece. It's slower and less flashy than fitting a boosted tree, but it's the only part of the pipeline that actually answers "what should we do."
The practical takeaway
Before building or trusting a model, ask which question you're actually answering. If the deliverable will be used to decide an intervention — a policy change, a product change, a pricing change — treat it as a causal question from the start, even if a predictive model is part of the toolkit used to answer it.