When the Model's Confidence Became the Company's

The Global Signal

Zillow announced on November 2, 2021, that it was shutting down Zillow Offers, its algorithmic home-buying business, after disclosing in its third-quarter results a $304 million inventory write-down within its home-buying segment, with an additional $240 million to $265 million in losses anticipated for the fourth quarter, a combined figure widely reported as exceeding $500 million. The company's own disclosure statement attributed the shutdown directly to "unpredictability in forecasting home prices" that "far exceed[ed] what we anticipated," and the wind-down included cutting approximately 25 percent of Zillow's workforce. Zillow had purchased thousands of homes at algorithm-generated prices during 2021's unusually volatile housing market and found itself unable to resell many of them without a loss.

Zillow's own Zestimate model has historically carried a published national median error rate in the high single digits, with a wider margin of error, commonly cited around 14 percent, for homes not currently listed for sale, a figure Zillow itself has disclosed as part of its standard model documentation. That baseline error rate is not new information; it existed before, during, and after the Offers program. What changed in 2021 was the cost of that error rate at scale, during a period when home values were moving faster than the historical data underlying the model could account for.

The Hidden Signal

A pricing model's published error rate is a statement about its behavior under the conditions it was built and tested on. It says nothing about how that error rate behaves once market conditions move outside that historical range, and a model has no internal mechanism for announcing that it has left its own reliable zone. Bank of America's own market analysis, reported alongside coverage of the shutdown, found Zillow paying prices in markets including Phoenix that reflected annual price increases exceeding 20 percent, a pattern an external analyst could identify by comparing Zillow's purchase prices against broader market data, well before Zillow's own aggregate reporting reflected the program as anything other than a going concern.

Why the Visible Metric Misleads

A model's published historical accuracy answers the question of how it performed on average across the conditions it has already seen. It does not answer the question that matters most in a volatile market: is the current environment still inside that range, or has it moved outside it in a way the model cannot detect on its own. The more revealing practice is not distrusting the model's accuracy statistic, which Zillow disclosed honestly and consistently, but building a separate, standing check comparing the model's purchase prices against real-time, on-the-ground signals, exactly the kind local agents were already producing publicly, before committing capital at the model's stated confidence level.

Implication for Operators

Any organization deploying a pricing or forecasting model at scale should assume that the model's published historical accuracy describes its past performance, not a guarantee that current conditions still resemble the data it was trained on. The practical shift is building a standing, separate check against real-time market signals, particularly during periods when external, on-the-ground observers are already reporting something the model's own aggregate output has not yet reflected, rather than waiting for the model's own error to surface in a quarterly financial disclosure.