How to Reality-Check “Hyper-Accurate” Sales Forecast Claims

You’ve seen the boasts. “Within 2%.” “Within 5%.” “Within 10%.” Whatever.

Forecast-accuracy headlines abound. Are they real, or performed with smoke and mirrors? How can you tell?

We’ve blogged about how unrealistic these claims are and even how undesirable they are. Yet the headlines continue. And I believe that most of the claims are based on some narrow conditions where the claims are valid. My goal therefore isn’t to debunk anyone’s claim; it’s to understand how it’s possible to make them. So you can assess if the claims are meaningful and relevant to your business.

These questions are critical. You can’t assess the accuracy claim without understanding these questions

1) Bookings or recognized revenue?

Bookings reflect signed deals for future business transactions; recognized revenue represents how a company "earns” sales as it fulfills its obligations to its customers. What is "earned" is determined by generally accepted accounting principles (GAAP) applied to the delivery of past sales. Recognized revenue much more manageable. And very loosely coupled to sales bookings—the recording of new transactions that are promises to deliver products and services. If the accuracy claim is about bookings, you’re evaluating sales forecasts; if it’s about recognized revenue, you’re evaluating ability to manage revenue based on GAAP rules.

2) When in the quarter was the forecast made?

Timing matters. Day one of the quarter is different from two weeks before close. At quarter-end, the active pipeline has matured—and uncertainty drops. You also have more already-booked business, so the forecasted business is a smaller share of the total. If you’re forecasting 20% of the final figure, then being off by 40% (a big error) is only 8% of the total. Sounds a lot better than it is. Many businesses get pretty accurate forecasts in the last two weeks of their quarters.

3) What was the deal type mix among new logos, expansion deals, renewals?

Renewals are steadier; new logos are lumpier; expansions sit in between. You can’t compare accuracy without understanding the mix of deal types. If the forecasted business is 95% renewals and the business has a 95% renewal rate (on a large number of similar-sized renewals), then the total business will be quite predictable.

4) How many deals closed—and how concentrated were they?

Deal transaction volume and deal size concentration matter a lot. Repeatedly predicting the outcome of single deals (far in advance of the close) is impossible. Conversely though, anyone can get perfect accuracy on a deal or two. Assess this by asking how many deals were forecasted and how many closed. A good rule of thumb is that you need to be closing 25+ deals in a forecast period from a pipeline of 100+ for the business to start to be forecastable—but that will not be hyper accurate. If the business has 1000s of similar-sized transactions then it will be much more forecastable. And of course deal-concentration matters. If a handful of large deals make up a large share of the quarter, it’s very hard to forecast accurately.

5) What is the quarter-over-quarter volatility?

Lower volatility (flat or steady growth) makes accurate forecasting easier—there’s less noise to explain, so a small error is more achievable. High volatility (from seasonality, big demand swings, macro shocks, big deal shocks) makes accuracy harder—a small error there is not likely. Ask for historical QoQ variability to judge claims against the business’s natural noise.

6) How much business is created and closed in the forecast period? 

Short sales cycles lead to a high percent of the business being created-and-closed-in-quarter (CIQ). This shifts the forecasting burden from pipeline progression to in-quarter demand generation. You’ll need a separate model for this. Claims that lean on “better signals for existing deals” are great. But they won’t help you if a large share of your bookings are CIQ.

7) How many forecasts are you picking from?

For most businesses, the chance of landing within a small band (say ±10%) of a single forecast is quite small. That said, it happens. You don’t even have to be lucky. Make a forecast every day with three different models. You’ll have 270 forecasts to choose from at the end of your quarter. It’s likely that more than one will be very accurate. If you’re making only one forecast per week you can still get lucky. Repeating that is harder—unless you cherry-pick in arrears the date on which the forecast comparison is made.

What to do

You can, of course, turn these points into a rubric to score accuracy claims. But our advice is to ignore accuracy claims altogether. They’re context-dependent and rarely portable. What matters is outcomes—and improving them.

Accuracy vs. Outcomes

The point isn’t to nail your forecast; it’s to beat your goal. Ask vendors to show you how their application improves your odds of beating your plan. Look for specific actionable insights and recommendations—like when to focus on which deals and markets, how to allocate resources.

If they can’t do that—and quantify the gains if you follow their recommended actions—then you’re buying a dashboard and a streamlined rollup process to report the news. Maybe that's helpful. We think you should focus on how to change the news.