How you know a forecast is good: its ending is hidden.
A forecast is not judged by how well it explains the past, but by how well it guesses what it has not seen. This chapter explains how each model is put to the test and why running out of product weighs more than having too much.
The test with the answers covered
If you ask a model about dates it has already seen, it always looks good: it is like grading a student with the same questions they studied. The honest approach is to hide the end of the history, ask the model to forecast it, and compare with what was actually sold.
StockAI does this not once but eight times. It cuts your history at eight different points; at each one, the model trains on everything before and forecasts what follows. That way a model does not win by luck on a single stretch.
What the error is
On each hidden date, the error is the difference between what was forecast and what was sold. There are several ways to summarize it; StockAI shows two that can be understood without formulas:
- Average error in units: on average, how many units off it is per period.
- Percentage error on sales (WAPE): everything it got wrong, added up, divided by everything sold. 20% means that, in total, the error was a fifth of sales.
Running short does not cost the same as having extra
Being wrong on the high side and on the low side does not hurt equally. If you forecast too much, you have extra stock sitting around for a while. If you forecast too little, you lose the sale and sometimes the customer.
That is why the winning model is not chosen by the plain error but by a cost in which falling short weighs three times as much as overshooting. Between two models that are wrong by the same amount, the one that errs in the direction that costs you less wins.
Naive forecasts, as a yardstick
Alongside the models, three deliberately simple forecasts are calculated: repeat the last value, repeat the previous week, and repeat the average of the whole history. They do not compete to be chosen: they serve as a yardstick.
If for a product one of those naive forecasts is wrong less than all the models, StockAI flags it. That is a useful signal: that product may have no pattern to learn, or its history may not be representative.
Ties
When two models are less than 10% apart, they are considered tied. The tie goes to the one that has performed best across the rest of your catalog in that same run, so the winner does not change over a minimal difference.
What you take away
- A model is judged on dates it has not seen, across eight different cuts.
- StockAI shows the error in units and as a percentage of what was sold.
- The winner is the one that costs least: falling short weighs three times as much as overshooting.
- If a naive forecast beats every model, the product is flagged so you can take a look.
Where you see it in StockAI
In Forecasts, each product shows its winning model and its error measured on the hidden dates. In Accuracy you can compare what was forecast with what was actually sold afterward.
Keep reading
- The models, explainedThe idea behind each model and why different ones compete depending on the product.
- LearnA guide for non-technical readers: what a time series is, how a forecast is measured, what each model does, intermittent demand, the forecast range, and how it turns into a purchase.
- How it is calculated, in briefThe best way to understand it is to see it with your own products.
Put your own AI to work forecasting your sales. Today.
Three ways to start, depending on how much you want to commit right now. All three lead to the same product.
Create your account
With the whole engine. Upload your sales file and see your first forecast, and with it your first list of what to order.
Create my accountLook before giving your email
An instant trial account with sample data. It lasts 24 hours and is then erased.
Try it without signing upTalk to us
For the corporate plan, the source code, or to see it with your data and someone beside you. We answer within 24 hours.
Sales and contact: contacto@stockai.es. Phone: +506 7186 2820.
Sales forecasting made in Costa Rica, for companies across Latin America.