Closing the Loop: How Humans and Forecast Agents Can Learn Together to Improve Continuous Forecasting

The Need for Continuous Forecast Improvement

In today's fast-changing business landscape, accurate forecasting is essential for effective decision-making. Every step in the forecasting process should enhance forecast value add (FVA) and, ultimately, improve forecast accuracy.

Traditional forecasting methods, where human planners manually adjust algorithmic or machine learning (ML) predictions, have several limitations. Planners often lack the time to review all forecasts thoroughly, struggle to determine which forecasts require adjustments and by how much, and, when making changes, frequently introduce bias. Moreover, there is little structured learning in these processes to drive ongoing improvements.

Decision intelligence can help overcome these challenges. A new approach — where humans and intelligent agents collaborate in a closed-loop forecasting process — offers continuous improvement, enhances explainability, and allows both humans and agents to learn together.

Forecast Automation and Explainability

To reduce the burden on planners, forecast automation should be prioritized whenever possible. Intelligent agents can handle forecasting for B and C products, as well as highly predictable A products with reliable data inputs. These agents leverage AutoML to generate forecasts while also improving data input quality where feasible, aiming to maximize No Touch Forecasts (NTF).

Building trust in automated forecasting is critical. ML explainability methods such as LIME (Local Interpretable Model-Agnostic Explanations) or SHAP (Shapley Additive Explanations) help by providing planners with insights into the key drivers behind ML-generated forecasts.

Despite automation, human planners bring unique strengths that agents lack—such as intuition, contextual understanding, industry expertise, and real-time awareness of external factors like supply chain disruptions or customer feedback. A successful collaboration between humans and agents integrates these strengths to refine forecasts effectively.

The Power of the Nudge

The concept of a “nudge,” inspired by behavioral economics, allows planners and agents to learn and improve together. A nudge prompts the planner to reconsider an adjustment — both in terms of whether it is necessary and to what extent.

Nudges can be triggered based on predefined business logic. For example, if the forecast agent detects a significant deviation from the previous forecast or a variance from budget that warrants attention, it can issue a nudge. More importantly, nudges can be activated when human bias is detected.

By using ML classification, the agent can identify when human judgment is likely to be non-value-adding. This classification may be based on factors such as the size of overrides, trends in positive versus negative overrides, or other indicators of bias. The nudge then encourages planners to make higher-quality adjustments by highlighting potential biases in their decision-making.

Planners may receive nudges in the form of recommendations to review forecasts when significant variations are detected or when past forecasting behaviors suggest a tendency toward bias. These recommendations include insights into the detected bias, helping planners learn and refine their decision-making processes.

The Closed-Loop Forecasting Process

The key to continuous forecast improvement lies in a structured automation, feedback, and learning loop:

• No Touch Forecast (NTF): The forecast agent generates an automated forecast for applicable products using historical data and AutoML.
• Feedback & Explanation: The agent provides explanations on how the NTF was created while also offering feedback on any biases detected in similar forecasting scenarios.
• Nudge & Adjustment: For products that are not NTF, and which require human intervention, the agent issues a nudge in the form of a recommendation. The planner then reviews and, if necessary, modifies the forecast with the agent’s guidance, adding a reason code..
• Value-Add and Bias Evaluation: The agent assesses the impact of planner adjustments, determining whether they add value or introduce bias. If bias is detected, this feedback is incorporated into future nudges and explainability features.
• Auto Bias Correction: Since judgmental forecasts tend to be more accurate when adjusted for bias, the agent automatically corrects for detected biases and applies this knowledge to future recommendations.
• Learning and Continuous Improvement: Through insights from explainability and feedback mechanisms, planners become more knowledgeable and skilled, leading to better-calibrated adjustments and continuous learning from the forecast feedback loop.

The Future of Human-Agent Collaboration

This integrated approach fosters a system where both planners and agents evolve together. The agent minimizes forecasting errors by reducing unnecessary human intervention, while planners gain valuable real-time learning opportunities. Over time, planners refine their judgmental forecasting skills, and ML models become increasingly precise, resulting in a more efficient and adaptable forecasting system.

By embracing this closed-loop process, businesses can enhance forecast accuracy, reduce bias, and develop a truly intelligent demand-planning system. Instead of viewing human and agent contributions as competing forces, this approach positions them as partners — each playing a crucial role in refining and improving forecasts.

For planners, this method not only enhances forecasting proficiency but also encourages self-reflection and personal development, both within the forecasting function and beyond.