Scalable Retail Demand Forecasting Using XGboost Fusion with Tabular Feature Interpretation

Abstract

Accurate demand forecasting is critical in the retail industry, as it directly influences inventory management, pricing strategies, and overall business profitability. The increasing complexity of customer behavior, seasonal trends, and competitive market dynamics has made traditional forecasting methods less effective. Traditional retail systems primarily relied on manual analysis, historical averages, spreadsheets, or simple statistical methods. These approaches depended heavily on human judgment and past sales trends, offering limited accuracy and scalability. They also lacked automation, real-time prediction capabilities, and the ability to analyze complex relationships among multiple influencing factors, often resulting in overstocking or stock shortages. To address these limitations, the proposed system introduces a machine learning-based solution that automates the entire demand forecasting process. At the core of the system is a hybrid TabNet-Enhanced eXtreme Gradient Boosting (XGBoost) regression model, which leverages the feature learning capabilities of TabNet alongside the predictive power of XGBoost. TabNet effectively extracts important feature representations from tabular retail data, while XGBoost captures non-linear relationships and interactions among features. In addition to the proposed model, baseline models including K-Nearest Neighbors (KNN), Decision Tree Regressor (DTR), and Gradient Boosting Regressor (GBR) are implemented for performance comparison. Experimental results show that the hybrid TabNet-XGBoost model outperforms traditional models, achieving higher accuracy, improved R² scores, and lower error metrics. The system is deployed through a Flask-based interactive desktop interface, providing role-based access that allows AIML Engineers to perform model training, exploratory data analysis, and performance evaluation, while Retailers can perform real-time demand and discount predictions.