RATL: Learning from Retrieved Residuals for Robust Multivariate Time-Series Forecasting

Retrieval-augmented generation (RAG) complements parametric models with retrieved external evidence. The same idea is attractive for continuous-output regression, but directly reusing retrieved target values is often not robust when samples differ in output level, numerical scale, or local dynamics. Moreover, conventional forecasting pipelines generally use residuals for model optimization and error diagnosis, but do not retain individual historical residual examples as memory that can be accessed at inference time.For multivariate time-series forecasting, we propose RATL, a plug-in residual-retrieval and feedback-correction method. RATL freezes a base forecaster to construct retrieval keys and turns its historical forecast residuals into a train-only memory specific to that base model. At inference time, RATL retrieves residual trajectories from similar historical contexts subject to causal availability constraints, then uses a set-aware router operating over forecast blocks and variables to select and combine these trajectories. Experiments show that historical residuals matched to the current context contain reusable forecasting information and that RATL improves frozen base forecasters in most experimental settings. Ablations further show that learned routing strengthens raw residual feedback, while validation-based correction-strength selection limits residual over-injection.On real-world benchmarks, we use iTransformer as the primary frozen base forecaster, compare against multiple strong forecasting baselines, and test transferability across backbones. The results show that RATL can further improve base-forecaster performance in most settings.Overall, RATL shifts the retrieved object from historical target values to base-model-specific historical forecast errors, providing a plug-in, residual-memory-based paradigm for learned feedback correction in continuous-output forecasting.

Residual Algebra for Representation-Preserving Learning

Learning from heterogeneous representations is often reduced to feature concatenation, erasing which representation produced each error. We propose residual algebra, in which each representation retains its coordinate system and owns its unresolved residual until an explicit aggregation boundary. Fold instantiates representations as point-in-time conditional-mean fields on 10x10 rank grids, and FPRC-PQ composes them through relax-aggregate-close: each field first fits a correction to its own residual, corrected fields then meet at a fixed mean, and a shared learner closes only the aggregate's fresh residual. We formalize aggregation as a quotient by the zero-sum redistribution kernel, characterizing legal post-aggregation operators as those constant on its cosets. The resulting composition separates representation, local residual estimation, and residual-of-residual estimation, with population variance reduction and first-order coupled-path mean orthogonality. Rumination-B and Rumination-H extend the algebra with quotient-legal finite correction and feedback. On 3.67M Chinese A-share stock-day observations (2023-2026) under a frozen point-in-time protocol, FPRC-PQ raises net-of-cost return from 13.52% to 19.10% and Sharpe from 1.42 to 2.09, outperforming matched-capacity, unified-residual, identity-free two-stage, and pairwise-only controls. The gain is thus attributable to explicit residual ownership and composition rather than additional features or trees.

One Step Closer to Ground Truth: A Multi-Scale Residual-Aware Representation Learning Pipeline for Predicting Time Series Data

Transformer-based models have emerged as leading paradigms in time-series forecasting in recent years, employing self-attention mechanisms to capture long-range dependencies. Despite their success, these single-stage forecasting architectures exhibit persistent systematic residual biases arising from structural discrepancies, unmodeled stochastic components, or inadequate multi-scale temporal representations. This limitation persists when residuals are treated as irreducible noise, precluding adaptive correction of structured error patterns. To address this limitation, we introduce a two-stage, model-agnostic framework that explicitly decouples forecasting and residual learning into distinct stages of representation learning. A base transformer first generates the initial predictions. Subsequently, a dedicated meta-corrector dynamically models structured error patterns across multivariate channels, preserves cross-variable dependencies, and iteratively refines the residual bias of the base transformer. By formalizing this pipeline as a hypothesis space expansion, our framework addresses approximation limitations inherent in single-stage architectures, removes reliance on restrictive assumptions, and enables end-to-end learning of complex error dynamics. Evaluated on eight popular benchmark datasets using established protocols, our approach achieves state-of-the-art performance, with significant improvements in standard metrics (MSE, MAE). The results demonstrate the framework's ability to mitigate systematic biases and enhance robustness to complex temporal dynamics, advancing the practical applicability of transformer-based forecasting models.