When Relevance Is Not Enough: Enhancing Financial Models through Redundancy-Aware Feature Selection

This article examines the trade-off between feature relevance and redundancy in feature selection for linear regression models. The analysis focuses on two widely used feature selection paradigms, F-test and Random Forest, each implemented in two variants: a relevance-only version and a redundancy-aware extension that jointly accounts for both factors. To evaluate these methods, the authors construct a synthetic data generation framework that enables precise control over average feature relevance and redundancy. By assessing out-of-sample performance across a grid of such configurations, they identify the conditions under which redundancy-aware methods offer meaningful advantages. These findings are further validated on three real-world financial datasets. Results show that relying on relevance alone is often insufficient, and explicitly addressing redundancy improves both the predictive performance and the explanatory power of linear models.