Macroeconomic Feature Extraction for S&P 500 Return Forecasting and Market Regime Classification Using PCA and Explainable Machine Learning

Financial market forecasting is challenging because equity returns are influenced by nonlinear macroeconomic conditions, valuation measures, and investor expectations. This study applies Principal Component Analysis (PCA) and machine learning to extract macroeconomic features for S&P 500 return forecasting and market regime classification. Monthly S&P 500 and macro-financial data are used, including price, dividends, earnings, CPI, long-term interest rate, real price, real dividends, real earnings, and CAPE/PE10. PCA is applied to reduce multicollinearity and transform correlated variables into orthogonal components. The first three principal components explain approximately 96.0% of the total variance, indicating that PCA effectively summarizes the main macroeconomic information in the dataset. Forecasting models are evaluated using RMSE and , while regime classification models are evaluated using accuracy and score. The empirical results show that PCA slightly improves selected models, particularly Linear Regression, Random Forest, and XGBoost. However, all return forecasting models produce negative out-of-sample values, suggesting that monthly S&P 500 returns remain difficult to predict using macroeconomic variables alone. For regime classification, PCA improves Logistic Regression slightly, but overall classification performance remains modest. The findings suggest that PCA is useful for reducing feature redundancy and improving model stability, although additional volatility, momentum, and rolling macroeconomic features are needed for stronger predictive performance.