Mitigating Portfolio Risk via Machine Learning Across Market Regimes: Hierarchical Clustering vs. PCA Matrix Denoising.

Portfolio optimization relies on the covariance matrix of asset returns that is difficult to estimate reliably in high-dimensional settings where estimation noise destabilizes the optimized weights. We contrast two families of covariance denoising methods : (a) Principal component analysis (PCA) based on Random Matrix Theory (RMT) and the Marchenko-Pastur law, and (b) a machine-learning approach based on hierarchical clustering, with the raw sample covariance and Ledoit--Wolf linear shrinkage. The four estimators are combined with four portfolio strategies, including a Minimum Conditional Value at Risk strategy where tail scenarios are generated from the denoised dependence structure via a Student-$t$ copula and evaluated on a strictly out-of-sample, cost-aware rolling-window backtest over four market regimes: mature (sector ETFs), volatile (cryptocurrencies), global (country ETFs) and individual equities. A controlled sub-sampling experiment isolates the effect of dimension. We find that denoising is more valuable in the risk-oriented stage of portfolio construction rather than return forecasting, and that the efficient method is dependent on the market structure: PCA denoising is best when one factor dominates, while clustering-based shrinkage is best in high-dimensional, group-structured universes. The best denoiser improves risk-adjusted performance relative to the raw covariance in nearly all settings, but these gains are modest and not robust to multiple-comparison correction. Ultimately, the relative performance of an optimized portfolio versus equal weighting depends on the fundamental fit of strategy to universe, not covariance denoising.