Identifying Market Dynamics Through the Hurst Exponent

This study investigates financial market dynamics by analyzing historical crude oil price data through the Hurst exponent, a measure of long-term dependence in time series. The objective is to assess the persistence and volatility structure of the market and to evaluate the predictive capability of the Hurst exponent in identifying distinct volatility regimes.

First, the study estimates realized volatility and the time-varying Hurst exponent to characterize market behavior. A K-means clustering algorithm is then employed to classify observations into three volatility regimes: low, moderate, and high. The predictive power of the Hurst exponent is assessed by training and evaluating multiple machine learning models, including logistic regression, random forests, XGBoost, LightGBM, and support vector machines (SVMs).

Empirical findings indicate that the Hurst exponent serves as a robust indicator of market turbulence. Among the models tested, LightGBM achieves the highest predictive performance, with an accuracy of 88%. Further optimization using Optuna's Bayesian hyperparameter tuning enhances the model’s performance, increasing accuracy to 91%. The model demonstrates high sensitivity in detecting high-volatility periods (recall = 1.00), although its precision in classifying these phases (0.80) suggests a degree of false-positive predictions.

To ensure the robustness of the results, a rolling-window validation strategy is implemented, preserving the temporal structure of the dataset. Additionally, isotonic regression is applied to refine the calibration of predicted probabilities, improving their reliability.

The findings of this study underscore the Hurst exponent’s effectiveness as a market efficiency and volatility indicator. By integrating statistical methods with machine learning techniques, this research provides a systematic framework for anticipating periods of financial instability, particularly in commodity markets such as crude oil. The proposed methodology offers valuable insights for risk management, portfolio allocation, and financial market forecasting.