Motivation
Algorithmic trading systems based on deep reinforcement learning (RL) have proliferated over the past five years, yet the representations consumed by these agents remain dominated by classical technical indicators — moving averages, oscillators, volatility bands — that compress price dynamics into local, largely linear summaries. Such representations are well understood and computationally cheap, but they discard the global geometric organisation of the trajectory: clustering structure, recurrence loops, and higher-dimensional cavities that emerge when a univariate price series is lifted into a reconstructed phase space. Topological data analysis (TDA), and persistent homology in particular, provides a principled, stability-guaranteed framework for extracting precisely these multiscale geometric features. Prior applications of TDA to finance have concentrated on descriptive analyses — early-warning signals before market crashes, persistence-landscape diagnostics across crises — or on supervised forecasting. To our knowledge, no prior study has systematically integrated persistence-based descriptors into the state representation of a reinforcement learning trading agent under a leakage-controlled experimental protocol. This work fills that gap.
Contribution
We present a deep Q-network (DQN) trading agent whose state vector is constructed from descriptors of the persistent homology of recent price dynamics, and we benchmark this agent against twelve alternative state representations spanning the standard taxonomy of technical, statistical, and temporal feature families. The novelty is threefold: (i) the first systematic coupling of persistent-homology descriptors with off-policy deep RL for sequential trading decisions; (ii) a strictly causal embedding construction that precludes look-ahead at the feature-extraction stage; and (iii) a unified experimental protocol that places topological features and conventional indicators under identical agent architecture, hyperparameters, and evaluation criteria, so that observed performance differences are attributable to the representation rather than to modelling choices.
Data and task
Hourly OHLC data are used for six USDT-quoted cryptocurrencies (BNB, BTC, ETH, XRP, ADA, LTC) covering January 2018 to January 2023, and for four traditional instruments — the CAC 40 equity index, WTI crude oil futures, the EUR/USD exchange rate, and the iShares Core MSCI Europe ETF — covering January 2023 to June 2024. For each segment, training and held-out test windows are defined non-overlappingly: July 2019 – June 2021 (train) and July – December 2021 (test) for cryptocurrencies; January – December 2023 (train) and January – June 2024 (test) for traditional assets. The trading task is formulated as a Markov decision process with a binary action space {long, short}, and the one-step reward is the realised log-return scaled by the agent's current position.

TDA pipeline. Each univariate price series is mapped into a reconstructed phase space via a strictly causal time-delay embedding: at decision time t, the embedded vector x_t = (p_{t−dτ}, …, p_{t−τ}) uses only observations strictly preceding t, so that the contemporaneous price is never exposed to the agent. The embedding dimension is fixed at d = 3 and the delay grid is τ ∈ {72, 168} hours, corresponding to roughly three days and one week respectively. Sliding windows of length w ∈ {336, 504, 720, 1080} hours (approximately two, three, four-and-a-half, and six weeks) are then applied to the embedded trajectory, generating a family of local point clouds in ℝ³. For each point cloud, the Vietoris–Rips filtration is constructed and persistent homology is computed up to dimension two, yielding persistence diagrams in H₀ (connected components), H₁ (loops), and H₂ (voids). Three scalar functionals — persistence entropy, maximum amplitude, and feature count — are extracted per homology dimension, producing nine descriptors per (τ, w) configuration. Concatenation across the 2 × 4 grid yields a 72-dimensional topological state vector, min–max normalised to [0, 1].
RL setup
The agent is a DQN with a multilayer perceptron approximator, learning rate 10⁻², batch size 128, replay buffer of 10 000 transitions, discount factor γ = 0.99, target-network synchronisation every 100 updates, and ε-greedy exploration over 10⁵ training steps. Off-policy value learning is chosen over policy-gradient alternatives such as PPO or SAC on three grounds: the action space is small and discrete, favouring value-based methods that provide direct per-action estimates; experience replay enables sample-efficient reuse of historical transitions under a fixed data budget; and the smaller hyperparameter footprint allows the contribution of the representation to be isolated from algorithm-specific tuning.
Baselines
Twelve baseline state representations are evaluated under identical DQN architecture and hyperparameters, spanning trend indicators (simple and exponential moving averages), momentum indicators (MACD, RSI, stochastic oscillator), volatility indicators (Bollinger Bands, average true range), and statistical or temporal feature families (lagged returns, calendar/datetime features, rate-of-change and difference features, STL trend–seasonal decomposition, and time-delay embedding followed by PCA). Each baseline preserves the strictly causal construction of the topological variant, ensuring fair comparison.
Results
The TDA-augmented agent achieves the strongest performance across all four evaluation metrics: mean reward 0.0032 versus 0.0016 for the next-best baseline (TDE-PCA), final net worth 1712 versus 1344, Sharpe ratio 1.39 versus 0.66, and Sortino ratio 2.19 versus 0.97. Wilcoxon signed-rank tests on paired per-asset metric values confirm that the topological agent outperforms every baseline at the 5 % significance level on the majority of metric–comparator pairs, and at the 10 % level on the remainder. Maximum drawdown is comparable across all methods, indicating that the return advantage of the topological representation is not obtained at the cost of additional downside risk.
Reproducibility and scope
All hyperparameters (embedding dimension, delays, window lengths, DQN configuration, training horizon) are specified, and feature extraction relies on standard open-source persistent-homology libraries. Evaluation is frictionless; the cost-sensitivity analysis is left as an extension. The framework is asset-agnostic, requiring no domain-specific calibration beyond the embedding grid. The pipeline integrates algebraic-topological structure extraction with modern deep RL applied to multi-asset hourly streams, aligning directly with the Big Data, Computing, and AI scope of CSAN.

References
[1] L. A. Leaverton, ‘Analysis of Financial Time Series using TDA: Theoretical and Empirical Results’.
[2] M. Gidea and Y. Katz, ‘Topological Data Analysis of Financial Time Series: Landscapes of Crashes’, Physica A: Statistical Mechanics and its Applications, vol. 491, pp. 820–834, Feb. 2018, doi: 10.1016/j.physa.2017.09.028.
[3] N. Ravishanker and R. Chen, ‘Topological Data Analysis (TDA) for Time Series’. arXiv, Sep. 23, 2019. Accessed: May 26, 2023. [Online]. Available: http://arxiv.org/abs/1909.10604
[4] P. T.-W. Yen and S. A. Cheong, ‘Using Topological Data Analysis (TDA) and Persistent Homology to Analyze the Stock Markets in Singapore and Taiwan’, Front. Phys., vol. 9, p. 572216, Mar. 2021, doi: 10.3389/fphy.2021.572216.
[5] S. W. Akingbade, M. Gidea, M. Manzi, and V. Nateghi, ‘Why Topological Data Analysis Detects Financial Bubbles?’ arXiv, Apr. 13, 2023. Accessed: May 26, 2023. [Online]. Available: http://arxiv.org/abs/2304.06877