Introduction
Rapid urbanisation and intensifying climate variability are straining existing energy infrastructures, while city-level planning remains largely static and reactive. Urban digital twins—high-fidelity, data-rich virtual replicas of cities—combined with reinforcement learning (RL) offer new possibilities for adaptive, climate-responsive energy management. This paper explores how RL agents embedded in urban digital twins can orchestrate demand, storage, and distributed generation across buildings and districts to reduce energy use and emissions while preserving occupant comfort.

Methods
A multi-scale urban digital twin is constructed, integrating GIS-based morphology, building archetypes, district energy networks, and microclimate modelling. Real and synthetic data streams feed into the twin. RL agents (deep Q-learning and proximal policy optimisation, trained over 500 episodes with learning rate 3×10⁻⁴ and discount factor γ=0.99) control HVAC setpoints, shading devices, thermal storage, and battery dispatch at building and district levels. Reward functions encode multiple objectives, including minimising carbon intensity and peak demand while maintaining thermal comfort (operative temperature 20–26°C, PMV ±0.7). Comfort violations were deemed acceptable when affecting <5% of occupied hours. Scenario experiments explore different climate futures and urban design configurations.

Results
Simulation results for a mixed-use urban district show that RL-driven control reduces peak electrical demand by 18–30% and operational CO₂ emissions by 15–25% compared with rule-based schedules, while keeping comfort violations within acceptable limits. Coordinated control of façades, storage, and flexible loads mitigates urban heat island impacts during heatwaves. Policy analysis demonstrates how tariff design and incentive structures strongly influence RL convergence and system performance.

Conclusions
Coupling urban digital twins with reinforcement learning can transform static masterplans into adaptive, learning energy infrastructures. However, results are calibrated to a temperate European climate and mid-density mixed-use typology; generalisation to other climatic or morphological contexts requires further validation. Such tools nonetheless provide architects and policymakers with testbeds for prototyping climate-responsive districts.