The aging, deterioration and failure of civil structures are nowadays challenges of paramount importance, increasingly motivating the search of advanced Structural Health Monitoring (SHM) tools. In this work, we propose a SHM strategy for real time structural damage detection and localization, combining Deep Learning (DL) and Model-Order Reduction (MOR). The developed data-based procedure is driven by the analysis of vibration and temperature recordings, shaped as multivariate time series and collected on the fly through pervasive sensor networks. Damage detection and localization are treated as a supervised classification task considering a finite number of predefined damage scenarios. During a preliminary offline phase, for each damage scenario, a collection of synthetic structural responses and temperature distributions is numerically generated through a physics-based model. Several loading and thermal conditions are considered thanks to a suitable parametrization of the problem, which controls the dependency of the model on operational and environmental conditions. Because of the huge amount of model evaluations, required by the construction of a dataset such as to guarantee a good enough exploration of the parametric space, MOR techniques are employed to relieve the computational burden of the procedure. Finally, a DL network, featuring a stack of convolutional layers, is trained by assimilating both the vibrational and thermal data. During the online phase, the trained DL network processes new experimental recordings to classify the actual state of the structure, and thus providing information about the presence and the localization of the damage, if any. Numerical performances of the proposed approach are assessed through a numerical example involving the monitoring of a two-storey frame under low intensity seismic excitation.