The recent advances in sensor technologies coupled with the development of machine/deep learning strategies is opening new frontiers in Structural Health Monitoring (SHM). Dealing with the structural vibrations recorded by pervasive sensor networks, the goal of SHM is to extract meaningful damage-sensitive features from the data, shaped as multivariate time series, and to make in real-time decisions connected to the safety level of the structures. Within this context, we discuss an approach able to detect and localize a structural damage avoiding any pre-processing of the acquired data. The method takes advantage of the capability of Deep Learning of Fully Convolutional Neural Networks, trained during an initial offline phase. A hybrid model- and data-based solution is looked for: for the former aspect, Reduced Order Models (ROMs) of the structure are built in the aforementioned initial offline phase of SHM, to strongly reduce the computational burden of the subsequent online monitoring phase. Through some numerical benchmarks, we show how the proposed method can recognize and localize damage, even when data are corrupted by measurement noise and environmental variability.