Lightweight aggregate concrete (LWAC) is a practical alternative to conventional concrete in civil engineering, offering advantages such as reduced density, enhanced insulation properties, and improved seismic performance. However, segregation during compaction remains a limitation, potentially leading to non-uniform material distribution and decreased compressive strength. This study addresses this issue by combining non-destructive techniques with deep learning methods to predict the compressive strength of LWAC. We propose an explainable approach involving a convolutional recurrent neural network architecture, enhanced by unsupervised clustering and SHapley Additive exPlanations (SHAP), to improve interpretability. To optimize predictive performance, we evaluate aggregation strategies from the recurrent layer before passing to the dense layers, including configurations that apply full-sequence flattening, max pooling, average pooling, or an attention mechanism over the full sequence. Experimental results show that our model outperforms conventional machine learning methods such as multilayer perceptron (MLP), random forest (RF), support vector regression (SVR), as well as ensemble methods like gradient boosting (GBR), XGBoost, LightGBM, and weighted average ensemble (WAE). Furthermore, when combined with unsupervised clustering, the model identifies latent behavioral patterns that are not observable through traditional evaluation techniques. This shows the potential of integrating this tool with interpretable deep learning as a reliable non-destructive approach for the structural assessment of LWAC.