Road cracks are an important damage for road administrators to maintain the road condition. Deep learning (DL) is common for detecting cracks in road surface images considering its classification accuracy. Previous research works focused on convolutional neural networks (CNNs) without non-crack features or crack analysis with limited accuracies. This study incorporates background classification into CNNs. Background image features are extracted in an unsupervised way by a deep convolutional autoencoder (CAE). A self-organizing map (SOM) map clusters features to obtain background categories. By increasing the number of non-crack categories, CNNs are motivated to learn non-crack features. The proposed method is validated using common road crack datasets. Modified deep CNN models significantly improved accuracy by 1 % - 4 % and f-measure by 3 % - 8 % compared to previous models. The modified VGG16 showed the top-level performance, 96 % accuracy and 84 % - 85 % f-measure.