It is important when evaluating concrete's mechanical properties to capture the internal structure. In previous studies, segmentation of concrete CT images based on deep learning has been addressed. Datasets of CT images contain extreme minority classes, such as crack and void, and show extreme class imbalance. This class imbalance forms a bias when predicting the majority classes. Improvement of the loss function is one approach to addressing this challenge. Class-Frequency-aware Focal Loss (CFL), which incorporates class frequency as a weight into Focal Loss, is proposed in this study. CFL is compared with conventional loss functions, including cross-entropy loss, focal loss, and combo loss, to improve model performance. CT images are acquired from a total of 146 cores, consisting of 89 cores taken from eight in-service concrete structures and 57 cores cast in the laboratory. A total of 693 CT images are manually annotated into five classes, namely crack, void, aggregate, mortar, and background. The CT images are divided into training, validation, and test datasets. U-Net is used as the semantic segmentation model. The macro F1-score, calculated by averaging the F1-score of each class, is used for evaluation. The macro F1-score results are 0.774 for cross-entropy loss, 0.783 for focal loss, 0.841 for combo loss, and 0.861 for CFL. The F1-scores for crack are 0.362, 0.433, and 0.649 for the three models using convolutional loss function, whereas that for CFL is 0.785 and remains the highest. This improvement is attributed to the use of class frequency as a weight, which allows the minority classes to be learned efficiently. These results indicate that CFL improves the performance of crack detection for concrete CT images.