A Reproducible Deep Learning Pipeline for Respiratory Imaging: Internal Benchmarking, Calibration, and External Validation Across Tuberculosis, Pneumonia, and Lung Cancer

Introduction: Deep-learning models have demonstrated promising performance for respiratory disease classification from medical images; however, many studies report only internal test results, limiting understanding of cross-dataset generalization. This study presents a reproducible, manifest-driven benchmarking pipeline for respiratory imaging tasks and evaluates the impact of external validation and dataset shift across multiple diseases.

Methods: A unified transfer-learning framework was developed for tuberculosis, pneumonia, and lung-cancer classification using publicly available chest X-ray and computed tomography datasets. The pipeline incorporates standardized preprocessing, patient-level train/validation/test separation, class-balancing strategies, confidence-interval estimation through bootstrap resampling, calibration analysis, and validation-based threshold optimization using Youden’s J statistic. DenseNet121, EfficientNetV2B0, MobileNetV3Small, and ResNet50 backbones were evaluated. External validation was performed using independent datasets not involved in model development.

Results: Internal evaluation demonstrated strong performance, including AUROC values of 0.947 for tuberculosis, 0.986 for pneumonia, and 0.976 for the best lung-cancer model. External validation revealed substantial disease-dependent generalization differences. Pneumonia achieved the strongest external transfer (AUROC 0.881), while tuberculosis performance decreased to AUROC 0.675 on TBX11K. The largest degradation was observed for lung cancer, where external AUROC decreased to 0.536, indicating significant domain and label shift between datasets. Calibration analysis and threshold optimization improved sensitivity in several tasks but did not eliminate external performance degradation.

Conclusions: The proposed framework provides a reproducible benchmark for respiratory imaging AI and demonstrates that strong internal performance may substantially overestimate real-world generalization. The results highlight the importance of external validation, calibration assessment, and domain-shift analysis before clinical interpretation. This work should be regarded as a retrospective research benchmark and not as a clinically validated diagnostic system.