Automatic Detection of Prohibited Items in X-ray Security Imaging Based on Enhanced YOLO11

With the rapid increase in throughput at airports, railways, subways and customs, X-ray transmission imaging has become the dominant modality for non-intrusive inspection of prohibited items. Traditional inspection approaches rely on human operators, whose detection accuracy degrades under fatigue, while overlapping baggage, pseudo-colored material maps and large-scale variation further complicate the manual process. This paper investigates the automatic detection of prohibited items in an X-ray security imaging system based on enhanced YOLO11n detectors. We first benchmark five YOLO detectors and RT-DETR based on a re-splitting and pre-processed public X-ray baggage dataset, confirming that YOLO11n provides the best accuracy–latency balance for this scenario. We then replace the default CIoU loss with Wise-IoU-v3, whose dynamic non-monotonic focusing mechanism re-weights training samples so that medium-quality ground-truth boxes receive amplified gradient gain. Inspired by ParameterNet, we replace standard convolutions in the YOLO11n backbone with Dynamic Convolution. We have investigated the optimal replacement position and evaluated the performance of the proposed method. This approach retains model capacity and reduces FLOPs. Experiments on our X-ray dataset show that enhanced YOLO11n can achieve good accuracy while using less computation. EigenCAM visualization further confirms that the improved model focuses more tightly on prohibited-item contours and suppresses spurious activations on luggage texture.