Target recognition in ghost imaging (GI) systems presents substantial challenges at low sample ratios, when traditional approaches need computationally expensive image reconstruction or have poor accuracy. Deep learning (DL) provides a promising solution by allowing direct feature extraction from GI measurements, however existing methods frequently rely on reconstructed images or are ineffective for natural objects. This study provides an image-free DL architecture for high-accuracy target recognition in GI without intermediate reconstruction, which dramatically improves efficiency and performance. We develop an end-to-end neural network architecture for processing raw GI bucket data (single-pixel measurements) and correlating them to predetermined target classes. The model blends spatial feature encoding with attention techniques to improve discriminative performance in noisy, low-sampling environments. Training uses synthetically augmented GI data to promote generalization, whereas testing is done on experimentally captured natural items. The proposed system outperforms standard GI classification algorithms that rely on reconstructed images, achieving over 90% recognition accuracy at sampling ratios. A comparative investigation reveals a >25% increase in accuracy over conventional procedures at the same sample rate. This study presents a viable DL framework for GI-based target recognition, which eliminates the requirement for image reconstruction while maintaining good accuracy at low sampling rates. The findings highlight the potential for image-free processing in computational imaging, which could enable real-time applications in surveillance, biomedical imaging, and remote sensing. Future work will expand the approach to include dynamic situations and 3D target recovery.