Deep Learning (DL) algorithms need extensive amounts of data for classification tasks, which can be costly in specialized fields like maritime monitoring. To address data scarcity, we propose a fine-tuning approach leveraging complementary Infrared (IR) and Synthetic Aperture Radar (SAR) datasets. We evaluated our method using the ISDD, HRSID, and FuSAR datasets, employing VGG16 as a shared backbone integrated with Faster R-CNN (for ship detection) and a three-layer classifier (for ship classification). Results showed significant improvements in IR ship detection (mAP: +20%, Recall: +17%) and modest but consistent gains in SAR ship tasks (F1-score: +3%, Recall: +1%, mAP:+1%). Our findings highlight the effectiveness of domain adaptation in improving DL performance under limited data conditions.