The emergence of Unmanned Aerial Vehicles (UAVs), commonly known as drones, has presented numerous transformative opportunities across sectors such as agriculture, commerce, and security surveillance systems. However, the proliferation of these technologies raises significant concerns regarding security and privacy, as they could potentially be exploited for unauthorized surveillance or even targeted attacks. Various research endeavors have proposed drone detection models for security purposes. Yet, deploying these models on edge devices proves challenging due to resource constraints, which limit the feasibility of complex deep learning models. The need for lightweight models capable of efficient deployment on edge devices becomes evident, particularly for the anonymous detection of drones in various disguises to prevent potential intrusions. This study introduces a lightweight deep learning-based drone detection model by fusing knowledge distillation with Bayesian optimization. Knowledge distillation is utilized to transfer knowledge from a complex model (teacher) to a simpler one (student), preserving performance while reducing computational complexity, thereby achieving a lightweight model. However, selecting optimal hyperparameters for knowledge distillation is challenging due to a large number of search space and complexity requirements. Therefore, through the integration of Bayesian optimization with knowledge distillation, we present an enhanced CNN-KD model. This novel approach employs an optimization algorithm to determine the most suitable hyperparameters, enhancing the efficiency and effectiveness of the drone detection model. Validation on a dedicated drone detection dataset illustrates the model's efficacy, achieving a remarkable accuracy of 96% while significantly reducing computational and memory requirements. With just 102,000 parameters, the proposed model is five times smaller than the teacher model, underscoring its potential for practical deployment in real-world scenarios.