Manual inspection of concrete structures, such as tall buildings, bridges, and huge infrastructures, is a time-consuming and risky process for human employees. Drones with sensor camera nodes have showed potential in gathering close-range footage, but the problem is rapidly analysing enormous volumes of data to detect and diagnose structural deterioration. This study deals with these challenges by presenting an Internet of Things (IoT), computer vision and deep learning-based automated solution. The primary issue addressed in this research is the requirement for a more efficient and reliable way of identifying structural damage on Concrete Structures. The traditional manual inspection technique is time-consuming and expensive, making timely repairs and maintenance impossible. As a result, an automated solution is necessary to speed up the damage assessment process while reducing dangers to human personnel. The proposed system focuses on detecting various types of damage, such as cracks, Alkali-Silica Reaction (ASR), concrete degradation, and others, on Concrete Structures using drone-captured video footage. The system's scope includes developing a Convolutional Neural Network (CNN) architecture tailored to this specific task and implementing a seamless process for automatically obtaining video data from drones. The primary objective of this work is to create and implement an automated damage detection system capable of identifying structural damage on Concrete Structures in an efficient and accurate manner. The technology intends to expedite the inspection process by utilising IoT, computer vision and deep learning techniques, enabling proactive maintenance and preservation activities. The novelty of the proposed system is a custom-designed CNN architecture that is optimised for detecting damage on Concrete Structures and a system architecture based on IoT to automatically capture data, perform analysis and reporting. The performance of the proposed automated damage detection system was evaluated using a diverse dataset of drone-captured video footage containing various types of damage on Concrete Structures. The CNN architecture demonstrated impressive results, achieving an accuracy of 94% in correctly identifying different types of structural damage. The results showed that the system could accurately detect and identify structural damage on cultural heritage sites. The system is much faster and more efficient than manual inspection, reducing the time and cost required for damage assessment. The proposed system has the potential to revolutionize the way damage assessment is performed on concrete structures. It can help to preserve and protect it by enabling early detection of damage and facilitating timely repairs.