Health monitoring of civil and industrial structures has been gaining importance since the collapse of the bridge in Genoa (Italy). It is vital for the creation and maintenance of reliable infrastructure. Traditional manual inspections for this task are crucial but time consuming.

We present a novel approach for combining Unmanned Aerial Vehicles (UAVs) and artificial intelligence to tackle the above-mentioned challenges. Modern architectures in Convolutional Neural Networks (CNNs) were adapted to the special characteristics of data streams gathered from UAV visual sensors. The approach allows for automated detection and localisation of various damages to steel structures, coatings and fasteners, e.g. cracks or corrosion, under uncertain and real-life environments.

The proposed model is based on a multi-stage cascaded classifier to account for the variety of detail level from the optical sensor captured during an UAV flight. This allows for reconciliation of the characteristics of gathered image data and crucial aspects from a steel engineer's point of view. To improve performance of the system and minimize manual data annotation, we use transfer learning based on the well-known COCO-dataset combined with field inspection images. This approach provides a solid data basis for object localisation and classification with state-of-the-art CNN architectures.