Video surveillance and augmented reality in maritime safety

Augmented reality (AR) is used more often in many maritime applications. In this paper, AR model is considered for improvement traffic monitoring in ports. It is useful for port authorities.

The model’s input is camera installed in the port. It provides a video stream over IP connection to the facility where the processing computer is placed. Ship’s detection is performed by YOLO (You only look once) artificial neural network. Developed YOLO detector detects small and large vessels. Hence, ships with automatic identification system (AIS) and non-AIS maritime traffic is detected. This creates realistic real world scenario for Mediterranean port traffic portfolio, which includes passengers’ ships, fishing ships, touristic ships, yachts, and other small-type of sailing objects.

Trajectories are estimated based on the central point of the detected vessel on the video stream from the surveillance camera. The intersection of the diagonals of the boundary box gives the central point of the vessel, the position of the central point remains remembered for each frame in the last 5 seconds. Position also depends on external influences. Hence, a linear regression is performed to get the direction of the vessel's movement between the memorized positions.

The collision risk assessment is made based on the distance between the vessels and the direction and speed of the vessel's movement. If the continued movement of the vessel according to the estimated trajectories and speeds will result in the intersection of the motion vector, it is suggested to change the course or speed of the vessel in order to eliminate the potential danger of collision.

In order to be useful for port authorities, the goal is to visualize collision risks in AR environment. AR is installed on smart phone and employee of port authority can check for possible problems easily without need for powerful computers and desk.