On the flight control of Flapping Wing Micro Air Vehicles with model-based reinforcement learning

Hummingbirds and insects can hover in disturbed conditions, escape from predators with a very fast response, fly for miles without landing, etc. These outstanding features are still unmatched by the most recent bio-inspired drones, due to complex aerodynamic phenomena that are underexploited by flapping wings. We propose an innovative control framework that blends model-free and model-based strategies to control the wing kinematics of Flapping Wing Micro Air Vehicles (FWMAVs) in a “take-off and hover” scenario.
The control strategy reunites a Reinforcement Learning approach (Deep Deterministic Policy Gradient), that mimics the trial-and-error learning process of natural species and an adjoint-based approach that interacts with a calibrated model of the environment. The approaches collaborate and learn from each other to be robust to highly dynamic maneuvers and sample-efficient. The approach is tested on a canonical drone formed of a spherical body and two semi-elliptical, rigid wings that operate within the hummingbird’s range. The drone flight is simulated combining the equations of motion with a data-driven, quasi-steady model that estimates the wing aerodynamic forces. The controller adapts those forces by varying the wing motion, parametrized by three degrees of freedom, to reach the flight objective and satisfy an energy-minimization constraint.
The results show that the drone efficiently reaches its target thanks to the complex adaptation of its wing kinematics. The physics of the flight was also analyzed thanks to a high-fidelity CFD environment. This contribution thus shows a first proof of concept of a control algorithm that aims to bridge the gap between natural flyers and bio-inspired drone flight maneuvers.