Introduction

Unmanned aerial vehicles (UAVs) offer a small-form solution to many aerial tasks and provide many benefits such as cost effectiveness and the lack of a pilot. Consequently, they are rapidly performing many more tasks in the civilian and military domains. UAVs can be controlled automatically by algorithms that serve to perform a given task with minimal control input to the UAV from its control system. Control algorithms that minimize the effort needed to control the UAV by actuation have been advanced in recent literature.

The 2-degree of freedom (2-DOF) helicopter, the Quanser AERO, exists as a low-budget solution to implement control algorithms in a testing lab setting. This project aims to advance motion control of a 2-DOF helicopter under various operating conditions using a model-based reinforcement learning strategy, approximate dynamic programming. Motion control strategies will be employed for a possible minimization of energy consumed, navigation time, or even the risk for an UAV to complete its pre-defined tasks in an environment.

Project Solution

One of the most challenging tasks in control applications is to control a nonlinear system. State-feedback control can be utilized, but determining the the optimal state-feedback gain for a nonlinear system can be difficult. It can be made especially difficult if either the system or the environment is changing. One such system is the 2-DOF helicopter due to its irregular precessional torques acting on both pitch and yaw.

To overcome this difficulty in determining the state-feedback gain, approximate dynamic programming will be experimented on the 2-DOF helicopter. Approximate dynamic programming, ADP, utilizes measured state error data and an approximate system model to determine the optimal value for the state-feedback gain. ADP uses the measured error state data as initial conditions to see how state error will propagate through the system model for various state-feedback gains. The state-feedback gain will then be updated based on these state error predictions and a performance measurement. Once the state-feedback is updated, the system may progress until the state-feedback gain is updated once again.

In order to implement ADP for the Quanser AERO, we followed standard electrical engineering methodologies. We started with a method of control and a system. The method of control, ADP, was modified for our particular system. Once mathematical modeling and analysis were completed, simulations were conducted in both MATLAB and V-REP to see if ADP was a viable control strategy. Upon promising simulation results, implementation could begin.

To show the true potential of ADP, it had to be implemented on an embedded system to show a proof of concept that the method could be applied to physical systems such as quadcopters and drones. Implementation began by modeling ADP in Simulink which could control the Quanser AERO directly via licensed Quanser software. Upon successful implementation, Simulink was used to generate C-code for an embedded system, a Raspberry Pi 3 in our case. Due to the unknown compiling solutions of Simulink, the desired states of the Quanser AERO could not be changed internally once the generated C-code was on the Raspberry Pi. In order to update the desired states, an Android smart phone application was developed to communicate with the Raspberry Pi via Wi-Fi. This implementation resulted in a complete wireless controller.

Outcome

In this project, we were able to experiment with an adaptive control strategy and a quasi-linear system (discovered after much research and analysis). We completed mathematical modeling and analysis of both the control strategy and the system. This allowed us to complete successful simulations in both MATLAB and V-REP. With promising simulations, we could then continue to implementation to show proof of concept. Successful implementation included control of the Quanser AERO via Simulink, an embedded system, and an embedded system with control from an Android smart phone.