Abstract

Due to their complex nonlinear dynamics and batch-to-batch variability, batch processes pose a challenge for process control. Due to the absence of accurate models and resulting plant-model mismatch, these problems become harder to address for advanced model-based control strategies. Reinforcement Learning (RL), wherein an agent learns the policy by directly interacting with the environment, offers a potential alternative in this context. RL frameworks with actor-critic architecture have recently become popular for controlling systems where state and action spaces are continuous. It has been shown that an ensemble of actor and critic networks further helps the agent learn better policies due to the enhanced exploration due to simultaneous policy learning. To this end, the current study proposes a stochastic actor-critic RL algorithm, termed Twin Actor Soft Actor-Critic (TASAC), by incorporating an ensemble of actors for learning, in a maximum entropy framework, for batch process control.

Abstract

The plants of nano aerial vehicles (NAVs) are inherently unstable. Hence, a NAV needs a flight controller to accomplish a mission. Furthermore, the sensing and computational capabilities of NAV's autopilot hardware are limited. Hence, the implementation of the full state feedback controllers with gain scheduling is difficult. This paper proposes a flight controller scheme that consists of two parts: a Simultaneously Stabilizing Output Feedback Linear (SSOFL) controller and a Proximal Policy Optimization (PPO) deep reinforcement learning agent, which is connected in parallel to the SSOFL controller. In this scheme, the single SSOFL controller provides stabilization and nominal tracking performance to the NAV throughout its flight envelope by accomplishing simultaneous stabilization (SS). Additionally, the PPO agent is trained using the closed-loop (CL) nonlinear plant with this SSOFL controller to enhance the tracking performance. The effectiveness of the proposed flight controller scheme is verified using the six-degree-of-freedom nonlinear simulations of the fixed-wing nano aerial vehicle.

Abstract

This paper proposes a Reinforcement Learning (RL) based technique to develop a simple neural network controller for the task of waypoint navigation in quadrotors. In this paper, the application of Twin Delayed Deep Deterministic (TD3) Policy Gradient algorithm for high and low-level control implementation for quadrotors is discussed. The proposed methods are tested on high fidelity Gym-Pybullet-Drones simulator. The effectiveness of the methods developed is validated through numerical simulations. The simulation results indicate that both control policies are successful in navigating through the assigned waypoint, with the low-level controller being accurate in the nominal flight conditions. In the presence of disturbance inputs, the high-level controller performs better when compared to the low-level controller.

Abstract

In this paper, the Multi-Swarm Cooperative Information-driven search and Divide and Conquer mitigation control (MSCIDC) approach is proposed for faster detection and mitigation of forest fire by reducing the loss of biodiversity, nutrients, soil moisture, and other intangible benefits. A swarm is a cooperative group of Unmanned Aerial Vehicles (UAVs) that fly together to search and quench the fire effectively. The multiswarm cooperative information-driven search uses a multi-level search comprising cooperative information-driven exploration and exploitation for quick/accurate detection of fire location. The search level is selected based on the thermal sensor information about the potential fire area. The dynamicity of swarms, aided by global regulative repulsion and merging between swarms, reduces the detection and mitigation time compared to the existing methods. The local attraction among the members of the swarm helps the non-detector members to reach the fire location faster, and divide-and-conquer mitigation control ensures a non-overlapping fire sector allocation for all members quenching the fire. The performance of MSCIDC has been compared with different multi-UAV methods using a simulated environment of pine forest. The performance clearly shows that MSCIDC mitigates fire much faster than the multi-UAV methods. The Monte-Carlo simulation results indicate that the proposed method reduces the average forest area burnt by 65% and mission time by 60% compared to the best result case of the multi-UAV approaches, guaranteeing a faster and successful mission. Index Terms—Swarm search, forest firefighting, cooperative information-driven search, divide and conquer mitigation control

Abstract

This paper presents detailed mathematical modeling, controller design, and flight test results for the autonomous mission of a nonconventional fixed-wing biplane micro air vehicle (MAV) called Skylark having span and chord length within 150 mm. Although numerous fixed-wing MAV designs are reported in the open literature, an MAV’s reliable autonomous flight is still a challenge. The primary difficulties in performing the autonomous mission of fixed-wing MAV are system integration within the weight and power budget, center of gravity (CG) management, and flight controller design for fast dynamics. In this paper, the key challenges are addressed by using the higher payload capacity of Skylark, suitable selection and design of avionics components, and design of controller after detailed development of the mathematical model, including the additional effect of propeller wash and motor countertorque. An …

Abstract

In this paper, we solve an optimal consensus control problem of maximizing the state-dependent communi- cation connectivity during a multi-agent consensus dynamics. A proportional-derivative type consensus controller is leveraged to drive agents into a symmetric formation. The asymptotic stability of the closed-loop system dynamics is established using Lyapunov theory, which helps us to deduce an intuitive time-varying gain profile based on a sufficient condition for convergence. Further, a Model Predictive Control approach is adopted to minimize a quadratic cost over a finite prediction horizon by adjusting the controller gains, such that the optimal connectivity is attained on the way with less control efforts, while handling constraints to agents’ states, inputs, turn-rates and disturbances injected into agent velocities. Simulation results with time-varying controller gains demonstrate the impact of our proposed technique

Abstract

A detailed design approach undertaken in the development of “Skylark” is presented in this paper. “Skylark” is a non-conventional fixed-wing biplane Micro Air Vehicle (MAV) with a wingspan and chord length within 150 mm. It is specially designed with the ability to host onboard vision-assisted autonomous navigation systems. Fixed-wing MAV with capabilities of vision assisted autonomous navigation is not reported in the open literature. To stay within the maximum dimensional constraint, flying wing configuration with a low aspect ratio is preferred for MAV design, and therefore, the stability is inadequate due to lower static margin when compared to bigger Unmanned Aerial Vehicles (UAVs). In this paper, the novel design strategy addresses the major challenges such as high payload-carrying capacity, stability, and onboard processing required for vision-assisted autonomous navigation. The higher payload …

Abstract

Unmanned Aerial Vehicle (UAV) based remote sensing system incorporated with computer vision has demonstrated potential for assisting building construction and in disaster management like damage assessment during earthquakes. The vulnerability of a building to earthquake can be assessed through inspection that takes into account the expected damage progression of the associated component and the component’s contribution to structural system performance. Most of these inspections are done manually, leading to high utilization of manpower, time, and cost. This paper proposes a methodology to automate these inspections through UAV-based image data collection and a software library for post-processing that helps in estimating the seismic structural parameters. The key parameters considered here are the distances between adjacent buildings, building plan-shape, building plan area, objects on the rooftop and rooftop layout. The accuracy of the proposed methodology in estimating the above-mentioned parameters is verified through field measurements taken using a distance measuring sensor and also from the data obtained through Google Earth. Additional details and code can be accessed from h

Abstract

In this paper, we propose a predictive and cooperative optimal control based autonomous navigation and collision avoidance for multiple fixed-wing aircraft (FWA) that also takes into account the physical constraints of the aircraft. The proposed method is implemented in a cooperative framework, analogous to the Reciprocal Velocity Obstacle (RVO) framework for autonomous navigation and collision avoidance for FWA swarm moving in a three dimensional (3D) space. Also, the change in performance with respect to varying prediction horizon is analyzed through numerical simulations. As a result of predictive optimal control and cooperation, we get less conservative maneuvers, reduced path lengths, and less deviation from the shortest path when compared to the conventional method without prediction and cooperation. Simulation results are provided, highlighting the advantages of the proposed method when compared to a popular method (Optimal reciprocal collision avoidance) for collision avoidance in 3D for FWA swarm and also against FGA algorithm (Fast geometric avoidance algorithm).

Abstract

Some of the issues associated with the prac- tical applications of consensus of multi-agent systems (MAS) include switching topologies, attrition and inclusion of agents from an existing network, and model uncertain- ties of agents. In this paper, a single distributed dynamic state-feedback protocol referred to as the Robust Attrition- Inclusion Distributed Dynamic (RAIDD) consensus proto- col, is synthesized for achieving the consensus of MAS with attrition and inclusion of linear time-invariant higher-order uncertain homogeneous agents and switching topologies. A state consensus problem termed as the Robust Attrition- Inclusion (RAI) consensus problem is formulated to find this RAIDD consensus protocol. To solve this RAI con- sensus problem, first, the sufficient condition for the exis- tence of the RAIDD protocol is obtained using the ν-gap metric-based simultaneous stabilization approach. Next, the RAIDD consensus protocol is attained using the Glover- McFarlane robust stabilization method if the sufficient con- dition is satisfied. The performance of this RAIDD protocol is validated by numerical simulations.