Abstract

This article presents a metacognitive decision-making (MDM) framework inspired by human-like metacognitive principles. The MDM framework is incorporated in unmanned aerial vehicles (UAVs) deployed for decentralized stochastic search without communication for detecting and confirming stationary targets (fixed/sudden pop-up) and dynamic targets. The UAVs are equipped with multiple sensors (varying sensing capability) and search for targets in a largely unknown area. The MDM framework consists of a metacognitive component and a self-cognitive component. The metacognitive component helps to self-regulate the search with multiple sensors addressing the issues of “which-sensor-to-use,” “when-to-switch-sensor,” and “how-to-search.” Based on the information gathered by sensors carried by each UAV, the self-cognitive component regulates different levels of stochastic search and switching levels for effective searching, where the lower levels of search aim to localize a target (detection) and the highest level of a search exploit a target (confirmation). The performance of the MDM framework with two sensors having a low accuracy for detection and increased accuracy to confirm targets is evaluated through Monte Carlo simulations and compared with six decentralized multi-UAV search algorithms (three self-cognitive searches and three self and social-cognitive-based searches). The results indicate that the MDM framework can efficiently detect and confirm targets in an unknown environment.

Abstract

This article proposes the multiswarm cooperative information-driven search and divide and conquer mitigation control (MSCIDC) approach for faster detection and mitigation of forest fires by reducing the loss of biodiversity, nutrients, soil moisture, and other intangible benefits. A swarm is a cooperative group of unmanned aerial vehicles (UAVs) flying together to search and quench the fire areas effectively. The multiswarm cooperative information-driven search uses a two-stage search comprising cooperative information-driven exploration and exploitation for quick/accurate detection of fire locations. The search level is selected based on the thermal sensor information about the potential fire area. The dynamic nature of swarms acquired from global regulative repulsion and merging between swarms reduces the detection and mitigation time compared to the existing methods. The local attraction among the swarm members helps the nondetector members reach the fire location faster, and divide-and-conquer mitigation control ensures a nonoverlapping fire sector allocation for all members quenching the fire. The performance of the MSCIDC has been compared with different multi- UAV methods using a simulated pine forest environment. The Monte-Carlo simulation results indicate that the MSCIDC reduces the average forest area burnt by 65% and mission time by 60% compared to the best case of the multi- UAV approaches, guaranteeing a faster and more successful mission.

Abstract

Developing a reinforcement learning (RL) framework that works satisfactorily in deterministic and stochastic environments is challenging. To address this problem, a twin agent RL framework is proposed in this work, wherein we amalgamate both stochastic and deterministic agents’ actions in a multiagent framework that works with a feedback mechanism that actively monitors the output. The proposed algorithm uses twin actor networks of different agents, corresponding to deterministic and stochastic agents, and an action selection critic network is used to choose the best action from both agents. Here, the algorithm blends the outcomes of two reinforcement learning (RL) agents, a stochastic agent and a deterministic agent, namely, Proximal Policy Optimization (PPO) and Twin Delayed Deep Deterministic Policy Gradient (TD3), respectively. We assess the effectiveness of the proposed algorithm by applying it to two case studies: (i) monoclonal antibody (mAb) production and (ii) production of propylene glycol (PG). The studies are conducted in the presence of parametric uncertainties, measurement noise, and nominal conditions. It is observed that for case study 1, the root-mean-square error (RMSE) value for the proposed algorithm is reduced by 40.9% when compared with TD3 and 27.57% when compared with PPO for the simulations. Similarly, for case study 2, the RMSE for the proposed algorithm is reduced by 8.87% when compared with TD3 and 5.8% with PPO. Based on extensive simulations, it is found that the proposed twin agent algorithm has faster convergence and better set-point tracking when compared to the agents operated individually.

Abstract

Remote labs are a groundbreaking development in the education industry, providing students with access to laboratory education anytime, anywhere. However, most remote labs are costly and difficult to scale, especially in developing countries. With this as a motivation, this paper proposes a new remote labs (RLabs) solution that includes two use case experiments: Vanishing Rod and Focal Length. The hardware experiments are built at a low-cost by retrofitting Internet of Things (IoT) components. They are also made portable by designing miniaturised and modular setups. The software architecture designed as part of the solution seamlessly supports the scalability of the experiments, offering compatibility with a wide range of hardware devices and IoT platforms. Additionally, it can live-stream remote experiments without needing dedicated server space for the stream. The software architecture also includes an automation suite that periodically checks the status of the experiments using computer vision (CV). The software architecture is further assessed for its latency and performance. RLabs is qualitatively evaluated against seven non-functional attributes - affordability, portability, scalability, compatibility, maintainability, usability, and universality. Finally, user feedback was collected from a group of students, and the scores indicate a positive response to the students’ learning and the platform’s usability.

Abstract

Advancements in Deep Reinforcement Learning (DRL) have shown significant promise for the development of self directed Unmanned Aerial Vehicles (UAVs). Our research focuses on examining the impact of time-varying noise such as that present in Inertial Measurement Unit (IMU) sensors on the efficacy of DRL-based waypoint navigation and obstacle avoidance in UAVs. Sensor noise affects localization and obstacle detection, and is assumed to follow a Gaussian probability distribution with an unknown non-zero time varying mean and variance. In this study, we consider environment with static obstacles where noise exhibits a time-varying bias, a characteristic commonly associated with IMU sensors. We evaluate the effectiveness of a DRL agent, trained using the Proximal Policy Optimization (PPO) technique, in the presence of such noise.

Abstract

This paper presents a simple method for stabilizing the quadrotor dynamics under complete loss of one actuator using two-stage optimal control. Detailed equilibrium analysis and subsequent selection of operating point under actuator loss are provided, incorporating constraints on the maximum available thrust. A detailed simulation study using a high-fidelity nonlinear model of the quadrotor is presented showing the stability and performance of the closed-loop system under complete actuator loss, in the presence of external disturbances.

Abstract

Crack segmentation plays a crucial role in ensuring the structural integrity and seismic safety of civil structures. However, existing crack segmentation algorithms encounter challenges in maintaining accuracy with domain shifts across datasets. To address this issue, we propose a novel deep network that employs incremental training with unsupervised domain adaptation (UDA) using adversarial learning, without a significant drop in accuracy in the source domain. Our approach leverages an encoder-decoder architecture, consisting of both domain-invariant and domain-specific parameters. The encoder learns shared crack features across all domains, ensuring robustness to domain variations. Simultaneously, the decoder's domain-specific parameters capture domain-specific features unique to each domain. By combining these components, our model achieves improved crack segmentation performance. Furthermore, we introduce BuildCrack, a new crack dataset comparable to sub-datasets of the well-established CrackSeg9K dataset in terms of image count and crack percentage. We evaluate our proposed approach against state-of-the-art UDA methods using different sub-datasets of CrackSeg9K and our custom dataset. Our experimental results demonstrate a significant improvement in crack segmentation accuracy and generalization across target domains compared to other UDA methods - specifically, an improvement of 0.65 and 2.7 mIoU on source and target domains respectively. Code, models, and dataset will be made available.

Abstract

This paper explores the critical issue of enhancing cybersecurity measures for low-cost, Wi-Fi-based Unmanned Aerial Vehicles (UAVs) against Distributed Denial of Service (DDoS) attacks. In the current work, we have explored three variants of DDoS attacks, namely Transmission Control Proto col (TCP), Internet Control Message Protocol (ICMP), and TCP + ICMPflooding attacks, and developed a detection mechanism that runs on the companion computer of the UAV system. As a part of the detection mechanism, we have evaluated various machine learning, and deep learning algorithms, such as XGBoost, Isolation Forest, Long Short-Term Memory (LSTM), Bidirectional-LSTM (Bi-LSTM), LSTM with attention, Bi LSTM with attention, and Time Series Transformer (TST) in terms of various classification metrics. Our evaluation reveals that algorithms with attention mechanisms outperform their counterparts in general, and TST stands out as the most efficient model with a run time of ∼0.1 seconds. TST has demonstrated an F1 score of 0.999, 0.997, and 0.943 for TCP, ICMP, and TCP + ICMP flooding attacks respectively. In this work, we present the necessary steps required to build an on board DDoS detection mechanism. Further, we also present the ablation study to identify the best TST hyperparameters for DDoS detection, and we have also underscored the advantage of adapting learnable positional embeddings in TST for DDoS detection with an improvement in F1 score from 0.94 to 0.99.

Abstract

Flapping-wing robots (FWR) have domain-specific applications, where the lack of a fast-rotating propeller makes them safer when operating in complex environments with human proximity. However, most existing research in flapping-wing robots focuses on improving range/endurance or increasing payload capacity. This paper proposes a modular powertrain-based flapping-wing robot as a versatile solution to a mission-specific priority switch between payload or range for the same FWR. As the flapping frequency and stroke amplitude directly influence the flight characteristics of the FWR, we exploit this relation when designing our swappable powertrain with different motorgearbox combinations and 4-bar crank lengths to obtain the desired frequency and amplitude. We calculate initial estimates for default configuration and simulate it using pterasoftware. We then fabricate two powertrain modules - a default configuration with a higher flapping frequency for payload purposes and an extended-range configuration with a tandem propeller for higher flight velocity for longer range and endurance. To verify the results, we compare the flight test data of both power train configurations using the same FWR platform.

Abstract

In this study, we address the challenge of obstacle avoidance for Unmanned Aerial Vehicles (UAVs) through an innovative composite imitation learning ap- proach that combines Proximal Policy Optimization (PPO) with Behavior Cloning (BC) and Generative Adversarial Imitation Learning (GAIL), enriched by the integration of ray-tracing techniques. Our research underscores the sig- nificant role of ray-tracing in enhancing obstacle detection and avoidance capabilities. Moreover, we demonstrate the effectiveness of incorporating GAIL in coordinating the flight paths of two UAVs, showcasing improved collision avoidance capabilities. Extending our methodology, we apply our combined PPO, BC, GAIL, and ray-tracing framework to scenarios involving four UAVs, illustrating its scalability and adaptability to more complex scenarios. The findings indicate that our approach not only improves the reliability of basic PPO based obstacle avoidance but also paves the way for advanced autonomous UAV operations in crowded or dynamic environments.