Abstract

In the field of aerial transport, specifically in operations involving quadrotors carrying suspended payloads, two critical stages are the attachment and detachment of the payload. These procedures are challenging due to various uncertainties inherent in the quadrotor, environmental variables, and payload swings caused by human or external interactions. These uncertainties can escalate quickly, posing significant risks to the quadrotor and payload and, crucially, to the human operator involved in the attachment/detachment process. Current advanced controllers in this field often need to adequately address these uncertainties, typically treating them as bounded, predictable variables. However, this approach must be revised to manage the unpredictable nature of payload swings during these critical stages. To overcome this limitation, our research introduces an innovative adaptive anti-swing controller. This controller uniquely considers uncertainties state-dependently, specifically tailored to address swing induced by the unpredictable aspects of attaching and detaching suspended payloads. In this work, we have conducted a thorough analytical evaluation of the closed-loop stability of this new system. Additionally, real-time experimental trials have been carried out. The results from these experiments demonstrate a marked performance improvement compared to existing state-of-the-art systems. This novel approach shows promise in enhancing safety and control effectiveness in aerial payload transportation, especially during the vital phases of payload and unclasping.

Abstract

Major goal of this thesis is to study the dialectal variations and improve the performance of speech recognition with an embeddings derived from improved dialect classification system. Initial studies focused on improvement of the dialect classification system with three major dialects (AU:Australian, UK:Britain, and US:American) of English. In order to improve the performance of dialect classification system and based on the analysis of dialectal variations, advanced signal processing approaches were proposed to investigate for dialect classification with traditional i-vector system. The features that provide high spectral resolution will help to capture subtle differences between dialects. So, this thesis proposed to use single frequency filtering (SFF) and zero-time windowing (ZTW) based features that provide high spectral resolution without compromising temporal resolution. Along with frame level spectral resolution, longer temporal context will constitute for dialect classification. So, approaches that enhance the temporal context of proposed features (SFF and ZTW) approaches such as delta and double delta coefficients (∆+∆∆), shifted delta coefficients (SDCs) are experimented. It is observed that dialect classification system has given promising performance with the proposed features with temporal context provided by ∆+∆∆ and SDCs. Further, signal processing approaches that can provide long temporal summarization such as frequency domain linear prediction (FDLP) are proposed for dialect classification. From experiments, with FDLP based features, it is observed that long temporal summarization provided by FDLP based features is advantageous for discriminating dialects. So, both the signal processing approaches that provide high spectral resolution (SFF and ZTW) and long temporal summarization (FDLP) have shown to give promising performance in dialect classification when compared to commonly used STFT based features. Further, due to promising performance by deep neural networks in classification tasks and its ability to provide longer temporal context, simpler (CNN) to advanced deep neural network (TCN, TDNN, and ECAPA-TDNN) architectures that provide different temporal contexts are investigated, it is observed that advanced neural network architectures improved the performance of dialect classification. Further, on evaluation of the best of both stages, it is observed that ECAPA-TDNN performed better with proposed features (SFF). The dialectal variations in speech degrade the performance of multi-dialectal automatic speech recognition (ASR) system. The embeddings derived from the best dialect classification system are applied to multi-dialect (with AU, UK, and US dialects) ASR and found to improve the performance of the ASR system. In most studies, Indian English is considered as a single dialect even though it has different native speakers. So, the inclusion of foreign dialectal embeddings improved the performance of the ASR system. The observations made in dialect classification systems with major dialects of English are extended to foreign dialect classification (i.e., native language (or L1) identification). The embeddings extracted from the improved dialect classification system are included along with the Indian English ASR system to improve the performance.

Abstract

About 57% of the land area in India lies within the moderate-to-severe seismic zone, where approximately 80% of the population resides. Consequently, a substantial proportion of India's buildings are at risk due to seismic hazard and building stock in seismically active regions. The pre-earthquake safety assessment of the built environment plays a crucial role in mitigating damage and losses by identifying structures that are susceptible to earthquakes and implementing suitable measures to minimize their impact. The process of rapid visual screening (RVS) plays a vital role in evaluating the seismic vulnerability and potential risks of existing buildings in urban areas. The preservation and safety of existing buildings assume greater significance as urban areas undergo continuous growth and expansion. The Rapid Visual Survey (RVS) is a crucial instrument used to identify structures that necessitate more comprehensive assessment and potential retrofitting in order to improve their ability to withstand seismic activity. In the present study, a total of 15 reinforced concrete framed buildings of above 10 m. Height were selected randomly from nearly 6 different locations in Vijayawada city falling in Seismic Zone III. Studies for Rapid Visual Assessments as well as detailed assessment were conducted and the results of both assessments are discussed. The study finds: (i) the construction practices in India must be checked regularly and also must evaluated regularly; (ii) there is urgent need to spread the awareness among engineers and builders about the seismic code provisions and municipal guidelines.

Abstract

The digital age presents an overwhelming deluge of multimodal data, underscoring the imperative need for effective text summarization techniques. Such techniques transform vast amounts of textual and visual data into concise, comprehensible, and insightful summaries, facilitating information retrieval, comprehension, and decision-making. This thesis pioneers innovative strategies to enhance text summarization by employing various forms of contextual guidance and multimodal data, contributing significantly to the evolution of the field and offering a cohesive narrative that links these diverse yet interconnected areas of study. The journey begins with an exploration of ”Popularity Forecasting” of sentences within news articles. This novel approach surpasses traditional salience-based extractive summarization by predicting the ’popularity’ or ’eye-catching’ potential of sentences. We create a popularity dataset which contains news articles from CNN/DM[47] with their sentence-popularity score mapping. We create this by comparing sentences with the search queries for the particular article. Then we adapt trained extractive summarizers to perform regression tasks and predict the popularity of a particular sentence within a news article. The result is a ranking of sentences based on their popularity scores Next, the research advances into the realm of ”Multimodal Summarization,” which synergizes textual and visual elements to create a more holistic summary. By pairing concise textual summaries with the most salient images from news articles, this technique delivers a richer and more comprehensive understanding of the content. In this work we also show that we can improve the accuracy of summarization models by using images to aid the summarization process. To do this we utilize visuolinguistic transformers like CLIP[54], OSCAR[36] to help in the interaction of the two modalities and we adapt general summarization models so that we can incorporate both textual and visual information in the summarization model Building on the foundation of extractive summarization, and using the core logic from the multimodal summarization work the study then introduces ”Guided Summarization.” This innovative method uses salience scores of sentences, obtained from an extractive summarizer, to guide an abstractive summarizer. This symbiotic relationship between the two forms of summarization results in more contextually relevant and focused abstract summaries. The research further pushes the boundaries of personalization with ”Persona-based Summarization,” applied to SEBI legal case files. This technique generates tailored summaries based on the specific information needs of different personas such as investors, defense lawyers, and judges. It underscores the potential of personalization in text summarization, making the information more accessible and relevant to each user profile. Finally, building on the insights gleaned from the exploration of multimodal summarization, the study culminates with the creation of an ”Indic Multimodal Text-Image Pair Dataset.” This unique resource is a rich assembly of text and image pairs of different Indian languages, serving as a critical foundation for the development and evaluation of visuolinguistic transformers, especially those focusing on data from the Indian subcontinent. In summary, this thesis provides a comprehensive exploration of how contextual guidance and multimodal data can significantly enhance text summarization. The innovative techniques and resources proposed and developed in this research, connected through a cohesive narrative, promise to significantly advance the field of text summarization, paving the way for more engaging, comprehensive, and personalized summary generation

Abstract

In the realm of biometric security, face recognition technology plays an increasingly pivotal role. However, as its adoption grows, so does the need to safeguard it against adversarial attacks. Attacks involve presenting images of a person printed on a medium or displayed on a screen. Detection of such attacks relies on identifying artifacts introduced in the image during the printing or display and capture process. Adversarial Attacks try to deceive the learning strategy of a recognition system using slight modifications to the captured image. Evaluating the risk level of adversarial images is essential for safely deploying face authentication models in the real world. Among these, physical adversarial attacks present a particularly insidious threat to face antispoofing systems. Popular approaches for physical-world attacks, such as print or replay attacks, suffer from some limitations, like including physical and geometrical artifacts. The presence of a physical process (printing and capture) between the image generation and the PAD module makes traditional adversarial attacks non-viable. Recently adversarial attacks have gained attraction, which try to digitally deceive the learning strategy of a recognition system using slight modifications to the captured image. While most previous research assumes that the adversarial image could be digitally fed into the authentication systems, this is not always the case for systems deployed in the real world. This thesis delves into the intriguing domain of physical adversarial attacks on face antispoofing systems, aiming to expose their vulnerabilities and implications. Our research unveils novel methodologies using white box and black box approaches to craft adversarial inputs capable of deceiving even the most robust face antispoofing systems. Unlike traditional adversarial attacks that manipulate digital inputs, our approach operates in the physical domain, where printed images and replayed videos are utilized to mimic real-world presentation attacks. By dissecting and understanding the vulnerabilities inherent in face antispoofing systems, we can develop more resilient defenses, contributing to the security of biometric authentication in an increasingly interconnected world. This thesis not only highlights the pressing need to address these vulnerabilities but also motivates towards a pioneering approach by exploring simple yet effective attack strategy to advancing the state of the art in face antispoofing security.

Abstract

In the infrastructure industry, the focus is shifting from constructing new suburban buildings to maintaining and rehabilitating existing structures, particularly high-rise buildings. These buildings are prone to various failures due to age, density, and altitude, making regular maintenance vital for safety and longevity. However, traditional inspection methods using heavy machinery and risky rappelling are time-consuming and costly. They often fail to provide comprehensive details for inaccessible surfaces. Finding safer and more efficient inspection approaches is essential to ensure building safety and durability. We introduce an innovative end-to-end pipeline designed specifically for high-rise building inspection. Our proposed method incorporates several key components to ensure effective inspection processes. Firstly, we develop a trajectory generation system for an unmanned aerial vehicle (UAV). This system optimizes the UAV’s trajectory, enabling it to reach the desired destination even in the presence of obstacles. The trajectory can be dynamically adjusted in real-time to ensure efficient navigation. During the UAV’s flight, it captures images of the high-rise building using predetermined camera and drone parameters. These images serve as the basis for building inspections, particularly in detecting cracks. Moreover, the collected pool of images is utilized to construct a detailed 3D mesh model of the high-rise building. This model allows for a comprehensive representation of the structure, facilitating the identification and visualization of detected cracks. By combining trajectory optimization, image capture, crack detection, and 3D mesh modeling, our proposed pipeline offers a comprehensive approach to high-rise building inspection. It presents a promising solution to enhance the efficiency, accuracy, and safety of inspection processes in the field of structural engineering. Our work explores the underexplored domain of deep learning in thermalimaging. It focuses on the classical problem of generating high-resolution images from low-resolution counterparts using Superresolution (SR) techniques. The objective is to extract more details from the original scene by increasing the pixel density, which is particularly valuable in computer vision applications, including pattern recognition and medical imaging. The proposed pipeline aims to achieve real-time video super-resolution using a thermal camera on an embedded edge device.

Abstract

In this thesis, we focus on the secure storage and sharing of Personal Health Records (PHRs) in Internet of Medical Things (IoMT). Due to the high value of personal health information, PHRs are one of the favorite targets of cyber attackers worldwide. Over the years, many solutions, including those based on blockchain, have been proposed; however, most solutions are inefficient for practical applications. For instance, several existing schemes rely on bilinear pairings, which incur high computational costs. To mitigate these issues, we propose a novel PHR-sharing scheme that is dynamic, efficient, and practical. Specifically, we combine searchable symmetric encryption, blockchain technology, and a decentralized storage system known as the Inter-Planetary File System (IPFS) to guarantee the confidentiality of PHRs, verifiability of search results, and forward security. We start by introducing cloud and blockchain and then provide a detailed literature survey of existing works that make use of these technologies for storage and sharing of health records. We then propose our own scheme which provides better security and better efficiency than existing schemes for storage and sharing of PHRs. Moreover, we provide formal security proofs for the proposed scheme and also provide extensive test-bed experimental results which demonstrate that the proposed scheme can be used in practical scenarios related to the IoMT environment.

Abstract

The combination of Internet of Things (IoT) and Machine Learning (ML) technologies has the potential to completely transform air pollution monitoring and management. Low-cost air pollution measurement sensors have grown in popularity in recent years because they enable a low-cost solution to measure air quality in real time. However, these sensors frequently have accuracy concerns and must be calibrated to ensure that their results are valid. This thesis mainly focuses on calibrating low-cost sensors using ML models and the detection of pollutant hot spots in urban areas using IoT devices on a mobile platform. In mobile monitoring of air pollution, IoT-enabled sensors assembled onto automobiles, and portable devices offer real-time data collecting when traveling between areas. This dynamic technique provides useful insights into real-world pollution fluctuations, allowing the identification of pollution sources and trends along traffic routes. The study highlights the importance of calibrating the IoT devices in mobile setting, even when the devices are calibrated in laboratory settings. Real-time ML algorithms can be used to calibrate sensor data based on location, weather, and additional relevant data. In this thesis, a methodology is proposed for detecting emission spikes of PM2.5, CO, and NO2 in polluted urban environments employing portable low-cost sensors. Identification of harmful pollutant concentrations is achieved using two different IoT device types (MegaSense One and Prana Air) mounted on a mobile platform. Reliable identification of the PM2.5, CO, and NO2 emission spikes can be attained by driving through the city on different days. IoT device measurement errors were corrected by ML based calibration against a reference instrument co-located on a mobile platform. ML regression models like simple linear regression (LR), multivariate linear regression (MLR), polynomial regression (PR), support vector regression (SVR), decision tree regression (DT), random forest regression (RF) were applied to calibrate the devices.Among these models RF was the most suitable technique to reduce the variability between the IoT devices due to heterogeneity in the mobile sensing datasets. The spatial variability of PM2.5, CO, and NO2 harmful emission spikes at a resolution of 50 m were identified, but their intensity changes on a daily basis according to meteorological conditions. The data from the PM2.5, CO, and NO2 emission spikes at points of interests that disturb traffic flows clearly show the need for public education about when it is hazardous for persons with respiratory conditions to be outside, as well as when it is unsafe for young children and the elderly to be outside for extended periods of time. This detection strategy is adaptable to any mobile platform used by individuals traveling by foot, bicycle or drones in any metropolis.

Abstract

Biometric technology has long relied on fingerprint recognition as a trusted means of identity verification. While fingerprint enhancement shares similarities with general image denoising tasks, the unique properties of biometric data make it crucial to treat fingerprints differently from typical realworld images. Deep learning methods have the capacity to capture and improve complex patterns and subtle details in fingerprint data, offering a comprehensive solution. This thesis proposes a deep learning-based method, specifically leveraging the U-Net architecture to achieve superior fingerprint enhancement. At its core, the approach employs a multi-task deep network with domain knowledge from minutia and orientation fields for the enhancement of rolled and plain fingerprint images. We investigate different configurations of the U-Net architecture, examine the effects of various architectural elements, and present extensive experimental evidence to support the effectiveness of our proposed approach. Subsequently, we explore a self-supervised learning paradigm with fingerprint biometrics for robust feature representation learning. In this direction, we introduce a pre-training technique by utilizing the learned information from the enhancement task. We adopt the enhancement pre-trained encoder for learning fixed-length fingerprint embeddings. We evaluate the performance of the learned embeddings in the verification task compared to standard self-supervised techniques without the explicit need for enhanced fingerprints. By merging the capabilities of deep learning with the specificity of fingerprint features, the proposed paradigm offers significant improvements in the robustness and accuracy of fingerprint verification. Experimental evaluations on standard benchmarks such as the NIST and FVC datasets demonstrate the efficacy of this approach. We present compelling evidence that our methodology not only improves the quality of fingerprint images but also facilitates a more effective embedding extraction, ultimately leading to enhanced recognition performance. This work highlights the promising role that deep learning, tailored to biometrics, can play in advancing the field of fingerprint recognition. In addition, it opens up avenues for future research in biometrics, particularly in optimizing computational efficiency and in tackling challenges related to partial and distorted fingerprints.

Abstract

Advancements in 3D sensing, driven by the adoption of LiDAR technology, have reignited innovation in autonomous navigation. LiDAR sensors offer real-time, large-scale point clouds, surpassing traditional vision solutions. Datasets like nuScenes and KITTI empower researchers in tasks such as Localization, Place Recognition, and Obstacle Trajectory Prediction. This thesis contributes by exploring the modeling and prediction of large-scale point cloud sequences. Additionally, it showcases a practical application, representing point-cloud sequences as occupancy grid maps and generating trajectories for autonomous navigation. This dual focus enhances our understanding of autonomous navigation complexities, providing valuable insights into real-world implementation challenges. The first study introduces ATPPNet, an innovative architecture tailored to predict future point cloud sequences using Conv-LSTM, channel-wise and spatial attention, and a 3D-CNN branch. Extensive experiments conducted on publicly available datasets demonstrate the model’s impressive performance, outperforming existing methods. The thesis includes a comprehensive ablative study of ATPPNet and an application study showcasing its potential for tasks such as odometry estimation. In the second study, the thesis proposes NeuroSMPC, a novel integration of data-driven frameworks with sampling-based optimal control for real-time applications, particularly onroad autonomous driving. The 3D-CNN layers in NeuroSMPC predicts optimal mean control without iterative resampling, reducing computation time. The approach proves effective in generating diverse control samples around the predicted optimal mean, facilitating real-time trajectory rollout in the presence of dynamic obstacles. The 3D-CNN architecture implicitly learns future trajectories of dynamic agents, ensuring collision-free navigation without explicit future trajectory predictions. Performance gains are demonstrated over multiple baselines in on-road scenes through closed-loop simulations in CARLA. Real-world applicability is showcased by deploying the system on a custom Autonomous Driving Platform (AutoDP). These studies collectively advance the understanding and implementation of autonomous navigation systems in dynamic environments.