Abstract

Invertible convolutions have been an essential element for building expressive normalizing flowbased generative models since their introduction in Glow. Several attempts have been made to design invertible k × k convolutions that are efficient in training and sampling passes. Though these attempts have improved the expressivity and sampling efficiency, they severely lagged behind Glow which used only 1 × 1 convolutions in terms of sampling time. Also, many of the approaches mask a large number of parameters of the underlying convolution, resulting in lower expressivity on a fixed run-time budget. We propose a k × k convolutional layer and Deep Normalizing Flow architecture which i.) has a fast parallel inversion algorithm with running time O(nk2 ) (n is height and width of the input image and k is kernel size), ii.) masks the minimal amount of learnable parameters in a layer. iii.) gives better forward pass and sampling times comparable to other k × k convolution-based models on real-world benchmarks. We provide an implementation of the proposed parallel algorithm for sampling using our invertible convolutions on GPUs. Benchmarks on CIFAR-10, ImageNet, and CelebA datasets show comparable performance to previous works regarding bits per dimension while significantly improving the sampling time.

Abstract

Cocaine use disorder (CUD) is a compulsive urge to seek and consume cocaine despite the inimical consequences. MRI studies from different modalities have shown that CUD patients exhibit structural and/or functional connectivity pathology among several brain regions. Nevertheless, both connectivities are commonly studied and analyzed separately, which may potentially obscure its relationship between them, and with the clinical pathology. Here, we compare and contrast structural and functional brain networks in CUD patients and healthy controls (HC). The existing body of research has predominantly concentrated on studying substance abuse within the Caucasian population. Gaining a comprehensive understanding of the distinctive challenges and cultural environments in which substance abuse disorders emerge among various populations is essential. For this study, we recruited 105 individuals of Mexican descent and carefully examined the T1-weighted, diffusion weighted imaging, resting-state fc-fMRI sequences and clinical metrics of these participants. Based on the previous work [53], we identified regions of interest (ROIs) that are known to have altered structural connectivity in CUD patients and examined their pairwise functional connectivity. The results demonstrated that CUD exhibited stronger connection between the right Posterior Cingulate and right Postcentral than HC which could suggest an increase in interoception potentially associated with compulsion behavior. We computed a battery of graph-based measures from multi-shell diffusion-weighted imaging (tc-dMRI) and resting state fc-fMRI to quantify local and global connectivity. We investigated the differences in functional and structural modalities between the two groups independently. Unimodal analysis showed an increase in the participation coefficient in Striatum among CUD compared to HC in the fc-fMRI modality which has been previously linked to the compulsive drug-seeking behavior observed in CUD. Multimodal fusion is a data driven approach that involves fusion of different brain modalities to gain a more comprehensive understanding of the brain’s structure and function. Multimodal fusion has played an important role in investigating joint functional and structural changes in neurological disorders like Alzheimer’s disease, schizophrenia, and epilepsy. This is the first study that uses multimodal fusion of graph theoretical measures to investigate topological alterations in CUD patients. We specifically used multimodal canonical component analysis plus joint independent component analysis (mCCA+jICA) to evaluate group differences and their association with clinical scores. When performing multimodal fusion analysis, we observed a higher betweenness centrality and lower participation coefficient in CUD patients indicating reorganization of functional networks. In addition to altered straiatal connectivity revealed by the unimodal approach, multimodal fusion revealed latent information about brain regions involved in impairment due to cocaine abuse. Overall, results revealed by our study not only concord with previous research in the field but also uncovered findings which could help in understanding the pathology of CUD and develop better pre-treatment/post-treatment intervention design.

Abstract

Unmanned Aerial Vehicles (UAVs) continue to penetrate diverse sectors, including agriculture, disaster management, communication, transportation, and defense. The robustness and reliability of their operation have become paramount due to the increasing ubiquity of these systems across numerous sectors. UAVs can undergo faults in their sensors, actuators (motors), and even structure - all of these have different levels of severity and effects on the system. Motor faults can be of three main types, namely, Motor Locking, Loss of Efficiency (LoE), and Loss of Actuation (LoA). Motor locking is a case where the actuator keeps spinning at a given RPM and cannot be changed, whereas LoE is a case where the actuator degrades over time and is not able to generate enough torque due to lower RPM. In this thesis, however, we focus on LoA, where an actuator stops working completely (gets stuck at zero RPM) and can result in catastrophic failure in the UAV. This work delves into the dynamics of UAVs under actuator fault conditions, examines the resulting instability issues, and explores potential methods for identification and control reconfiguration for safe operation. Once an actuator failure occurs, the Fault Detection & Isolation (FDI) module should be able to locate the fault and pinpoint it with high accuracy and in a very short time, to allow the UAV to stabilize the fault. Various approaches can be used for the FDI system, namely, rule-based, model-based, and data-driven. Implementation of these methods can be relying on RPM sensors, battery monitors, and software-only solutions. We propose a data-driven software module for this purpose, since it is UAV frame agnostic, does not require any additional hardware (software-only), and can be run with minimal setup. The proposed Rotation Forest based classifier can detect, classify, and report the fault within 120-250 milliseconds of occurrence of the fault. This is an acceptable delay, since we have observed that the vehicle cannot handle delays upwards of 500 milliseconds in simulations. Once a fault is reported by the FDI module, the Fault Tolerant Control (FTC) module reconfigures the control system to stabilize the vehicle and continue the mission, or prevent a crash by peforming a safe landing. This thesis focuses on cases of complete actuator failure in Hexacopter UAVs, specifically, for single motor failure scenarios. The proposed FTC module performs UAV stabilization using control reconfiguration, after fault occurrence. We perform a thorough analysis for the FDI and FTC modules separately, extensively in simulation, and demonstrate the same in real flight tests. We also combine the two modules and analyze the response in the simulation. In real flights, the classifier (FDI module) responds to the fault in 2-5 sensor data samples (at a 60ms rate per sample) and has a high true positive rate of 92.6%. Also, in real flights, the performance of the FTC module is measured in terms of tracking error, percentage overshoot, and settling time. Integration of the FDI and FTC modules is performed and simulation results are presented showing satisfactory operation of each of these modules with guaranteed stable flight.

Abstract

Automatic Speech Recognition (ASR) systems are increasingly prevalent in our daily lives, with commercial applications such as Siri, Alexa, and Google Assistant. However, the focus of these systems has been largely angled towards English, leaving a considerable portion of non-English speakers underserved. This is particularly evident in India, a linguistically diverse country with many languages classified as low-resource in the context of ASR due to the scarcity of annotated speech data. This thesis aims to bridge this gap, focusing on enhancing ASR systems for Indian languages using deep learning methodologies. India is a land of language diversity. There are approximately 2000 languages spoken around, and among those officially registered are 23. Of those, very few have ASR capability. This is because building an ASR system requires thousands of hours of annotated speech data, a vast amount of text, and a lexicon that can span all the words in the languages. The necessity for a comprehensive presence in the diverse Indian markets demands the development of multilingual Automatic Speech Recognition (ASR) systems. It’s a common scenario where ASR systems for Indian languages have to be implemented in low-resourced contexts. Furthermore, the complexity of the linguistic landscape is amplified due to the high prevalence of bilingualism in the Indian population, leading to frequent instances of code-switching and linguistic borrowing between languages. Operating concurrent ASR systems that can handle code-switching in the Indian context presents a considerable challenge. This predicament has spurred our research endeavors, driving us to focus on constructing a large corpus for one language and leveraging its phonetic space on other language families in monolingual and multilingual ASR scenarios. This thesis incorporates a crowd-sourcing strategy to collect an extensive speech corpus, particularly for Telugu. Using this approach, around 2000 hours of Telugu speech data, capturing regional variations through three modes: spontaneous, conversational, and read, under various background conditions, has been collected. This data served as a foundation for developing and evaluating neural network architectures tailored to the characteristics of Indian languages. We also explored the potential of self-supervised learning to understand and enhance learned representations, fine-tuning them to suit different language families and data sizes. This approach led to insights into the shared phonetic space among Indian languages, allowing for the development of a multilingual ASR system utilizing a joint acoustic model approach. These studies mark a significant stride towards overcoming the challenges of multilingualism in the Indian context, setting a path for creating more inclusive and effective ASR systems. The research findings presented in this thesis not only contribute towards building efficient and accurate ASR systems for low-resource Indian languages but also underscore the power of deep learning approaches in linguistic technology. It is our hope that this work will motivate and aid further research in this direction, promoting linguistic diversity and broadening access to information and communication technologies for speakers of low-resource languages.

Abstract

In this thesis, we try to tackle the problem of erroneous English-Hindi machine translation (MT) outputs due to the presence of the Verb Phrase Ellipsis (VPE) in English. The phenomenon of VPE is prominent in spoken English, and the antecedent to the ellipsis can come from previous sentences in a conversation as well. MT systems translate sentences as a whole and ignore the contextual information from the previous sentences. For these two reasons, spoken English-Hindi translations suffer. We approached this problem by manually annotating 1200 two-person conversations that contain VPE and by studying how their resolution affects the translation qualities. Based on this analysis, we designed a rule-based system for the detection and resolution of VPE in English with the goal of improving their subsequent Hindi translation qualities. Our rule-based system is capable of the following: 1) Detection of VPE, 2) Resolution of Elided Head verb, 3) Resolution of Elided Head verb’s children, 4) Resolution of non-verbal predicates of a copula or a ’be’ main verb, 5) Modifying original sentence in the conversation with the resolved verb phrase. In order to assess the scalability of our rule-based system, we also tested the system’s performance on VPE datasets outside of our annotated data. The system’s performance was also tested on nonconversational data in this research. The scalability test allowed us to utilize our rule-based system for the purpose of data augmentation as well. The augmented data was utilized for the training of a Transformer-Based Seq2Seq generation model. This model helped us investigate whether a sequenceto-sequence model can be designed to handle instances of conversational English VPE, which could be utilized as a pre-processing step before carrying out English-Hindi Machine Translation. Furthermore, a comparative study between the rule-based and the Transformer-based model was carried out as a part of the research supporting this thesis.

Abstract

Multi-armed bandits (MABs) have various applications in real life, however, most of these applications require certain variations to the classic MAB problem. In this thesis, we focus on one such variant - budgeted combinatorial MAB (BCMAB). A BCMAB allows for multiple arms to be pulled in each round with no restrictions on the number of arms selected per round or the budget consumed per round, as long as the arms pulled do not exceed the total budget. The reward structure is taken to be additive, i.e., the reward obtained on pulling a set of arms in a round is the sum of the rewards obtained by the individual arms. We study relevant problems and develop our solutions to BCMAB. We come up with the algorithms CBwK-Greedy-UCB and CBwK-LP-UCB. We mathematically prove regret bound for CBwKLP-UCB. We experimentally analyze our algorithms and compare them with each other and with the previous most suited algorithm.

Abstract

Cooperative human-human communication becomes challenging when restrictions such as difference in communication modality and limited time are imposed. In this thesis, we present the popular cooperative social game Pictionary as an online multimodal test bed to explore the dynamics of humanhuman interactions in such settings. Pictionary is a multiplayer game where the players attempt to convey a word or phrase through drawing. The restriction imposed on the mode of communication gives rise to intriguing diversity and creativity in the players’ responses. To explore the player activity in Pictionary, an online browser-based Pictionary application is developed and utilized to collect a Pictionary dataset. We conduct an exploratory analysis of the dataset, examining the data across three domains: global session-related statistics, target word-related statistics, and user-related statistics. We also present our interactive dashboard to visualize the analysis results. We identify attributes of player interactions that characterize cooperative gameplay. Using these attributes, we find stable role-specific playing style components independent of game difficulty. In terms of gameplay and the larger context of cooperative partially observable communication, our results suggest that too much interaction or unbalanced interaction negatively impacts game success. Additionally, the playing style components discovered via our analysis align with select player personality types proposed in existing frameworks for multiplayer games. Furthermore, this thesis explores atypical sketch content within the Pictionary dataset. We present various baseline models for detecting such atypical content. We conduct a comparative analysis of three baseline models, namely BiLSTM+CRF, SketchsegNet+, and modified CRAFT. Results indicate that the image segmentation-based deep neural network outperforms recurrent models that rely on stroke features or stroke coordinates as input.

Abstract

Ultrasound guided regional anesthesia (UGRA) involves approaching target nerves through a needle in real-time, enabling precise deposition of drug with increased success rates and fewer complications. Development of autonomous robotic systems capable of administering UGRA is desirable for remote settings and localities where anesthesiologists are unavailable. Real-time segmentation of nerves, needle tip localization and needle trajectory extrapolation are required for developing such a system. In the first part of this thesis, we developed models to localize nerves in the ultrasound domain using a large dataset. Our prospective study enrolled 227 subjects who were systematically scanned for brachial plexus nerves in various settings using three different ultrasound machines to create a dataset of 227 unique videos. In total, 41,000 video frames were annotated by experienced anaesthesiologists using partial automation with object tracking and active contour algorithms. Four baseline neural network models were trained on the dataset and their performance was evaluated for object detection and segmentation tasks. Generalizability of the best suited model was then tested on the datasets constructed from separate ultrasound scanners with and without fine-tuning. The results demonstrate that deep learning models can be leveraged for real time segmentation of brachial plexus in neck ultrasonography videos with high accuracy and reliability. Using these nerve segmentation predictions, we define automated anesthesia needle targets by fitting an ellipse to the nerve contours. The second part of this thesis focuses on localization of the needles and development of a framework to guide the needles toward their targets. For the segmentation of the needle, a natural RGB pre-trained neural network is first fine-tuned on a large ultrasound dataset for domain transfer and then adapted for the needle using a small dataset. The segmented needle’s trajectory angle is calculated using Radon transformation and the trajectory is extrapolated from the needle tip. The intersection of extrapolated trajectory with the needle target guides the needle navigation for drug delivery. The needle trajectory’s average angle error was 2 o , average error in trajectory’s distance from center of the image was 10 pixels (2 mm) and the average error in needle tip was 19 pixels (3.8 mm) which is within acceptable range of 5 mm as per experienced anesthesiologists. The entire dataset has been released publicly for further study by the research community.

Abstract

Object Detection and Tracking computer vision algorithms have remarkably progressed in the past few years, owing to the rapid progress in the field of deep learning. These algorithms find numerous applications in surveillance, robotics, autonomous driving, sports analytics, and human-computer interaction. They achieve near-perfect performance on the benchmark datasets they have been evaluated on. However, deploying such algorithms in real-world poses a large number of challenges, and they do not work as well as we expect them to work by looking at their performance on benchmark datasets. In this thesis, we present details of deploying one such Multi Camera Multi-Object Detection and Tracking system to track players in the bird’s eye (top) view in a sports event, specifically in the game of cricket. Our system is able to generate the top-view map of the fielders from cameras placed on crowd stands of a stadium, making it extremely cost-effective compared to using spider cameras. We present details on the challenges and hurdles we faced while deploying our systems and the solutions we designed to overcome these challenges. Ultimately, we tailored a neatly engineered end-to-end system that went live in the Asia Cup of 2022. Deploying such a multi-camera detection and tracking system poses many challenges. The first of them is related to camera placement. We had to devise strategies to place cameras optimally so that all the objects of interest could be captured by one or more of the cameras placed, at the same time ensuring that they do not appear so small that it becomes difficult for a detector to localize them. Constructing the bird’s eye view of the detected players required camera calibration. In the real world, we may not find reliable point correspondences to calibrate a camera. Even if we might find point correspondences, sometimes they may be really hard to see to accurately calibrate a camera. Detecting players in a setup like this proved to be really challenging because the objects of interest appeared very small in the camera views. Moreover, since the detections were coming from multiple cameras, algorithms had to be devised to associate them correctly across camera views. Tracking in a setting like this was even harder because we had to rely only on motion cues to track the objects of interest. Appearance features did not add any value because of the fact all the players being tracked wore jerseys of the same color. Each of these challenges became even more, harder to solve because of the real-time requirements of our use case. Finally, setting up the hardware architecture to receive the real-time live feeds from each camera in a synchronized manner and implementing a fast communication protocol for transmitting data between the various system components required careful design choices to be made, all of which have been presented in detail in this thesis.

Abstract

In the past decade, India witnessed a surge in concentrated urban growth to manifolds. The same trend is visible in hilly regions where the seismic safety of buildings is partially answered. Also, the peak ground accelerations observed in the past earthquakes are in accordance with the design PGA, the associated damages observed are brittle, which is undesirable. This can be mainly attributed to the current design codes not providing sufficient recommendations for the safety of buildings on hill slopes. For example, the code suggests modifications to consider the height of buildings resting on slopes in calculating the lateral forces, but do not discuss the ambiguity in the shear force distribution that is inevitable at the shorter column. The parameters responsible for their ill behaviour must be well understood to improve the safety of hill buildings. Therefore, a methodology is formulated for understanding the effect of varying building dimensions on (i) design stress ratios, (ii) dynamic response, i.e., drifts, and (iii) dynamic characteristics, i.e., modal properties. Correlation matrices are plotted to identify the parameters influencing the behaviour. Further, nonlinear dynamic analysis is performed using a reference structure to detect the failure pattern. It is observed that the predominant failure is due to shear in all uphill columns, followed by the yielding of an immediate story. Based on the parameters identified, a framework is proposed to (i) restrict the shear failure in the uphill columns and (ii) improve the base shear distribution, flexural deformations, and modal properties along and across the valley. A similar nonlinear analysis is performed to confirm the improvement in the behaviour of buildings resting on slopes