Abstract

“Your brain is constantly rewiring itself, adapting with each new experience. It’s a dynamic, ever-changing network of connections, shaped by the world around you.” - David Eagleman This thesis investigates the reorganization of brain networks during the aging process, particularly in the transition from young to middle-aged to older adults. The study utilizes the CAMCAN dataset, a comprehensive cross-sectional dataset with multimodal data, including pre-processed resting-state fMRI (rs-fMRI) data. The aim is to understand how brain networks evolve with age and identify key brain regions and network properties that undergo changes. Data-driven statistical and graph-theoretic measures are employed to study modular segregation and integration in the brain. The results reveal characteristic nodes forming stable cores and flexible peripheries in both young and old age groups. Notably, regions within the Default Mode network (DMN) show a negative correlation with modularity in the old age group, while regions from the Limbic, SensoriMotor (SMN), and Salience networks display a positive correlation. Machine learning models based on flexibility scores further validate the relevance of these regions, providing promising insights for future investigations. Additionally, the study uncovers age-related changes in brain connectivity and network properties. Modularity increases with age, indicating greater functional specialization in the aging brain, accompanied by a decrease in flexibility, suggesting reduced adaptability to changing cognitive demands. The negative correlation between flexibility and modularity across all age groups implies that as the brain becomes less modular, it becomes more flexible in its organization. Certain brain regions show significant connectivity alterations, with increased participation coefficient in some frontal and temporal regions and decreased participation coefficient in several frontal and parietal regions. Hemispheric differences indicate that age-related connectivity changes may vary between hemispheres. Furthermore, the complexity of the relationship between cognitive abilities, task performance, and brain network dynamics is highlighted. The absence of strong correlations between task scores and network measures at the nodal level, along with the weak correlation between Cattell scores and global flexibility, underscore the multifaceted nature of these associations. Age alone cannot fully account for the observed dynamics, suggesting the involvement of other factors in shaping the relationship between cognition and brain network measures. However, this study has limitations. The use of cross-sectional data hinders the exploration of individual changes over time, and longitudinal data would provide more robust insights. The dataset’s relatively small size and the categorization of age into discrete groups may limit the generalizability of the findings. Moreover, relying solely on resting-state fMRI data may not fully capture the dynamic nature of brain function during various cognitive processes. Additionally, causal inferences should be made with caution, as the study is based on observational data, and other factors may influence the observed brain network changes. In conclusion, this thesis contributes to understanding age-related changes in brain networks and their impact on cognitive aging. The findings highlight the importance of specific brain regions in maintaining functional networks during aging and underscore the complexity of brain network dynamics. Addressing the limitations in future research will enhance our knowledge of brain aging and the interplay between brain networks, cognitive abilities, and behavior.

Abstract

Code-switching or code-mixing occurs when ”lexical items and/or grammatical features from two languages appear in one sentence”. With the rising popularity of social media platforms such as Twitter, Facebook, and Reddit, the volume of texts on these platforms has also grown significantly. Twitter alone has over 500 million text posts (tweets) per day. India, a country with over 300 million multilingual speakers, has over 23 million users on Twitter as of January 2022, and code-switching can be observed heavily on this social media platform. Code-mixed social media posts present unique challenges due to the mixing of languages, slang, and informal expressions. Extracting valuable information from code-mixed data enables a better understanding of user sentiments, preferences, and trends across different language communities. It helps identify named entities, detect events, and extract relevant information from multilingual conversations. By effectively extracting and analyzing code-mixed content, businesses, researchers, and language processing systems can gain valuable insights into diverse language patterns, linguistic phenomena, and cultural nuances, contributing to more accurate language processing, sentiment analysis, and targeted communication strategies. Extracting valuable information from social media content has numerous benefits that impact various domains. This has led to a growing interest in Information Extraction (IE) as an active research area in artificial intelligence. However, research efforts are often hindered by the lack of automated NLP tools to analyse massive amounts of code-mixed data, especially for resource-poor languages. There have been significant efforts towards understanding some languages, such as English, German, French, and Spanish, by creating annotated resources for various tasks, while some languages have not received the same focus, making them resource-poor. One such language is Kannada, which has over 58 million native and secondarylanguage (L2) speakers worldwide. Like in most places around the world, Kannada speakers tend to use code-mixed language, mixed with English, in informal settings including social media and texting platforms, but the same has not received much focus from researchers. In this thesis, we work towards information extraction from Kannada-English code-mixed data from social media. We work on the problems of Named Entity Recognition (NER) and Event Detection. Named Entity Recognition (NER) and Event Detection on social media code-mixed data are crucial for understanding and analyzing user-generated content in diverse languages. They enable the identification of named entities, such as names of people, organizations, and locations, as well as the detection of events, allowing for deeper insights into multilingual conversations, cross-cultural trends, and effective information retrieval from code-mixed social media posts. We have collected Kanglish code-mixed data from social media, Twitter, and curated annotated datasets for both NER and Event Detection tasks while mentioning the annotation guidelines for the same. We have analysed the challenges that are unique to Kannada-English code-mixed data and have provided annotation guidelines for the same. We have also proposed a few supervised approaches towards these tasks on our dataset with careful feature selection and critically analysed the results in hopes to promote more focus from the research community on such low-resource languages.

Abstract

Videos have become an integral part of our daily digital consumption. With the widespread adoption of mobile devices, internet connectivity, and social media platforms, the number of online users and consumers has risen exponentially in recent years. This has led to an unprecedented surge in video content consumption and creation, ranging from short-form content on TikTok to educational material on Coursera and entertainment videos on YouTube. Consequently, there is an urgent need to study videos as a modality in Computer Vision, as it can enable a multitude of applications across various domains, including virtual reality, education, and entertainment. By understanding the intricacies of video content, we can unlock its potential and leverage its benefits to enhance user experiences and create innovative solutions. Producing video content at scale can be challenging due to various practical issues. The recording process can take several hours of practice, and setting up the right studio and camera equipment can be time-consuming and expensive. Moreover, recording requires manual effort, and any mistakes made during the shoot can be difficult to rectify or modify, often requiring the entire video to be re-shot. In this thesis, we aim to ask the question “Can synthetically generated videos take the place of real videos?” as automatic content creation can significantly scale digital media production and ease the process of content creation that can aid several applications. A form of human-centric representation that is becoming increasingly popular in the research community is the ability to generate talking-head videos automatically. Talking-head generation refers to the ability to generate realistic videos of a person speaking, where the generated video can be of a person that may not exist in reality or may exhibit significantly different characteristics than the original person. Recent deep learning approaches can synthesize synthetic talking-head videos at tremendous scale and quality, with diverse content and styles, that are visually indistinguishable from real videos. Therefore, it is imperative to study the process of generating talking-head videos as these videos can be used for a variety of applications, such as video conferencing, movie-making, broadcasting news, vlogging, and language learning among others. Consider a digital avatar reading news from a text transcript being broadcasted on news. In this vein, this thesis aims to explore two prominent use cases of generating synthetic talkingheads automatically - the first one towards generating large-scale synthetic content to aid people in lipreading at scale. The second use case is for automating the task of actor-double face-swapping in the moviemaking industry. We study and elucidate the challenges and limitations of the existing approaches, propose solutions based on synthetic talking head generation, and show the superiority of our methods through extensive experimental evaluation and user studies. In the first task, we address the challenges associated with learning to lipread. Lipreading is a primary mode of communication for people suffering from some form of hearing loss. Therefore, learning to lipread is an important aspect for hard-of-hearing people. However, learning to lipread is not an easy task and finding resources to improve one’s lipreading skills can be challenging. Existing lipreading training websites that provide basic online resources to improve lipreading skills, are unfortunately, limited by real-world variations in the talking faces, cover only a limited vocabulary, and are available in a few select languages and accents. This leaves the vast majority of users without access to adequate lipreading training resources. To address this challenge, we propose an end-to-end pipeline to develop an online lipreading training platform using state-of-the-art talking head video generator networks, textto-speech models, and computer vision techniques, to increase the amount of online content on the LRT platforms in an automated and cost-effective manner. We show that incorporating existing talking heading generator networks for the task of lipreading is not trivial, and requires careful adaptation. For instance, we develop an audio-video alignment module that aligns the speech utterance on the region with the mouth movements and adds silence around the aligned utterance. Such modifications are necessary to generate realistic-looking videos that don’t cause distress to the lipreaders. We also design carefully thought out lipreading training exercises, conduct extensive user studies, and perform statistical analysis to show the effectiveness of the generated content in replacing the manually recorded lipreading training videos. In the second problem, we address challenges in the entertainment industry. Body doubles play an indispensable role in the moviemaking industry. They take

Abstract

Word problem Solving is a popular NLP task that deals with solving mathematical problems described in natural language. Mathematical Word Problems cover problems over a large mathematical domain with various complexities ranging from Arithmetic and Algebraic to Geometry and Calculus. While most word problems are entirely textual, some word problems, like geometric word problems, may also have a visual component. Much research has been carried out to solve different genres of word problems with various complex- ity levels in recent years. However, most publicly available datasets and work are in English. Recently there has been a surge in word problem-solving in Chinese with the creation of large benchmark datasets. Labelled benchmark datasets for low-resource languages are very scarce. The first requirement for solving word problems, like any other problem, is data. To the best of our knowledge, no datasets are available for any Indian Languages for Word Problem Solving. Such limitations on data availability not only encouraged us to create a new dataset for an Indian Language but also made us explore techniques by which data for other Indian Languages can be created with ease. In this work, we present a diverse dataset containing 2336 Arithmetic word problems in Hindi built by manually crafting word problems and using word problems augmented from benchmark datasets of other languages. For augmentation, we used translation of word problems (of other languages in which Word Problem Solving data was developed) as a tool to generate diverse word problems. In this process, we studied the translated word problems, gathered the patterns of issues seen in these translations and defined the steps to eliminate the errors and improve the quality of the translated output for it to be suited to be a set of Hindi word problems that look as natural as word problems that are studied across India in Hindi medium schools. We also developed baseline systems for solving these word problems - a rule-based solver that uses verbs to identify operations for generating the answers to word problems and an end-to-end deep learning-based solver that generates equations for word problems. We also propose a new evaluation technique for word problem solvers taking equation equivalence into account. This will form the basis for future work for Word Problem Solving in Indian Languages, especially in Hindi.

Abstract

Autonomous driving requires accurate reasoning of the location of objects from raw sensor data. Recent end-to-end learning methods go from raw sensor data to a trajectory output via Bird’s Eye View (BEV) segmentation as an interpretable intermediate representation. Motion planning over cost maps generated via Birds Eye View (BEV) segmentation has emerged as a prominent approach in autonomous driving. However, current approaches have two critical gaps. First, the optimization process is simplistic and involves just evaluating a fixed set of trajectories over the cost map, which are not adapted based on their associated cost values. Second, the existing cost maps do not account for the uncertainty arising from noise in RGB images, BEV annotations. As a result, these approaches can struggle in challenging scenarios where there is abrupt cut-in, stopping, overtaking, and merging from neighboring vehicles. In this thesis, we propose UAP-BEV, a novel approach that models the noise in Spatio-Temporal BEV predictions to create uncertainty-aware occupancy grid maps. Using queries of the distance to the closest occupied cell, we obtain a sample estimate of the collision probability of the ego-vehicle. Subsequently, our approach uses gradient-free sampling-based optimization to compute low-cost trajectories over the cost map. Notably, the sampling distribution is adapted based on the optimal cost values of the sampled trajectories. By explicitly modeling probabilistic collision avoidance in the BEV space, our approach can outperform the cost-map-based baselines in collision avoidance, route completion, time to completion, and smoothness. To further validate our method, we also show results on the real-world dataset NuScenes, where we report collision avoidance and smoothness improvements. It also outperforms other uncertainty-based methods in conservatism.

Abstract

The ability to accurately identify the direction-of-arrival (DOA) and other signal parameters of an incoming radio signal is crucial in many applications, such as military and commercial communication systems. To meet this need, this paper proposes a system for direction finding and pulse parameter estimation that has been designed and implemented using a novel real-time multi-channel parallel Fast Fourier Transform (FFT) on a Field Programmable Gate Array (FPGA). The proposed system employs a unique approach that allows for the implementation of the FFT operation 5.5 times faster than the traditional IP-core-based implementation, with fewer arithmetic operations and a throughput of 1.35 Giga samples per second (Gsps). The parallel processing capabilities of the FPGA are utilized to enable the system to process multiple signals simultaneously in real-time. To achieve high precision in DOA estimation, the system employs the sum-difference monopulse technique. This technique utilizes the difference in the amplitudes of two signals received by two antennas. By comparing the amplitude difference with the sum of the signals, the angle of arrival of the incoming radio signal can be estimated with high precision, up to 1 degree . The proposed system’s hardware implementation has been thoroughly studied and verified experimentally through the development of a real-time prototype of the DOA estimation system. The prototype is capable of processing multiple signals in real-time and accurately estimating the direction-of-arrival of the incoming radio signal. Overall, the proposed system provides an efficient and reliable solution for direction finding and pulse parameter estimation in various commercial and military applications. The use of a novel realtime multi-channel parallel FFT on an FPGA enables fast and accurate signal processing, while the sum-difference monopulse technique ensures high precision in DOA estimation. This makes the system suitable for use in various fields, including communication systems, radar systems, and electronic warfare systems.

Abstract

The Standard Model of particle physics explains the particle content of matter and their interactions at the fundamental level. While the theory predicts and explains most subatomic events quite accurately, it has some limitations; also, some anomalies have been observed. Consequently, various theories beyond the standard model have been developed. Leptoquarks (LQs) and Right-handed Neutrinos (RHNs) are two hypothetical particles appearing in many of these new theories. They are well-known for their potential to address multiple issues. Experiments at the Large Hadron Collider (LHC) have put bounds on their parameters (if they exist) with certain assumptions. However, no direct experimental constraints exist on Leptoquark (LQ) couplings with quarks and right-handed neutrinos (RHNs) yet. If the RHNs are lighter than the LQs, they can be produced from the LQ decays. This simultaneous probe for RHNs and the LQs that dominantly couple to RHNs has never been searched for in experiments. In this thesis, we investigate the signals arising from the decays of Right-Handed Neutrinos (RHNs) coming from the decays of a scalar Leptoquark (LQ) that dominantly couples to RHNs. Our investigation covers both pair and single productions of the LQ and also RHN pair production through t-channel LQ exchange. These typically lead to two RHNs which can subsequently decay to multiple charged leptons and jets. While the dilepton final state is the most promising, our analysis centres on the more challenging monolepton final state to compute the discovery and exclusion reach of LQs and RHNs at the LHC. Here, we consider a scalar Leptoquark with an electromagnetic charge of either 1/3 or 2/3 (referred to as ϕ1 and ϕ2 respectively), primarily coupling to second-generation quarks and RHNs. We discuss the Standard Model and some Beyond Standard Model theories in chapter 1. We discuss particle accelerators and their simulation software in chapter 2. We analyse the monolepton final state to estimate the discovery reach at the high-luminosity LHC in chapter 3. In chapter 4, we visualise the distributions of various parameters and then look at them collectively to maximise the discovery reach. We present the results of the work in chapter 5. In chapter 6, we digress from the main topic and study a computation technique that can be used to automatically compute the bounds on LQ parameters from the current LHC data. We discuss an alternative methodology to compute the test on a contingency table where the table elements involve couplings as variables. These individual elements are summed up to form polynomials, and the polynomial form of χ2is computed. This allows for using computational techniques (like gradient descent) that cannot be utilised if the value of χ2 is computed using the regular tabular method at discrete values of the couplings.

Abstract

With the increased data availability, the biggest problem is getting accurate labeled data for training models. However, most of the time, we get the missing values. It is preferred to consider weak supervised learning methods to avoid the cost of quality labeled data. One such technique is called partial labeled data classification. We present a novel second-order cone programming framework to create robust classifiers that can tolerate uncertainty in the observations of partially labeled multiclass classification problems. Only the presence of second-order moments is necessary for the proposed formulations independent of the underlying distribution. The case of missing values in observations for multiclass classification problems is thus specifically addressed by these formulations. Adversarial examples are machine learning model inputs that an attacker has purposefully constructed to cause the model to make a mistake. Recent research aimed to improve the computational efficiency of adversarial training for deep learning models. Projected Gradient Descent (PGD) and Fast Gradient Sign Method (FGSM) are popular current techniques for generating adversarial examples efficiently. There is a trade-off between these two regarding robustness or training time. Adversarial training with the PGD is considered as one of the most efficient adversarial defense techniques to achieve moderate adversarial robustness. However, PGD requires high training time since it takes multiple iterations to generate perturbations. On the other hand, adversarial training with the FGSM takes much less training time since it takes one step to generate perturbations but fails to increase adversarial robustness. On the CIFAR-10/100 datasets, our approach outperforms PGD strong adversarial training techniques in terms of robustness to adversarial training and speed. While deep learning systems trained on medical images have demonstrated cutting-edge performance in various clinical prediction tasks, recent research indicates that cleverly created hostile images can fool these systems. Deep learning-based medical image classification algorithms have been questioned regarding their practical deployment. To attack this problem, we provide an unsupervised learning technique for detecting adversarial attacks on medical images. Without identifying the attackers or reducing classification performance, our suggested strategy FNS (Features Normalization and Standardization), can detect adversarial attacks more effectively than earlier methods.

Abstract

Neural Networks are known to be black box models which are explained by interpretability based approaches. ML Interpretability methods to explain the complex Neural Networks reasoning in human understandable form. Humans think in abstract concepts like color, texture, shapes, etc. Explaining black box models in these simple concepts aids human understanding of models leading to more transparency, reliability and proactive identification of risks/biases in the models. The recent interpretability based methods have started using concepts for explaining complex models in simple human understandable terms. In computer vision, concepts are defined as a set of images having a human-understandable meaning associated with them (eg. striped images form stripiness concept). Current concept based interpretability methods use the encoding of concepts in an intermediate layer of the model to define concept representations. They further use these representations by perturbing model activations in the intermediate layer and gauging its sensitivity to the perturbations. We use these CAV based sensitivity calculations for evaluation of desired properties by using the very definitions from the problem as concepts. We further extend the use of CAVs from post-hoc analysis to ante-hoc training via our novel concept loss and distillation paradigm. Concretely, we present a concept sensitivity based method to measure disentanglement in an ill-posed ambiguous problem of Intrinsic Image Decomposition (IID) followed by a novel method for concept based training of DNNs which utilizes conceptual knowledge from large pre-trained models in a distillation paradigm. IID involves decomposing an image into its constituent Reflectance and Shading components, which are illumination-invariant and albedo-invariant, respectively. We use this definition of IID to define our concepts and propose to use the sensitivity scores to directly measure the alignment of models predictions with the definition. For this, we measure the illumination invariance of Reflectance prediction and albedo invariance of Shading prediction by gazing model’s sensitivity to relevant concepts. We thus define the evaluation of IID in abstract human centered concepts. We define Concept Sensitivity Metric (CSM) to measure the disentanglement of Reflectance Shading in the models predictions for evaluating IID methods. We evaluate and interpret three recent IID methods on our synthetic benchmark of controlled albedo and illumination invariance sets. We also compare our metric with existing IID evaluation metrics on both natural and synthetic scenes and report our observations. Our metric not only surpasses several limitations of the existing metrics but is also consistent in both synthetic and real-world datasets. The concept based interpretability methods are post-hoc (after training) and can be used for analyzing the models. We aim to use the feedback provided by these post-hoc methods to train the model for further improvements in an ante-hoc manner. For this we propose a novel concept loss based on the CAV sensitivity. We argue that CAV learning in same model is not efficient and propose to use a separate model having knowledge of concepts in a knowledge distillation paradigm. We present Concept Distillation: a novel method for concept sensitive training of Deep Neural Networks. Concept Distillation can be used to sensitize or desensitize the student model towards user desired concepts. We show applications of our concept-sensitivity based training in debiasing in classification problems and prior induction in IID. We also introduce the TextureMNIST dataset to evaluate the presence of complex texture biases. Our concept-sensitive training can improve model interpretability, reduce biases, and induce prior knowledge.

Abstract

In AI, the ability of intelligent agent to model human player in games such as Backgammon, Chess and Go has been an important metric in benchmarking progress. Fundamentally, the games mentioned above can be characterized as competitive and zero-sum. In contrast, games such as Pictionary and Dumb Charades falls into the category of ‘social’ games. Unlike competitive games, the emphasis is on cooperative and co-adaptive game-play in a relaxed setting. Such social games can form the basis for the next wave of game-driven progress in AI. Pictionary™ is a wonderful example of cooperative game play to achieve a shared goal in communication-restricted settings. This popular sketch-based guessing game, which we employ as a use case, provides an opportunity to analyze shared goal cooperative game play in restricted communication settings. To enable the study of Pictionary and to understand various aspects associated with the game play, we designed a software ecosystem for web-based online game of Pictionary dubbed PICTGUESS. To overcome several technological and logistic barriers, which the actual game presents, we implemented a simplified setting for PICTGUESS wherein a game consists of a time-limited episode involving two players - a Drawer and a Guesser. The Drawer is tasked with conveying a given target phrase to a counterpart Guesser by sketching on a whiteboard within that time limit. However, occasionally some players in Pictionary draw atypical sketch content. While such content is occasionally relevant in the game context, it sometimes represents a rule violation and impairs the game experience. To address such situations in a timely and scalable manner, we introduce DRAWMON, a novel distributed framework for automatic detection of atypical sketch content in concurrently occurring Pictionary game sessions. We build specialized online interfaces to annotate atypical sketch content, resulting in ATYPICT, the first ever atypical sketch content dataset. We use ATYPICT to train CANVASNET, a deep neural atypical content detection network. We utilize CANVASNET as a core component of DRAWMON. Our analysis of post deployment game session data indicates DRAWMON’s effectiveness for scalable monitoring and atypical sketch content detection. Beyond Pictionary, our contributions can also serve as a design guide for customized atypical content response systems involving shared and interactive whiteboards.