Abstract

Precise recognition of previously encountered locations is crucial for embodied agents to effectively localize and traverse their environment. This necessitates visual representations that remain distinctive despite substantial changes in camera perspective and environmental conditions. Current visual place recognition systems typically encode entire images for comparison. This conventional approach presents a significant hurdle when attempting to match two images of an identical location taken from disparate viewpoints: the similarity of what overlaps can be dominated by the dissimilarity of what does not overlap. To address this issue, we propose encoding and querying image segments rather than complete images. We leverage open-set image segmentation to decompose an image into ‘meaningful’ entities (encompassing things and stuff). This allows us to construct an novel image representation compris- ing multiple overlapping subgraphs that connect a segment to its neighbouring segments, which we term SuperSegment. Additionally, to efficiently encode these SuperSegments into condensed vector representations, we introduce a novel factorized formulation of feature aggregation. We demonstrate that retrieving these partial representations yields improved recognition accu- racy compared to conventional whole-image retrieval methods. Our segment-based technique, named SegVLAD, establishes new state-of-the-art in place recognition across a wide array of diverse datasets, while remaining compatible with both general-purpose and task-specific image encoders. Lastly, we showcase our method’s versatility in “revisiting anything” by assessing its performance on an object instance retrieval task. This application bridges two distinct research domains: visual place recognition and object-goal navigation, through their shared objective of identifying specific objects within a given location.

Abstract

This thesis explores the transformative potential of artificial intelligence (AI) in healthcare, particularly through enhancing diagnostic accuracy in medical imaging and streamlining data annotation processes. Central to this thesis is the development of an AI system using Unsupervised Domain Adaptation (UDA) to accurately grade retinal development in pediatric patients across different optical coherence tomography (OCT) devices. This innovative approach overcomes device-specific limitations, offering a robust and generalisable classification model without the need for device-specific data, matching the diagnostic accuracy of experienced clinicians. Additionally, the thesis addresses the variability in fundus image interpretation among medical professionals by employing a novel annotation tool. This tool facilitates a variability study among medical graders, underscoring the necessity for AI systems to account for such variability to enhance diagnostic reliability. This thesis also delves into the application of Batch Active Learning (BAL) for optimizing seed grading mechanisms in agriculture. By systematically selecting and labeling the most informative samples, the research demonstrates a significant improvement in the model’s accuracy while reducing the time and cost associated with manual data annotation. The integration of BAL with advanced machine vision techniques highlights its potential to revolutionize agricultural practices, particularly in enhancing the efficiency and accuracy of seed quality classification. This contribution underscores the versatility and scalability of AI-driven approaches in both healthcare and agriculture. Overall, the thesis provides a comprehensive evaluation of AI’s capability to handle domain shift—a significant challenge when training data differs from deployment data. It offers a framework for future models to be device-agnostic, enhancing applicability across different devices and settings. The findings not only contribute to the field of medical diagnostics but also offer insights into the broader application of AI in real-world clinical and agricultural settings.

Abstract

Machine Translation (MT), the automated process of translating text or speech between languages using computational linguistic algorithms, has drastically transformed global communication. By breaking down language barriers, MT has enabled seamless collaboration across different social, cultural, and geographical backgrounds. Although several decades of research have produced state-of-the-art models like Google Translate, these systems often falter when translating domain-specific content, such as specialized academic texts or technical manuals. These models, trained primarily on large, general-purpose datasets, struggle to accurately capture domain-specific vocabulary and linguistic nuances. For example, terms like ”chromosome” in a microbiology textbook are often poorly translated by these systems. In this thesis, we propose DomainTrans, a MT model designed to enhance translation accuracy for domain-specific texts while maintaining performance on general content. The core idea here is to form a bijection between multi-domain translation and multi-lingual translation. By treating different domain datasets as analogous to distinct languages, we leverage multi-lingual translation techniques to develop a more versatile and adaptable MT model. This thesis also studies and experiments with other key peripherals around this topic. First being domain classification, where we identify the domain to which a given text belongs as a precursor to translation. We particularly lay special emphasis on fine grained domain classification. Next we look at the effect of data availability while fine tuning models for machine translation. Towards end end of the thesis we also explore Knowledge Distillation - the process of condensing the learning done by a large or a set of large models onto a much smaller efficient to deploy model while trying to maintain similar accuracy.

Abstract

The neural mechanisms underlying the comprehension of meaningful sounds are yet to be fully understood. While previous research has shown that the auditory cortex can classify auditory stimuli into distinct semantic categories, the specific contributions of the primary (A1) and the secondary auditory cortex (A2) to this process are not well understood. We used songbirds as a model species, and analyzed their neural responses as they listened to their entire vocal repertoire (∼ 10 types of vocalizations). We first demonstrate that the distances between the call types in the neural representation spaces of A1 and A2 are correlated with their respective distances in the acoustic feature space. Then, we show that while the neural activity in both A1 and A2 is equally informative of the acoustic category of the vocalizations, A2 is significantly more informative of the semantic category of those vocalizations. Additionally, we show that the semantic categories are more separated in A2. These findings suggest that as the incoming signal moves downstream within the auditory cortex, its acoustic information is preserved, whereas its semantic information is enhanced.

Abstract

The development of computational methods for the efficient extraction of relevant legal information to aid legal practitioners is an active research area. In this regard, research efforts are being made to develop efficient methods by leveraging different kinds of information in a legal document, such as meta-data, citations, keywords, sentences, paragraphs and so on. Similar to other text documents, legal documents are composed of paragraphs. In this paper, we have analyzed how paragraph-level information could capture the similarity among judgments and improve the performance of precedence retrieval. By considering paragraph-level similarity methods with document level as the baseline, the statistical analysis on the Supreme Court of India judgment dataset shows that the paragraph-level methods with only a few paragraph interactions could capture the similarity among the judgments and exhibit more discriminating power over the baseline document-level method. Moreover, the comparison results on two benchmark datasets for the precedence retrieval on the Indian supreme court judgments task show that the paragraph-level methods exhibit comparable performance with the state-of-the-art methods by utilizing only a few paragraph interactions. Overall, the results show that paragraph-level granularity can better represent the legal document, and there is a scope to exploit them to improve the retrieval systems in the legal domain.

Abstract

Reconfigurable intelligent surfaces (RIS) are emerging as a promising technology for 6G networks due to their ability to shape the radio propagation environment. This ability helps to overcome propagation challenges, such as high path loss, absorption loss, etc., that are particularly useful at millimetre wave (mmWave) bands. Thus, RIS is envisioned to play a critical role in next-generation mmWave communication networks to enable a variety of applications, including communication, localization, and sensing. However, one critical aspect in the design of RIS-aided systems is channel estimation, as it involves estimating two channels, i.e., base station (BS) to RIS and RIS to user terminal (UT), separately based on the compound channel observed by the receiver. In this paper, we address this problem by proposing an algorithm that estimates both channels separately by leveraging the advantages of sparsity of the mmWave channel. Specifically, the proposed algorithm uses the parallel factor (PARAFAC) decomposition of the tensor, which is constructed using the received signal matrices observed under different RIS phase shift configurations. Our numerical analysis shows that the proposed algorithm provides significantly smaller normalized mean square error (NMSE) than the widely used alternating least squares (ALS) algorithm, particularly when the number of RIS phase shift configurations is smaller than the number of RIS elements. In addition, we also derived Cramer Rao Lower Bound (CRLB) for the joint parameter vector of RIS-aided mmWave channels, assuming a uniform planar array (UPA) for RIS antenna elements and uniform linear arrays (ULAs) for BS and UT antennas. Next, we also derived the CRLB for the transform parameter vector to obtain the CRLBs of BS-RIS and RIS-UT mmWave channels. Our numerical analysis demonstrates that the proposed algorithm for estimating both channels achieves their respective CRLBs for a wide range of SNRs.

Abstract

This thesis explores the intricate relationship among color saturation, salience, and visual perception, with a focus on defining color salience and investigating the potential for universal color salience. The research delves into how color saturation—defined as the intensity and purity of color—affects visual perception and contributes to the noticeability of objects in their surroundings. Special attention is given to addressing the limitations of existing theories concerning individual differences in color perception, particularly in scenarios involving color ambiguous stimuli like the internet phenomenon #thedress, where perceptions varied widely. The study draws inspiration from diverse sources, including a saree featuring 50,000 colors, prompting an examination of perceptual limits in color vision and the impact of psychophysical properties on color dominance within ensembles. Through experiments involving tasks to select dominant colors and eye-tracking analyses, the research investigates factors influencing perceived color dominance, such as hue, saturation, and value. The role of individual differences in color perception is explored using clustering techniques to group observers based on their color choices, revealing distinct perceptual strategies and trends within these clusters. Key findings challenge previous assumptions, particularly regarding the salience of hues like blue, which are found to exhibit significant dominance effects contrary to previous research. The study introduces the Relative Perceived Hue Difference (RPHD) metric to quantify color dominance, showing instances where perceived dominance does not align with actual color distribution in stimuli. Eye-tracking data further suggests that sustained attention, reflected in largest fixation points, may be influenced by dominant hues in binary stimuli, highlighting complexities in dominant color perception beyond initial visual capture. This research contributes to advancing our understanding of color perception, salience, and the underlying factors driving visual attention in individuals, offering insights into the intricate dynamics of how we perceive and prioritize colors in our visual environment.

Abstract

Interactive visualization and analysis of the class boundaries is important because it tells us how and why the classes differ. However, the problem of modeling the boundary of classes of arbitrary size, shape and density is challenging. The boundary of a class should not be limited to the points/shape which encloses the points within the class but it should be, the points/shape which encloses the region of influence of a class. The “region of influence” refers to the space around the class where any point lying within the region is likely to be classified to the class based on a nearest neighbor classifier. We have developed interactive boundary visualization toolkit for classified datasets which provides insights about the nature of the dataset and the classifier model used on the dataset. Our algorithm first generates a candidate boundary set for each class based on reverse k-nearest neighbors approach and extends this boundary iteratively through the region of influence of the class. Further, we present these boundary points enclosing the region of influence as a linear approximated shape using triangulation techniques. We show experimental results on 2D and 3D datasets. We also present a distributed map-reduce based algorithm to speed up generation of regions of influence.

Abstract

This research explores Bayesian methods for predicting model uncertainty in remote sensing data, focusing on two distinct approaches: Bayesian linear regression and a novel Bayesian Nonlinear Neural Network (BN3) model. Both methods utilize SENTINEL-2 Normalized Difference Vegetation Index (NDVI) data from agricultural fields in Uttar Pradesh and Madhya Pradesh, India, for time series predictions. Bayesian linear regression, a probabilistic deep learning approach, evaluates model uncertainty through prior and posterior probability parameters using Bayesian inference. The analysis reveals high model uncertainty in predicted NDVI values, which diminishes as the data volume increases. This reduction in uncertainty is accompanied by sharper posterior density, indicating reduced variance. Regression analysis with Gaussian distribution further validates this trend, confirming that greater data availability leads to improved model certainty. Complementing this approach, the study develops the BN3 model to address model uncertainty in neural networks. BN3 employs probabilistic modeling with variational hidden layers and output layers characterized by Gaussian distributions for variational posteriors. It assesses model uncertainty using dense variational layers with continuous, differentiable nonlinear activation functions and varying degrees of freedom. The study evaluates the goodness of fit for nonlinear models, including polynomials and neural networks, using RMSE as a loss function. Comparison with Deep Ensemble methods highlights the BN3 models effectiveness, particularly in reducing uncertainty for cubic models. Visual analyses demonstrate BN3s consistent performance in minimizing model uncertainty across datasets, even with varying data availability. These findings demonstrate BN3s potential as a robust tool for improving uncertainty quantification in decision support systems.

Abstract

The widespread use of artificial intelligence has resulted in significant advancements in the field of Natural Language Processing, with text summarization and question answering models being some of the most sought-after applications. By understanding the Causal Relationships between Events within a text, we can gain valuable insights into how Events influence each other, leading to more accurate and sophisticated model outcomes. This thesis is an attempt towards improving Event Causality through various ways. The thesis commences with a comprehensive examination of Events and States, outlining their various classifications and the relationships between them. This foundational discussion establishes the context for the exploration of Event Causality. It then delves into existing research on Event Causality identification highlighting areas for potential contribution Recognizing the scarcity of datasets focused on Event Causality in the Hindi language, this thesis introduces the Hindi Causal TimeBank. This dataset is built upon the existing Hindi TimeBank containing about 1000 news articles and can serve as a crucial resource for developing and training Event Causality models for Hindi-language text. The thesis elaborates on the different types of Causal Relations annotated within the dataset. The primary focus of this thesis lies in the advancement of Event Causality identification. Specifically, it explores the use of graph-based models (Graph Attention Networks (GAT)). GATs are well-suited for representing complex relationships between entities and Events. This thesis presents a series of experiments designed to improve results on Event Causality identification, aiming to outperform the current state-of-the-art methods. The performance of these models is evaluated across a range of datasets, including the newly developed Hindi Causal TimeBank. This thesis makes a significant contribution to the field of Event Causality. Improved Event Causality understanding has implications for information retrieval, machine translation, and knowledge graph construction. The introduction of the Hindi Causal TimeBank further expands the scope of this research into a crucial and understudied language.