Abstract

Machine Translation is an area of Natural Language Processing that involves building systems that translate information from one language to another. Over the past few decades, various approaches and their variants have been proposed, such as Rule-based systems, Statistical Machine Translation (SMT), and Neural Machine Translation (NMT) have been presented to automate the Translation process. Researchers explored several approaches to analyze the changes in translation quality. While some methodologies yielded an overall improvement in translation, others declined the performance over baselines. This thesis presents our attempts and outcomes in exploring several notions to improve the overall translation quality. We propose selecting a better hypothesis for Neural Machine Translation Systems. After building a baseline system, we take the last few iterations outputs as the N-best list, and it is observed that it has an oracle score higher by +1.01 BLEU points. Hence, we propose a technique of Checkpoint Reranking, which selects a better hypothesis than the baseline system without any language model or data by focusing on the decoder’s ability to generate distinct tokens. Next, we investigate the effects of filtering the synthetic data in Iterative BackTranslation.We introduce a notion of reutilizing synthetic data generated across different passes of the system and do a Round-Trip Filtering to choose the best scorer. Our experiments observe that the procedure performs slightly better over the baseline Iterative BackTranslation system.

Abstract

It is often observed that certain human diseases exhibit similarities in some form while having different prognoses and requiring treatment strategies. These similarities may be in the form of risk factors towards the diseases, symptoms observed, visual similarity in imaging studies, or in some cases, similarity in molecular associations. Computer-Aided Diagnosis (CAD) of closely related diseases is challenging and requires tailored approaches to discriminate between such closely related diseases accurately. This thesis looks at two sets of closely related diseases of two different organs, identified from two different modalities. It develops novel approaches to achieve explainable and accurate CAD for these two close diseases. These two problems are discrimination of healthy, mild cognitive impairment (MCI) and Alzheimer’s Disease (AD) from brain MRI-derived surface mesh and classifying healthy, non-COVID pneumonia and COVID from chest X-ray images. In the first part of this thesis, we present a novel 2D image representation for the brain mesh surface, called a height map. Further, we explore the use of height maps towards the hierarchical classification of healthy, MCI, and AD cases. We also compare different strategies of extracting features and regions of interest from height maps and their performance towards healthy vs. MCI vs. AD classification. We demonstrate that the proposed method achieves fast classification of AD and MCI with minor loss of accuracy compared to state of the art. In the second half of this thesis, we present a novel deep learning architecture called Multi-scale Attention Learning Residual Learning (MARL) and a new conicity loss for training the MARL architecture. We utilize MARL and the conicity loss for achieving hierarchical classification of normal, non-COVID pneumonia and COVID pneumonia from Chest X-ray images. We present classification results on three public datasets and demonstrate that the proposed method achieves comparable or marginally better performance than state-of-the-art in all cases. Further, we demonstrate that the proposed framework achieves clinically consistent explanations with extensive experimentation. Qualitatively, this is shown by comparing GradCAM heatmaps for the proposed method to those for the state-of-the-art method. It is observed that the heatmaps overlap better with the bounding boxes for pneumonia marked by experts compared to the overlap achieved by the state-of-the-art method. Next, we show quantitatively that the GradCAM heatmaps for the proposed method generally lie within inner regions of the lung for nonCOVID pneumonia. However, the same heatmaps lie in outer regions in the case of COVID pneumonia. Thus, we establish the clinical consistency of explanations provided by the proposed framework.

Abstract

Since Neural Machine Translation (NMT) performs better than traditional statistical machine translation (SMT) models, it has become prevalent in recent years. NMT systems need a large amount of training data and perform poorly relative to Phrase-Based Machine Translation (PBMT) systems in low resource and domain adaptation scenarios. One of the challenges in NMT is domain adaptation. It becomes more challenging for scarce resource Indic languages, and technical domains like Artificial Intelligence(AI) and Chemistry as these domains may contain many technical terms and equations, etc. In a typical domain adaptation scenario like ours, we have a large amount of out-of-domain bilingual training data for which we need to train an NMT model, and we can treat this as a general model. Now given only an additional small amount of in-domain data, the challenge is to improve the translation performance on the new domain. We present two kinds of approaches for domain adaptation in this thesis: data-based and model-based, which have their own advantages and disadvantages. From the experiments, it was observed that the data-based approach is performing significantly better than the model-based approach in our case. Even though the translation performance on domain data is lower for the model-based approach than the data-based approach, it is higher than the general model. Based on this, we want to make two critical points of this thesis, 1. Suppose a large amount of general corpus is available. In that case, we can choose a data-based approach where the new model is trained on combined data(general domain and in-domain data), 2. if there is no such kind of parallel data, but there is a general model with some configurations. Then we can choose a model-based approach where it does not require any general data. However, it uses the available general model and continues the learning with new domain data. As we have general parallel data available and the first mentioned approach outperformed the model-based approach, we continued different experiments using the data-based approach itself. The domains are Chemistry and Artificial Intelligence in our work, and the language pair is Hindi and Telugu. As the mentioned domains can be treated as technical domains containing many domain-specific terms and equations, there is a need to improve the domain term translation. Therefore, we used a domain dictionary as parallel data and trained it along with training data. We created a domain dictionary for the chemistry domain for Hindi-Telugu. Using an automatic domain term extraction algorithm, we extracted the domain terms. First, we extracted domain terms in English and manually converted them into Telugu and Hindi. Using the domain dictionary as parallel data, the translation performance improved significantly in terms of domain terms and BLEU. The domains we adopted in this are similar in terms of vocab overlap. Hence, we used a combination of domains to improve the translation. The addition of different domain data also matters in the performance. We also showed how general data is powerful to complete the translation of a domain-specific text. We showed the order of combining the domain data sets and how it impacts the overall translation. In this work, we mainly concentrated on the data-based approach where we combine the available tiny amount of domain data to a large amount of general data. Combining only a small amount of domain data yields good performance, but it can be improved with backtranslation. The trivial back translation won’t help here because the noisy synthetic data may prevent the model from producing domain-specific translation properly. Therefore we need a practical approach that deals with this domain-specific data. We address the same issue with an algorithm called ”Domain-Specific Back Translation.” Using this algorithm we achieved a significant improvements in BLEU scores

Abstract

Decades of relentless efforts in manufacturing integrated circuits and reducing their cost has enabled the design of almost every system which was thought impossible before. As the semiconductor industry thrives to expand its market every year, a question keeps popping up every decade or so asking what kind of technology will continue expanding the market. Previous technological waves include laptops and smart-phones, however they are tending towards market saturation. The convergence of social and technological trends suggest that there is a growing need to efficiently utilize the resources on earth and create smart environments. This is possible when every object on earth, ex. cars, drones, buildings, etc. share the appropriate data and provide suggestions for optimal usage of the resources and things around us. It is evident that these objects need to be connected to the Internet to perform such tasks and hence the term Internet-of-Things (IoT) rose into picture in 1999 [1]. The IoT is still in its technological infancy and the growing IoT wave will soon dominate the semi-conductor market in the upcoming years. The IoT application space is vast and includes wide variety of applications like biomedical, surveil- lance, environmental, etc. These applications have stringent power and area constraints and hence the circuits present in the IoT nodes need to comply with these conditions. The average power consump- tion is a critical factor in deciding the system’s lifetime, which can be reduced to an extent by applying the concept of duty-cycling. However, blocks which are always-on, are required for the application of duty-cycling and they contribute to the sleep mode power consumption inevitably. Hence, they need to be designed with extremely low power consumption. Apart from the power constraint, they are desired to be process and supply invariant, and have low area. This thesis discusses the design of always-on blocks that include voltage reference, current reference and analog temperature sensor. Chapter 1 introduces the concept and evolution of IoT over the past few years, its applications, chal- lenges that will be faced by it to dominate the semi-conductor market and its future. After understanding IoT applications, we introduce voltage reference, current reference, analog temperature sensor and ex- plain their functionalities in Chapter 2. With this background, we move on to the design of the always-on blocks in the successive chapters. The references and sensor incorporate thin-oxide devices to achieve pico-watt power consumption while consuming low-area. Resistor based architectures are eliminated as their values should be in the order of GΩs to achieve pico-watt power consumption. Other resistor-less architectures suffer from the drawbacks as mentioned in the corresponding chapters. The thin-oxide devices or gate-leakage tran- sistors serve as effective replacements for resistors. However, they have non-linear characteristics with various parameters and hence cannot be directly visualized as resistors. The temperature dependencies of gate-leakage current in inversion and accumulation region are studied in Chapter 3 and then exploited in the designs of always-on blocks. Chapter 4 discusses the design of pico-watt current reference using gate-leakage transistors in inver- sion region while the design in Chapter 5 operates them in accumulation region to achieve a bandgap reference. It is beneficial if the functionality of both voltage and current reference are included in a single circuit. Chapter 6 deals with such an architecture by again exploting accumulation-mode gate-leakage current characteristics. Chapter 7 deals with the design of a pico-watt process-invariant temperature sensor using gate-leakage transistors in accumulation region. PVT variations are inevitable in any kind of circuit and most of the circuits try to achieve PVT in- variant designs. Design techniques for achieving process-invariant voltage reference and temperature sensor are discussed in Chapters 5 and 7. Current references cannot usually be made process-invariant and hence process-invariant current reference is out of scope of this thesis. Techniques for achieving high supply noise invariance are discussed in Chapters 8 and 9. Chapter 8 proposes high PSRR volt- age reference, current reference and temperature sensor in a single circuit while Chapter 9 deals with achieving high PSRR in composite-pair based temperature sensors. Chapter 10 finally concludes the thesis and mentions the scope of improvement for this work.

Abstract

We interact with the world around us through multiple sensory streams of information such as audio, vision, and text (language). Each of these streams complement each other, but also contain redundant information, albeit in different forms. For example, the content of a person speaking can be captured by listening to the sounds in the speech, or partially understood by looking at the speaker’s lip movements, or by reading out the text transcribed from the vocal speech. This redundancy across modalities is utilized in human perceptual understanding that helps us to solve various practical problems. However, in the real-world, more often than not, information in individual streams is corrupted by various types of degradation like electronic transmission, background noise, and blurring which lead to deterioration in the content quality. In this work, we aim to recover the distorted signal in a given stream by exploiting the redundant information in another stream. Specifically, we deal with talking-face videos involving vision and speech signals. We propose two core ideas to explore cross-modal redundancy: (i) denoising speech using visual assistance, and (ii) upsampling very low-resolution talking-face videos using audio assistance. The first part focuses on the task of speech denoising. We show that the visual stream helps in distilling the clean speech from the corrupted signal by suppressing the background noise. We identify the key issues prevailing in the existing state-of-the-art speech enhancement works: (i) most of the current works use only the audio stream and are limited in their performance in a wide range of realworld noises, and (ii) few recent works use the lip-movements as additional cues with an aim to improve the quality of the generated speech over “audio-only” methods. However, they cannot be applied for several applications where the visual stream is unreliable or completely absent. Thus, in this work, we propose a new paradigm for speech enhancement: “pseudo-visual” approach, where the visual stream is synthetically generated from the noisy speech input. We demonstrate that the robustness and the accuracy boost obtained from our model lead to various real-world applications which were previously not possible. In the second part, we explore an interesting question of what can be obtained from an 8 × 8 pixel video sequence by utilizing the corresponding speech of the person talking. Surprisingly, it turns out to be quite a lot. We show that when processed with the right set of audio and image priors, we can obtain a full-length talking video sequence with a 32× scale-factor. When the semantic information about the identity, including basic attributes like age and gender, are almost entirely lost in the input low-resolution video, we show that utilizing the speech that accompanies the low-resolution video aids in recovering the key face attributes. Our proposed audio-visual upsampling network generates realistic, accurate, and high-resolution (256 × 256 pixels) talking-face videos from an 8 × 8 input video. Finally, we demonstrate that our model can be utilized in video conferencing applications where the network bandwidth consumption can be drastically reduced. We hope that our work in cross-modal content recovery enables exciting applications such as smoother video calling, accessibility of video content in low-bandwidth situations, and restoring old historical videos. Our work can also pave the way for future research avenues for cross-modal enhancement of talking-face videos

Abstract

Code-mixing is a phenomenon of natural language where multilingual speakers mix two or more of the languages they speak, within the same utterance. This mixture is not made at random, but governed by systematic rules, and is part of highly effective communication. Code-mixing is very common in exchanges between multilinguals in day-to-day conversation, mass-media, pop-culture and on social media networks. NLP systems have historically struggled with processing and understanding code-mixed speech due to the non-standardized nature of code-mixing in informal contexts, such as in informal speech and on social media. The lack of standardization results in extremely high variation in how code-mixing is employed, making the design and creation of rule based NLP systems impractical. The solution has been to look towards statistical systems of machine learning which can automatically learn from data. While there is a large trove of code-mixed data available on the internet, the data is extremely noisy and needs preprocessing and cleaning before it can be made viable for use as input in a machine learning pipeline and be learnt from. Due to the painstaking process of data normalization involved, resources for code-mixed language are scarce and often suffer from poor quality. In this thesis we detail our attempts at improving results on code-mixed NLP in scenarios where the amount of data is the limiting factor. First, we develop an approach for sentiment analysis for Hindi-English code-mixing using neural networks. We describe an architecture for the hierarchical analysis of code-mixed texts at the word and the sentence level, and use it for sentiment classification. Second, we introduce a novel new dataset intended for evaluating code-mixed language models, with parallel translations for Hindi-English code-mixed tweets in both monolingual Hindi and monolingual English. We then establish baselines for the codemixed translation task and describe how one can use such datasets to leverage the Hindi and English pretraining in multilingual models, via our hypothesis that multilingual models learning to code-mix from both component languages will outperform those learning only from a single component language. Finally, we put this hypothesis to the test and show our results for the same.

Abstract

The widespread popularity of over-parameterized deep neural networks (NNs) is backed by its ‘unreasonable’ performance on unseen data that is Independent, Identically Distributed with respect to the train data. The generalization of NNs cannot be reasoned with the traditional machine learning wisdom that the increase in the number of parameters leads to overfitting on train samples and subsequently reduced generalization. Various generalization measures have been proposed in recent times to explain the generalization property of deep learning networks. Despite some promising investigations, there is little consensus on how we can explain the generalization of NNs. Furthermore, the ability of neural networks to fit any train data for any random configuration of labels makes it more challenging to explain their generalization performance. Despite this ability to completely fit any given data, the neural network seems to be able to ‘cleverly’ learn a generalizing solution. We hypothesize that the ‘simple’ solution lies in a constrained subspace of the hypothesis space. We propose a constraint formulation of neural networks to close the generalization gap. We show that, through a principled constraint, we can achieve comparable train test performance for neural networks. We propose a way to constrain each output of the neural network as the convex combination of its inputs to ensure certain desirable geometry of the decision boundaries. This document covers two major aspects. Firstly, we show the improved generalization of neural networks using convex constraints. The second section goes beyond the IID setting and investigates the generalization of neural networks on the Out Of Distribution test sets.In the first section of the document, we investigate the constrained formulation of neural networks where the output is a convex function of the input. We show that the convexity constraints can be enforced on both fully connected and convolutional layers, making them applicable to most architectures. The convexity constraints include restricting the weights (for all but the first layer) to be non-negative and using a non-decreasing convex activation function. Albeit simple, these constraints have profound implications on the generalization abilities of the network. We draw three valuable insights: (a) Input Output Convex Neural Networks (IOC-NNs) self regularize and significantly reduce the problem of overfitting; (b) Although heavily constrained, they outperform the base multi-layer perceptrons and achieve similar performance as compared to base convolutional architectures and (c) IOCNNs show robustness to noise in train labels. We demonstrate the efficacy of the proposed idea using thorough experiments and ablation studies on six commonly used image classification datasets with three different neural network architectures.In the second section, we revisit the ability of networks to completely fit any given data and yet ‘cleverly’ learn the generalizing hypothesis from the large number of variances that can explain the train data. In accordance with concurrent findings, our explorations show that neural networks learn the most ‘low-lying’ variance in the data. They learn the features that easily correlate with the label and do no further exploration after finding such a solution. With this insight, we reinvent the need to understand the generalization of neural networks and improve them. We go beyond traditional IID and OOD evaluation benchmarks to further our understanding of learning in deep networks. Through our explorations, we give a possible explanation as to why neural networks can do well in certain benchmarks and why other inventive methods fail to give any consistent improvement over a simple neural network. Domain Generalization (DG) requires a model to learn a hypothesis from multiple distributions that generalizes to an unseen distribution. DG has been perceived as a front face of OOD generalization. We present empirical evidence to show that the primary reason for generalization in DG is the presence of multiple domains while training. Furthermore, we show that methods for generalization in IID are equally important for generalization in DG. Tailored methods fail to add performance gains in the Traditional DG (TDG) evaluation. Our experiments prompt if TDG has outlived its usefulness in evaluating OOD generalization? To further strengthen our investigation, we propose a novel evaluation strategy, ClassWise DG (CWDG), where for each class, we randomly select one of the domains and keep it aside for testing. We argue that this benchmarking is closer to human learning and relevant in real-world scenarios. Counter-intuitively, despite being exposed to all domains during training, CWDG is more challenging than TDG evaluation. While explaining the observations, our work makes a case for more fundamental analysis around the DG problem before exploring new ideas to tackle it K

Abstract

Agriculture Ontology for Telugu language (AOT) is a new ontology that we have created for plants and their diseases that models semantic relations between the entities in this domain. The ontology provides a formal way for the representation of entities and their mutual relations. Several methodologies that exploit numerous techniques of various fields (machine learning, text mining, knowledge representation and reasoning, information retrieval, and natural language processing) proposed to bring some level of automation in ontology acquisition from unstructured text. Question Answering Systems, Anaphora Resolution, Dialogue Systems are some of the realworld applications of ontology. This thesis presents a dataset of Ontology for the Telugu language and a Question Answering system which generates its responses by accessing the ontology as its database. We create an architecture for a Telugu agriculture-based Question Answering system using Ontology as the data resource in this thesis. We used Machine Learning techniques for Question Classification, Named Entity Recognition for Keyword Extraction, to generate a query for ontology used for answer generation. A Question Answering system’s question analysis is a critical component. We attempted to categorise Telugu queries both semantically and architecturally. Structural classification aids us in accurately parsing questions and, as a result, correctly identifying question arguments. Semantic classification helps in the mapping of question arguments to Ontologies and, as a result, the extraction of replies from the data resource. We give an architecture for an Interactive QA system as an extension to our QA system. For this we are extending our Ontology to accommodate a vector, which contains the symptoms of the disease in order to identify the disease when described by the farmer. We introduced a dialog manager to accommodate the new changes.

Abstract

We look at Quantum query complexity, a model which has been prominent in evaluating quantum algorithms, and other related complexity measures. There exist two popular lower bounding techniques on quantum query complexity: polynomial method and adversary method. They have been used to determine the quantum query complexity of a function in the case of most quantum algorithms. These, with the recent addition of spectral sensitivity act as concrete lower bounding techniques for quantum query complexity. We look at the quantum query complexity of Gap Majority, a partially symmetric Boolean function. We characterize ts query complexity using Adversary method. We use this result and show a lower bound on noisy randomised query complexity in terms of quantum query complexity. This also forms a base to explore the quantum query complexity of partially symmetric functions. We also give an explicit solution to the positive adversary method for total symmetric Boolean functions. This acts as a stronger evidence for the fact that all the aforementioned concrete lower bounding techniques give the same value for every total symmetric Boolean function [Nai21]. Lastly, we also see how big the separation could be between the block sensitivity and the sensitivity for any total symmetric Boolean function. Block sensitivity and sensitivity have been studied extensively in relation to query complexity. We show the biggest possible separation, even up to constants, and also show a function that achieves this thereby making the bound tight.

Abstract

With the ever-increasing size of documents on the web, tasks such as relevant information extraction with a query and similar document retrieval have become big challenges. Typical information retrieval and classification systems have been posed with questions like – Which documents match the given short text query? Which documents are similar to the given query document? Are the given two query documents possible duplicates? Researchers have been trying to represent documents and queries to answer these questions. Building frameworks for extracting such representations have been crucial to information retrieval research. Handcrafted textual features have been traditionally utilized to represent documents. Recently, advanced learning models such as transformers relying on attention mechanism have formed the basis of learning techniques proven to push the state of the art in language processing. These models have been built to sentence and document representations, however, their applications in extracting domainspecific inter-document information have been limited. Documents are found in various domains – medical, scientific journals, software engineering etc. Short documents are more prevalent in empirical medicine in the form of evidence-based systematic reviews and large scale software repository in the form of bug reports. The data entry design of both these systems has been primarily free-text with weak structure guidance. The large and growing number of published studies makes the task of identifying relevant studies in an unbiased way becomes complex and challenging. Similarly in the context of bug repository, with the rapid increase in features and codebase in large scale repositories, the task of triaging and extracting potential duplicates becomes time-consuming. Thus, this thesis aims at providing document retrieval and representation methods by enhancing query formulation and generating effective document embeddings. Firstly we devise a method of assisting Cochrane experts with writing systematic reviews of a study. In order to write a systematic review, researchers have to conduct a thorough search over the published articles relevant to a study. We conduct experiments that test the effectiveness of information retrieval methods to overcome the limitations of boolean search and title/abstract screening of documents. We provide a method using expansion techniques according to Relevance Feedback to retrieve relevant documents. The feedback mechanism uses term boosting over multiple iterations constructs grading feedback for ranking documents. Secondly, we focus on representing documents over large databases to improve retrieval performance while capturing semantic concept described in natural language. We present a novel method of representing a bug report by utilizing inter-document context. We use pre-trained transformers with a unique choice of triplet formulation to learn rich document embeddings. Experimentation with downstream tasks like duplicate bug detection and classification performs better than the existing methods in terms of recall rates.