Abstract

In this modern world, due to the increased mobility of humans, encountering a foreign language has become a common challenge for many people around the world. This causes a language barrier in their regular lives, which makes communication quite difficult. This makes machine translation a facility that helps people to overcome this language barrier. Research on machine translation has been going on for many decades, and there are many MT models that give good-quality translations, but even the best of these models fail to produce quality output when a domain-specific input is given to them. These models are trained on large general domain data, which makes their domain-specific translations not up to par. This brings up the issue of domain adaptation for different areas. For Indian languages, the issue arises with the lack of domain-specific data and good baseline models. This thesis will try to put forward an approach to improving the scores of domain-specific translations with efficient use of domain data. Before getting into domain adaptation, this thesis will try to understand how a domain is defined and how domain information is stored in these documents or sentences. For this study, we will discuss two tasks that will help us to understand the importance of domain terminologies. The first one discussed fine-grained domain classification as a task. It tries to get information out of similar domains and what makes those domains different by classifying an unknown document into a similar set of domains. The other task helps to find domain terms in a document in an unsupervised manner. It used an improved TextRank approach, where n-grams are used to get the most important terms in a document. Both of these approaches helped in understanding domain terms and their importance in defining a domain. After understanding domains, we detail different approaches done for domain adaptation and give a comparative analysis of them. We started with a very basic domain adaptation approach that gave us a good result but proved to be an inefficient task for multiple domains. All the approaches were for the English-Hindi language pairs, but the basic domain adaptation of individual domains was also done for the English-Telugu and English-Bengali language pairs. After multiple experiments, we show in this thesis how we can get better performances in an efficient manner when we use the domain knowledge of different domains in the task of domain adaptation. Different approaches will be discussed for all the domains to create different translation models for our task of domain adaptation.

Abstract

The electroencephalography (EEG) technique measures scalp electrical activity with high temporal resolution; required to map the neuronal global functional brain networks supporting motor movements. The time series of the change in electric potential across electrodes is analysed using frequency (and/or) time domain methods. An alternate approach is to represent a dynamic system with states and transitions between them defined using microstates. These are quasi-stable (60-120 ms) scalp potential configurations that represent spontaneous EEG as a sequence of a small number of scalp potential field maps and provide insight into brain health exhibiting sub-second coherent activations of brain. The present study investigates microstates extracted from EEG signals of motor-tasks. We investigate the extracted microstate parameters (occurrence, duration, coverage and no. of microstate) for Tip-pinch movement (index finger & thumb) and Wrist flexion & extension performed by both hands independently. These movements are selected due to its relevance in benchmarking rehabilitation of paretic arm post-stroke. The data was extracted from 17 electrodes (of a 32 channel EEG system) covering the pre-frontal, frontal, central and motor regions. Twenty-four right-handed and five left-handed healthy male subjects (age = 18-30 years; mean = 24.25 years; SD = 3.96 years) participate in this study. The approach of analysing each dynamic variable (occurrence, duration, coverage, and number of microstate) values during pre-event, event, and post-event conditions with reported resting state microstates highlights the similarities or differences as a function of motor action. We identified new microstate topographies specific to each motor task. The number of microstates for both motor activities is independent of the handedness of the participant (right-handed and left-handed). Task dependent variation in the topography of the microstates is observed, deviating from the standard maps reported for resting state EEG. The tip-pinch movement for event condition demonstrates higher microstate class stability than Wrist flexion & extension, indicative of lower disruption in the processing for the former action. The identification of disruptions or novel topologies for a task indicates the potential to function as a physiological marker, even considering the arguments on microstate’s state change being discrete or continuous. The findings will serve as a template for future studies analysing data from stroke patients.

Abstract

Data-Driven approaches are becoming increasingly popular for robot control problems. In this thesis, we present two approaches that make use of deep learning frameworks to solve classic robot control problems. In the first part(chapter 3), we propose a lightweight visual servoing MPC framework that generates optimal control near real-time at a frequency of 10.52 Hz. This work utilizes the differential crossentropy sampling method for quick and effective control generation along with a lightweight neural network, significantly improving the servoing frequency. We also propose a flow depth normalization layer that ameliorates the issue of inferior predictions of two view depths from the flow network. We conduct extensive experimentation on the Habitat simulator and show a notable decrease in servoing time compared to other approaches that optimize over a time horizon. We achieve the right balance between time and performance for visual servoing in six degrees of freedom (6DoF), while retaining the advantageous MPC formulation. In the second part(chapter 4), we present a real-time algorithm for computing the optimal sequence and motion trajectories for a fixed-base manipulator to pick and place a set of given objects. The optimality is defined in terms of the total execution time of the sequence or its proxy, the arc length in the joint space. The fundamental complexity stems from the fact that the optimality metric depends on the joint motion, but the task specification is in the end-effector space. Moreover, mapping between a pair of end-effector positions to the shortest arc-length joint trajectory is not analytic; instead, it entails solving a complex trajectory optimization problem. Existing works ignore this complex mapping and use the Euclidean distance in the end-effector space to compute the sequence. In this chapter, we overcome the reliance on the Euclidean distance heuristic by introducing a novel data-driven technique to estimate the optimal arc-length cost in joint space (a.k.a the value function) between two given endeffector positions. We parametrize the value function as a Neural Network and motivate a niche choice for its architecture, inspired by the works on metric learning. The learned value function is then used as an edge cost in a capacitated vehicle routing problem (CVRP) set-up to compute the optimal visitation sequence. Finally, we optimize over the input space of the learned value function network to propose a novel Inverse Kinematics (IK) algorithm that produces substantially shorter joint arc-length trajectories than existing approaches while executing the computed optimal sequence. We show that our sequence planner, in combination with our proposed IK contoller, offers a substantial improvement in joint arc length over existing state-of-the-art while maintaining scalability to a large number of objects

Abstract

Natural language generation has gained tremendous popularity in recent times primarily due to the advent of large pretrained language models trained on vast amount of data. However, most of this progress has only been limited to few high resource languages like English. Almost all low resource(LR) languages still suffer from the lack of sufficient training data and hence the lack of usable generative models. Furthermore, multiple business scenarios also require an automated generation of descriptive human-readable long text from structured input data, where the source is typically a high-resource language and the target is a low or medium resource language. In this work, we present systems and approaches which can be utilised to ultimately enrich the structured and unstructured content available for low resource languages in the encyclopedic domain. In order to do so, we introduce cross lingual techniques which efficiently utilise the abundant structured data available in high resource languages. We also introduce systems to further enrich this structured data using the information present in the form of natural language text in low resource languages. Firstly we propose novel problem of cross lingual fact to text alignment in order to construct the XAlign dataset for the purpose of cross lingual fact to text generation and fact extraction. We explore several methods to automatically align English facts from Wikidata to sentences from native language Wikipedia. We experiment with approaches accounting for syntactic and semantic matches between the fact and the sentence and propose a two stage pipeline for automated alignment and evaluate it on a manually annotated high quality test set. We also experiment with distant supervision and transfer learning based techniques in order to achieve quality alignment. We use the best approach to create the XAlign dataset which consists of more than half a million aligned (sentence, facts) pairs across 12 Indian languages. Following the construction of the dataset we propose the problem of Cross Lingual Fact Extraction (CLFE). Recent approaches concentrate on automatically enriching large knowledge graphs like Wikidata and DBPedia from text. However a lot of information present in the form of natural text in low resource languages is often missed out. Furthermore, considering the potential use case of utilising structured data for generating content in various LR languages, the CLFE task aims at extracting factual information in the form of English triples from LR Indian Language text. Despite its massive potential, progress made on this task is lagging when compared to Monolingual Information Extraction. We propose strong baselines and an end-to-end generative approach for the CLFE task which achieves an overall F1 score of 77.46. We then introduce and explore the problem of cross lingual fact to text generation (XF2T). We extensively explore multiple approaches for the task and analyse different components of the pipeline. Starting from the choice of pretrained transformer model used, we explore the impact of different continued pretraining strategies. We also show that building cross lingual systems results in better performance than translation based approaches or multiple bi-lingual modes, thus validating the necessity of the proposed problem. We introduce novel techniques like fact-aware embeddings to further improve the generation quality. We demonstrate that these methods produce coherent and precise sentences. Our experiments with the XF2T task lead to the observation that these generative models suffer from hallucination and due to the training setup, are only limited to generating a single sentence at a time. In order to mitigate these limitations, we extend the XF2T task to the problem of Cross-Lingual Fact to Long Text Generation (XFLT). The task involves generating descriptive and human readable long text in a target language from structured input data (such as fact triples) in a source language. XFLT is challenging because of (a) hallucinatory nature of the state-of-the-art NLG models, (b) lack of good quality training data, and (c) lack of a suitable cross-lingual NLG metric. Unfortunately previous work focuses on different related problem settings like monolingual graph to text and has made no specific efforts to handle hallucinations. Hence, we propose a novel solution to the XFLT task which addresses these challenges by training multilingual Transformer-based encoder-decoder models with coverage prompts and grounded decoding. Further, it improves on the XFLT quality by defining taskspecific reward functions and training on them using reinforcement learning. On a dataset with over 64,000 paragraphs across 12 different languages, we compare this novel solution with several strong baselines using a new metric, cross-lingual PARENT. Overall, we work on multiple related tasks aimed at automating the gene

Abstract

Artificial time delay controller was conceptualised for nonlinear systems to reduce dependency on precise system modelling unlike the conventional adaptive and robust control strategies. In this approach unknown dynamics is compensated by using input and state measurements collected at immediate past time instant (i.e., artificially delayed). The advantage of this kind of approach lies in its simplicity and ease of implementation. However, the applications of artificial time delay controllers in robotics, which are also robust against unknown statedependent uncertainty, are still missing at large. This thesis presents the study of this control approach toward two important classes of robotic systems, namely a fully actuated bipedal walking robot and an underactuated quadrotor system. In the first work, we explore the idea of a unified control design instead of multiple controllers for different walking phases in adaptive bipedal walking control (i.e. control taking care of unknown robot parameters) while bypassing computing constraint forces, since they often lead to complex designs. State-of-the art attempts to design a single controller for all walking phases either ignored or oversimplified the state-dependent constraint forces which may lead to conservative performance or even instability. This work proposes an innovative artificial time delay based adaptive control method, which covers the entire bipedal walking phase and provides robustness against state-dependent unmodelled dynamics such as constraint forces and external impulsive forces arising during walking. Studies using a high fidelity simulator under various forms of disturbances show the effectiveness of the proposed design over the state of the art. The second work focuses on quadrotors employed for applications such as payload delivery, inspection and search-and-rescue. Such operations pose considerable control challenges, especially when various (a priori unbounded) state-dependent unknown dynamics arises from payload variations, aerodynamic effects and from reaction forces while operating close to the ground or in a confined space. The existing adaptive control strategies for quadrotors, unfortunately, are suitable to handle unknown state-dependent uncertainties. We address such unsolved control challenge in this work via a novel adaptive artificial time delay controller. The effectiveness of this controller is validated using experimental results.

Abstract

Face Reenactment and Synthetic Talking Head works have been widely popular for creating realistic face animations by using a single image of a person. In light of the recent developments in processing facial features in images and videos, as well as the ability to create realistic talking heads, We are focusing on two promising applications. These applications include utilizing face reenactment for movie dubbing and compressing video calls where the primary object is a talking face. We propose a novel method to generate realistic talking head videos using audio and visual streams. We animate a source image by transferring head motion from a driving video using a dense motion field generated using learnable keypoints. We use audio as an additional input for high-quality lip sync, by helping the network to attend to the mouth region. We use additional priors using face segmentation and face mesh to preserve the structure of the reconstructed faces. Finally, we incorporate a carefully designed identity- aware generator module to get realistic quality of talking heads. The identity-aware generator takes the source image and the warped motion features as input to generate a high-quality output with fine-grained details. Our method produces state-of-the-art results and generalizes well to unseen faces, languages, and voices. We comprehensively evaluate our approach using multiple metrics and outperforming the current techniques both qualitative and quantitatively. Our work opens up several applications, including enabling low-bandwidth video calls and movie dubbing. We leverage the advancements in talking head generation to propose an end-to-end system for video call compression. Our algorithm transmits pivot frames intermittently while the rest of the talking head video is generated by animating them. We use a state-of-the-art face reenactment network to detect keypoints in the non-pivot frames and transmit them to the receiver. A dense flow is then calculated to warp a pivot frame to reconstruct the non-pivot ones. Transmitting keypoints instead of full frames leads to significant compression. We propose a novel algorithm to adaptively select the best-suited pivot frames at regular intervals to provide a smooth experience. We also propose a frame-interpolater at the receiver’s end to improve the compression levels further. Finally, a face enhancement network improves reconstruction quality, significantly improving several aspects, like the sharpness of the generations. We evaluate our method both qualitatively and quantitatively on benchmark datasets and compare it with multiple compression techniques

Abstract

Multi-robot formation control has various applications in domains such as vehicle troops, platoons, payload transportation, and surveillance. Maintaining formation in a vehicle platoon requires designing a suitable control scheme that can tackle external disturbances and uncertain system parameters while maintaining a predefined safe distance between the robots. A crucial challenge in this context is dealing with the unknown/uncertain friction forces between wheels and the ground, which vary with changes in road surface, wear in tires, and speed of the vehicle. Although state-of-the-art adaptive controllers can handle a priori bounded uncertainties, they struggle with accurately modeling and identifying frictional forces, which are often state-dependent and cannot be a priori bounded. This thesis proposes a new adaptive sliding mode controller for wheeled mobile robot-based vehicle platoons that can handle the unknown and complex behavior of frictional forces without prior knowledge of their parameters and structures. The controller uses the adaptive sliding mode control techniques to regulate the platoon’s speed and maintain a predefined inter-robot distance, even in the presence of external disturbances and uncertain system parameters. This approach involves a two-stage process: first, the kinematic controller calculates the desired velocities based on the desired trajectory; and second, the dynamics model generates the commands to achieve the desired motion. By separating the kinematics and dynamics of the robot, this approach can simplify the control problem and allow for more efficient and robust control of the wheeled mobile robot. The stability of the closed-loop system employing both the proposed controllers are studied analytically via Lyapunov theory. The effectiveness of the proposed controller is demonstrated through simulations using Gazebo, a popular robot simulation tool. The simulations show that the proposed controller outperforms existing state-of-the-art controllers in terms of stability, convergence, and robustness to changes in frictional forces. The simulations also demonstrate the controller’s ability to maintain formation under various road conditions, including slopes, curves, and rough terrain.

Abstract

Natural Language Generation (NLG) focuses on the automatic generation of natural language text, which should ideally be coherent, fluent, and stylistically appropriate for a given communicative goal and target audience. The tasks in NLG are varied, whether it be summarization, headline generation, dialogue generation etc., and are also heavily dependent on the domain being considered. Recent research has focused on creating domain-specific datasets and developing domain-specific models to make NLP systems more suited to real-world applications. Training models on data specific to a domain has been observed to yield significantly better results across different domains, whether it be legal, financial or biomedical. However, we observe that there has not been much work done on problems in the tourism domain. The tourism industry is important for the benefits it brings and due to its role as a commercial activity that creates demand and growth for many more industries. Currently, there does not exist any standard benchmark for the evaluation of travel and tourism-specific data science tasks and models. To address this gap, we propose a benchmark, TOURISMNLG, of five natural language generation (NLG) tasks for the tourism domain and release corresponding datasets with standard train, validation and test splits. Moreover, as NLG systems are diversifying across languages, the datasets we create and the models we contribute are also multilingual in nature, which is beneficial for the tourism industry globally. Further, previously proposed data science solutions for tourism problems do not leverage the recent benefits of transfer learning. Thus, in this thesis, we also contribute the first rigorously pretrained mT5 and mBART model checkpoints for the tourism domain. The models have been pretrained on four tourismspecific datasets covering different aspects of tourism. Using these models, we present initial baseline results on the benchmark tasks, that indicate an improvement in performance as compared to the respective models without domain-specific pretraining. Additionally, we consider the problem of summarization for Indian languages, as described in the ILSUM (Indian Language SUMmarization) shared task, which focuses on summarising content from the news domain in three important Indian languages: Indian English, Hindi, and Gujarati. We evaluate the performance of existing pretrained models for the task and present our results and findings. We also talk about steps that must be taken to create high-quality summarization datasets for Indian languages. We hope that the contributions of this thesis will promote active research for natural language generation for travel and tourism, as well as other domain-specific and language-specific tasks and models

Abstract

Climate change is one of the most pressing issues of our time, and understanding the discourse surrounding it is crucial for effective communication and action. The discourse encircling climate change can circumscribe a wide range of perspectives, attitudes and opinions. It is essential to analyze this discourse to identify current challenges, road-maps, and systematic changes governments, organizations, and institutions require to combat the effects of climate change. Social media is an important platform for climate change discourse due to its widespread use and real-time nature. This makes it possible to analyze the discourse in near real-time, providing valuable insights into public opinions, attitudes, assess topic framing, event dependent attention to the issue and concerns surrounding climate change. We evaluate the contextual and social features that play key role in the coverage on different platforms. In this thesis, we focus on the fine-grained classification and stance detection of climate changerelated social media text surrounding the United Nations Climate Change Conference. We establish two corpora, ClimateStance and ClimateEng with the help of tweets posted during the 2019 United Nations Framework Convention on Climate Change with Intergovernmental Panel in Geneva. We comprehensively outline the dataset collection, pre-processing, annotation methodology, and dataset composition. We have put together a set of guidelines and specifications for creating expandable corpora ClimateEng, ClimateStance which is a collection of 3777 tweets that have been manually labeled with information about events, states, the categories they belong to, and their corresponding stance. We benchmark both datasets for climate change prevention stance detection and fine-grained classification using state-ofthe-art methods in text classification and experiments along with results are discussed in detail. In addition, we create a dataset called ClimateReddit, which is based on Reddit and includes 6262 comments from climate-change related subreddits. We perform semi-supervised learning on the corpus with pseudo-labelling and manually annotate 329 comments for the tasks of fine-grained classification and stance detection of climate-change data. We compare the results with the best-performing models for both tasks from the supervised experiments. Finally, we provide linguistic analysis of ClimateEng, ClimateStance and ClimateReddit using techniques such as part-of-speech tagging and named-entity recognition. Further, we extend our work in code-mixed setting. We collect Hindi and English code-mixed data from twitter during 2020 and construct a corpus of code-mixed Twitter data. We define the task of finegrained classification for the same and outline data-collection and annotation methodology for code-mix data.

Abstract

The Large Hadron Collider (LHC) experiment is searching for exotic beyond the Standard Model signal processes. These processes occur at extremely low rates compared to the Standard Model ones making it difficult to extract these signals effectively. Classical cut-based methods that rely heavily on hand-crafted strategies fail to perform well on these exotic signals. For the next stage of runs at the LHC, there is an urgent need to develop newer algorithms more capable of identifying these rare signals. Deep learning methods have recently gained traction to assist these searches due to their exceptional ability to analyse complex patterns and show immense potential to improve the sensitivity of experiments. In this thesis, I describe a few approaches to adapt and incorporate deep learning techniques into collider experiments. First, I present a case study of a phenomenological search for a hypothetical heavy quark, where I augment the standard search strategy with a simple deep-learning model to yield better sensitivity and reach. In the case study, I also discuss the caveats of such methods and point to a potential solution. Secondly, I present a simple adaptation of the deep learning training strategy to align it with physics goals. Specifically, I study modifications to the standard Cross Entropy loss and propose a new loss function, based on the Lovasz extension, to directly optimise the sensitivity metric (Z-score). Finally, I present a novel generative model based on the GAN (Generative Adversarial Framework) to generate particle jets. This model can quickly generate jets from different processes and assist jet studies.