Abstract

In recent years, due to the advent of technology and the Internet, the amount of opinionated data targeting products has increased exponentially. With this increase, there has emerged a need to understand the opinions and their associated sentiment to enhance the feedback loop for organizations manufacturing the products and users looking for opinions about the usability of the same to base their future purchasing decisions on. The major bottleneck in achieving the above is the constraint on resources available for certain languages. We explore ways to work through these constraints and attempt to achieve good results for the tasks of Sentiment Analysis and it’s variants. In this thesis, We first attempt to extract language invariant features for downstream tasks like sentiment analysis to be able to retain decent performance in a low resources setting. We find that the ability to do so is task-dependent and hypothesize patterns in tasks where the approach can and cannot work effectively. We then take a closer look at the reasons for the poor performance of models in Aspect Term Extraction and Aspect Term Polarity Classification for Hindi, which is a variant of Sentiment Analysis based on Opinion Mining. After a detailed analysis of the same, we conclude that there is a gap in the state of the existing gold standard dataset for Hindi. We then go ahead and describe our methodology for developing a high-quality dataset parallel to the Gold English dataset for these tasks and establish that the new dataset adequately represents the task. To further improve the state of Aspect Term Extraction (ATE) and Aspect Term Polarity Classification (ATPC), we develop a novel architecture that achieves new state-of-the-art results for Hindi and near state-of-the-art results for English. We also show the fullness of our method in solving the task in a multilingual setting and achieving near-state-of-art results and hence establishing that for tasks where we cannot extract language invariant features, we can develop models that can learn features crucial for the task in a manner which can be leveraged to give high performance reliably.

Abstract

The rapid advancement of autonomous driving systems is transforming the future of transportation and urban mobility. However, these systems face significant challenges when deployed in complex and unstructured traffic environments, such as those found in many cities in south-east Asian countries like India. This thesis aims to address the challenges related to data collection, management, and generation for autonomous driving systems operating in such scenarios. The primary contributions of this work include the development of a data collection toolkit, the creation of a comprehensive driving dataset for unstructured environments (IDD-3D), and the proposal of a synthetic data generation framework (TRoVE) based on real-world data. The data collection toolkit presented in this thesis enables sensor fusion for driving scenarios by treating sensor APIs as separate entities from the data collection interface. This framework allows for the use of any sensor configuration and demonstrates the smooth creation of driving datasets using our framework. This toolkit is adaptable to different environments and can be easily scaled, making it a crucial step towards creating a large-scale dataset for Indian road scenarios. The IDD-3D dataset provides a valuable resource for studying unstructured driving scenarios with complex road situations. We present a thorough statistical and experimental analysis of the dataset, which includes high-quality annotations for 3D object bounding boxes and instance IDs for tracking. We highlight the diverse object types, categories, and complex trajectories found in Indian road scenes, enabling the development of robust autonomous driving systems that can generalize across different geographical locations. In addition, we provide benchmarks for 3D object detection and tracking using state-of-the-art approaches. The TRoVE synthetic data toolkit offers a framework for the automatic generation of synthetic data for visual perception, leveraging existing real-world data. By combining synthetic data with real data, we show the potential for improved performance in various computer vision tasks. The data generation process can be extended to different locations and scenarios, avoiding the limitations of bounded data volumes and variety found in manually designed virtual environments. This thesis contributes to the development of systems in complex environments by addressing the challenges of data acquisition, management, and generation. By bridging the gap between the current state-of-the-art and the needs of unstructured traffic scenarios, this work paves the way for more robust and versatile intelligent transportation systems that can operate safely and efficiently in a wide range of situations

Abstract

The internet serves as a vast repository of information covering a diverse array of topics, ranging from blogs and articles to websites. However, it is important to note that not all of this information is valuable or relevant. Navigating through the plethora of content in order to gain a comprehensive understanding of a particular topic can be a daunting and time-consuming task. Furthermore, it is all too common to invest time in reading content that ultimately proves to be unimportant or irrelevant. Given the inherent limitations of the human cognitive capacity to process large quantities of information, concise and relevant summaries are highly sought after in order to efficiently and effectively comprehend complex subjects. Summarization is a computational task that condenses textual information into a concise version by including only the most essential and relevant information. There are two main approaches to summarization: extractive and abstractive. Extractive summarization involves selecting sentences based on their importance, while abstractive summarization involves introducing new words or phrases in the summary. Document summarization has been studied for over three decades by the NLP community. However, progress in Indian language summarization has been limited due to the lack of high-quality datasets and benchmark models, which has motivated us to work towards developing resources and benchmarks for Indian languages. In this thesis, we have developed text summarization resources for Indian languages in three different settings: mono-lingual, cross-lingual, and multi-lingual. The initial focus of this thesis is on mono-lingual summarization, specifically creating a high-quality dataset for the popular south Indian language, Telugu. We propose a pipeline that crowd-sourced summarization data and then aggressively filtered the content via: automatic and partial expert evaluation. Using the pipeline, we create a high-quality Telugu abstractive summarization dataset (TeSum). The dataset consists of 20,329 document-summary pairs, which were created by 347 annotators and evaluated by 3 raters. We carefully designed annotation guidelines that consider the parameters of Relevance, Readability, and Creativity. Additionally, we compared our dataset with existing Telugu summarization datasets. By training a summarization system on multiple languages, the system can learn to represent concepts in a shared space, regardless of the language in which they are expressed. This shared representation learning can be useful for transfer learning, as it enables the model to apply knowledge gained from one language to another language. To achieve this, we perform the multi-lingual and cross-lingual summarization for Indian languages. For multi-lingual summarization, we utilized the Indian Language Summarization (ILSUM) dataset to create baselines, which includes Hindi, Gujarati, and Indian English. We test the proposed filters on ILSUM data to perform the quality assessment. We conducted experiments with different pre-trained sequence-to-sequence models to identify the best-performing model for each language. Our work also involved an in-depth analysis of the impact of k-fold cross-validation when dealing with limited data. Additionally, we performed experiments using a combination of the original and filtered versions of the data to assess the effectiveness of the pre-trained models. We present the PMIndiaSum, a new cross-lingual and highly parallel summarization dataset for languages in India. The dataset covers 4 language families, 14 languages, and 196 language pairs. We detail the approaches taken to derive this dataset, including data acquisition, cleaning, quality assurance, and inspection. In addition, we publish benchmarks for various methodologies, such as fine-tuning pre-trained language models and summarization-and-translation. Experimental results suggest that the provision of multilingual data enhances cross-lingual summarization between Indian languages. Furthermore, this thesis also delves into multi-perspective scientific document summarization. Our objective is to develop a model that can generate a generic summary encompassing various aspects covered by multiple reference summaries of a scientific document. We describe the different pre-trained models used in this task, as well as the challenges encountered during the process.

Abstract

Synthesis, capture and analysis of a highly complex 3D terrain structure are essential for critical applications such as river/flood modelling, disaster mitigation planning, landslide modelling and flight simulation. On the other hand, synthesis of natural-looking 3D terrains finds its applications in the entertainment industry such as computer gaming and VFX. This thesis explores novel learning-based techniques for the generation of immersive and realistic 3D virtual environments, catering to the needs of the aforementioned applications. The generation of virtual worlds involves multiple components, including terrain, vegetation, and other objects. We primarily focus on three key aspects for virtual world generation: 1) develop novel AI-enabled 3D terrain authoring solutions based on real-world satellite and aerial data using a learning-based framework, 2) L-systems grammar-based 3D tree generation and 3) rendering techniques that are used to create high-quality visualizations of the generated world. Terrain generation is a critical component of 3D virtual world generation, as it provides the foundational structure for the environment. Traditional techniques for 3D terrain generation involve procedural generation, which relies on mathematical algorithms to generate landscapes. However, deep learning techniques have shown promise in generating more realistic terrain, as they can learn from real-world data to produce new, varied, and realistic landscapes. In this thesis, we explore the use of deep learning techniques for 3D terrain generation, which can produce realistic and varied terrains with high visual fidelity. Specifically, we propose two learning-based novel frameworks for Interactive 3D Terrain Authoring & Manipulation and Adaptive Multi-Resolution Infinite Terrain Generation. In addition to terrain generation, vegetation is another important component of virtual world generation. Trees and other plants provide visual interest and can help create a more immersive environment. L-systems are a popular technique for generating realistic vegetation, as they are capable of generating complex structures that resemble real-world plants. In this thesis, we propose a variant of the L-systems for 3D tree generation and compare the results to traditional procedural generation techniques. Finally, rendering is a critical component of 3D virtual world generation, as it is responsible for creating the final visual output that users will see. In addition to terrain and tree generation, this thesis also covers rendering techniques used to visualize the generated virtual world. We explore the use of real-time rendering techniques in conjunction with terrain generation to achieve high-quality visual results while maintaining performance. Overall, the research presented in this thesis aims to advance the state-of-the-art in virtual world generation and contribute to the development of more realistic and immersive virtual environments. We performed extensive empirical evaluation on publicly available datasets to report details qualitative and quantitative results demonstrating the superiority of the proposed methods over existing solutions in the literature.

Abstract

Accessibility to text for machines i.e. for downstream tasks, has been improved greatly using mainstream NLP tasks such as Machine Translation and Summarization. In this dissertation, however, we delve into two lesser-explored research areas: Text Simplification (reducing the linguistic complexity of text) and Conversation Disentanglement (extracting individual conversation threads from multi-party dialogues). We pursue text simplification by analyzing the claim of utilizing control tokens in input text to produce simplified versions with precise variations along key text attributes [1]. The effect of control tokens on simplifications was isolated individually and in particular combinations through dedicated models. We show that though control tokens can help mold the outputs closer to the target simplifications, they often cause undesired interactions between control tokens and text attributes as well as within control tokens. This sometimes results in outputs being trivial source text variants that adhere to the control token constraints but do not furnish true simplifications. We offer methods to curb some of these trivial outputs e.g. masking named-entities to shield them from unwanted replacements and to curb the extent of data sparsity using pre-trained input embeddings. Our modifications produce fewer trivial outputs and show better resilience to unseen contexts. As part of our work in conversation disentanglement, we explore the most widely used expertannotated resource in the field: Ubuntu-IRC (IRC) dataset [2]. We question IRC’s utility for training generalizable models for other domains which currently lack similar large-scale, high-quality datasets, e.g., meeting transcripts & movie dialogues. We address this concern by analyzing IRC for potential sources of platform-based specificity. We find that using direct mentions (explicit references to a message’s recipient) in IRC is one such instance of platform specificity that occurs abundantly in the dataset. To measure its impact on model performance, we create Ubuntu-IRC-Hard – a variant of IRC without direct mentions. We show that the performance of past approaches (and proposed baselines) degrades significantly without direct mentions. An in-depth analysis of models is conducted using qualitative metrics based on dialogue properties which reveal brittleness and variations among models across properties. We introduce two methods that leverage these insights to alleviate this brittleness: (i) A weighted-loss formulation to better represent less frequent sub-ranges of dialogue properties in the data, and (ii) An informed-ensembling technique that determines a subset of models best suited for each property sub-range. Both methods surpass SOTA [3] performance and score more consistently across dialogue properties.

Abstract

Air pollution monitoring is crucial for assessing the health risks posed by pollutants, identifying pollution sources, and developing effective strategies for reducing pollution and protecting public health. The Indian government has taken initiatives, which provides real-time air quality data to the public and raises awareness of air pollution levels. The initiative measures several pollutants, including Particulate Matter (PM), and categorizes Air Quality Index (AQI) level into six categories ranging from “Good” to “Severe”. The air pollution sensors employed for calculating the AQI are associated with several limitations. Consequently, the central objective of this thesis is to estimate the AQI without relying on any pollution sensor. To achieve this goal, the proposed methodology employs real-time traffic data and images to estimate the AQI in real-time. Firstly, this thesis propose an image processing based technique to estimate the AQI levels using traffic images and weather parameters, which can be used in rural and sub-urban areas where sensors are hard to deploy and maintain. This approach allows for real-time estimation of AQI through smartphones and can be used portably. The proposed method achieves up to 90% accuracy for the AQI classification. Furthermore, a feature-rich dataset is made publicly available to encourage further research. After that, a novel method based on the Internet-of-Things (IoT) and Machine Learning (ML) is proposed to estimate the AQI using real-time traffic data. To build a rich traffic dataset, PM monitoring nodes were deployed in 15 diverse traffic scenarios across Indian roads, and digital map service providers were utilized. Three ML models, namely Random Forest (RF), Support Vector machine (SVM), and Multi Layer Perceptron (MLP), were trained on this dataset to predict AQI categories into five levels. Experimental results demonstrate an accuracy of 82.60% and an F1-score of 83.67% on the complete dataset. In addition, individual node datasets were used to train ML models, and the behavior of AQI levels was observed.

Abstract

Sign languages have possibly been around for longer than spoken languages. What is interesting about them is that they evolve similarly to spoken languages and tend to have complex grammar and vocabulary. They are mostly used in day-to-day communications by the hearing-impaired community or by people who interact with them on a daily basis. It is even considered by some to be their mother tongue. Indian Sign Language is used mainly in the Indian subcontinent, mostly by the hearing-impaired community. Apart from investigations that have a more or less sociological approach, research results about the linguistic structure of sign language in India had been very scarce until the last decade. However, it has gained prominence in recent years. The 11th Five Year Plan (2007-2012) acknowledged that the needs of people with hearing disabilities had been relatively neglected and envisaged the development of a sign language research and training center, to promote and develop sign language and training of teachers and interpreters. As such, some work is being done in translating spoken languages in various forms into ISL. But most of the work done in this regard always focuses on the accuracy of the signed videos while grammar takes a back step. In 2 we show that these tools are mainly used by hearing-impaired students who are trying to pick up English as a second language. Our work hence focuses on getting as much grammatical accuracy as possible for converting English to Indian Sign Language. Here we tackle the problem of converting written text into ISL video by using Text for Indian Sign Language as an intermediary step. To do this, we use rule-based translation methods and apply techniques such as multi-word expression detection and synonym substitution in the source language. In this thesis, I shall try to explain the advantages of taking this route as well as provide some background into the world of sign languages with a focus on Indian Sign Language (ISL).

Abstract

Unsupervised text embedding models are ubiquitous in the field of Natural language Processing. These models embed words, sentences or documents as vectors in an Euclidean space with the principle that textual information that are semantically similar have similar representations i.e they lie close to each other in the semantic space. Word2vec and GLoVe are the most popular examples of such models. Both these models provide efficient training of word embeddings and provide evaluation methods like word similarity and word analogy tasks that proves the effectiveness of these models. However, they have their fair share of drawbacks. For instance, there is little to no explanation on why word vector summation solves word analogy. These models also suffer from meaning conflation deficiency where multiple senses of a word are represented by one single vector and thus lead to inaccurate semantic modelling. This is also an issue with document and sentence embeddings which span multiple words, phrases and topics. Existing unsupervised models which tackle meaning conflation deficiency perform sense representation where each senses of a word are represented by an individual vector. This is done by modifying the Word2vec skip-gram algorithm and adding some extra constraints on the vector embedding space. Instead of adding extra constraints on the Word2vec algorithm, we aim to address this issue of by exploiting the geometry of the embedding space. We propose three unsupervised text embedding models that embed texts in Riemannian manifolds by integrating the linguistic principles of Word2vec and GLoVe with optimization tools from differential geometry. The first and the second model uses the same joint modelling framework but embeds both words and documents in the Grassmannian manifold and a custom product manifold respectively. Both models generate quality document embeddings given by their evaluation results in document clustering and document classification tasks. The third model embed words in Spectahedron manifold where each word is a matrix whose eigenvectors correspond with the multiple senses of that word. Finally, we provide a Lie-group theoretic understanding of linear substructures present in Word2vec and GLoVe that solves word analogy. Lie groups are differentiable manifolds with a group structure. Some basic properties of these groups show that the linear substructures are present due to the tangent space of the Identity element of the group.

Abstract

Code-mixing (CM) is a common linguistic phenomenon, especially in multilingual communities, where speakers blend two or more languages or dialects in a single sentence or conversation. With English being the second language in India, code-mixing has become a predominant practice, particularly in urban areas. It is common for a Hindi or Telugu speaker to switch to English in the same sentence or utterance. The surge in code-mixing can be attributed to the rise of social media, which has provided a platform for people from diverse linguistic backgrounds to communicate. In such contexts, code-mixing has become an effective way to express oneself using words and phrases from multiple languages. Code-mixing has been of interest to sociolinguists for a long time. Studying its functional form from a sociolinguistic perspective has become a significant phenomenon along with its linguistic structures. In a multilingual country like India, politicians use language and linguistic mechanisms to establish relationships with people. Our study focuses on using code-mixing in Telugu political speeches to understand the factors responsible for their usage levels in various social settings and communicative contexts. As part of our analysis, we have compiled a set of rules that capture dialectal variations between the Standard and Telangana dialects of Telugu. Building NLP systems for code-mixing can help analyze sentiment and conversations in social media. It can also facilitate machine translation, helping people who converse in multiple languages to communicate more efficiently. The informal nature of code-mixed text presents a significant challenge for NLP systems. It results in a wide range of language variations that include non-standard abbreviations, contracted spellings, and informal grammatical structures. These challenges collectively result in a scarcity of quality code-mixed data that can be used for building NLP systems. We propose a hybrid methodology to generate code-mixed text. The proposed hybrid methodology combines rule-based and statistical approaches to convert a monolingual Telugu sentence into a codemixed sentence in English and Telugu dialects. Due to the complexity of the proposed hybrid approach, we propose a fine-tuned neural machine translation method to generate high-quality code-mixed sentences using minimal gold-standard corpus. We use filters from the gold corpus to ensure that the synthetic training data for the models is of high quality, resulting in improved performance of the neural machine translation models. Considering the recent success of pre-trained models such as mT5 and mBART, we fine-tuned these models. Moreover,our approach outperforms the current systems trained on synthetic data for code-mixed generation in Hindi-English. Apart from Hindi-English, the approach performs well when applied to Telugu, a lowresource language, to generate Telugu-English code-mixed sentences. It is crucial to investigate the effectiveness of filtering techniques for generating high-quality code-mixed data, especially in the case of low-resource languages. Moreover, exploring the application of one-shot and zero-shot learning techniques to determine whether the models are trained to generate code-mixed sentences in general or are specific to the languages they are trained on is also a promising future direction.

Abstract

The exchange of information between humans involves the use of speech and text and is known as Natural Language. Each day, people communicate with each other in various languages through speech or text, sharing a vast amount of information. The data produced through natural language communication can offer valuable insights despite being ambiguous, unstructured, and noisy. However, computers cannot interpret natural language on their own yet. To fully comprehend this data and respond intelligently, computers need the ability to understand and emulate human language. This is where Natural Language Processing (NLP) comes in - it is a branch of Artificial Intelligence that focuses on enabling machines to read, comprehend and derive meaning from human languages. NLP integrates the disciplines of linguistics and computer science. It decodes language, its structure, rules, and creates models capable of comprehending, analyzing, and extracting important information from both text and speech. An abundance of information is available in the form of text, including books, documents, articles, social media posts, and more. A document, one of the oldest forms of information exchange, refers to written, printed, or electronic material that is created to facilitate the exchange of information from its author to its intended audience. These files contain valuable information that can significantly benefit business activities. With the use of NLP applications, insights can be extracted from text data. Enterprises utilize NLP applications for various purposes, ranging from document understanding, information extraction, or providing answers to common questions. In this thesis, we develop techniques for deeper analysis and understanding of documents that are commonly used in enterprises. A contract is a frequently used type of document in the corporate world. Contracts are agreements between two or more parties, that govern what each party can or cannot do and are usually dense in information. Automatically extracting key components or components that contain rare or novel information from these large documents makes reviewing contracts easier. Nevertheless, it can be a challenging task as the key and novel components are not present in isolation within the contract. Extraction of significant components (key components + novel components) from contracts aims to simplify the end user’s comprehension and reduce dependency on legal experts for reviewing contracts. In this thesis, we introduce approaches for the automatic identification and extraction of significant components from a contract. We propose a Bidirectional Encoder Representations from Transformers (BERT) based model that automatically identifies or highlights significant components of a contract. In the corporate world, reports are also a frequently encountered type of document. A report is a document that provides information and analysis on a particular topic or issue. Reports are used to convey important information to stakeholders, such as managers, executives, investors, and customers. The vast data available in these reports have the potential to revolutionize datadriven analysis. Causality identification and span detection is one such data-driven task. The relationship between two entities where one causes another event to happen is known as cause and effect. We explored various transformer-based models that help in classifying sentences as well as identifying spans in a sentence.