Abstract

The internet serves as a vast repository of information covering a diverse array of topics, ranging from blogs and articles to websites. However, it is important to note that not all of this information is valuable or relevant. Navigating through the plethora of content in order to gain a comprehensive understanding of a particular topic can be a daunting and time-consuming task. Furthermore, it is all too common to invest time in reading content that ultimately proves to be unimportant or irrelevant. Given the inherent limitations of the human cognitive capacity to process large quantities of information, concise and relevant summaries are highly sought after in order to efficiently and effectively comprehend complex subjects. Summarization is a computational task that condenses textual information into a concise version by including only the most essential and relevant information. There are two main approaches to summarization: extractive and abstractive. Extractive summarization involves selecting sentences based on their importance, while abstractive summarization involves introducing new words or phrases in the summary. Document summarization has been studied for over three decades by the NLP community. However, progress in Indian language summarization has been limited due to the lack of high-quality datasets and benchmark models, which has motivated us to work towards developing resources and benchmarks for Indian languages. In this thesis, we have developed text summarization resources for Indian languages in three different settings: mono-lingual, cross-lingual, and multi-lingual. The initial focus of this thesis is on mono-lingual summarization, specifically creating a high-quality dataset for the popular south Indian language, Telugu. We propose a pipeline that crowd-sourced summarization data and then aggressively filtered the content via: automatic and partial expert evaluation. Using the pipeline, we create a high-quality Telugu abstractive summarization dataset (TeSum). The dataset consists of 20,329 document-summary pairs, which were created by 347 annotators and evaluated by 3 raters. We carefully designed annotation guidelines that consider the parameters of Relevance, Readability, and Creativity. Additionally, we compared our dataset with existing Telugu summarization datasets. By training a summarization system on multiple languages, the system can learn to represent concepts in a shared space, regardless of the language in which they are expressed. This shared representation learning can be useful for transfer learning, as it enables the model to apply knowledge gained from one language to another language. To achieve this, we perform the multi-lingual and cross-lingual summarization for Indian languages. For multi-lingual summarization, we utilized the Indian Language Summarization (ILSUM) dataset to create baselines, which includes Hindi, Gujarati, and Indian English. We test the proposed filters on ILSUM data to perform the quality assessment. We conducted experiments with different pre-trained sequence-to-sequence models to identify the best-performing model for each language. Our work also involved an in-depth analysis of the impact of k-fold cross-validation when dealing with limited data. Additionally, we performed experiments using a combination of the original and filtered versions of the data to assess the effectiveness of the pre-trained models. We present the PMIndiaSum, a new cross-lingual and highly parallel summarization dataset for languages in India. The dataset covers 4 language families, 14 languages, and 196 language pairs. We detail the approaches taken to derive this dataset, including data acquisition, cleaning, quality assurance, and inspection. In addition, we publish benchmarks for various methodologies, such as fine-tuning pre-trained language models and summarization-and-translation. Experimental results suggest that the provision of multilingual data enhances cross-lingual summarization between Indian languages. Furthermore, this thesis also delves into multi-perspective scientific document summarization. Our objective is to develop a model that can generate a generic summary encompassing various aspects covered by multiple reference summaries of a scientific document. We describe the different pre-trained models used in this task, as well as the challenges encountered during the process.

Abstract

Synthesis, capture and analysis of a highly complex 3D terrain structure are essential for critical applications such as river/flood modelling, disaster mitigation planning, landslide modelling and flight simulation. On the other hand, synthesis of natural-looking 3D terrains finds its applications in the entertainment industry such as computer gaming and VFX. This thesis explores novel learning-based techniques for the generation of immersive and realistic 3D virtual environments, catering to the needs of the aforementioned applications. The generation of virtual worlds involves multiple components, including terrain, vegetation, and other objects. We primarily focus on three key aspects for virtual world generation: 1) develop novel AI-enabled 3D terrain authoring solutions based on real-world satellite and aerial data using a learning-based framework, 2) L-systems grammar-based 3D tree generation and 3) rendering techniques that are used to create high-quality visualizations of the generated world. Terrain generation is a critical component of 3D virtual world generation, as it provides the foundational structure for the environment. Traditional techniques for 3D terrain generation involve procedural generation, which relies on mathematical algorithms to generate landscapes. However, deep learning techniques have shown promise in generating more realistic terrain, as they can learn from real-world data to produce new, varied, and realistic landscapes. In this thesis, we explore the use of deep learning techniques for 3D terrain generation, which can produce realistic and varied terrains with high visual fidelity. Specifically, we propose two learning-based novel frameworks for Interactive 3D Terrain Authoring & Manipulation and Adaptive Multi-Resolution Infinite Terrain Generation. In addition to terrain generation, vegetation is another important component of virtual world generation. Trees and other plants provide visual interest and can help create a more immersive environment. L-systems are a popular technique for generating realistic vegetation, as they are capable of generating complex structures that resemble real-world plants. In this thesis, we propose a variant of the L-systems for 3D tree generation and compare the results to traditional procedural generation techniques. Finally, rendering is a critical component of 3D virtual world generation, as it is responsible for creating the final visual output that users will see. In addition to terrain and tree generation, this thesis also covers rendering techniques used to visualize the generated virtual world. We explore the use of real-time rendering techniques in conjunction with terrain generation to achieve high-quality visual results while maintaining performance. Overall, the research presented in this thesis aims to advance the state-of-the-art in virtual world generation and contribute to the development of more realistic and immersive virtual environments. We performed extensive empirical evaluation on publicly available datasets to report details qualitative and quantitative results demonstrating the superiority of the proposed methods over existing solutions in the literature.

Abstract

Accessibility to text for machines i.e. for downstream tasks, has been improved greatly using mainstream NLP tasks such as Machine Translation and Summarization. In this dissertation, however, we delve into two lesser-explored research areas: Text Simplification (reducing the linguistic complexity of text) and Conversation Disentanglement (extracting individual conversation threads from multi-party dialogues). We pursue text simplification by analyzing the claim of utilizing control tokens in input text to produce simplified versions with precise variations along key text attributes [1]. The effect of control tokens on simplifications was isolated individually and in particular combinations through dedicated models. We show that though control tokens can help mold the outputs closer to the target simplifications, they often cause undesired interactions between control tokens and text attributes as well as within control tokens. This sometimes results in outputs being trivial source text variants that adhere to the control token constraints but do not furnish true simplifications. We offer methods to curb some of these trivial outputs e.g. masking named-entities to shield them from unwanted replacements and to curb the extent of data sparsity using pre-trained input embeddings. Our modifications produce fewer trivial outputs and show better resilience to unseen contexts. As part of our work in conversation disentanglement, we explore the most widely used expertannotated resource in the field: Ubuntu-IRC (IRC) dataset [2]. We question IRC’s utility for training generalizable models for other domains which currently lack similar large-scale, high-quality datasets, e.g., meeting transcripts & movie dialogues. We address this concern by analyzing IRC for potential sources of platform-based specificity. We find that using direct mentions (explicit references to a message’s recipient) in IRC is one such instance of platform specificity that occurs abundantly in the dataset. To measure its impact on model performance, we create Ubuntu-IRC-Hard – a variant of IRC without direct mentions. We show that the performance of past approaches (and proposed baselines) degrades significantly without direct mentions. An in-depth analysis of models is conducted using qualitative metrics based on dialogue properties which reveal brittleness and variations among models across properties. We introduce two methods that leverage these insights to alleviate this brittleness: (i) A weighted-loss formulation to better represent less frequent sub-ranges of dialogue properties in the data, and (ii) An informed-ensembling technique that determines a subset of models best suited for each property sub-range. Both methods surpass SOTA [3] performance and score more consistently across dialogue properties.

Abstract

Air pollution monitoring is crucial for assessing the health risks posed by pollutants, identifying pollution sources, and developing effective strategies for reducing pollution and protecting public health. The Indian government has taken initiatives, which provides real-time air quality data to the public and raises awareness of air pollution levels. The initiative measures several pollutants, including Particulate Matter (PM), and categorizes Air Quality Index (AQI) level into six categories ranging from “Good” to “Severe”. The air pollution sensors employed for calculating the AQI are associated with several limitations. Consequently, the central objective of this thesis is to estimate the AQI without relying on any pollution sensor. To achieve this goal, the proposed methodology employs real-time traffic data and images to estimate the AQI in real-time. Firstly, this thesis propose an image processing based technique to estimate the AQI levels using traffic images and weather parameters, which can be used in rural and sub-urban areas where sensors are hard to deploy and maintain. This approach allows for real-time estimation of AQI through smartphones and can be used portably. The proposed method achieves up to 90% accuracy for the AQI classification. Furthermore, a feature-rich dataset is made publicly available to encourage further research. After that, a novel method based on the Internet-of-Things (IoT) and Machine Learning (ML) is proposed to estimate the AQI using real-time traffic data. To build a rich traffic dataset, PM monitoring nodes were deployed in 15 diverse traffic scenarios across Indian roads, and digital map service providers were utilized. Three ML models, namely Random Forest (RF), Support Vector machine (SVM), and Multi Layer Perceptron (MLP), were trained on this dataset to predict AQI categories into five levels. Experimental results demonstrate an accuracy of 82.60% and an F1-score of 83.67% on the complete dataset. In addition, individual node datasets were used to train ML models, and the behavior of AQI levels was observed.

Abstract

Sign languages have possibly been around for longer than spoken languages. What is interesting about them is that they evolve similarly to spoken languages and tend to have complex grammar and vocabulary. They are mostly used in day-to-day communications by the hearing-impaired community or by people who interact with them on a daily basis. It is even considered by some to be their mother tongue. Indian Sign Language is used mainly in the Indian subcontinent, mostly by the hearing-impaired community. Apart from investigations that have a more or less sociological approach, research results about the linguistic structure of sign language in India had been very scarce until the last decade. However, it has gained prominence in recent years. The 11th Five Year Plan (2007-2012) acknowledged that the needs of people with hearing disabilities had been relatively neglected and envisaged the development of a sign language research and training center, to promote and develop sign language and training of teachers and interpreters. As such, some work is being done in translating spoken languages in various forms into ISL. But most of the work done in this regard always focuses on the accuracy of the signed videos while grammar takes a back step. In 2 we show that these tools are mainly used by hearing-impaired students who are trying to pick up English as a second language. Our work hence focuses on getting as much grammatical accuracy as possible for converting English to Indian Sign Language. Here we tackle the problem of converting written text into ISL video by using Text for Indian Sign Language as an intermediary step. To do this, we use rule-based translation methods and apply techniques such as multi-word expression detection and synonym substitution in the source language. In this thesis, I shall try to explain the advantages of taking this route as well as provide some background into the world of sign languages with a focus on Indian Sign Language (ISL).

Abstract

Unsupervised text embedding models are ubiquitous in the field of Natural language Processing. These models embed words, sentences or documents as vectors in an Euclidean space with the principle that textual information that are semantically similar have similar representations i.e they lie close to each other in the semantic space. Word2vec and GLoVe are the most popular examples of such models. Both these models provide efficient training of word embeddings and provide evaluation methods like word similarity and word analogy tasks that proves the effectiveness of these models. However, they have their fair share of drawbacks. For instance, there is little to no explanation on why word vector summation solves word analogy. These models also suffer from meaning conflation deficiency where multiple senses of a word are represented by one single vector and thus lead to inaccurate semantic modelling. This is also an issue with document and sentence embeddings which span multiple words, phrases and topics. Existing unsupervised models which tackle meaning conflation deficiency perform sense representation where each senses of a word are represented by an individual vector. This is done by modifying the Word2vec skip-gram algorithm and adding some extra constraints on the vector embedding space. Instead of adding extra constraints on the Word2vec algorithm, we aim to address this issue of by exploiting the geometry of the embedding space. We propose three unsupervised text embedding models that embed texts in Riemannian manifolds by integrating the linguistic principles of Word2vec and GLoVe with optimization tools from differential geometry. The first and the second model uses the same joint modelling framework but embeds both words and documents in the Grassmannian manifold and a custom product manifold respectively. Both models generate quality document embeddings given by their evaluation results in document clustering and document classification tasks. The third model embed words in Spectahedron manifold where each word is a matrix whose eigenvectors correspond with the multiple senses of that word. Finally, we provide a Lie-group theoretic understanding of linear substructures present in Word2vec and GLoVe that solves word analogy. Lie groups are differentiable manifolds with a group structure. Some basic properties of these groups show that the linear substructures are present due to the tangent space of the Identity element of the group.

Abstract

Code-mixing (CM) is a common linguistic phenomenon, especially in multilingual communities, where speakers blend two or more languages or dialects in a single sentence or conversation. With English being the second language in India, code-mixing has become a predominant practice, particularly in urban areas. It is common for a Hindi or Telugu speaker to switch to English in the same sentence or utterance. The surge in code-mixing can be attributed to the rise of social media, which has provided a platform for people from diverse linguistic backgrounds to communicate. In such contexts, code-mixing has become an effective way to express oneself using words and phrases from multiple languages. Code-mixing has been of interest to sociolinguists for a long time. Studying its functional form from a sociolinguistic perspective has become a significant phenomenon along with its linguistic structures. In a multilingual country like India, politicians use language and linguistic mechanisms to establish relationships with people. Our study focuses on using code-mixing in Telugu political speeches to understand the factors responsible for their usage levels in various social settings and communicative contexts. As part of our analysis, we have compiled a set of rules that capture dialectal variations between the Standard and Telangana dialects of Telugu. Building NLP systems for code-mixing can help analyze sentiment and conversations in social media. It can also facilitate machine translation, helping people who converse in multiple languages to communicate more efficiently. The informal nature of code-mixed text presents a significant challenge for NLP systems. It results in a wide range of language variations that include non-standard abbreviations, contracted spellings, and informal grammatical structures. These challenges collectively result in a scarcity of quality code-mixed data that can be used for building NLP systems. We propose a hybrid methodology to generate code-mixed text. The proposed hybrid methodology combines rule-based and statistical approaches to convert a monolingual Telugu sentence into a codemixed sentence in English and Telugu dialects. Due to the complexity of the proposed hybrid approach, we propose a fine-tuned neural machine translation method to generate high-quality code-mixed sentences using minimal gold-standard corpus. We use filters from the gold corpus to ensure that the synthetic training data for the models is of high quality, resulting in improved performance of the neural machine translation models. Considering the recent success of pre-trained models such as mT5 and mBART, we fine-tuned these models. Moreover,our approach outperforms the current systems trained on synthetic data for code-mixed generation in Hindi-English. Apart from Hindi-English, the approach performs well when applied to Telugu, a lowresource language, to generate Telugu-English code-mixed sentences. It is crucial to investigate the effectiveness of filtering techniques for generating high-quality code-mixed data, especially in the case of low-resource languages. Moreover, exploring the application of one-shot and zero-shot learning techniques to determine whether the models are trained to generate code-mixed sentences in general or are specific to the languages they are trained on is also a promising future direction.

Abstract

The exchange of information between humans involves the use of speech and text and is known as Natural Language. Each day, people communicate with each other in various languages through speech or text, sharing a vast amount of information. The data produced through natural language communication can offer valuable insights despite being ambiguous, unstructured, and noisy. However, computers cannot interpret natural language on their own yet. To fully comprehend this data and respond intelligently, computers need the ability to understand and emulate human language. This is where Natural Language Processing (NLP) comes in - it is a branch of Artificial Intelligence that focuses on enabling machines to read, comprehend and derive meaning from human languages. NLP integrates the disciplines of linguistics and computer science. It decodes language, its structure, rules, and creates models capable of comprehending, analyzing, and extracting important information from both text and speech. An abundance of information is available in the form of text, including books, documents, articles, social media posts, and more. A document, one of the oldest forms of information exchange, refers to written, printed, or electronic material that is created to facilitate the exchange of information from its author to its intended audience. These files contain valuable information that can significantly benefit business activities. With the use of NLP applications, insights can be extracted from text data. Enterprises utilize NLP applications for various purposes, ranging from document understanding, information extraction, or providing answers to common questions. In this thesis, we develop techniques for deeper analysis and understanding of documents that are commonly used in enterprises. A contract is a frequently used type of document in the corporate world. Contracts are agreements between two or more parties, that govern what each party can or cannot do and are usually dense in information. Automatically extracting key components or components that contain rare or novel information from these large documents makes reviewing contracts easier. Nevertheless, it can be a challenging task as the key and novel components are not present in isolation within the contract. Extraction of significant components (key components + novel components) from contracts aims to simplify the end user’s comprehension and reduce dependency on legal experts for reviewing contracts. In this thesis, we introduce approaches for the automatic identification and extraction of significant components from a contract. We propose a Bidirectional Encoder Representations from Transformers (BERT) based model that automatically identifies or highlights significant components of a contract. In the corporate world, reports are also a frequently encountered type of document. A report is a document that provides information and analysis on a particular topic or issue. Reports are used to convey important information to stakeholders, such as managers, executives, investors, and customers. The vast data available in these reports have the potential to revolutionize datadriven analysis. Causality identification and span detection is one such data-driven task. The relationship between two entities where one causes another event to happen is known as cause and effect. We explored various transformer-based models that help in classifying sentences as well as identifying spans in a sentence.

Abstract

Hospital buildings are expected to remain Occupiable after earthquakes to cater to the post-disaster response and rescue efforts. This requires a stiffer and stronger structural system compared to the commonly adopted moment frames in normal buildings; wallframe systems are recommended in these buildings. But, in addition to merely providing the wall-frame structural system, it is also essential to adequately design the wall-frames to meet the desirable Occupiable seismic performance. Appropriate seismic structural configuration, Structural Plan Density (SPD) of structural walls, and seismic design parameters are presented of typical hospital building located in Seismic Zone IV, that helped meet the preferred performance. A displacement-based limit state of structural damage in line with classic displacement demand estimation is proposed, to confirm the performance. Fiber inelasticity in walls and moment frames in study frames and buildings are defined, for performing displacement controlled nonlinear static pushover analyses and nonlinear time history analyses in commercial software PERFORM3D. Alongside, limit states of structural damages, namely yielding of longitudinal reinforcement in beams or columns in tension, yielding of vertical longitudinal reinforcement in structural walls in tension, crushing of confined concrete in compression in beams, crushing of confined concrete in compression in columns, and spalling of unconfined concrete in compression in structural walls are also monitored to grade the damages, and in turn the seismic performance of wall-frames. Results suggest, numerical models with fiber inelasticity help predict nonlinear behaviour reasonably well. Moment frames do not provide preferred seismic performance, but wall-frames with plan-aspect ratio more than 4 enhances lateral stiffness, strength, and ductility of wall-frames, thereby meeting the preferred performance. The major findings of the study are at least 3% SPD of structural walls is essential in typical wall-frame hospital buildings to ensure Occupiability performance after earthquakes. Furthermore, Column-to-Beam Strength Ratio of minimum 2 is also required in hospital buildings to remain Occupiable.

Abstract

We, humans, are social beings, and our communication is the most evolved and wellstructured form of a communication system that we are aware of. An essential aspect of human communication that helps humans bond faster and develop a sense of closeness is the use of humor. Humor that occurs as part of a conversation is known as conversational humor. Conversational humor is a type of humor that is unique and contrary to what it may seem like. It is more than just plugging canned jokes into a conversation. It requires the use of certain techniques and the presence of at least two interlocutors who understand the context of the conversation. The first step towards understanding conversational humor is to identify the different types into which it can be categorized and the techniques that are used to generate each type of humor. Current studies on this front either consider only a subset of these types and techniques or are domain specific. To tackle these challenges, we first propose a hierarchical annotation schema which allows us to get a comprehensive overview of conversational humor. For this task, we use a famous Telugu play, Kanyasulkam, and consider humorous utterances from this play as the dataset. This schema includes tags for type, technique, and benignity and considers cultural nuances in the text, making it an extensive schema for conversational humor. Further, to test the universality of the schema, we built a dataset of a different domain (Covid-19-based humor) and language (English). This dataset was annotated using a part of the annotation schema containing the type and technique tags. Two more tags viz. “Situation” and “Relevance” were added in the schema to help make the dataset more valuable as a standalone dataset which can be used by researchers from other fields like marketing, sociology, etc. The effectiveness of this dataset is tested with the help of various experiments for binary as well as multi-label multi-class classification using state-of-the-art ML models including but not limited to BERT, RoBERTa, BerTweet, etc. Based on the accuracy and analysis from the experiments, we can show that the annotation schema is universal in terms of language and domain. Such a classification of data can be used to accelerate the annotation process for humor data, and this annotated data can be used for various purposes like marketing, connecting with a target audience based on the relevance tag, and aiding research in the field of conversational humor for building humorous chatbots, and more human-like interactive systems.