Abstract

In the recent past, math word problem solvers have received wide attention from the NLP community at large. With the advancement of deep learning techniques, solvers have started to show better performance than traditional rule based semantic parsing techniques. Standard accuracy metrics have shown that math word problem solvers have achieved high performance on benchmark datasets. However, these performances are based on datasets that have limited problem statements and equation templates, thus providing very limited diversity and a low probability of generalization on different word problems for practical purposes. Hence, the extent to which existing MWP solvers truly understand natural language problem statements and its relationship with numerical quantities is still unclear. In this work, we first generate adversarial attacks to evaluate the robustness of state-of-the-art MWP solvers. We propose two methods Question Reordering and Sentence Paraphrasing to generate adversarial attacks. We conduct experiments across three neural MWP solvers over two benchmark datasets. On average, our attack method is able to reduce the accuracy of MWP solvers by over 40 percentage points on these datasets. Our results demonstrate that existing MWP solvers are sensitive to linguistic variations in the problem text. We verify the validity and quality of generated adversarial examples through human evaluation. These results showcase that math word solvers do not generalize well and rely on superficial cues to achieve high performance. Next, we conduct experiments to showcase that this behaviour is mainly associated with the limited size and diversity present in existing MWP datasets. We modify the problem statements by altering the text in different settings such that either the problem statement does not make much sense or no question has been asked in the problem statement. The preliminary results from these analysis did not show significant drop in accuracy metric as was expected. Then, we propose several data augmentation techniques broadly categorized into Substitution and Paraphrasing based methods to mitigate the issues found by our analysis. By deploying data augmentation methods we increase the size of existing datasets by five folds. Extensive experiments on two benchmark datasets across three state-of-the-art MWP solvers show that the proposed methods increase the generalization and robustness of existing solvers. On average, the proposed methods significantly increase the state-of-the-art results by over five percentage points on benchmark datasets. Further, the solvers trained on the augmented dataset performs comparatively better on the challenge test set. We also show the effectiveness of proposed techniques through ablation studies and verify the quality of augmented samples through human evaluation

Abstract

In this study, we carried out a comprehensive analysis of SARS-CoV-2 mutations and their spread in India over the past two years of the pandemic (27th Jan 2020 – 8th Mar 2022). The analysis covers four important timelines, viz., the early phase, followed by the first, second, and third waves of the pandemic in the country. Phylogenetic analysis of the isolates indicated multiple independent entries of coronavirus in the country, while principal component analysis identified few state-specific clusters. Genetic analysis of isolates during the first year revealed that though lockdown helped in controlling the spread of the virus, region-specific sets of shared mutations were developed during the early phase due to local community transmissions. We thus report the evolution of state-specific subclades, namely, I/GJ20A (Gujarat), I/MH-2 (Maharashtra), I/Tel-A-20B, I/Tel-B-20B (Telangana), and I/AP-20A (Andhra Pradesh) that explain the demographic variation in the impact of COVID-19 across states. In the second year of the pandemic, India faced an aggressive second wave while the third wave was quite mild in terms of severity. Here we also discuss the prevalence and impact of different lineages and Variants of Concerns/Interests, viz., Delta, Kappa, Omicron, etc. observed during this period. From the genetic analysis of mutation spectra of Indian isolates, the insights gained into its transmission, geographic distribution, containment, and impact are discussed. Next, we evaluated the impact of some important India-specific mutations on protein function using structure and network-based analysis. Finally, we performed epidemiological modeling of the dominant variants in India during the Second (Delta and Kappa) and Third (Omicron) waves of the pandemic using multi-strain SEIR models to estimate the transmission factor (β) for the different variants. This interdisciplinary study provides valuable insights into the demographic, structural, and epidemiological factors influencing the spread of COVID-19 in India. The findings can inform and aid policymakers and public health officials in responding to future waves of COVID-19 or any other pandemic caused by a novel pathogen.

Abstract

Spatial Data has increased tremendously in the last two decades from various data sources including satellite sensor data, Internet of Things data, data collected from remote sensing instruments and crowd sourced data. With the advent of mobile phones, devices and internet applications, a vast amount of spatial data is also available from various mobile and web applications. These datasets have lots of applications in different fields like Urban planning, waste management, Traffic monitoring and planning, Climate science studies, Climate resilience, Agriculture, Forest survey, disaster management, smart city planning etc. These datasets have huge volume, variety, veracity, and velocity, making the problem of spatial data analysis a big data problem. To convert these datasets to usable applications in timely manner will ensure the effective utilization of the application. A unified framework that collects, processes, and helps users analyze the output over internet will be beneficial to the geospatial domain researchers and policy makers for enabling faster decision making. Apart from these, web based geospatial applications also has applications for general citizens like smart homes, transportation etc. Climate change has been significantly affecting our environment and society since ages. Another important application of web based geospatial framework is climate change study. This is one of the urgent needs for which informed action has to be taken immediately. Sustainable Development Goal 13 (SDG-13) calls for urgent climate action to combat climate change [1]. This calls for focused study to create mitigation plans at global, country and regional levels. For this, climate research should be carried out in faster way. Also, along with climate science expertise, traditional climate research involves significant data analysis and knowledge in computing is essential. Considering this, there is a need for common platform that provide web-based climate services for climate researchers, decision makers and policy makers that enable faster research and availability of research output results. It is also required for these services to be discoverable, available, interoperable, and reusable. Hence, using published and widely used standards for this platform is also essential for enabling wider use. Open geospatial Consortium (OGC) has been developing and maintaining the standards needed for Geospatial data management, publishing, sharing, and processing such datasets. OGC Web Services standards WMS, WFS, and WPS are suitable for providing data access to users in the form of maps and data. OGC Web Services standard WPS, allows users to utilize the geospatial data processes that are published. Hence OGC Web Services standards are suitable for implementing this framework. Also, these services are widely used and available as open-source solutions. However, OGC Web Services are implemented in Remote-Procedure-Call (RPC) architectural style using Extensible Markup Language (XML) over HTTP protocol. Currently, resource-oriented architecture like RESTful API is being used by various modern web applications and are competing against traditional service-oriented architectures like RPC. Hence, standards that implement RESTful APIs like OGC API standards are more suitable for this framework. OGC API components maps, tiles, features, coverages, and processes offer similar services to OGC Web services. Additionally, OGC API records implements a mechanism for the available datasets and tools to be discoverable. Considering this, OGC APIs are more suitable for this framework. Extensive work is being carried out by the OGC community to publish all the standards of OGC APIs and many opensource software like pygeoap, GeoServer and MapServer are also implementing these standards. Considering this, we chose both OGC API and OGC Web Services standards for the design of this framework. Also, most of the geospatial software worldwide offers tools and services compatible with OGC Web Services currently. By implementing both standards, our design will follow the W3C best practices for sharing data as well as being in compliant with most available and widely used geospatial software. Phasing out the OGC web services can be carried out when the OGC APIs are published and implemented by major geospatial software and service providers. SensorThings API is another standard that is used in this design for ingesting sensor data into the portal. After ingestion, this data is designed to be served using OGC API Features, Maps and Environmental Data Retrieval components making sensor data useful to different sectors of users. Considering the current requirements and the above-mentioned standards, in our research work, we identify the challenges involved in developing a unified web based spatial data framework and design a web based spatial data framework that will address these challenges. Further, we chose climate science as a use case to

Abstract

While face recognition and verification in controlled settings is already a solved problem for machines, the uniqueness of face as a biometric is that the mode of capture is highly diverse. A face could be captured nearby or at distance, at different poses, with different lighting, and by different devices. Face recognition/verification has several challenges to overcome to effectively perform under these varying conditions. Most current methods, try to find salient features of an individual by ignoring these variations. This can be looked at from the paradigm of signal and noise. The signal here refers to that information that is unique to an individual, but not varying as per the condition. Noise represents those aspects that are not related to the identity itself and are influenced by the capture mechanism, physical setting, etc. This is usually done through metric learning approaches in addition to the use of loss functions such as cross-entropy (e.g., Siamese networks, angular loss, and other margin losses such as ArcFace). There are certain aspects that lie between signal and noise such as facial attributes (such as eyeglasses). These may or may not be unique to the individual subject, but introduces artifacts into the face image. The question then arises, why can’t these variations be detected using learning methods, and the knowledge thus attained about the variations be put to good use during the matching process? It is this curiosity that has resulted in aggregation strategies for matching, which were previously implemented for aspects such as pose, age, etc. However, in the wild, humans demonstrate significant variability in facial attributes such as facial hair, eyeglasses, hairstyles, and make-up. This is common as one of the primary mechanisms of face image acquisition is covert capture in public (with ethics of consent in place), where people usually display significant variability in facial attributes. Hence it is very important to address this variability during the matching process. This work attempts to do the same. The curious question that arises however is if indeed matching performance varies if the attribute prior is known. Even if it does, how does one conceptualize a system that exploits the same? It is here that this thesis proposed two frameworks. One of the configuration-specific operating points and the other involves suppression of attribute information in face embedding prior to matching. The attribute suppression is attempted both directly at the final embedding, and suppression of intermediary layers of a Vision Transformer Deep Neural network. Both of these require the facial attribute of each image to be detected prior to passing the images into the proposed framework for matching. The above naturally adds another task to the face verification pipeline. It is therefore extremely necessary to find efficient and effective ways of performing face attribute detection (and face template generation), since efficiently performing parts, mitigates the pipeline expansion overhead and makes this a viable pipeline to consider for face verification. We observe that face attribute detection usually employs end-to-end networks, which results in a lot of parameters for inference. A feasible alternative is to constantly leverage the SOTA (state-of-the-art) face recognition networks and use the earlier feature layers to perform the face attribute classification task. Since the highly accurate SOTA is currently DNNs (Deep Neural Networks) for face, the same is dealt with in this thesis. More narrowly, we focus on open-set face verification, where DNNs aim to find unique representation even for subjects not used for training the DNN.

Abstract

Physical exercises and gross motor skills support the spine and the body’s lower extremities. Un- derstanding the relationship between physical characteristics and activity to kinematic properties of the spine has implications for sports, occupational tasks, clinical diagnosis and prognosis. This study aimed to categorize spine kinematics and the lumbopelvic-hip segments from flexion, lateral-bend, twist, and squat exercises by examining the correlations to body-mass index (BMI), waist-to-hip ratio (WHR), and physical activity levels in healthy young adults. The data was collected from a marker-based opto- electronic motion capture setup and a marker-less motion capture with the RGB camera of a standard mobile phone. Comparing the accuracies of the two techniques on the same data set is important to de- velop inexpensive (RGB camera) diagnostics. Sixty-two participants (40 Male, 22 female) participated in the study. The angular displacement of the spine and knee/hip for each exercise was extracted by vector analysis using a single reference node/marker. The analysis showed no significant correlations of angular displacements values for the four movements with BMI or WHR. The physical activity level of male participants is significantly correlated with angular displacements for the flexion (p<0.001) and lateral-bend (p-value ranges from 0.001-0.04) and weakly correlated (but not significant) for twist ex- ercises. The physical activity level shows a significant correlation for only flexion (p<0.001) in female participants. BMI is negatively correlated in male participants for the squat movement, while WHR and physical activity show positive associations. In the female cohort, a negative correlation with BMI, WHR, and physical activity level is observed for the squat movement. The findings emphasize the criti- cal role of physical activity on musculoskeletal flexibility in young, healthy adults. The angles estimated using both techniques were comparable and significantly correlated across participants and exercises.

Abstract

Legal contracts are formal agreements between parties that create legally binding obligations and rights, typically between two parties involved in a transaction. Legal clauses form the fundamental units of discourse of a legal contract, which are individual provisions that set out specific terms or conditions of the agreement, which collectively make up the entire contract. Contracts are typically drafted by lawyers or other legal professionals who have expertise in contract law through a process that involves a series of steps to ensure that the interests of both the involved parties are captured, while also ensuring the legal correctness of the content. The domain specific content in legal contracts is an interesting area for the application of NLP techniques, not only because of the peculiar nature of legalese, but also because of the challenges that lie in processing the long length of the content involved. Compared to the general domain of language such as news, encyclopedic content, stories, social media content or some other domain-specific verticals such as scientific articles, judicial proceedings, the domain of legal contracts has seen much less research when it comes to the application of deep learning techniques in NLP, with most of it focusing on problems in contract understanding and review. However, a study of the application of generative methods to this domain has been severely lacking. This thesis aims to establish a stepping stone towards the AI-aided generation of legal content, by presenting a study of two generation problems. We also focus on involving user customizability in the process, for easy tailoring of legal content with respect to the parties involved. The thesis starts off with a simple exploratory idea focused on a small set of rental agreements. In this pilot study, we built and studied an agreement drafting tool that could match informal user intents to rental agreement clauses. Observations from this study brought forth the challenges that lie in the generation of legal content, and the need for explicit finetuning in the face of the scarcity of supervised data while leveraging the presence of large unlabeled corpus for content generation. Following this, we study and extend a prior work done towards contract drafting on the recommendation of legal clauses. In this work, we experiment with addition of a new clause to an existing, incomplete contract draft and experiment several strategies to study the effect of different informative signals for clause recommendation. While we model the contexts for recommendation using mean-based pooling of representations from BERTbased architectures, we also explore methods to achieve a better modeling with the use of long range transformers and present the difficulties involved. In another major contribution of this thesis, we devise a pipeline for generation of legal clauses from minimal, keyword-level information. We take inspiration from the content planning for story generation vii viii paradigm, and study the application of coherence-based techniques in devising a content plan for the generation of legal clauses. Our approach is centered on the idea of generating a legal clause, given the topic (or the type) of the clause and a few keywords for customization. Our proposed pipeline uses the topic and keyword input to use a lightweight graph-based mechanism for creating a content plan to act as the outline for the clause to be generated. The content plan is used by a transformer-based generative model for producing an appropriate legal clause, interpolating through the content plan keywords. We propose an ordered content plan consisting of clause keywords, ranked with respect to their generic-ness, with the keywords more generic to the topic ranked higher. We show the benefit of this order as opposed to a natural, sequential order. The study also compares the ordered plan-based generation in contrast to baselines involving prompt-based generation and generation from unordered content plans. We also show the robustness of our approach across a broad range of topics. With our techniques based completely on pretrained transformer architectures, we also contribute a small chapter on the effectiveness of these architectures on four other tasks.

Abstract

Wireless sensor networks are a network of small or large, dedicated sensors designed for particular and specialized purposes. These purposes can range from monitoring environmental behavior like temperature, pressure change, etc., to monitoring man-made physical objects and activities like vehicle speed, sound, density in a region, etc. Sensor networks have a vast application in fields that includes environmental monitoring, natural or artificial object tracking, fire and water detection, natural calamity detection, and traffic monitoring. These groups of sensors and actuators collect information and data for the purpose they have been designed for and then send all the information to a central location for processing or general storage in databases for future use. Based on the topologies of the networks and routing protocols, a particular sensor network can be classified as hierarchical-based routing, location-based routing, and flat-based routing. These protocols are very simple and thus are very prone to attacks like Selective forwarding, HELLO flood attack, Sinkhole attack, Sybil attack, Wormholes, replaying routing information, and Acknowledgement spoofing or altering. Also, since one of the properties of these sensors is limited power, the wireless sensor networks consisting of such sensors generally face power shortages or outages altogether. Wireless Sensors that are small and limited battery capacity are widely used for several different applications. For this reason, energy efficiency, as well as the cost of the overall network, are the two main factors influencing the Wireless Sensor Networks (WSN) performance. Low Energy Adaptive Clustering Hierarchy (LEACH) is an energy-efficient hierarchical-based routing protocol. The prime focus of this thesis is to analyze various existing LEACH protocols and to propose a new LEACH Protocol to resolve some of the issues mentioned above related to wireless sensor networks. This thesis presents an optimized cluster head selection mechanism protocol based on a weighted centroid approach. It is a clustering-based routing protocol that minimizes total energy consumption. Our LEACH-WC protocol algorithm outperforms other LEACH protocol algorithms by making nodes offer themselves candidates for high-energy cluster-heads and adapting the corresponding clusters based on the nodes that choose to be cluster-heads at a given time. We have also discussed an exciting, combined cost and energy wastage minimization problem, and an interesting solution is proposed. Along with the improvement in the existing LEACH protocol, we have also discussed an Along with the improvement in the existing LEACH protocol, we have also discussed an approach that indexes the physical location of various entities across the globe. In the future, combining this improvement in LEACH protocol with the proposed indexing system approach and their application may result in an even better and more intelligent transportation system using wireless sensors network.

Abstract

Music is an important art form which has been present for centuries. Lyrics are a crucial part of a song conveying thoughts, messages and also a wide range of emotions. Individuals listen to songs to satisfy their emotional needs. The task of identifying emotions from a given music track has been an active pursuit in the Music Information Retrieval (MIR) community for years. Music emotion recognition has typically relied on acoustic features, social tags, and other metadata to identify and classify music emotions. The role of lyrics in music emotion recognition remains under-appreciated in spite of several studies reporting superior performance of music emotion classifiers based on features extracted from lyrics. In the first study, we use the transformer-based approach model using XLNet as the base architecture, which has not been used to identify emotional connotations of music based on lyrics. Our proposed approach outperforms existing methods for multiple datasets. We also used a robust methodology to enhance web crawlers’ accuracy for extracting lyrics. There are no datasets of Indian language songs that contain both valence and arousal manual ratings of lyrics. We present a new manually annotated dataset of Telugu songs’ lyrics collected from Spotify with valence and arousal annotated on a discrete scale. A fairly high interannotator agreement was observed for both valence and arousal. Subsequently, we create two music emotion recognition models by using two classification techniques to identify valence, arousal and respective emotion quadrant from lyrics. Support vector machine (SVM) with term frequency-inverse document frequency (TF-IDF) features and fine-tuning the pre-trained XLMRoBERTa (XLM-R) model were used for valence, arousal and quadrant classification tasks. Fine-tuned XLMRoBERTa performs better than the SVM by improving macro-averaged F1-scores of 54.69%, 67.61%, 34.13% to 77.90%, 80.71% and 58.33% for valence, arousal and quadrant classifications, respectively, on 10-fold cross-validation. In addition, we compare our lyrics annotations with Spotify’s annotations of valence and energy (same as arousal), which are based on entire music tracks. We also compare the performance emotion recognition models on the original, translated, transliterated texts. The implications of our findings are discussed. We make the dataset publicly available with lyrics, annotations and Spotify IDs. We also used the XLM-R model to identify emotional connotations in Indian language song lyrics datasets of Hindi and Telugu. We also conducted a perceptual validation study on misclassified lyrics. Lastly, we conduct a study to understand the individual differences between the preferences of music with and without lyrics

Abstract

Data on the internet is growing at ever increasing rates. People share content with each other (or the world at once) over social media platforms such as the likes of Twitter, and consume content from online media outlets such as news agencies like CNN and Dailymail. Gone are the days of monolithic text-only blogs. Online content today generally encompasses multiple modes, or modalities, of communication coming together to convey information. These modalities include text along with images, audio and video. Machine learning models have long been able to capture and understand images and text as separate entities. The first neural network to be used on images dates to before the advent of internet itself. However, only recently, have machine learning researchers started to adopt the notion that multiple modalities can be understood better in a shared setting and under a common architecture, not as isolated black boxes. In this thesis, we attempt to use data-driven approaches towards understanding and improving the interaction of modalities within neural network architectures. The first problem we tackle is that of fake news detection in tweets and posts on microblogging websites such as Weibo. Existing works on the problem focus on independently encoding the different modalities and there is a lack of emphasis on shared learning. Our model attempts to generate richer embeddings through a combination of embeddings generated from pre-trained models. We managed to achieve results that beat the state of the art architecture on the problem statement, and the work was accepted as a full paper in the ASONAM 2021 conference. The second problem we tackle is that of image-aided summarization. While text summarization is a problem that has existed for an eternity, it is not enough to condense information in modality rich sources like news articles. Our model tries to generate textual summaries of articles that demonstrate overlap with image content present in the article, along with selecting the most relevant image from the entire article. We make use of multimodal information retrieval models such as OSCAR to aid in the intermixing of modal information. The third problem we dive into in this thesis is that of content recommendation. Undertaken as a project at LinkedIn, in this problem we try to improve the ranking of LinkedIn Learning content for each user. Since user history is causal, we enable use of time as a modality through making use of techniques such as Time2Vec and train ranking models jointly to enable better vi vii representation of user history and prediction of future action. Through this methodology, we were able to create a strong recommendation system. In the fourth problem, we take a look at the availability of multimodal datasets in Indic languages. To enable and enrich research in this domain in an Indic setting, we try to create the first authentic (not translated) dataset of Image-Text pairs in 11 Indian languages. We use deep-learning based caption filtration techniques to prune down the Samanantar dataset, and then use a query simplification algorithm to create queries to download images related to those sentences. Our work enables the creation of large multimodal models such as CLIP and OSCAR within an Indian setting.

Abstract

Malaria, which is spread by the female anopheles mosquito, is a highly fatal disease that affects many parts of the world, with up to 0.4 million deaths reported worldwide. The detection of malaria infection levels is based on vital gene expressions. Experts quantify malaria parasite-infected RBCs and classify their life cycle stages at the macroscopic level in order to make informed decisions. Several computational approaches have recently been proposed to avoid the dimensionality problem and produce accurately predicted results. Our study presents a theoretical framework to select diagnosis markers and drug targets by implementing ML techniques on sc-RNA-seq data. The main objective is to select the top-ranked genes from the scRNA-seq profiles at different stages of the Plasmodium falciparum (Pf) life cycle inside infected RBC. We employ a supervised learning algorithm coupled with feature selection algorithms to extract the most relevant genes to predict the life cycle stages of Pf inside RBC. The first stage of modeling is to optimize the quality of data from the dataset (5066 features) by removing the irrelevant features. Genetic Algorithm (GA) based search technique is popularly used for feature selection and dealing with high dimensionality datasets. This reduced subset (378) is further utilized in the second stage of high accuracy multi-class classification. In this work, a GA-based dimensionality reduction technique is used on single-cell transcriptomics to obtain an optimised subset of features from a larger data set. To separately transform the selected elements into a lower dimension, features are chosen based on their class variants, taking into account increased efficiency and accuracy. We constructed the protein-protein interaction network (PPIN) of these genes and performed topological analysis using the Search Tool for the Retrieval of Interacting Genes/ Proteins database (STRING 11.0 b) and Gephi software to provide hierarchies according to the importance of the genes in the network. Various topological measures are estimated to evaluate the node characteristics in the PPINs, including degree, between centrality, eccentricity, closeness centrality, eigenvector centrality, and clustering coefficient. Proteins having a high degree and betweenness centrality tend to assert more control over the network function. We also performed gene ontology analysis to determine the role of proteins in the parasite’s life cycle progression. For the multi-class classification of the life cycle of malaria parasite based on oriented gradients and local binary pattern features, a three-pronged approach employing the multi-class Support Vector Machine (SVM), Logistic Regression (LR), and Random Forest (RF) techniques are used. On using these 378 features, RF performed best with a classification accuracy of 92% while SVM had a 91% vi vii accuracy and LR gave 88% accuracy. By merely using the 378 features, we achieved similar or better performance scores for all four classes, across all three models. Further, randomly chosen features from our dataset of 378 were also evaluated using the SVM, LR, and RF models. We achieved an accuracy of 81%, 79%, and 80% for the three respective models. This proves the robustness of the features selected using the GA-based approach. The proposed research methodology can be likely used for improved malaria diagnosis and drug targets.