Abstract

Analysis of Indian English (IE) pronunciation variabilities is useful in ASR and TTS modelling for the Indian context. Prior works characterised IE variabilities by reporting qualitative phonetic rules relative to Received Pronunciation (RP). However, such characterisations lack quantitative descriptors and data-driven analysis of diverse IE pronunciations, which could be due to the scarcity of phonetically labelled data. Furthermore, the versatility of IE stems from the influence of a large diversity of the speakers’ mother tongues and demographic region differences. To address these issues, we consider the corpus Indic TIMIT and manually obtain 13, 632 phonetic transcriptions in addition to those parts of the corpus. By performing a data-driven analysis on 15, 974 phonetic transcriptions of 80 speakers from diverse regions of India, we present a new set of phonetic rules and validate them against the existing phonetic rules to identify their relevance. Finally, we test the efficacy of Grapheme-to-Phoneme (G2P) conversion developed based on the obtained rules considering Phoneme Error Rate (PER) as the metric for performance.

Abstract

The task of headline generation within the realm of Natural Language Processing (NLP) holds immense significance, as it strives to dis- till the true essence of textual content into con- cise and attention-grabbing summaries. While noteworthy progress has been made in head- line generation for widely spoken languages like English, there persist numerous challenges when it comes to generating headlines in low- resource languages, such as the rich and diverse Indian languages. A prominent obstacle that specifically hinders headline generation in In- dian languages is the scarcity of high-quality annotated data. To address this crucial gap, we proudly present Mukhyansh, an extensive mul- tilingual dataset, tailored for Indian language headline generation. Comprising an impressive collection of over 3.39 million article-headline pairs, Mukhyansh spans across eight promi- nent Indian languages, namely Telugu, Tamil, Kannada, Malayalam, Hindi, Bengali, Marathi, and Gujarati. We present a comprehensive evaluation of several state-of-the-art baseline models. Additionally, through an empirical analysis of existing works, we demonstrate that Mukhyansh outperforms all other mod- els, achieving an impressive average ROUGE-L score of 31.43 across all 8 languages.

Abstract

Generative Large Language Models (LLMs) have achieved remarkable advancements in var- ious NLP tasks. In this work, our aim is to ex- plore the multilingual capabilities of large lan- guage models by using machine translation as a task involving English and 22 Indian languages. We first investigate the translation capabilities of raw large language models, followed by ex- ploring the in-context learning capabilities of the same raw models. We fine-tune these large language models using parameter efficient fine- tuning methods such as LoRA and additionally with full fine-tuning. Through our study, we have identified the best performing large lan- guage model for the translation task involving LLMs, which is based on LLaMA.

Abstract

The global impact of coal mining and associated activities on land use/land cover (LULC) changes is significant. This study used Landsat satellite images from 1990 to 2020 to assess LULC changes and their impact on land surface temperature (LST) in four districts of Chhattisgarh state, India. Over three decades, Korba and Raigarh districts saw expansion in coal mines, built-up areas, and water bodies, while forest areas diminished by 711.3 km2 and 212.87 km2, respectively. Koriya district saw coal mine expansion of 5.68 km2 (1990–2010), later declining to 2.85 km2, alongside growth in built-up regions, and forest cover reduction by 251.31 km2 in 2020. Surguja district experienced coal mine and built-up area expansion (1990–2020), with initial forest decline of 160.21 km2 in 2010 followed by recovery in 2020. LST was determined using the Mono-window algorithm. LST increased during winter and summer, with the most significant rise in summer. Vegetation-rich regions had lower LST, while coal mines had the highest temperatures. There was a positive relationship between mining land patch size and patch temperatures. This study underscores the need for vegetation restoration in mining areas, particularly abandoned sites, and sustainable mining practices to mitigate coal mining's warming effects.

Abstract

Unconstrained Asian roads often involve poor infrastructure, affecting overall road safety. Missing traffic signs are a regular part of such roads. Missing or non-existing object detection has been studied for locating missing curbs and estimating reasonable regions for pedestrians on road scene images. Such methods involve analyzing task-specific single object cues. In this paper, we present the first and most challenging video dataset for missing objects, with multiple types of traffic signs for which the cues are visible without the signs in the scenes. We refer to it as the Missing Traffic Signs Video Dataset (MTSVD). MTSVD is challenging compared to the previous works in two aspects i) The traffic signs are generally not present in the vicinity of their cues, ii) The traffic signs’ cues are diverse and unique. Also, MTSVD is the first publicly available missing object dataset. To train the models for identifying missing signs, we complement our dataset with 10K traffic sign tracks, with 40% of the traffic signs having cues visible in the scenes. For identifying missing signs, we propose the Cue-driven Contextual Attention units (CueCAn), which we incorporate in our model’s encoder. We first train the encoder to classify the presence of traffic sign cues and then train the entire segmentation model end-to-end to localize missing traffic signs. Quantitative and qualitative analysis shows that CueCAn significantly improves the performance of base models.

Abstract

Large-scale deployment of fully autonomous vehicles re- quires a very high degree of robustness to unstructured traf- fic, weather conditions, and should prevent unsafe mispre- dictions. While there are several datasets and benchmarks focusing on segmentation for drive scenes, they are not specifically focused on safety and robustness issues. We introduce the IDD-AW dataset, which provides 5000 pairs of high-quality images with pixel-level annotations, cap- tured under rain, fog, low light, and snow in unstructured driving conditions. As compared to other adverse weather datasets, we provide i.) more annotated images, ii.) paired Near-Infrared (NIR) image for each frame, iii.) larger label set with a 4-level label hierarchy to capture unstructured traffic conditions. We benchmark state-of-the-art models for semantic segmentation in IDD-AW. We also propose a new metric called “Safe mean Intersection over Union (Safe mIoU)” for hierarchical datasets which penalizes danger- ous mispredictions that are not captured in the traditional definition of mean Intersection over Union (mIoU). The re- sults show that IDD-AW is one of the most challenging datasets to date for these tasks. The dataset and code will be available here: http://iddaw.github.io

Abstract

In the Structural Health Monitoring (SHM) framework, model updating procedure is an iterative or cyclical process which gives numerical model of structure having modal behaviour close to the dynamic behaviour of real structure. Normally for structures with or without any expansion joints typical model updating procedure is used. But the typical procedure may not be applicable for long uniform buildings having expansion joints since the individual blocks in structure connected with expansion joints tend to vibrate in coherence. This behaviour is confirmed by observing the identicalness of modal frequencies of individual blocks in long uniform buildings. This phenomenon was observed in three selected long uniform buildings. When structures connected with expansion joints vibrate in coherence or when the modal frequency of the connected structures are identical, the numerical models of these structures cannot be updated using typical procedure for the following reasons: (1) Modal parameters from Modal identification of structure doesn’t represent the actual parameters of the individual block since the ambient vibration is not independent and is influenced by interaction between blocks. (2) Modal analysis of analytical model of individual structural blocks gives dynamic parameters which are true to the model. And these two sets of parameters can’t be compared for convergence in model updating procedure. Therefore, to update a numerical model of long uniform buildings with expansion joints having the coherence phenomenon among its individual blocks, a different is needed. Hence, a new approach is proposed where instead of using modal parameters from analytical analysis, the parameters obtained for the structure from modelled floor responses to weak earthquake using Frequency Domain Decomposition (FDD) algorithm are used for checking convergence in model updating procedure. To demonstrate the proposed procedure, two long uniform buildings in which the coherence phenomenon was observed were updated by applying the proposed procedure and results are discussed. Here for the selected buildings, only modal frequencies were used for checking convergence, since mode shapes obtained were not reliable as floor vibration data of intermediate floors were not available.

Abstract

The discovery of potential therapeutic agents for life-threatening diseases has become a significant problem. There is a requirement for fast and accurate methods to identify drug-like molecules that can be used as potential candidates for novel targets. Existing techniques like high-throughput screening and virtual screening are time-consuming and inefficient. Traditional molecule generation pipelines are more efficient than virtual screening but use time-consuming docking software. Such docking functions can be emulated using Machine Learning models with comparable accuracy and faster execution times. However, we find that when pre-trained machine learning models are employed in generative pipelines as oracles, they suffer from model degradation in areas where data is scarce. In this study, we propose an active learning-based model that can be added as a supplement to enhanced molecule generation architectures. The proposed method uses uncertainty sampling on the molecules created by the generator model and dynamically learns as the generator samples molecules from different regions of the chemical space. The proposed framework can generate molecules with high binding affinity with ∼ a 70% improvement in runtime compared to the baseline model by labeling only ∼ 30% of molecules compared to the baseline oracle.

Abstract

We propose CLIP-Head, a novel approach towards text-driven neu- ral parametric 3D head model generation. Our method takes simple text prompts in natural language, describing the appearance & facial expressions, and generates 3D neural head avatars with ac- curate geometry and high-quality texture maps. Unlike existing approaches, which use conventional parametric head models with limited control and expressiveness, we leverage Neural Parametric Head Models (NPHM), offering disjoint latent codes for the dis- entangled encoding of identities and expressions. To facilitate the text-driven generation, we propose two weakly-supervised map- ping networks to map the CLIP’s encoding of input text prompt to NPHM’s disjoint identity and expression vector. The predicted latent codes are then fed to a pre-trained NPHM network to gener- ate 3D head geometry. Since NPHM mesh doesn’t support textures, we propose a novel aligned parametrization technique, followed by text-driven generation of texture maps by leveraging a recently pro- posed controllable diffusion model for the task of text-to-image syn- thesis. Our method is capable of generating 3D head meshes with arbitrary appearances and a variety of facial expressions, along with photoreal texture details. We show superior performance with ex- isting state-of-the-art methods, both qualitatively & quantitatively, and demonstrate potentially useful applications of our method.

Abstract

Researchers have extensively studied the field of vision and language, discovering that both visual and textual content is crucial for understanding scenes effectively. Particularly, comprehending text in videos holds great significance, requiring both scene text understanding and temporal reasoning. This paper focuses on exploring two recently introduced datasets, NewsVideoQA and M4-ViteVQA, which aim to address video question answering based on textual content. The NewsVideoQA dataset contains question-answer pairs related to the text in news videos, while M4-ViteVQA comprises question-answer pairs from diverse categories like vlogging, traveling, and shopping. We provide an analysis of the formulation of these datasets on various levels, exploring the degree of visual understanding and multi-frame comprehension required for answering the questions. Additionally, the study includes experimentation with BERT-QA, a text-only model, which demonstrates comparable performance to the original methods on both datasets, indicating the shortcomings in the formulation of these datasets. Furthermore, we also look into the domain adaptation aspect by examining the effectiveness of training on M4-ViteVQA and evaluating on NewsVideoQA and vice-versa, thereby shedding light on the challenges and potential benefits of out-of-domain training.