Abstract

The scarcity of high-quality annotated data presents a sig- nificant challenge in developing effective end-to-end speech- to-text translation (ST) systems, particularly for low-resource languages. This paper explores the hypothesis that weakly la- beled data can be used to build ST models for low-resource lan- guage pairs. We constructed speech-to-text translation datasets with the help of bitext mining using state-of-the-art sentence encoders. We mined the multilingual Shrutilipi corpus to build Shrutilipi-anuvaad, a dataset comprising ST data for lan- guage pairs Bengali-Hindi, Malayalam-Hindi, Odia-Hindi, and Telugu-Hindi. We created multiple versions of training data with varying degrees of quality and quantity to investigate the effect of quality versus quantity of weakly labeled data on ST model performance. Results demonstrate that ST systems can be built using weakly labeled data, with performance compa- rable to massive multi-modal multilingual baselines such as SONAR and SeamlessM4T.

Abstract

Despite advancements in Speech-to-Speech Translation, maintaining expressiveness between the source and target speech, particularly between English and Indian languages, remains challenging. This study investigates prosodic similarities and variations between English and Malayalam, a language spoken in southern India. A set of 22 prompts from the IViE corpus, covering five categories—simple sentences, WH-questions, questions without morphosyntactic markers, inversion questions, and coordinations—was selected. These prompts, originally spoken by UK speakers, were translated into Malayalam and both language prompts were recorded by bilingual Malayalam speakers while preserving expressiveness. Word-level prominence was manually annotated, and comparisons were made across Indian English, Malayalam, and UK English. The analysis reveals that prominence is retained at key points in Indian English, whereas in Malayalam, it is low due to question specific diacritics and agglutinative nature.

Abstract

We present our shared task on text-based emotion detection, covering more than 30 languages from seven distinct language families. These languages are predominantly low-resource and are spoken across various continents. The data instances are multi-labeled with six emotional classes, with additional datasets in 11 languages annotated for emotion intensity. Participants were asked to predict labels in three tracks: (a) multilabel emotion detection, (b) emotion intensity score detection, and (c) cross-lingual emotion detection. The task attracted over 700 participants. We received final submissions from more than 200 teams and 93 system description papers. We report baseline results, along with findings on the best-performing systems, the most common approaches, and the most effective methods across different tracks and languages. The datasets for this task are publicly available.

Abstract

People worldwide use language in subtle and complex ways to express emotions. Although emotion recognition--an umbrella term for several NLP tasks--impacts various applications within NLP and beyond, most work in this area has focused on high-resource languages. This has led to significant disparities in research efforts and proposed solutions, particularly for under-resourced languages, which often lack high-quality annotated datasets. In this paper, we present BRIGHTER--a collection of multilabeled, emotion-annotated datasets in 28 different languages and across several domains. BRIGHTER primarily covers low-resource languages from Africa, Asia, Eastern Europe, and Latin America, with instances labeled by fluent speakers. We highlight the challenges related to the data collection and annotation processes, and then report experimental results for monolingual and crosslingual multi-label emotion identification, as well as emotion intensity recognition. We analyse the variability in performance across languages and text domains, both with and without the use of LLMs, and show that the BRIGHTER datasets represent a meaningful step towards addressing the gap in text-based emotion recognition.

Abstract

We present a diffusion-based generative model for coordinated trajectory planning in multi-UAV swarms. The proposed method represents each UAV’s trajectory in a Bernstein polynomial coefficient space and employs a de- noising diffusion process with self-attention layers to generate diverse, feasible motion plans. A safety filter is integrated into the generation pipeline to refine candidate trajectories, enforcing inter-drone collision avoidance and other feasibility constraints. The model is trained offline on a large set of expert demonstration trajectories, eliminating the need for reinforcement learning and manual reward function design. In experiments with a 16-UAV swarm using a dataset of collision-free trajectories, the approach achieved a high success rate in producing safe and smooth flight paths. These results demonstrate that the learned planner can rapidly generate a diverse set of smooth, collision-free trajectories for the swarm. Keywords- Swarm, Diffusion, UAV path planning, Joint Opti- mization

Abstract

Current computational approaches for analysing or generating code-mixed sentences do not explicitly model ``naturalness'' or ``acceptability'' of code-mixed sentences, but rely on training corpora to reflect distribution of acceptable code-mixed sentences. Modelling human judgement for the acceptability of code-mixed text can help in distinguishing natural code-mixed text and enable quality-controlled generation of code-mixed text. To this end, we construct Cline - a dataset containing human acceptability judgements for English-Hindi~(en-hi) code-mixed text. Cline is the largest of its kind with 16,642 sentences, consisting of samples sourced from two sources: synthetically generated code-mixed text and samples collected from online social media. Our analysis establishes that popular code-mixing metrics such as CMI, Number of Switch Points, Burstines, which are used to filter/curate/compare code-mixed corpora have low correlation with human acceptability judgements, underlining the necessity of our dataset. Experiments using Cline demonstrate that simple Multilayer Perceptron (MLP) models when trained solely using code-mixing metrics as features are outperformed by fine-tuned pre-trained Multilingual Large Language Models (MLLMs). Specifically, among Encoder models XLM-Roberta and Bernice outperform IndicBERT across different configurations. Among Encoder-Decoder models, mBART performs better than mT5, however Encoder-Decoder models are not able to outperform Encoder-only models. Decoder-only models perform the best when compared to all other MLLMS, with Llama 3.2 - 3B models outperforming similarly sized Qwen, Phi models. Comparison with zero and fewshot capabilitites of ChatGPT show that MLLMs fine-tuned on larger data outperform ChatGPT, providing scope for improvement in code-mixed tasks. Zero-shot transfer from En-Hi to En-Te acceptability judgments are better than random baselines.

Abstract

Large Language Models (LLMs) have made significant contributions to cognitive science research. One area of application is narrative understanding. Sap et al. (2022) introduced sequentiality, an LLM-derived measure that assesses the coherence of a story based on word probability distributions. They reported that recalled stories flowed less sequentially than imagined stories. However, the robustness and generalizability of this narrative flow measure remain unverified. To assess generalizability, we apply sequentiality derived from three different LLMs to a new dataset of matched autobiographical and biographical paragraphs. Contrary to previous results, we fail to find a significant difference in narrative flow between autobiographies and biographies. Further investigation reveals biases in the original data collection process, where topic selection systematically influences sequentiality scores. Adjusting for these biases substantially reduces the originally reported effect size. A validation exercise using LLM-generated stories with "good'' and "poor'' flow further highlights the flaws in the original formulation of sequentiality. Our findings suggest that LLM-based narrative flow quantification is susceptible to methodological artifacts. Finally, we provide some suggestions for modifying the sequentiality formula to accurately capture narrative flow.

Abstract

Creating photorealistic, animatable, and relightable 3D head avatars traditionally requires expensive Lightstage with multiple calibrated cameras, making it inaccessible for widespread adoption. To bridge this gap, we present a novel, cost-effective approach for creating high-quality relightable head avatars using only a smartphone equipped with polaroid filters. Our approach involves simultaneously capturing cross-polarized and parallel-polarized video streams in a dark room with a single point-light source, separating the skin's diffuse and specular components during dynamic facial performances. We introduce a hybrid representation that embeds 2D Gaussians in the UV space of a parametric head model, facilitating efficient real-time rendering while preserving high-fidelity geometric details. Our learning-based neural analysis-by-synthesis pipeline decouples pose and expression-dependent geometrical offsets from appearance, decomposing the surface into albedo, normal, and specular UV texture maps, along with the environment maps. We collect a unique dataset of various subjects performing diverse facial expressions and head movements.

Abstract

We consider the problem of single-source domain generalization. Existing methods typically rely on extensive augmentations to synthetically cover diverse domains during training. However, they struggle with semantic shifts (e.g., background and viewpoint changes), as they often learn global features instead of local concepts that tend to be domain invariant. To address this gap, we propose an approach that compels models to leverage such local concepts during prediction. Given no suitable dataset with per-class concepts and localization maps exists, we first develop a novel pipeline to generate annotations by exploiting the rich features of diffusion and large-language models. Our next innovation is TIDE, a novel training scheme with a concept saliency alignment loss that ensures model focus on the right per-concept regions and a local concept contrastive loss that promotes learning domain-invariant concept representations. This not only gives a robust model but also can be visually interpreted using the predicted concept saliency maps. Given these maps at test time, our final contribution is a new correction algorithm that uses the corresponding local concept representations to iteratively refine the prediction until it aligns with prototypical concept representations that we store at the end of model training. We evaluate our approach extensively on four standard DG benchmark datasets and substantially outperform the current state-of-the-art (12% improvement on average) while also demonstrating that our predictions can be visually interpreted.

Abstract

For people with hearing impairments, lipreading is a crucial skill, as it bridges the gap between spoken language and understanding, improving the ability to communicate effectively. However, the creation of lipreading resources is pervasive and labor-intensive, leading to a scarcity of high-quality, structured, and widely accessible educational materials. To address this, we developed LipMOOC, a Massive Open Online Course (MOOC) tailored for lipreading education. Our approach leverages synthetically generated content, reducing the dependence on human-recorded videos and building an accessible and effective learning option. Using the state-of-the-art models, we created a comprehensive lipreading course and conducted a study to evaluate its effectiveness. We statistically proved that (1) Synthetic videos can replace human-recorded videos without affecting the learning outcomes, and (2) Learner performance on a structured course, i.e., LipMOOC, is much better than a standard course. A total of 120 participants with hearing impairment and no prior lipreading experience volunteered. These results highlight the potential of AI-driven content generation and scientific instructional design.