Abstract

Traditionally, dysarthric speech intelligibility assessment systems have focused on speech as the primary input, utilizing methods such as extraction of relevant speech features, classification models, alignment of Automatic Speech Recognition (ASR) outputs, and comparisons between speech representations of dysarthric and healthy speakers. However, to achieve an automated intelligibility assessment that closely mirrors the auditory-perceptual evaluations conducted by clinicians, a model that captures both the acoustic characteristics of dysarthric speech and the linguistic structure related to word pronunciation are needed. Inspired by the practices of clinicians, this study introduces a novel text-guided dysarthric speech intelligibility assessment framework that leverages custom keyword spotting (DySIA-CKWS). The model evaluates intelligibility by detecting specific keywords and is extensively tested using UA-Speech database for speaker-wise analysis and across word groups of varying complexity. To ensure robustness, the system's performance is further validated on TORGO database, demonstrating its adaptability in cross-database settings. Statistical analysis demonstrates strong alignment between predicted and subjective intelligibility scores, with a Pearson Correlation Coefficient (PCC) of 0.9588 and a Spearman's Correlation Coefficient (SCC) of 0.9141, achieved using the proposed system on the UA-Speech database. The findings emphasize the importance of word selection and showcase the model's effectiveness in diagnosing dysarthric speech, offering a significant advancement in intelligibility assessment methodologies.

Abstract

This paper investigates the nuanced distinctions between typical and atypical speech disfluencies, focusing on features captured by self-supervised models. Typical disfluencies are natural, non-pathological irregularities in speech, while atypical disfluencies are linked to speech disorders like stuttering, characterized by more frequent and severe disruptions. Despite progress in automatic disfluency detection, limited research addresses the direct classification of these two types. This study leverages intermediate representations from four pretrained models Wav2Vec2.0, HuBERT, WavLM, and TERA, to analyze and classify typical and stuttered disfluencies. The experiments utilize two novel Indian English datasets, IIITH-IEDE and IIITH-TISA, enabling a comprehensive analysis of disfluency patterns in a linguistically diverse context. Classification experiments with support vector machines (SVM) and convolutional neural networks (CNN) reveal that features from HuBERT’s 5th layer—balancing low-level acoustic and high-level semantic information—achieve a peak F1 score of 0.97. These findings highlight the importance of intermediate layer representations of self-supervised models in distinguishing nuanced speech variations and contribute to robust and interpretable systems for speech disfluency classification.

Abstract

Dysarthric speech poses significant challenges in developing as- sistive technologies, primarily due to the limited availability of data. Recent advances in neural speech synthesis, especially zero-shot voice cloning, facilitate synthetic speech generation for data augmentation; however, they may introduce biases to- wards dysarthric speech. In this paper, we investigate the effec- tiveness of state-of-the-art F5-TTS in cloning dysarthric speech using TORGO dataset, focusing on intelligibility, speaker sim- ilarity, and prosody preservation. We also analyze potential biases using fairness metrics like Disparate Impact and Parity Difference to assess disparities across dysarthric severity lev- els. Results show that F5-TTS exhibits a strong bias toward speech intelligibility over speaker and prosody preservation in dysarthric speech synthesis. Insights from this study can help integrate fairness-aware dysarthric speech synthesis, fostering the advancement of more inclusive speech technologies.

Abstract

Automatic detection and classification of stuttering events remain a challenging problem in speech processing, particularly due to the variability of disfluency types and the scarcity of large annotated datasets. This paper presents a signal processing-based approach for stuttering classification that avoids the data-intensive requirements of machine learning models. Our method focuses on syllable-level analysis using key acoustic parameters such as energy patterns, pitch contours, spectral stability, and temporal features, implemented through rule-based classification with predefined thresholds. The methodology involves automatic syllable segmentation followed by rule-based classification using thresholds on acoustic features. The dataset comprises annotated read and spontaneous speech from 106 Kannada-speaking adults, including 26 new participants evaluated during clinical trials. Three certified Speech-Language Pathologists (SLPs) conducted perceptual evaluations, and system outputs were validated using intraclass correlation (ICC), yielding strong agreement (textgreater 0.82) across stuttering types. The system achieved classification accuracies of 89% for blocks, 83% for repetitions, and 81% for prolongations. However, the system has several limitations: it relies on static threshold values, exhibits sensitivity to noise, and lacks automated severity grading. Additionally, generalizability across languages and speaking conditions requires further exploration. Machine learning models such as SVM, TDNN, and LSTM were trained and achieved a performance of around 30 to 60% accuracy. Our approach offers high interpretability, low computational cost, and real-time feasibility compared to machine learning methods. Future work will focus on adaptive thresholding, noise-robust processing, and hybrid approaches integrating signal processing with lightweight learning-based refinement.

Abstract

This paper presents the submission of IIITHBUT to the IWSLT 2025 shared task on speech translation for the low-resource Bhojpuri-Hindi language pair. We explored the impact of hyperparameter optimisation and data augmentation techniques on the performance of the SeamlessM4T model fine-tuned for this specific task. We systematically investigated a range of hyperparameters including learning rate schedules, number of update steps, warm-up steps, label smoothing, and batch sizes; and report their effect on translation quality. To address data scarcity, we applied speed perturbation and SpecAugment and studied their effect on translation quality. We also examined the use of cross-lingual signal through joint training with Marathi and Bhojpuri speech data. Our experiments reveal that careful selection of hyperparameters and the application of simple yet effective augmentation techniques significantly improve performance in low-resource settings. We also analysed the translation hypotheses to understand various kinds of errors that impacted the translation quality in terms of BLEU

Abstract

The scarcity of high-quality annotated data presents a sig- nificant challenge in developing effective end-to-end speech- to-text translation (ST) systems, particularly for low-resource languages. This paper explores the hypothesis that weakly la- beled data can be used to build ST models for low-resource lan- guage pairs. We constructed speech-to-text translation datasets with the help of bitext mining using state-of-the-art sentence encoders. We mined the multilingual Shrutilipi corpus to build Shrutilipi-anuvaad, a dataset comprising ST data for lan- guage pairs Bengali-Hindi, Malayalam-Hindi, Odia-Hindi, and Telugu-Hindi. We created multiple versions of training data with varying degrees of quality and quantity to investigate the effect of quality versus quantity of weakly labeled data on ST model performance. Results demonstrate that ST systems can be built using weakly labeled data, with performance compa- rable to massive multi-modal multilingual baselines such as SONAR and SeamlessM4T.

Abstract

Computer-Assisted Language Learning systems provide speech exaggerating at mispronounced word locations as a feedback to the L2 learners. Traditionally, expert speakers recordings (which limit the scalability) are considered for this task though there are advancements in text-to-speech (TTS) that can generate native like natural sounding speech. To address these, this work proposes two novel controllable strategies for scalable speech exaggeration. One strategy is direct speech exaggeration that incorporates the proposed label conditioned tokenization in GlowTTS. Another strategy is cascading the state-of-the-art TTS to a WORLD vocoder with proposed energy and duration modifications. A subset of Tatoeba corpus, that we annotated with prominent words, is used for experimentation. Automatic and manual assessment reveals that the exaggerated speech quality from both direct and cascaded strategy with duration modification is closer to the prominent words in the native speaker's speech.

Abstract

Despite advancements in Speech-to-Speech Translation, maintaining expressiveness between the source and target speech, particularly between English and Indian languages, remains challenging. This study investigates prosodic similarities and variations between English and Malayalam, a language spoken in southern India. A set of 22 prompts from the IViE corpus, covering five categories—simple sentences, WH-questions, questions without morphosyntactic markers, inversion questions, and coordinations—was selected. These prompts, originally spoken by UK speakers, were translated into Malayalam and both language prompts were recorded by bilingual Malayalam speakers while preserving expressiveness. Word-level prominence was manually annotated, and comparisons were made across Indian English, Malayalam, and UK English. The analysis reveals that prominence is retained at key points in Indian English, whereas in Malayalam, it is low due to question specific diacritics and agglutinative nature.

Abstract

Abstract— Transgenic crops have become widespread but have associated health concerns. Testing for transgenic corn is slow and lengthy. In this paper, we used UV-Vis spectroscopy in combination with chemometrics and the PCA-SVM classifier to identify different types of corn samples from each other, including transgenic varieties. The presented methodology and the classifier can differentiate very well between the three different varieties of corn considered in this paper. Although there is a slight improvement in prediction accuracies compared to the state-of-the-art methods, our method is considerably more cost-effective and less time-consuming.

Abstract

Automatic detection of prominence at the word and syllable-levels is critical for building computer-assisted language learning systems. It has been shown that prosody embeddings learned by the current state-of-the-art (SOTA) text-to-speech (TTS) systems could generate word- and syllable-level prominence in the synthesized speech as natural as in native speech. To understand the effectiveness of prosody embeddings from TTS for prominence detection under nonnative context, a comparative analysis is conducted on the embeddings extracted from native and non-native speech considering the prominence-related embeddings: duration, energy, and pitch from a SOTA TTS named FastSpeech2. These embeddings are extracted under two conditions considering: 1) only text, 2) both speech and text. For the first condition, the embeddings are extracted directly from the TTS inference mode, whereas for the second condition, we propose to extract from the TTS under training mode. Experiments are conducted on native speech corpus: Tatoeba, and non-native speech corpus: ISLE. For experimentation, word-level prominence locations are manually annotated for both corpora. The highest relative improvement on word & syllable-level prominence detection accuracies with the TTS embeddings are found to be 13.7% & 5.9% and 16.2% & 6.9% compared to those with the heuristic-based features and self-supervised Wav2Vec-2.0 representations, respectively.