Abstract

Speech enhancement models optimised for percep-
tual quality metrics such as PESQ do not necessarily improve
automatic speech recognition (ASR) performance, as aggres-
sive noise suppression can distort the phonemic cues that
acoustic models rely on. We propose a time-domain speech
frontend based on the Mamba selective state space model,
trained exclusively with ASR-oriented loss functions to pre-
serve phonetically discriminative structure rather than max-
imise perceptual quality. The model follows a U-Net encoder-
decoder architecture with 4× temporal downsampling and six
stacked bidirectional Mamba blocks operating on the raw wave-
form, trained with a multi-component loss combining multi-
resolution spectral supervision, log-mel feature matching, speech
correlation preservation, and ideal-ratio-mask spectral objec-
tives. On VoiceBank-DEMAND, the proposed model achieves
10.26% WER with wav2vec2-base-960h and 7.68% with
whisper-base, achieving statistically equivalent WER to SE-
Mamba (10.30% / 7.73%) while producing a lower PESQ
of 3.10 versus 3.69. Phoneme-class analysis confirms that the
proposed model matches or outperforms SEMamba on five
of seven articulatory classes, demonstrating that ASR-oriented
training objectives achieve equivalent robustness to perceptual
enhancement models without modifying the downstream ASR
system.

Abstract

Developing Automatic Speech Recognition (ASR)
systems for specialized medical domains is challenging in low-
resource settings due to the limited availability of annotated
speech data. Synthetic speech generated using text to speech
(TTS) systems is often used to augment training data, but directly
mixing synthetic and real speech can introduce distribution
mismatch that degrades encoder representations. This work
introduces Real-Governed Representation-Calibrated Training
(RG-RCT), a training strategy that regulates the influence of syn-
thetic speech at the representation level. The proposed framework
combines reliability-based confidence estimation, representation
alignment, and layer-wise representation governance to stabilize
the encoder feature space while preserving acoustic diversity from
multi-speaker synthetic data. Experiments on medical-domain
speech datasets using Wav2Vec2 and Whisper demonstrate that
RG-RCT consistently outperforms conventional strategies such
as direct mixing and confidence-aware training. The proposed
method achieves the lowest Word Error Rate (WER), reaching
17.1% and 16.6% for Telugu and 23.3% and 25.6% for Kannada.
These results indicate that regulating encoder representation
geometry enables more effective utilization of synthetic speech
for low-resource medical ASR

Abstract

Text-to-speech systems are improving fast, but measuring how natural they sound still requires expensive human listening tests; existing automatic methods struggle to generalise well across different datasets, so we present QMOS, a MOS prediction framework that extracts hierarchical speech quality features from a frozen Qwen2-Audio large audio language model and combines them with layer-weighted WavLM SSL representations through a learned cross-attention fusion, capturing both high-level semantic naturalness and low-level acoustic distortions in a unified model, and on two standard benchmarks, SOMOS and BVCC, QMOS achieves competitive system-level SRCC of 0.923 and 0.921 respectively while using no system-ID conditioning or listener embeddings, with cross-domain evaluation yielding a competitive SRCC of 0.702, suggesting the learned representations generalise across acoustic domains.

Abstract

Current systems for stuttering detection are limited by task-agnostic feature compression and flat layer-fusion strategies. Traditional methods rely on Principal Component Analysis (PCA) to reduce self-supervised representations, which often suppresses subtle disfluency patterns in favor of speaker-identity cues. We propose ResGDS-HLF, a framework that replaces static compression with Residual Gumbel Dimension Selection to recover discriminative dimensions directly optimized for stuttering classification. By partitioning the transformer backbone into a functional hierarchy, i.e.~acoustic, phonetic, and lexical, the model preserves specialized information that is otherwise diluted by weighted averaging. Evaluated on the SEP-28K corpus, ResGDS-HLF achieves a Macro-F1 of 0.61, representing an absolute gain of 0.14 over single-layer PCA baselines. Zero-shot transfer to FluencyBank confirms that task-driven Gumbel selection captures robust, generalizable disfluency signatures across diverse acoustic environments.

Abstract

Automatic Speech Recognition (ASR) for low resource Dravidian languages like Telugu and Kannada faces significant challenges in specialized medical domains due to limited annotated data and morphological complexity. This work proposes a novel confidence-aware training framework that integrates real and synthetic speech data through a hybrid confidence mechanism combining static perceptual and acoustic similarity metrics with dynamic model entropy. Unlike direct fine-tuning approaches, the proposed methodology employs both fixed-weight and learnable-weight confidence aggregation strategies to guide sample weighting during training, enabling effective utilization of heterogeneous data sources. The framework is evaluated on Telugu and Kannada medical datasets containing both real recordings and TTS-generated synthetic speech. A 5gram KenLM language model is applied for post-decoding correction. Results show that the hybrid confidence-aware approach with learnable weights substantially reduces recognition errors: Telugu Word Error Rate (WER) decreases from 24.3 % to 15.8 % (8.5 % absolute improvement), while Kannada WER drops from 31.7% to 25.4% (6.3% absolute improvement), both significantly outperforming standard fine-tuning baselines. These findings confirm that combining adaptive confidence-aware training with statistical language modeling delivers superior performance for domain-specific ASR in morphologically complex Dravidian languages.

Abstract

Automatic stuttering detection is hindered by two major challenges: extreme class imbalance, where minority stutter types represent less than five percent of the data, and long-range temporal dependencies that span multiple non-adjacent speech segments. We address these issues with HyDRA, a hierarchical hypergraph neural network that factorizes the detection task into binary disfluency recognition followed by subtype classification. HyDRA constructs view-specific hypergraphs from wav2vec2 and HuBERT embeddings, where hyperedges connect multiple acoustically similar segments, enabling modeling of repetitive and prosodic patterns that pairwise graphs cannot represent. Experiments on the SEP-28k dataset show that HyDRA achieves 47.2 macro F1 for subtype classification, compared to 31.4 for flat baselines, with gains exceeding 16 points on minority classes. Cross-dataset evaluation on FluencyBank confirms generalization across domains. These results demonstrate that hierarchical factorization, higher-order hypergraph modeling, and structured multi-view integration provide a principled solution to the imbalance and dependency challenges in automatic stuttering detection.

Abstract

Despite Telugu being spoken by over 80 million
people, speech translation research for this morphologically rich language remains severely
underexplored. We address this gap by developing a high-quality Telugu-English speech
translation benchmark from 46 hours of manually verified CSTD corpus data (30h/8h/8h
train/dev/test split). Our systematic comparison of cascaded versus end-to-end architectures shows that while IndicWhisper + IndicMT achieves the highest performance due
to extensive Telugu-specific training data, finetuned SeamlessM4T models demonstrate remarkable competitiveness despite using significantly less Telugu-specific training data.
This finding suggests that with careful hyperparameter tuning and sufficient parallel data
(potentially less than 100 hours), end-to-end
systems can achieve performance comparable
to cascaded approaches in low-resource settings. Our metric reliability study evaluating
BLEU, METEOR, ChrF++, ROUGE-L, TER,
and BERTScore against human judgments reveals that traditional metrics provide better
quality discrimination than BERTScore for
Telugu-English translation. The work delivers
three key contributions: a reproducible Telugu-
English benchmark, empirical evidence of competitive end-to-end performance potential in
low-resource scenarios, and practical guidance
for automatic evaluation in morphologically
complex language pairs.

Abstract

Automatic speech recognition for low-resource
languages remains fundamentally constrained
by the scarcity of labeled data and compu-
tational resources required by state-of-the-art
models. We present a systematic investigation
into cross-lingual continuous pretraining for
low-resource languages, using Perso-Arabic
languages (Persian, Arabic, and Urdu) as our
primary case study. Our approach demon-
strates that strategic utilization of unlabeled
speech data can effectively bridge the resource
gap without sacrificing recognition accuracy.
We construct a 3,000-hour multilingual corpus
through a scalable unlabeled data collection
pipeline and employ targeted continual pretrain-
ing combined with morphologically-aware tok-
enization to develop a 300M parameter model
that achieves performance comparable to sys-
tems 5 times larger. Our model outperforms
Whisper Large v3 (1.5B parameters) on Persian
and achieves competitive results on Arabic and
Urdu despite using significantly fewer parame-
ters and substantially less labeled data. These
findings challenge the prevailing assumption
that ASR quality scales primarily with model
size, revealing instead that data relevance and
strategic pretraining are more critical factors for
low-resource scenarios. This work provides a
practical pathway toward inclusive speech tech-
nology, enabling effective ASR for underrepre-
sented languages without dependence on mas-
sive computational infrastructure or proprietary
datasets.

Abstract

Educational technologies like reading tutors and assistive applications rely on accurate pronunciation modeling. Low resource Perso-Arabic languages such as Persian and Urdu lack sufficient Grapheme-to-Phoneme resources compared to Arabic, limiting use in educational tools.
We address this by exploiting phonological similarities between Arabic, Persian, and Urdu to construct a unified Common Phone Space (CPS-APU) that enables cross-lingual knowledge transfer. Using linguistic expertise, we develop a shared phoneme mapping across the three languages.
To evaluate the hypothesis that lower-resource languages benefit from such shared representations, we test and enrich our G2P parser across three stages: (A) expert-authored rules, (B) corpus-mined rules from alignments, and (C) LLM-powered generation using lightweight instruction-tuned models. CPS
framework consistently outperforms monolingual approaches for low-resource languages, with the LLM- augmented stage yielding the best results.

Abstract

Stuttering is a complex speech disorder that challenges both ASR systems and clinical assessment. We propose a multimodal stuttering detection and classification model that integrates acoustic and linguistic features through a two-stage fusion mechanism. Fine-tuned Wav2Vec 2.0 and HuBERT extract acoustic embeddings, which are early fused with MFCC features to capture fine-grained spectral and phonetic variations, while Llama-2 embeddings from Whisper ASR transcriptions provide linguistic context. To enhance robustness against out-of-distribution speech patterns, we incorporate Retrieval-Augmented Generation or adaptive classification. Our model achieves state-of-the-art performance on SEP-28k and FluencyBank, demonstrating significant improvements in detecting challenging stuttering events. Additionally, our analysis highlights the complementary nature of acoustic and linguistic modalities, reinforcing the need for multimodal approaches in speech disorder detection.