Abstract

Stroke can often result in Dysarthria, a neuromotor speech impairment that diminishes intelligibility and adversely affects communication and quality of life. Manual evaluation of intelligibility by speech therapists is subjective and labor-intensive, complicating detection of subtle articulatory differences. In this preliminary study, we propose an unsupervised, objective framework for evaluating speech intelligibility in Telugu speaking post-stroke patients. Speech data were collected from ten participants using a phonemically structured set of 38 Telugu words designed to involve different articulatory muscles. Intelligibility was measured as the Dynamic Time Warping (DTW) distance between patient and reference embeddings derived from pretrained Wav2Vec2-Large-XLSR-53-Telugu model.

Our results demonstrate that the proposed PI score reliably distinguishes intelligible and unintelligible speech samples, achieving an F1 score of 94% with over 90% classification accuracy against human perceptual intelligibility ratings. While traditional assessments methods yield a single cumulative score per individual, our methodological framework and stimulus design enables word-level analysis that identifies phoneme-specific misarticulations, thereby supporting targeted therapy for post stroke patients. Collectively, our findings present an automated intelligibility evaluation technique by incorporating unsupervised speech representation learning for post-stroke speech assessment.

Abstract

Automatic syllable stress detection is a key component of computer-assisted language learning (CALL) systems, as it enables precise feedback to second-language (L2) learners. Most of the existing detection methods depend on explicit phoneme or syllable-level boundaries derived from manual annotation or forced alignment. However, manual annotation is costly, and the forced alignment methods are prone to errors. To overcome these limitations, we introduce a fully boundary-independent framework for syllable stress detection based on hierarchical time-compression. Our approach employs an encoder-decoder architecture with Hierarchical compact self-Attention based Time-compression (HAT) to progressively compress the frame-level embeddings into syllable-level representations. This enables the model to capture stress cues directly from speech without any boundary supervision. We evaluate the proposed framework on the ISLE corpus of second language English learners (German and Italian), leveraging self-supervised wav2vec 2.0 embeddings. Experimental results show that our approach consistently outperforms both boundary-dependent and prior partially boundary-independent baselines. Our approach achieves a maximum accuracy of 95.07% on GER and 96.18% on ITA, with highest absolute improvements of up to 23% and 19%.

Abstract

Depression and PTSD often co-occur, and share substantial symptom overlap, making accurate assessment challenging when each disorder is modeled in isolation. However, most existing works treat them independently overlooking comorbidity and symptom-level dynamics. To address this gap, we propose a comorbidity-aware, mulit-tasking framework to jointly model depression and PTSD from clinical interviews. We use ASR decoded text to generate symptom-dependent summaries with a large language model(LLM) guided by symptom-conditioned prompts aligned with Patient Health Questionnaire (PHQ-8) and
PTSD Checklist- Civilian Version (PCL-C). These symptom based summaries are fed into a shared multi-task encoder that jointly predicts symptom-level and overall severity scores for both the disorders. Experiments on the EDAIC dataset show that explicitly modeling comorbidity improves both joint and individual disorder prediction, outperforming prior baselines. Furthermore, stronger depression-related cues provide complementary supervision that enhances PTSD prediction under joint learning highlighting the importance of comorbidity-aware modeling for reliable automated mental health assessment.

Abstract

Lexical stress detection is vital for effective Computer-Assisted Language Learning (CALL) systems and is typically modeled as a binary task, labeling syllables as stressed or unstressed. However, detecting stress in non-native speech is challenging due to deviations from native (canonical) patterns and limited annotated data. We propose LexiRep, a one-shot framework utilizing linguistic knowledge for learning lexical stress representations. It combines a linguistically-guided contrastive framework-based discriminative stress representation learning with Improved Deep Embedded Clustering (IDEC) and iteratively refines the representations under linguistic constraints. We evaluated German and Italian non-native English speech with transfer learning from native data and observed that LexiRep achieves a 3% absolute improvement over the respective state-of-the-art while reducing model complexity by 99%. Further, LexiRep outperforms by 7.62% in an unsupervised setting, demonstrating the effectiveness of linguistically constrained contrastive learning for low-resource stress detection.

Abstract

Automatic syllable stress detection traditionally relies on
heuristic features like energy, duration, and pitch. Recent self-supervised models, such as wav2vec 2.0, capture stress cues more effectively, enhancing scalability. These models encode different information across layers, yet existing research focuses mainly on final-layer features, leaving intermediate layers unexplored. Stress detection is particularly challenging for non-native speakers due to deviations from correct stress patterns. This study analyzes all layers of self-supervised models to identify the most effective for stress detection. We evaluate wav2vec 2.0 (base, large, xlsr) on supervised and unsupervised classifiers using the ISLE corpus of non-native German and Italian speakers. Results show that wav2vec 2.0-large achieves the highest accuracy (96.43% for German, 96.68% for Italian) at layers 16 and 13, respectively, improving by 2.9% and 3.3%
over the baseline.

Abstract

Grammar error detection in one's speech, referred to as spoken grammar error detection (SGED), is a critical component for building computer-assisted language learning systems. Traditionally, ASR followed by text-based GED has been used, but ASR errors can limit SGED performance. In this work, we analyse SGED under three speaking conditions: spontaneous, semi-spontaneous, and memorized, using five state-of-the-art ASRs. We examine the decoded text and ASR cues such as confidence scores and aligner probabilities, out of which, for the latter one, the ground-truth grammatically correct (GGC) text is used. Experiments revealed that autoregressive ASRs show bias toward GGC text, leading to suboptimal performance. It is observed that there is an increase in performance in response to the decreased freedom of speech, with semi-spontaneous speech outperforming memorized speech. Aligner probabilities outperform confidence scores despite the alignment and overconfidence issues.

Abstract

Typically, in computer assisted language learning systems, pronunciation assessment of learners' reading is a critical component when they're practicing with long paragraphs. Thus, automatic methods need to be developed for assessing spoken paragraphs i.e. monologues. However, there is a limitation on the
availability of monologues data, corresponding ratings and the works to assess them. To address the data and label scarcity, this work proposes a method to obtain vector (ProMoVec) representations to assess the pronunciation of monologue considering
the i-vector modelling based on synthetic monologues data. The synthetic monologues are proposed to obtain by concatenating the utterances which are semantically similar and corresponding ratings predicted based on entropy criterion which uses utterance ratings. Experiments with SpeechOcean762 reveal that the PromoVecs used in downstream assessment task with clustering shows a relative improvement of 11.6% compared to the baseline method.

Abstract

Automatic pronunciation assessment is a critical component in computer assisted language learning. Typically, modeling pronunciation assessment tasks need labels, which are difficult to obtain as it requires expert annotators. Thus, it is essential to build an accurate model with less annotated data. In this work,
an approach is proposed that considers a few speech samples using the i-vector framework. Each sample, first, is lengthened by T factor by concatenating the augmented samples of the same speech. The augmentation is obtained using time-scale modification (TSM), pitch-scale modification (PSM) and both.
Next, phoneme-level goodness-of-pronunciation scores of concatenated speech are converted to a vector (GoP2Vec) with the i-vector framework. Experiments on two datasets revealed that the proposed GoP2Vec outperforms the state-of-the-art (SOTA) unsupervised methods and is on par with the SOTA supervised
methods when it is used to train a simple neural model with a few samples.

Abstract

Computer-Assisted Language Learning systems provide speech exaggerating at mispronounced word locations as a feedback to the L2 learners. Traditionally, expert speakers recordings (which limit the scalability) are considered for this task though there are advancements in text-to-speech (TTS) that can generate native like natural sounding speech. To address these, this work proposes two novel controllable strategies for scalable speech exaggeration. One strategy is direct speech exaggeration that incorporates the proposed label conditioned tokenization in GlowTTS. Another strategy is cascading the state-of-the-art TTS to a WORLD vocoder with proposed energy and duration modifications. A subset of Tatoeba corpus, that we annotated with prominent words, is used for experimentation. Automatic and manual assessment reveals that the exaggerated speech quality from both direct and cascaded strategy with duration modification is closer to the prominent words in the native speaker's speech.

Abstract

Despite advancements in Speech-to-Speech Translation, maintaining expressiveness between the source and target speech, particularly between English and Indian languages, remains challenging. This study investigates prosodic similarities and variations between English and Malayalam, a language spoken in southern India. A set of 22 prompts from the IViE corpus, covering five categories--simple sentences, WH-questions, questions without morphosyntactic markers, inversion questions, and coordinations--was selected. These prompts, originally spoken by UK speakers, were translated into Malayalam and both language prompts were recorded by bilingual Malayalam speakers while preserving expressiveness. Word-level prominence was manually annotated, and comparisons were made across Indian English, Malayalam, and UK English. The analysis reveals that prominence is retained at key points in Indian English, whereas in Malayalam, it is low due to question specific diacritics and agglutinative nature.