Abstract

In stress-timed languages like English, word stress placement can change meaning, making automatic syllable stress
detection vital for Computer Assisted Language Learning
(CALL) systems, as L2 learners often misplace stress. Existing approaches rely on manually annotated boundaries or
forced alignment, which require extensive labeling and suffer
from alignment inaccuracies. We propose a fully boundaryindependent framework using convolutional neural networks
(CNNs) for hierarchical time-compression, converting framelevel embeddings into compact syllable-level representations
without explicit segmentation. To enforce the linguistic constraint that each word contains exactly one stressed syllable, we
also incorporate a Post-net2.0 loss. Experiments on the ISLE
corpus with German (GER) and Italian (ITA) learners show
consistent improvements over boundary-dependent and prior
boundary-independent SOTA models, achieving 94.86% (GER)
and 96.24% (ITA), with gains up to 18.67% and 16.12%.

Abstract

This paper presents a novel framework for fine-grained classification of emphasis in spontaneous speech by moving beyond traditional binary stress detection, which rely on handcrafted features like MFCCs, pitch and formant frequencies. By utilizing a lecture mode multi-speaker speech corpus annotated with two stress levels: `High' and `Low', we propose a methodology that utilises a fusion of acoustic embeddings along with contextual linguistic features from pretrained Self-Supervised models. This feature set is processed by a Temporal CNN-BiLSTM network to learn the intricate temporal patterns of stress. The results demonstrate that the proposed approach outperforms the traditional baselines with an accuracy of 98% on the Stressed vs Unstressed task and 96% on the Low vs High-stress task. The framework achieved an accuracy of 95% on the multi-class (Unstressed vs Low vs High) task, validating its ability to perform real-world nuanced detection of emphasis at varied levels and speakers.

Abstract

utomatic pronunciation assessment (APA) of L2 learners is
critical in computer-assisted language learning. Existing ap-
proaches typically rely on canonical phonemes, its aligned
boundaries and large labeled data, which is costly and cumber-
some. To address these, we propose ALFreeD, a segmentation-
free APA framework guided by teacher's speech without the
need of canonical phonemes using minimal labeled data. In
this, pronunciation quality is assessed by quantifying deviation
between learners' and teacher's utterances. First, frame-level
embeddings are extracted using HuBERT, and deviation scores
are computed between them after obtaining their temporal cor-
respondence with DTW. Next, utterance level pronunciation de-
viation vector is obtained by suppressing variabilities other than
pronunciation quality with i-vector framework and is passed to
a MLP for predicting quality. Experiments on SpeechOcean762
demonstrate competitive performance with recent E2E methods
using minimal labeled data

Abstract

Stroke can often result in Dysarthria, a neuromotor speech impairment that diminishes intelligibility and adversely affects communication and quality of life. Manual evaluation of intelligibility by speech therapists is subjective and labor-intensive, complicating detection of subtle articulatory differences. In this preliminary study, we propose an unsupervised, objective framework for evaluating speech intelligibility in Telugu speaking post-stroke patients. Speech data were collected from ten participants using a phonemically structured set of 38 Telugu words designed to involve different articulatory muscles. Intelligibility was measured as the Dynamic Time Warping (DTW) distance between patient and reference embeddings derived from pretrained Wav2Vec2-Large-XLSR-53-Telugu model.

Our results demonstrate that the proposed PI score reliably distinguishes intelligible and unintelligible speech samples, achieving an F1 score of 94% with over 90% classification accuracy against human perceptual intelligibility ratings. While traditional assessments methods yield a single cumulative score per individual, our methodological framework and stimulus design enables word-level analysis that identifies phoneme-specific misarticulations, thereby supporting targeted therapy for post stroke patients. Collectively, our findings present an automated intelligibility evaluation technique by incorporating unsupervised speech representation learning for post-stroke speech assessment.

Abstract

Automatic syllable stress detection is a key component of computer-assisted language learning (CALL) systems, as it enables precise feedback to second-language (L2) learners. Most of the existing detection methods depend on explicit phoneme or syllable-level boundaries derived from manual annotation or forced alignment. However, manual annotation is costly, and the forced alignment methods are prone to errors. To overcome these limitations, we introduce a fully boundary-independent framework for syllable stress detection based on hierarchical time-compression. Our approach employs an encoder-decoder architecture with Hierarchical compact self-Attention based Time-compression (HAT) to progressively compress the frame-level embeddings into syllable-level representations. This enables the model to capture stress cues directly from speech without any boundary supervision. We evaluate the proposed framework on the ISLE corpus of second language English learners (German and Italian), leveraging self-supervised wav2vec 2.0 embeddings. Experimental results show that our approach consistently outperforms both boundary-dependent and prior partially boundary-independent baselines. Our approach achieves a maximum accuracy of 95.07% on GER and 96.18% on ITA, with highest absolute improvements of up to 23% and 19%.

Abstract

Depression and PTSD often co-occur, and share substantial symptom overlap, making accurate assessment challenging when each disorder is modeled in isolation. However, most existing works treat them independently overlooking comorbidity and symptom-level dynamics. To address this gap, we propose a comorbidity-aware, mulit-tasking framework to jointly model depression and PTSD from clinical interviews. We use ASR decoded text to generate symptom-dependent summaries with a large language model(LLM) guided by symptom-conditioned prompts aligned with Patient Health Questionnaire (PHQ-8) and
PTSD Checklist- Civilian Version (PCL-C). These symptom based summaries are fed into a shared multi-task encoder that jointly predicts symptom-level and overall severity scores for both the disorders. Experiments on the EDAIC dataset show that explicitly modeling comorbidity improves both joint and individual disorder prediction, outperforming prior baselines. Furthermore, stronger depression-related cues provide complementary supervision that enhances PTSD prediction under joint learning highlighting the importance of comorbidity-aware modeling for reliable automated mental health assessment.

Abstract

Lexical stress detection is vital for effective Computer-Assisted Language Learning (CALL) systems and is typically modeled as a binary task, labeling syllables as stressed or unstressed. However, detecting stress in non-native speech is challenging due to deviations from native (canonical) patterns and limited annotated data. We propose LexiRep, a one-shot framework utilizing linguistic knowledge for learning lexical stress representations. It combines a linguistically-guided contrastive framework-based discriminative stress representation learning with Improved Deep Embedded Clustering (IDEC) and iteratively refines the representations under linguistic constraints. We evaluated German and Italian non-native English speech with transfer learning from native data and observed that LexiRep achieves a 3% absolute improvement over the respective state-of-the-art while reducing model complexity by 99%. Further, LexiRep outperforms by 7.62% in an unsupervised setting, demonstrating the effectiveness of linguistically constrained contrastive learning for low-resource stress detection.

Abstract

Automatic syllable stress detection traditionally relies on
heuristic features like energy, duration, and pitch. Recent self-supervised models, such as wav2vec 2.0, capture stress cues more effectively, enhancing scalability. These models encode different information across layers, yet existing research focuses mainly on final-layer features, leaving intermediate layers unexplored. Stress detection is particularly challenging for non-native speakers due to deviations from correct stress patterns. This study analyzes all layers of self-supervised models to identify the most effective for stress detection. We evaluate wav2vec 2.0 (base, large, xlsr) on supervised and unsupervised classifiers using the ISLE corpus of non-native German and Italian speakers. Results show that wav2vec 2.0-large achieves the highest accuracy (96.43% for German, 96.68% for Italian) at layers 16 and 13, respectively, improving by 2.9% and 3.3%
over the baseline.

Abstract

Grammar error detection in one's speech, referred to as spoken grammar error detection (SGED), is a critical component for building computer-assisted language learning systems. Traditionally, ASR followed by text-based GED has been used, but ASR errors can limit SGED performance. In this work, we analyse SGED under three speaking conditions: spontaneous, semi-spontaneous, and memorized, using five state-of-the-art ASRs. We examine the decoded text and ASR cues such as confidence scores and aligner probabilities, out of which, for the latter one, the ground-truth grammatically correct (GGC) text is used. Experiments revealed that autoregressive ASRs show bias toward GGC text, leading to suboptimal performance. It is observed that there is an increase in performance in response to the decreased freedom of speech, with semi-spontaneous speech outperforming memorized speech. Aligner probabilities outperform confidence scores despite the alignment and overconfidence issues.

Abstract

Typically, in computer assisted language learning systems, pronunciation assessment of learners' reading is a critical component when they're practicing with long paragraphs. Thus, automatic methods need to be developed for assessing spoken paragraphs i.e. monologues. However, there is a limitation on the
availability of monologues data, corresponding ratings and the works to assess them. To address the data and label scarcity, this work proposes a method to obtain vector (ProMoVec) representations to assess the pronunciation of monologue considering
the i-vector modelling based on synthetic monologues data. The synthetic monologues are proposed to obtain by concatenating the utterances which are semantically similar and corresponding ratings predicted based on entropy criterion which uses utterance ratings. Experiments with SpeechOcean762 reveal that the PromoVecs used in downstream assessment task with clustering shows a relative improvement of 11.6% compared to the baseline method.