Abstract

Traditionally, dysarthric speech intelligibility assessment systems have focused on speech as the primary input, utilizing methods such as extraction of relevant speech features, classification models, alignment of Automatic Speech Recognition (ASR) outputs, and comparisons between speech representations of dysarthric and healthy speakers. However, to achieve an automated intelligibility assessment that closely mirrors the auditory-perceptual evaluations conducted by clinicians, a model that captures both the acoustic characteristics of dysarthric speech and the linguistic structure related to word pronunciation are needed. Inspired by the practices of clinicians, this study introduces a novel text-guided dysarthric speech intelligibility assessment framework that leverages custom keyword spotting (DySIA-CKWS). The model evaluates intelligibility by detecting specific keywords and is extensively tested using UA-Speech database for speaker-wise analysis and across word groups of varying complexity. To ensure robustness, the system's performance is further validated on TORGO database, demonstrating its adaptability in cross-database settings. Statistical analysis demonstrates strong alignment between predicted and subjective intelligibility scores, with a Pearson Correlation Coefficient (PCC) of 0.9588 and a Spearman's Correlation Coefficient (SCC) of 0.9141, achieved using the proposed system on the UA-Speech database. The findings emphasize the importance of word selection and showcase the model's effectiveness in diagnosing dysarthric speech, offering a significant advancement in intelligibility assessment methodologies.

Abstract

Humans have the ability to utilize visual cues, such as lip movements and visual scenes, to enhance auditory perception, particularly in noisy environments. However, current Automatic Speech Recognition (ASR) or Audio-Visual Speech Recognition (AVSR) models often struggle in noisy scenarios. To solve this task, we propose a model that improves transcription by correlating noise sources to visual cues. Unlike works that rely on lip motion and require the speaker's visibility, we exploit broader visual information from the environment. This allows our model to naturally filter speech from noise and improve transcription, much like humans do in noisy scenarios. Our method re-purposes pretrained speech and visual encoders, linking them with multi-headed attention. This approach enables the transcription of speech and the prediction of noise labels in video inputs. We introduce a scalable pipeline to develop audio-visual datasets, where visual cues correlate to noise in the audio. We show significant improvements over existing audio-only models in noisy scenarios. Results also highlight that visual cues play a vital role in improved transcription accuracy.

Abstract

Automatic syllable stress detection traditionally relies on heuristic features like energy, duration, and pitch. Recent self-supervised models, such as wav2vec 2.0, capture stress cues more effectively, enhancing scalability. These models encode different information across layers, yet existing research focuses mainly on final-layer features, leaving intermediate layers unexplored. Stress detection is particularly challenging for non-native speakers due to deviations from correct stress patterns. This study analyzes all layers of self-supervised models to identify the most effective for stress detection. We evaluate wav2vec 2.0 (base, large, xlsr) on supervised and unsupervised classifiers using the ISLE corpus of non-native German and Italian speakers. Results show that wav2vec 2.0-large achieves the highest accuracy (96.43% for German, 96.68% for Italian) at layers 16 and 13, respectively, improving by 2.9% and 3.3% over the baseline.

Abstract

Brachiation, inspired by ape locomotion, involves swinging from one substrate to another. Existing approaches typically rely on simple grippers and computationally expensive optimal control to compute feasible states and control trajectories. In contrast, learning-based methods often lack physical modeling and require extensive training data. We present a brachiating system using high degree-of-freedom anthropomorphic hands to generate swing trajectories and perform stable grasps in a physics-based simulation environment (MuJoCo). An optimal open-loop trajectory is first generated via trajectory optimization based on a desired grasp location. A tracking controller follows this reference, while a grasping controller activates upon proximity to ensure secure contact. To reduce computational cost, we train a Generative Adversarial Imitation Learning (GAIL) policy using expert trajectories from the optimization framework. The GAIL-based controller generalizes to perturbed conditions and eliminates the need for repeated re-optimization, significantly lowering computation time. It also adapts to varying initial configurations, removing the requirement to rerun optimization for each case. We compare the learned model with a traditional optimal controller and demonstrate marked improvements in both computational efficiency and versatility.

Abstract

Stable aerial manipulation during dynamic tasks such as object catching, perching, or contact with rigid surfaces necessarily requires compliant behavior, which is often achieved via impedance control. Successful manipulation depends on how effectively the impedance control can tackle the unavoidable coupling forces between the aerial vehicle and the manipulator. However, the existing impedance controllers for aerial manipulator either ignore these coupling forces (in partitioned system compliance methods) or require their precise knowledge (in complete system compliance methods). Unfortunately, such forces are very difficult to model, if at all possible. To solve this long-standing control challenge, we introduce an impedance controller for aerial manipulator which does not rely on a priori knowledge of the system dynamics and of the coupling forces. The impedance control design can address unknown coupling forces, along with system parametric uncertainties, via suitably designed adaptive laws. The closed-loop system stability is proved analytically and experimental results with a payload-catching scenario demonstrate significant improvements in overall stability and tracking over the state-of-the-art impedance controllers using either partitioned or complete system compliance.

Abstract

Do LLMs genuinely incorporate external definitions, or do they primarily rely on their parametric knowledge? To address these questions, we conduct controlleD experiments across multiple explanation benchmark datasets (general and domain-specific) and label definition conditions, including expert-curated, LLM- generated, perturbed, and swapped definitions. Our results reveal that while explicit label definitions can enhance accuracy and explainability, their integration into an LLM’s task-solvinG processes is neither guaranteed nor consistent, suggesting reliance on internalized representations in many cases. Models often default to their internal representations, particularly in general tasks, whereas domain-specific tasks benefit more from explicit definitions. These findings underscore the need for a deeper understanding of how LLMs process external knowledge alongside their pre-existing capabilities.

Abstract

Knowledge Graph Foundation Models (KGFMs) have shown promise in enabling zero-shot reasoning over unseen graphs by learning transferable patterns. However, most existing KGFMs rely solely on graph structure, overlooking the rich semantic signals encoded in textual attributes. We introduce SEMMA, a dual-module KGFM that systematically integrates transferable textual semantics alongside structure. SEMMA leverages Large Language Models (LLMs) to enrich relation identifiers, generating semantic embeddings that subsequently form a textual relation graph, which is fused with the structural component. Across 54 diverse KGs, SEMMA outperforms purely structural baselines like ULTRA in fully inductive link prediction. Crucially, we show that in more challenging generalization settings, where the test-time relation vocabulary is entirely unseen, structural methods collapse while SEMMA is 2x more effective. Our findings demonstrate that textual semantics are critical for generalization in settings where structure alone fails, highlighting the need for foundation models that unify structural and linguistic signals in knowledge reasoning.

Abstract

This paper presents an 18.87–24.19 GHz dual-core, dual-mode voltage-controlled oscillator (VCO) using a multi-tap (MT) inductor-based 2-port resonator with capacitive coupling via a mode-switching block. The 2-port resonator introduces a gate-to-drain phase shift, suppressing flicker phase noise (PN), while the high-quality factor of the tank minimizes thermal PN. The mode-switching block enables dual resonance modes—high-frequency band (HB) and low-frequency band (LB), achieving a wide frequency tuning range (FTR) and low PN in both the 1/f^2 and 1/f^3 regions. The proposed VCO is implemented in 65 nm CMOS, and post-layout results show a PN of –68.7 dBc/Hz at a 10 kHz offset and –118.2 dBc/Hz at a 1 MHz offset from an 18.87 GHz carrier frequency, with a power consumption of 25.65 mW and an area of 0.154 mm^2. The peak figure of merit (FoM) is 189.6 dBc/Hz, and the maximum 1/f^3 noise corner frequency is 347 kHz across the FTR.

Abstract

We explore language-agnostic deep text embeddings for severity classification of dysarthria in Amyotrophic Lateral Sclerosis (ALS). Speech recordings are transcribed by human and ASR and embeddings of the transcripts are considered. Though speech recognition accuracy has been studied for grading dysarthria severity, no effort has yet been made to utilize text embeddings of the transcripts. We perform severity classification at different granularity (2, 3, and 5-class) using data obtained from 47 ALS subjects. Experiments with dense neural network based classifiers suggest that, though text features achieve nearly equal performances as baseline speech features, like statistics of mel frequency cepstral coefficients (MFCC), for 2-class classification, speech features outperform for higher number of classes. Concatenation of text embeddings and MFCC statistics attains the best performances with mean F1 scores of 88%, 68%, and 53%, respectively, in 2, 3, and 5-class classification.

Abstract

As robotic manipulators start performing more daily tasks, striking can be a useful method for transporting objects because it significantly increases the reachable workspace. However, striking methods are underexplored compared to pick-and-place because of the difficulty of modeling and executing striking interactions. In this paper, we develop an algorithm for striking objects so that they stop at a target location. We start with an optimizer in simulation that solves for the striking velocity given the relative target position, and perform system identification to set the simulation parameters. However, real-world striking with this model does not have high accuracy because it is unclear which parameters should be considered for identification in practice. Therefore, we finally developed a residual learning approach that subsumes all unmodeled differences between the simulation and the real-world environment into action, that is, striking velocity residuals. Our real-world experiments show that the residual learning model results in 81.6% more accurate object strikes.