Abstract

Despite the progress seen in classification methods, current approaches for handling videos with distribution shifts in source and target domains remain source-dependent as they require access to the source data during the adaptation stage. In this paper, we present a self-training based source-free video domain adaptation approach to address this challenge by bridging the gap between the source and the target domains. We use the source pre-trained model to generate pseudo-labels for the target domain samples, which are inevitably noisy. Thus, we treat the problem of source-free video domain adaptation as learning from noisy labels and argue that the samples with correct pseudo-labels can help us in adaptation. To this end, we leverage the crossentropy loss as an indicator of the correctness of the pseudo-labels and use the resulting small-loss samples from the target domain for fine-tuning the model. We further enhance the adaptation performance by implementing a teacher-student framework, in which the teacher, which is updated gradually, produces reliable pseudo-labels. Meanwhile, the student undergoes fine-tuning on the target domain videos using these generated pseudo-labels to improve its performance. Extensive experimental evaluations show that our methods, termed as CleanAdapt, CleanAdapt + TS, achieve state-of-the-art results, outperforming the existing approaches on various open datasets. Our source code is publicly available at https://avijit9.github.io/CleanAdapt.

Abstract

Stuttering is a speech disorder that affects speech fluency and rhythm, with millions worldwide experiencing it. Early diagnosis and treatment can significantly improve speech fluency and the quality of life for individuals who stutter. Automatic detection of stuttering events can help diagnose, monitor, and develop effective interventions. Therefore, this paper aims to propose a feature space-based classifier for detecting stuttering events in speech. To achieve this, we have investigated Zero Time Windowing Cepstral Coefficients (ZTWCC) as a feature set for stutter detection using classifiers such as SVM, LSTM, and Bidirectional LSTM. We compared the performance of ZTWCC with the standard handcrafted features, such as MFCC, CQCC, and SFFCC, on the SEP- 28K dataset with and without including phase information. The results in both cases indicate that ZTWCC is giving a higher F-1 score than baseline MFCC features.

Abstract

The control system design specifications for dynamical systems are typically provided in terms of the desired transient response and steady-state response. Meeting such requirements for dynamical systems whose mathematical models are unavailable is a challenging task. In this paper, we propose a learning-based controller to achieve the desired control system design specifications for unknown dynamical systems. We consider a SoTA model-free reinforcement learning agent (TD3) in the continuous state and action space setting where the agent has no knowledge of system dynamics. The selection of an appropriate reward function is a key factor that shapes the response of the dynamical system when the learning-based controller is implemented. The resulting controller is trained and tested for first-order and second-order linear systems, as well as a nonlinear system. The resulting trajectories of the closed-loop systems indicate that the transient and steady-state response can be altered by choosing the appropriate reward function while adhering to the constraints imposed on the control input.

Abstract

Emergency signages are crucial for safe evacuations but have been challenged by inconsistent and ambiguous designs. We conducted two experiments to evaluate a specific emergency signage design, i.e., running man's pictogram, in ease of conveying the direction to exit an unknown building. We compared the running man's pictogram with traditional arrow signage in exp.1, and in exp. 2, we examined the key features that contribute to the running man's composition. In both conditions, the participant's task was to determine the running man's/ arrow's orientation and press the instructed corresponding keys. We observed a faster and more accurate response for the right arrow than the right running man in experiment 1. Further, in exp. 2, the head/face component showed an advantage in identifying the running man's orientation over the leg/hand. Results suggest running man alone in emergency signage may be risky, especially when it is incomplete without a head and face.

Abstract

In this paper, we introduce a novel approach to address the task of synthesizing speech from silent videos of any in-the-wild speaker solely based on lip movements. The traditional approach of directly generating speech from lip videos faces the challenge of not being able to learn a robust language model from speech alone, resulting in unsatisfactory outcomes. To overcome this issue, we propose incorporating noisy text supervision using a state-of-the-art lip-to-text network that instills language information into our model. The noisy text is generated using a pre-trained lip-to-text model, enabling our approach to work without text annotations during inference. We design a visual text-to-speech network that utilizes the visual stream to generate accurate speech, which is in-sync with the silent input video. We perform extensive experiments and ablation studies, demonstrating our approach's superiority over the current state-of-the-art methods on various benchmark datasets. Further, we demonstrate an essential practical application of our method in assistive technology by generating speech for an ALS patient who has lost the voice but can make mouth movements. Our demo video, code, and additional details can be found at http://cvit.iiit.ac.in/research/projects/cvit-projects/ms-l2s-itw.

Abstract

Visual servoing has been gaining popularity in various real-world vision-centric robotic applications. Autonomous robotic grasping often deals with unseen and unstructured environments, and in this task, Visual Servoing has been able to generate improved end-effector control by providing visual feedback. However, existing Servoing-aided grasping methods tend to fail at the task of grasping in dynamic environments i.e. - moving objects. In this paper, we introduce DynGraspVS, a novel Image-based Visual Servoing-aided Grasping approach that models the motion of moving objects in its interaction matrix. Leveraging a single-step rollout strategy, our approach achieves a remarkable increase in success rate, while converging faster and achieving a smoother trajectory, while maintaining precise alignments in six degrees of freedom. By integrating the velocity information into the interaction matrix, our method is able to successfully complete the challenging task of robotic grasping in the case of dynamic objects, while outperforming existing deep Model Predictive Control (MPC) based methods in the PyBullet simulation environment. We test it with a range of objects in the YCB dataset with varying range of shapes, sizes, and material properties. We report various evaluation metrics such as photometric error, success rate, time taken, and trajectory length.

Abstract

Deployment of Face Recognition systems on the edge has seen significant growth due to advancements in hardware design and efficient neural architectures. However, tailoring SOTA Face Recognition solutions to a specific edge device is still not easy and is vastly unexplored. Although, benchmark data is available for some combinations of model, device, and framework, it is neither comprehensive nor scalable. We propose an approximation to determine the relationship between a model and its inference time in an edge deployment scenario. Using a small number of data points, we are able to predict the throughput of custom models in an explainable manner. The prediction errors are small enough to be considered noise in observations. We also analyze which approaches are most efficient and make better use of hardware in terms of accuracy and error rates to gain a better understanding of their behaviour. Related & necessary modules such as Face Anti-Spoofing are also analyzed. To the best of our knowledge, we are the first to tackle this issue directly. The data and code along with future updates to the models and hardware will be made available at https://github.com/AyanBiswas19/Resource_Efficient_FR.

Abstract

Visual Place Recognition (VPR) is vital for robot localization. To date, the most performant VPR approaches are environment- and task-specific : while they exhibit strong performance in structured environments (predominantly urban driving), their performance degrades severely in unstructured environments, rendering most approaches brittle to robust real-world deployment. In this work, we develop a universal solution to VPR – a technique that works across a broad range of structured and unstructured environments (urban, outdoors, indoors, aerial, underwater, and subterranean environments) without any re-training or finetuning. We demonstrate that general-purpose feature representations derived from off-the-shelf self-supervised models with no VPR-specific training are the right substrate upon which to build such a universal VPR solution. Combining these derived features with unsupervised feature aggregation enables our suite of methods, AnyLoc , to achieve up to 4× significantly higher performance than existing approaches. We further obtain a 6% improvement in performance by characterizing the semantic properties of these features, uncovering unique domains which encapsulate datasets from similar environments. Our detailed experiments and analysis lay a foundation for building VPR solutions that may be deployed anywhere , anytime , and across anyview . We encourage the readers to explore our project page and interactive demos: https://anyloc.github.io/ .

Abstract

Biomimetic technologies are an important part of the soft robotics research area because of their ability to replicate naturally occurring structures. Researchers have been developing and improving many such technologies ranging from prosthetics to artificial skin. In this work, we present the fabrication and characterization of an artificial muscle flapping wing using dielectric elastomer actuators (DEAs). The actuator was fabricated using a urethane acrylate polymer elastomer with carbon nanotube (CNT) based electrodes. The 40 mm × 20 mm rectangular actuator was powered through two electrodes at the top. It was subjected to different voltages in the range of 500 - 2500 V and the displacement of the bottom edge was observed. We report the actuator edge displacement to be 20 mm, i.e., 0.5 body lengths (BL) at static 2.5 kV. The dynamic behavior of the device was also analyzed by actuating it with a square wave of various frequencies (0 to 10 Hz). We report the resonant frequency of the actuator to be 4 Hz, with a sub-harmonic at 2 Hz

Abstract

Abstract The proliferation of speech technology applications has led to an unprecedented demand for effective speech enhancement techniques, particularly in low Signal-to-Noise Ratio (SNR) conditions. This research presents a novel approach to speech enhancement, specifically designed for very low SNR scenarios. Our technique focuses on speaker embedding at a granular level and highlights its consistent impact on enhancing speech quality and improving Automatic Speech Recognition (ASR) performance, a significant downstream task. Experimental findings demonstrate competitive speech quality and substantial enhancements in ASR accuracy compared to alternative methods in low SNR situations. The proposed technique offers promising advancements in addressing the challenges posed by low SNR conditions in speech technology applications.