Abstract

Second language learners often make grammatical mistakes, which can impact the performance of Spoken Language Understanding (SLU) systems. SLU systems consist of two key components: Automatic Speech Recognition (ASR) and Natural Language Understanding (NLU). This work focuses on the effects of grammatical errors which were synthetically generated by manipulating Parts of Speech (POS) tokens on the NLU module. Our analysis comprises three main aspects. Firstly, we assess the impact of grammatical errors on the overall performance of NLU systems, specifically in the domains of Intent Detection and Slot Filling. Secondly, we investigate NLU performance concerning POS tags. Lastly, we utilize an Attention-based NLU model to evaluate the significance of different POS. This study evaluates NLU models on ATIS and SNIPS datasets for Intent Detection and Slot Filling tasks with introduced grammatical errors, leading to a significant performance drop as large as 4.7% for Intent Detection and 5.1% for Slot Filling. However, training on datasets with synthetic errors helps mitigate this drop, and the models are shown to be sensitive to increasing levels of corrupted tokens. Additionally, attention analysis reveals Proper Nouns’ higher weights in determining intent, while specific POS corruption, particularly Auxiliary Verbs and Nouns, is more likely to cause intent misclassification.

Abstract

The Industrial Internet of Things (IIoT) is able to connect machines, analytics and people with IoT smart devices, gateway nodes and edge devices to create powerful intuitivenesses to drive smarter, faster and effective business agreements. IIoT having interconnected machines along with devices can monitor, gather, exchange, and analyze information. Since the communication among the entities in IIoT environment takes place insecurely (for instance, wireless communications and Internet), an intruder can easily tamper with the data. Moreover, physical theft of IoT smart devices provides an intruder to mount impersonation and other attacks. To handle such critical issues, in this work, we design a new private blockchain-envisioned access control scheme for Pervasive Edge Computing (PEC) in IIoT environment, called PBACS-PECIIoT. We consider the private blockchain consisting of the transactions and registration credentials of the entities related to IIoT, because the information is strictly confidential and private. The security of PBACS-PECIIoT is significantly improved due to usage of blockchain as immutability, transparency and decentralization along with protection of various potential attacks. A meticulous comparative analysis exhibits that PBACS-PECIIoT achieves greater security and more functionality features, and requires low costs for communication and computational as compared to other pertinent schemes.

Abstract

This paper presents a trim-free, low-power, and high-precision voltage/current reference for IoT and biomedical applications. The voltage reference is generated by the proper addition of CTAT and PTAT voltages, whereas, the current reference is generated using a MOSFET in the active region, which acts like a resistor and helps in lowering power consumption without increasing the form factor. Implemented in the 90nm CMOS process, the proposed voltage/current reference outputs a voltage and current of 820mV and 3.23nA, respectively. Without trimming, the process variation of the proposed voltage/current reference is 1.34%(σ/μ) / 1.75%(σ/μ) and the temperature coefficient of the voltage/current reference is 62ppm/oC / 332ppm/oC over the temperature range of -40oC to 100oC. The line sensitivity of the voltage/current reference is 0.296%/V / 0.414%/V for a wide supply range of 1V - 3.5V. The area occupied by the total circuit is 0.0112mm2, while the total power consumption of the design is 37nW at the typical corner of 27oC and 1V supply.

Abstract

Abstract The proliferation of speech technology applications has led to an unprecedented demand for effective speech enhancement techniques, particularly in low Signal-to-Noise Ratio (SNR) conditions. This research presents a novel approach to speech enhancement, specifically designed for very low SNR scenarios. Our technique focuses on speaker embedding at a granular level and highlights its consistent impact on enhancing speech quality and improving Automatic Speech Recognition (ASR) performance, a significant downstream task. Experimental findings demonstrate competitive speech quality and substantial enhancements in ASR accuracy compared to alternative methods in low SNR situations. The proposed technique offers promising advancements in addressing the challenges posed by low SNR conditions in speech technology applications.

Abstract

Visual Place Recognition (VPR) is vital for robot localization. To date, the most performant VPR approaches are environment- and task-specific : while they exhibit strong performance in structured environments (predominantly urban driving), their performance degrades severely in unstructured environments, rendering most approaches brittle to robust real-world deployment. In this work, we develop a universal solution to VPR – a technique that works across a broad range of structured and unstructured environments (urban, outdoors, indoors, aerial, underwater, and subterranean environments) without any re-training or finetuning. We demonstrate that general-purpose feature representations derived from off-the-shelf self-supervised models with no VPR-specific training are the right substrate upon which to build such a universal VPR solution. Combining these derived features with unsupervised feature aggregation enables our suite of methods, AnyLoc , to achieve up to 4× significantly higher performance than existing approaches. We further obtain a 6% improvement in performance by characterizing the semantic properties of these features, uncovering unique domains which encapsulate datasets from similar environments. Our detailed experiments and analysis lay a foundation for building VPR solutions that may be deployed anywhere , anytime , and across anyview . We encourage the readers to explore our project page and interactive demos: https://anyloc.github.io/ .

Abstract

Deployment of Face Recognition systems on the edge has seen significant growth due to advancements in hardware design and efficient neural architectures. However, tailoring SOTA Face Recognition solutions to a specific edge device is still not easy and is vastly unexplored. Although, benchmark data is available for some combinations of model, device, and framework, it is neither comprehensive nor scalable. We propose an approximation to determine the relationship between a model and its inference time in an edge deployment scenario. Using a small number of data points, we are able to predict the throughput of custom models in an explainable manner. The prediction errors are small enough to be considered noise in observations. We also analyze which approaches are most efficient and make better use of hardware in terms of accuracy and error rates to gain a better understanding of their behaviour. Related & necessary modules such as Face Anti-Spoofing are also analyzed. To the best of our knowledge, we are the first to tackle this issue directly. The data and code along with future updates to the models and hardware will be made available at https://github.com/AyanBiswas19/Resource_Efficient_FR.

Abstract

In this paper, we introduce a novel approach to address the task of synthesizing speech from silent videos of any in-the-wild speaker solely based on lip movements. The traditional approach of directly generating speech from lip videos faces the challenge of not being able to learn a robust language model from speech alone, resulting in unsatisfactory outcomes. To overcome this issue, we propose incorporating noisy text supervision using a state-of-the-art lip-to-text network that instills language information into our model. The noisy text is generated using a pre-trained lip-to-text model, enabling our approach to work without text annotations during inference. We design a visual text-to-speech network that utilizes the visual stream to generate accurate speech, which is in-sync with the silent input video. We perform extensive experiments and ablation studies, demonstrating our approach's superiority over the current state-of-the-art methods on various benchmark datasets. Further, we demonstrate an essential practical application of our method in assistive technology by generating speech for an ALS patient who has lost the voice but can make mouth movements. Our demo video, code, and additional details can be found at http://cvit.iiit.ac.in/research/projects/cvit-projects/ms-l2s-itw.

Abstract

Stuttering is a speech disorder that affects speech fluency and rhythm, with millions worldwide experiencing it. Early diagnosis and treatment can significantly improve speech fluency and the quality of life for individuals who stutter. Automatic detection of stuttering events can help diagnose, monitor, and develop effective interventions. Therefore, this paper aims to propose a feature space-based classifier for detecting stuttering events in speech. To achieve this, we have investigated Zero Time Windowing Cepstral Coefficients (ZTWCC) as a feature set for stutter detection using classifiers such as SVM, LSTM, and Bidirectional LSTM. We compared the performance of ZTWCC with the standard handcrafted features, such as MFCC, CQCC, and SFFCC, on the SEP- 28K dataset with and without including phase information. The results in both cases indicate that ZTWCC is giving a higher F-1 score than baseline MFCC features.

Abstract

Despite the progress seen in classification methods, current approaches for handling videos with distribution shifts in source and target domains remain source-dependent as they require access to the source data during the adaptation stage. In this paper, we present a self-training based source-free video domain adaptation approach to address this challenge by bridging the gap between the source and the target domains. We use the source pre-trained model to generate pseudo-labels for the target domain samples, which are inevitably noisy. Thus, we treat the problem of source-free video domain adaptation as learning from noisy labels and argue that the samples with correct pseudo-labels can help us in adaptation. To this end, we leverage the crossentropy loss as an indicator of the correctness of the pseudo-labels and use the resulting small-loss samples from the target domain for fine-tuning the model. We further enhance the adaptation performance by implementing a teacher-student framework, in which the teacher, which is updated gradually, produces reliable pseudo-labels. Meanwhile, the student undergoes fine-tuning on the target domain videos using these generated pseudo-labels to improve its performance. Extensive experimental evaluations show that our methods, termed as CleanAdapt, CleanAdapt + TS, achieve state-of-the-art results, outperforming the existing approaches on various open datasets. Our source code is publicly available at https://avijit9.github.io/CleanAdapt.

Abstract

Language learning involves the correct acquisition of grammar skills. To facilitate learning with computer-assisted systems, automatic spoken grammatical error detection (SGED) is necessary. This work explores Automatic Speech Recognition (ASR), which decodes text from speech, for SGED. With current advancements in ASR technology, often it can be believed that these systems could capture spoken grammatical errors in the decoded text. However, these systems have an inherent bias from the language model towards the grammatically correct text. We explore the ASR-decoded text from commercially available current state-of-the-art systems considering a text-based GED algorithm and also its word-level confidence score (CS) for SGED. We perform the experiments on the spoken English data collected in-house from 13 subjects speaking 4110 grammatically erroneous and correct sentences. We found the highest relative improvement in SGED with CS is 15.36% compared to that with decoded text plus GED.