Abstract

We introduce an SSMT (Speech to Speech Machine Translation, aka Speech to Speech Video Translation) Pipeline1 , as a web application for translating videos from one language to another by cascading multiple language modules. Our speech translation system combines highly accurate speech to text (ASR) for Indian English, pre-possessing modules to bridge ASRMT gaps such as spoken disfluency and punctuation, robust machine translation (MT) systems for multiple language pairs, SRT module for translated text, text to speech (TTS) module and a module to render translated synthesized audio on the original video. It is user-friendly, flexible, and easily accessible system. We aim to provide a complete configurable speech translation experience to users and researchers with this system. It also supports human intervention where users can edit outputs of different modules and the edited output can then be used for subsequent processing to improve overall output quality. By adopting a human-in-theloop approach, the aim is to configure technology in such a way where it can assist humans and help to reduce the involved human efforts in speech translation involving English and Indian languages. As per our understanding, this is the first fully integrated system for English to Indian languages (Hindi, Telugu, Gujarati, Marathi, and Punjabi) video translation. Our evaluation shows that one can get 3.5+ MOS score using the developed pipeline with human intervention for English to Hindi. A short video demonstrating our system is available at https://youtu.be/MVftzoeRg48.

Abstract

Modelling and understanding time remains a challenge in contemporary video understanding models. Time also appears in language through temporal relations. Video-language models can benefit from having a sense of time, especially since language provides an interface for generalization. In this paper, we consider a specific aspect of temporal understanding: consistency of time order as elicited by before/after relations. We construct a simple synthetic dataset to measure such temporal understanding in video-language models and find that six existing models struggle to understand even such simple relations. We then posit whether it is feasible to equip these foundation models with temporal awareness without re-training them from scratch. Towards this, we propose a temporal adaptation recipe on top of one such model, VideoCLIP, based on postpretraining on a small amount of video-text data. Our work serves as a first step towards probing and instilling a sense of time in existing video-language models without needing data- and compute-intense training from scratch. Project page:

Abstract

Unconstrained Asian roads often involve poor infrastructure, affecting overall road safety. Missing traffic signs are a regular part of such roads. Missing or non-existing object detection has been studied for locating missing curbs and estimating reasonable regions for pedestrians on road scene images. Such methods involve analyzing task-specific single object cues. In this paper, we present the first and most challenging video dataset for missing objects, with multiple types of traffic signs for which the cues are visible without the signs in the scenes. We refer to it as the Missing Traffic Signs Video Dataset (MTSVD). MTSVD is challenging compared to the previous works in two aspects i) The traffic signs are generally not present in the vicinity of their cues, ii) The traffic signs cues are diverse and unique. Also, MTSVD is the first publicly available missing object dataset. To train the models for identifying missing signs, we complement our dataset with 10K traffic sign tracks, with 40 percent of the traffic signs having cues visible in the scenes. For identifying missing signs, we propose the Cue-driven Contextual Attention units (CueCAn), which we incorporate in our model encoder. We first train the encoder to classify the presence of traffic sign cues and then train the entire segmentation model end-to-end to localize missing traffic signs. Quantitative and qualitative analysis shows that CueCAn significantly improves the performance of base models.

Abstract

Efficient certification and quantification of high dimensional entanglement of composite systems are challenging both theoretically as well as experimentally. Here, we demonstrate how to measure the linear entropy, negativity and the Schmidt number of bipartite systems from the visibility of Mach–Zehnder interferometer using single copies of the quantum state. Our result shows that for any two qubit pure bipartite state, the interference visibility is a direct measure of entanglement. We also propose how to measure the mutual predictability experimentally from the intensity patterns of the interferometric set-up without having to resort to local measurements of mutually unbiased bases. Furthermore, we show that the entanglement witness operator can be measured in a interference setup and the phase shift is sensitive to the separable or entangled nature of the state. Our proposal bring out the power of Interferometric set-up in entanglement detection of pure and several mixed states which paves the way towards design of entanglement meter.

Abstract

Electric vehicles (EVs) represent the long-term green substitute for traditional fuel-based vehicles. To encourage EV adoption, the trust of the end-users must be assured. In this work, we focus on a recently emerging privacy threat of profiling and identifying EVs via the analog electrical data exchanged during the EV charging process. The core focus of our work is to investigate the feasibility of such a threat at scale. To this end, we first propose an improved EV profiling approach that outperforms the state-of-the-art EV profiling techniques. Next, we exhaustively evaluate the performance of our improved approach to profile EVs in real-world settings. In our evaluations, we conduct a series of experiments including 25032 charging sessions from 530 real EVs, sub-sampled datasets with different data distributions, etc. Our results show that even with our improved approach, profiling and individually identifying the growing number of EVs appear extremely difficult in practice; at least with the analog charging data utilized throughout the literature. We believe that our findings from this work will further foster the trust of potential users in the EV ecosystem, and consequently, encourage EV adoption.

Abstract

The management and regulation of the waste generated from coal-based power plants have been a key issue throughout the world. There is a need for the voluminous utilization of coal ash in India. The bulk utilization of the industrial waste materials like coal ash can control the exploitation of the natural resources like soil in diferent aspects. Here, it has been attempted to evaluate the potential use of coal ash as a foundation/fll material. In addition, physical, geotechnical, morphological, and chemical characterization of the pond ash has been done. Then, fnite element modeling of the vibratory roller compaction was simulated using PLAXIS3D software for the recommendation of the pond ash as a construction material in place of soil. Also, the reduction in CO2 emission with pond ash was compared with that of the soil. The reduction in CO2 emission was found to be about 15% considering several factors like distance from the source station to the test site, types of coal ash used, types of construction equipment employed, etc. The utilization of pond ash as fll material was found to be economical, and its utilization saves around INR 32 to INR 399 per cubic meter. Keywords Coal ash · Foundation/fll material · Finite element modeling · Vibratory roller compaction · CO2 emission

Abstract

The authenticated key agreement is one of the major security services that can be used to secure an Internet of Things (IoT) environment, where the devices collect the data and the data is then aggregated at the cloud server, and then a user needs to access the data stored at the server(s) securely. For this purpose, after a mutual authentication performed between a user and the accessed server, a session key needs to be established among them for secure communication. In this article, we design an efficient lattice-based authenticated key exchange protocol using ring-based version of learning with errors assumption for the IoT-enabled smart devices. The proposed protocol is basically a key exchange that uses the reconciliation mechanism. The detailed security analysis under the standard model has been performed along with the informal security analysis to show that the proposed protocol is robust against different attacks. We then simulate the proposed protocol under the NS-3 simulator to measure the network performance parameters like network throughput and latency. A comparative analysis shows that the proposed protocol has superior security, less computational cost, and comparable communication cost when compered these parameters with the other competing schemes.

Abstract

Cognitive Internet of Things (CIoT) supports the organizations to learn from the information (data) arriving from various connected devices, sensors, machines and other sources, and at the same time it inspires intelligence into different business operations, products, customer experiences, and people. Data poising attacks are very serious concerns because they may play a significant factor for businesses and organizations for both financial terms and damaging their reputations, when the Big data analytics on the analyzed data is itself corrupted. To mitigate this issue, in this paper, we suggest a blockchain-based Artificial Intelligence(AI)/Machine Learning(ML)-enabled Big data analytics mechanism for CIoT environment. The comprehensive experimental results have been provided under two circumstances: (1) performance of the ML model under data poisoning attacks and (2) performance of the ML model without data poisoning attacks. In the first case, we show how the data poison attacks can effect the ML model when the data is on some cloud storage (i.e. not in the blockchains), whereas in the second case we show the effect when the data is in the blockchains (i.e., without data poisoning attacks). The experimental results demonstrate that we have significant gains in performance in terms of accuracy, recall, precision and F1 score when there are no data poisoning attacks on the data. Moreover, a detailed blockchain simulation has carried out to demonstrate the practical aspects of the proposed security framework.

Abstract

—With Cardiovascular Diseases on the rise around the world, Electrocardiograms (ECGs) play a crucial role in their diagnosis owing to their non-invasive nature and simplicity. Medical professionals typically use 12-lead ECGs for medical analysis but gathering 12-lead ECG data is an arduous task outside clinical setting. Modern wearables can collect an ECG with fewer leads than the standard 12 leads. However, medical professionals and conventional ECG analysis software find this reduced lead set data challenging to interpret. By using the reduced lead set data to create standard 12-lead ECG data, ECG reconstruction can solve this issue. This paper proposes a novel single-lead to multi-lead ECG reconstruction solution using a modified Attention U-net framework. Using only the lead II of ECG, our model is able to reproduce the other 11 leads of conventional 12-lead ECG with a Pearson correlation, Mean square error and R-squared error of 0.805, 0.0122 and 0.639, respectively. Further, a single combined model is used to reconstruct all 11 leads simultaneously, improving performance and simultaneously reducing the computational resources needed for training compared to current literature in the field. In comparison to previous works, which only reconstruct small ECG segments, our model is trained to reconstruct longer 10-second ECG signals. We demonstrate our model’s ability for real-life utilisation using a cardiovascular disease classification task. A deep learning model was trained for multi-disease classification on actual 12-lead ECG data and was tested on both original and reconstructed 12-lead ECG signals. The classification accuracies for the original and reconstructed signals were comparable, portraying that our reconstruction model can preserve diagnostically relevant artefacts in its reconstructed signals. This work provides a new promising solution in the field of single-lead ECG reconstruction, taking us a step closer to bridging the divide between reduced lead set data and existing 12-lead ECG end users like clinicians and automatic ECG classifiers. Index Terms—Healthcare, Cardiology, Electrocardiograms, Machine Learning, Neural Networks

Abstract

The particle flow Gaussian particle filter (PFGPF) uses an invertible particle flow to generate a proposal density. It approximates the predictive and posterior distributions as Gaussian densities. In this paper, we use a bank of PFGPF filters to construct a Particle flow Gaussian sum particle filter (PFGSPF), which approximates the prediction and posterior as Gaussian mixture model. This approximation is useful in complex estimation problems where a single Gaussian approximation is inadequate. We compare the performance of this proposed filter with the PFGPF and others in challenging numerical simulations. Index Terms— Gaussian mixture, invertible particle flow, Gaussian sum particle filer, Gaussian particle filter, PFGPF.