Abstract

In this work, we investigate the problem of lip-syncing a talking face video of an arbitrary identity to match a target speech segment. Current works excel at producing accurate lip movements on a static image or videos of specific people seen during the training phase. However, they fail to accurately morph the lip movements of arbitrary identities in dynamic, unconstrained talking face videos, resulting in significant parts of the video being out-of-sync with the new audio. We identify key reasons pertaining to this and hence resolve them by learning from a powerful lip-sync discriminator. Next, we propose new, rigorous evaluation benchmarks and metrics to accurately measure lip synchronization in unconstrained videos. Extensive quantitative evaluations on our challenging benchmarks show that the lip-sync accuracy of the videos generated by our Wav2Lip model is almost as good as real synced videos. We provide …

Abstract

Recent approaches for weakly supervised instance segmentations depend on two components:(i) a pseudo label generation model which provides instances that are consistent with a given annotation; and (ii) an instance segmentation model, which is trained in a supervised manner using the pseudo labels as ground-truth. Unlike previous approaches, we explicitly model the uncertainty in the pseudo label generation process using a conditional distribution. The samples drawn from our conditional distribution provide accurate pseudo labels due to the use of semantic class aware unary terms, boundary aware pairwise smoothness terms, and annotation aware higher order terms. Furthermore, we represent the instance segmentation model as an annotation agnostic prediction distribution. In contrast to previous methods, our representation allows us to define a joint probabilistic learning objective that minimizes the …

Abstract

Indian language machine translation performance is hampered due to the lack of large scale multi-lingual sentence aligned corpora and robust benchmarks. Through this paper, we provide and analyse an automated framework to obtain such a corpus for Indian language neural machine translation (NMT) systems. Our pipeline consists of a baseline NMT system, a retrieval module, and an alignment module that is used to work with publicly available websites such as press releases by the government. The main contribution towards this effort is to obtain an incremental method that uses the above pipeline to iteratively improve the size of the corpus as well as improve each of the components of our system. Through our work, we also evaluate the design choices such as the choice of pivoting language and the effect of iterative incremental increase in corpus size. Our work in addition to providing an automated framework also results in generating a relatively larger corpus as compared to existing corpora that are available for Indian languages. This corpus helps us obtain substantially improved results on the publicly available WAT evaluation benchmark and other standard evaluation benchmarks.

Abstract

We introduce a new dataset for graphical object detection in business documents, more specifically annual reports. This dataset, iiit-ar-13k, is created by manually annotating the bounding boxes of graphical or page objects in publicly available annual reports. This dataset contains a total of 13k annotated page images with objects in five different popular categories—table, figure, natural image, logo, and signature. It is the largest manually annotated dataset for graphical object detection. Annual reports created in multiple languages for several years from various companies bring high diversity into this dataset. We benchmark iiit-ar-13k dataset with two state of the art graphical object detection techniques using faster r-cnn [20] and mask r-cnn [11] and establish high baselines for further research. Our dataset is highly effective as training data for developing practical solutions for graphical object detection in both …

Abstract

This paper proposes a novel Gap Reduced Minimum Error Robust Simultaneous (GRMERS) estimator for resource-constrained Nano Aerial Vehicle (NAV) that enables a single estimator to provide simultaneous and robust estimation for a given N unstable and uncertain NAV plant models. The estimated full state feedback enables a stable flight for NAV. The GRMERS estimator is implemented utilizing a Minimum Error Robust Simultaneous (MERS) estimator and Gap Reducing (GR) compensators. The MERS estimator provides robust simultaneous estimation with minimal largest worst-case estimation error even in the presence of a bounded energy exogenous disturbance signal. The GR compensators reduce the gap between the graphs of N linear plant models to decrease the estimation error generated by the MERS estimator. A sufficient condition for the existence of a simultaneous estimator is established using LMIs and robust estimation theory. Further, MERS estimator and GR compensator design are formulated as non-convex tractable optimization problems and are solved using the population-based genetic algorithms. The performance of the GRMERS estimator consisting of MERS estimator and GR compensators from the population-based genetic algorithms is validated through simulation studies. The study results indicate that a single GRMERS estimator can produce state estimates with reduced errors for all flight conditions. The results indicate that the single GRMERS estimator is robust than the individually designed H inifinity filters.

Abstract

Control of fixed-wing Unmanned Aerial Vehicles (UAVs) is typically organized according to two layers: the low-level control or autopilot, and the high-level control or guidance. The disadvantage of this modular design is that an intelligent guidance layer may become ineffective if the autopilot layer cannot deal with uncertainty. In fact, the required knowledge derived from linearization of equations of motion (trimming points) makes most autopilots sensitive to uncertainty. In this work, we study an autopilot framework where the knowledge of the UAV dynamics and of trimming points is not required. The proposed design, tested with complex UAV dynamics, can emulate the behavior of a carefully tuned off-the-shelf autopilot, without using its a priori knowledge.

Abstract

Multi-area load frequency control (LFC) selects and controls a few generators in each area of the power system in an effort to dampen inter-area frequency oscillations. To effectively dampen such oscillations, it is required to enhance and lower the control activity dynamically during operation, so as to adapt to changing circumstances. Changing circumstances should cover not only parametric uncertainties and unmodelled dynamics (e.g. aggregated area dynamics and bus dynamics), but also the increasing structural flexibility of modern power systems (e.g. protection mechanisms against faults and cyber-attacks, or topology reconfiguration mechanisms for demand response). As formal stability guarantees around such an attractive adaptive multi-area LFC concept are still lacking, this work proposes framework in which adaptation and switching are combined in a provably stable way to handle parametric uncertainty, unmodelled dynamics, and dynamical interconnections of the power system. Stability is studied in the Lyapunov theory sense using the standard structure-preserving modelling approach, and the resulting adaptive multi-area LFC design is validated using an IEEE 39-bus benchmark.

Abstract

With the advent of intelligent transport, quadrotors are becoming an attractive solu-tion while lifting or dropping of payloads during emergency evacuations, construc-tion works etc. During such operations, dynamic variations in (possibly unknown)payload cause considerable changes in the system dynamics. However, a systematic control solution to tackle such interchanging dynamical behaviour is still missing.This paper proposes a switched dynamical framework to capture the interchanging dynamics of a quadrotor during vertical operations and a robust adaptive control solution to tackle such dynamics when it is unknown. The stability of the closed-loop system is studied analytically and the effectiveness of the proposed solution is verified via simulations

Abstract

In the presence of unmodelled dynamics and uncertainties with no a priori constant bounds, conventional robust adaptation strategies for switched systems cannot allow the control gains of inactive subsystems to remain constant during inactive intervals: vanishing gains are typically required in order to prove bounded stability. As a consequence, these strategies, known in literature as leakage-based adaptive methods, might introduce poor transients at each switching instant. Leakage-based adaptive control becomes even more problematic in the switched nonlinear case, where non-conservative state-dependent upper bounds for uncertainties and unmodelled dynamics are required. This work shows that, for a class of switched Euler–Lagrange systems, such difficulties can be mitigated: a novel leakage-based stable mechanism is introduced which allows the gains of inactive subsystems to remain constant. At the same time, unmodelled dynamics and uncertainties with no a priori bounds can be handled by a quadratic state-dependent upper bound structure that reduces conservativeness as compared to state-of-the-art structures. The proposed design is validated analytically and its performance is studied in simulation with a pick-and-place robotic manipulator example.

Abstract

Automated vision-based score estimation models can be used to provide an alternate opinion to avoid judgment bias. Existing works have learned score estimation models by regressing the video representation to ground truth score provided by judges. However, such regression-based solutions lack interpretability in terms of giving reasons for the awarded score. One solution to make the scores more explicable is to compare the given action video with a reference video, which would capture the temporal variations vis-á-vis the reference video and map those variations to the final score. In this work, we propose a new action scoring system termed as Reference Guided Regression (RGR), which comprises (1) a Deep Metric Learning Module that learns similarity between any two action videos based on their ground truth scores given by the judges, and (2) a Score Estimation Module that uses the first module to find