Abstract

Current visual question answering datasets do not consider the rich semantic information conveyed by text within an image. In this work, we present a new dataset, ST-VQA, that aims to highlight the importance of exploiting high-level semantic information present in images as textual cues in the VQA process. We use this dataset to define a series of tasks of increasing difficulty for which reading the scene text in the context provided by the visual information is necessary to reason and generate an appropriate answer. We propose a new evaluation metric for these tasks to account both for reasoning errors as well as shortcomings of the text recognition module. In addition we put forward a series of baseline methods, which provide further insight to the newly released dataset, and set the scene for further research

Abstract

Understanding videos of people speaking across internationalborders is hard as audiences from different demographies donot understand the language. Such speech videos are oftensupplemented with language subtitles. However, these ham-per the viewing experience as the attention is shared. Simpleaudio dubbing in a different language makes the video appearunnatural due to unsynchronized lip motion. In this paper, wepropose a system for automated cross-language lip synchro-nization for re-dubbed videos. Our model generates superiorphotorealistic lip-synchronization over original video in com-parison to the current re-dubbing method. With the help of auser-based study, we verify that our method is preferred overunsynchronized videos

Abstract

Abstract— Driver inattention is one of the leading causesof vehicle crashes and incidents worldwide. Driver inattentionincludes driver fatigue leading to drowsiness and driver distrac-tion, say due to use of cellphone or rubbernecking, all of whichleads to a lack of situational awareness. Hitherto, techniquespresented to monitor driver attention evaluated factors suchas fatigue and distraction independently. However, in orderto develop a robust driver attention monitoring system allthe factors affecting driver’s attention needs to be analyzedholistically. In this paper, we presentAutoRate, a system thatleverages front camera of a windshield-mounted smartphoneto monitor driver’s attention by combining several features.We derive a driver attention rating by fusing spatio-temporalfeatures based on the driver state and behavior such as headpose, eye gaze, eye closure, yawns, use of cellphones, etc.We perform extensive evaluation ofAutoRateon real-world driving data and also data from controlled, static vehiclesettings with 30 drivers in a large city. We compareAutoRate’sautomatically-generated rating with the scores given by 5human annotators. Further, we compute the agreement betweenAutoRate’s rating and human annotator rating using kappacoefficient.AutoRate’s automatically-generated rating has anoverall agreement of 0.87 with the ratings provided by 5 humanannotators on the static dataset.

Abstract

AbstractFully supervised deep learning-based methods have created a profound im-pact in various fields of computer science. Compared to classical methods, superviseddeep learning-based techniques face scalability issues as they require huge amountsof labeled data and, more significantly, are unable to generalize to multiple domainsand tasks. In recent years, a lot of research has been targeted towards addressingthese issues within the deep learning community. Although there have been extensivesurveys on learning paradigms such as semi-supervised and unsupervised learning,there are few timely reviews after the emergence of deep learning. In this paper, weprovide an overview of the contemporary literature surrounding alternatives to fullysupervised learning in the deep learning context. First, we summarize the relevanttechniques that fall between the paradigm of supervised and unsupervised learning.Second, we take autonomous navigation as a running example to explain and com-pare different models. Finally, we highlight some shortcomings of current methodsand suggest future directions.

Abstract

Abstract—This paper presents final results of ICDAR 2019Scene Text Visual Question Answering competition (ST-VQA).ST-VQA introduces an important aspect that is not addressedby any Visual Question Answering system up to date, namelythe incorporation of scene text to answer questions asked aboutan image. The competition introduces a new dataset comprising23,038images annotated with31,791question / answer pairswhere the answer is always grounded on text instances presentin the image. The images are taken from7different publiccomputer vision datasets, covering a wide range of scenarios.The competition was structured in three tasks of increas-ing difficulty, that require reading the text in a scene andunderstanding it in the context of the scene, to correctlyanswer a given question. A novel evaluation metric is presented,which elegantly assesses both key capabilities expected from anoptimal model: text recognition and image understanding.A detailed analysis of results from different participants isshowcased, which provides insight into the current capabilitiesof VQA systems that can read. We firmly believe the datasetproposed in this challenge will be an important milestone toconsider towards a path of more robust and general models thatcan exploit scene text to achieve holistic image understanding.

Abstract

Cross-modal retrieval methods have been significantly improvedin last years with the use of deep neural networks and large-scaleannotated datasets such as ImageNet and Places. However, collect-ing and annotating such datasets requires a tremendous amountof human effort and, besides, their annotations are limited to dis-crete sets of popular visual classes that may not be representativeof the richer semantics found on large-scale cross-modal retrievaldatasets. In this paper, we present a self-supervised cross-modalretrieval framework that leverages as training data the correlationsbetween images and text on the entire set of Wikipedia articles.Our method consists in training a CNN to predict: (1) the semanticcontext of the article in which an image is more probable to appearas an illustration, and (2) the semantic context of its caption. Ourexperiments demonstrate that the proposed method is not only ca-pable of learning discriminative visual representations for solvingvision tasks like classification, but that the learned representationsare better for cross-modal retrieval when compared to supervisedpre-training of the network on the ImageNet dataset.

Abstract

We present a simple, yet effective, Neural Ma-chineTranslationsystemforIndianlanguages.Wedemonstratethefeasibilityformultiplelan-guage pairs, and establish a strong baseline forfurther research.

Abstract

Given a target image as query, person re-identification systems retrieve a ranked list of candidate matches on a per-camera basis. In deployed systems, a human operator scans these lists and labels sighted targets by touch or mouse-based selection. However, classical re-id approaches generate per-camera lists independently. Therefore, target identifications by operator in a subset of cameras cannot be utilized to improve ranking of the target in remaining set of network cameras. To address this shortcoming, we propose a novel sequential multi-camera re-id approach. The proposed approach can accommodate human operator inputs and provides early gains via a monotonic improvement in target ranking. At the heart of our approach is a fusion function which operates on deep feature representations of query and candidate matches. We formulate an optimization procedure custom-designed to incrementally …

Abstract

Abstract—The ICDAR 2019 Challenge on “Scanned receiptsOCR and key information extraction” (SROIE) covers importantaspects related to the automated analysis of scanned receipts.The SROIE tasks play a key role in many document analysissystems and hold significant commercial potential. Although a lotof works have been published over the years on administrativedocument analysis, the community has advanced relatively slowly,as most datasets have been kept private. One of the keycontributions of SROIE to the document analysis community is tooffer a first, standardized dataset of 1000 whole scanned receiptimages and annotations, as well as an evaluation procedure forsuch tasks. The Challenge is structured around three tasks,namely Scanned Receipt Text Localization (Task 1), ScannedReceipt OCR (Task 2) and Key Information Extraction fromScanned Receipts (Task 3). The competition opened on 10thFebruary, 2019 and closed on 5th May, 2019. We received 29,24 and 18 valid submissions received for the three competitiontasks, respectively. This report presents the competition datasets,define the tasks and the evaluation protocols, offer detailedsubmission statistics, as well as an analysis of the submittedperformance. While the tasks of text localization and recognitionseem to be relatively easy to tackle, it is interesting to observethe variety of ideas and approaches proposed for the informationextraction task. According to the submissions’ performance webelieve there is still margin for improving information extractionperformance, although the current dataset would have to growsubstantially in following editions. Given the success of theSROIE competition evidenced by the wide interest generatedand the healthy number of submissions from academic, researchinstitutes and industry over different countries, we consider thatthe SROIE competition can evolve into a useful resource for thecommunity, drawing further attention and promoting researchand development efforts in this field

Abstract

Semantic Segmentation Datasets for ResourceConstrained TrainingAshutosh Mishra1?, Sudhir Kumar1,2?, Tarun Kalluri1,3?, Girish Varma1,Anbumani Subramaian4, Manmohan Chandraker3, and CV Jawahar11IIIT Hyderabad2University at Buffalo, State University of New York3University of California, San Diego4Intel BangaloreAbstract.Several large scale datasets, coupled with advances in deepneural network architectures have been greatly successful in pushingthe boundaries of performance in semantic segmentation in recent years.However, the scale and magnitude of such datasets prohibits ubiquitoususe and widespread adoption of such models, especially in settings withserious hardware and software resource constraints. Through this work,we propose two simple variants of the recently proposed IDD dataset,namelyIDD-miniandIDD-lite, for scene understanding in unstructuredenvironments. Our main objective is to enable research and benchmark-ing in training segmentation models. We believe that this will enablequick prototyping useful in applications like optimum parameter andarchitecture search, and encourage deployment on low resource hardwaresuch as Raspberry Pi. We show qualitatively and quantitatively that withonly 1 hour of training on 4GB GPU memory, we can achieve satisfactorysemantic segmentation performance on the proposed datasets.