Abstract

Modern feature extraction pipelines, especially the on esusing deep networks, involve an increasing variety of elements. With layered approaches heaping abstraction upon abstraction, it becomes difficult to understand what it is that these features are capturing. One appealing way of solving this puzzle is feature visualization, where features are mapped back to the image domain. Our work improves the generic approach of performing gradient descent (GD) in the image space to match a given set of features to achieve a visualization. Specifically, we note that coarse features of an image like blobs, outlines etc. are useful by them-selves for classification purposes. We develop an inversion scheme based on this idea by recovering coarse features of the image before finer details. This is done by modeling the image as the composition of a Laplacian Pyramid. We show that by performing GD on the pyramid in a level-wise manner, we can recover meaningful images. Results are presented for inverting a shallow network: the densely calculated SIFT as well as a deep network: Krizehvsky et al.’sImagenet CNN (Alexnet).

Abstract

Most of the traditional sketch-based image retrieval systems compare sketches and images using morphological features. Since these features belong to two different modalities, they are compared either by reducing the image to a sparse sketch like form or by transforming the sketches to a denser image like representation. However, this cross-modal transformation leads to information loss or adds undesirable noise to the system. We propose a method, in which, instead of comparing the two modalities directly, across-modal correspondence is established between the images and sketches. Using an extended version of Canonical Correlation Analysis (CCA), the samples are projected onto a lower dimensional subspace, where the images and sketches of the same class are maximally correlated. We test the efficiency of our method on images from Caltech, PAS-CAL and sketches from TU-BERLIN dataset. Our resultsshow significant improvement in retrieval performance with the cross-modal correspondence.

Abstract

Understanding the precise 3D structure of an environ-ment is one of the fundamental goals of computer vision and is challenging due to a variety of factors such as appearance variation, illumination, pose, noise, occlusion and scene clutter. A generic solution to the problem is ill-posed due to the loss of depth information during imaging. In this paper, we consider a specific but common situation, where the scene contains known objects. Given 3D models of a set of known objects and a cluttered scene image, we try to detect these objects in the image, and align 3D models to their images to find their exact pose. We develop an approach that poses this as a 3D-to-2D alignment problem.We also deal with pose estimation of 3D articulated objects in images. We evaluate our proposed method on Big Bird dataset and our own tabletop dataset, and present experimental comparisons with state-of-the-art methods.

Abstract

Recently, exemplar based approaches have been successfully applied for face detection in the wild. Contrary to traditional approaches that model face variations from a large and diverse set of training examples, exemplar-based approaches use a collection of discriminatively trained exemplars for detection. In this paradigm, each exemplar casts a vote using retrieval framework and generalized Hough voting, to locate the faces in the target image. The advantage of this approach is that by having a large database that covers all possible variations, faces in challenging conditions can be detected without having to learn explicit models for different variations. Current schemes, however, make an assumption of independence between the visual words, ignoring their relations in the process. They also ignore the spatial consistency of the visual words. Consequently, every exemplar word contributes equally during voting regardless of its location. In this paper, we propose a novel approach that incorporates higher order information in the voting process. We discover visual phrases that contain semantically related visual words and exploit them for detection along with the visual words. For spatial consistency, we estimate the spatial distribution of visual words and phrases from the entire database and then weigh their occurrence in exemplars. This ensures that a visual word or a phrase in an exemplar makes a major contribution only if it occurs at its semantic location, thereby suppressing the noise significantly. We perform extensive experiments on standard FDDB, AFW and G-album datasets and show significant improvement over previous exemplar …

Abstract

In this work, we propose an online handwriting solution, where the data is captured with the help of depth sensors. Users may write in the air and our method recognizes it in real time using the proposed feature representation. Our method uses an efficient fingertip tracking approach and reduces the necessity of pen-up/pen-down switching. We validate our method on two depth sensors, Kinect and Leap Motion Controller. On a dataset collected from 20 users, we achieve a recognition accuracy of 97.59% for character recognition. We also demonstrate how this system can be extended for lexicon recognition with reliable performance. We have also prepared a dataset containing 1,560 characters and 400 words with the intention of providing common benchmark for handwritten character recognition using depth sensors and related research.

Abstract

The problem of labeling the edges present in a single color image as convex, concave, and occluding entities is one of the fundamental problems in computer vision. It has been shown that this information can contribute to segmentation, reconstruction and recognition problems. Recently, it has been shown that this classification is not straightforward even using RGBD data. This makes us wonder whether this apparent simple cue has more information than a depth map? In this paper, we propose a novel algorithm using random forest for classifying edges into convex, concave and occluding entities. We release a data set with more than 500 RGBD images with pixel-wise ground labels. Our method produces promising results and achieves an F-score of 0.84 on the data set.

Abstract

Representation of fingerprints is one of the key factors that limits the accuracy and efficiency of matching algorithms. Most popular methods represent each fingerprint as an unordered set of minutiae with variable cardinality and the matching algorithms are left with the task of finding the best correspondence between the two sets of minutiae. While this makes the representation more flexible and matching more accurate, the task becomes computationally intensive. Fixed length representations with aligned features are highly efficient to match. However, creating an aligned representation without the knowledge of the sample to which it is to be matched, makes the problem of representation more complex. Some of the fixed-length representations only provide partial alignment, leaving the rest to the matching stage. In this paper, we propose a fixed length representation for fingerprints that provides exact alignment between the features, thus enabling high-speed matching with minimal computational effort. The representation extends the idea of object representation using bag of words into a bag of minutiae neighborhoods. The representation is provably invariant to affine transformations (rotation, translation and uniform scaling), and is shown to be highly discriminative for the task of verification. Experimental results on FVC 2002 and 2004 datasets clearly show the superiority of the representation with competing methods. As the proposed representation can be computed from the standard minutiae templates, the method is applicable to existing datasets, where the original fingerprint images are not available.

Abstract

The ability to detect floor regions from an image enables a variety of applications such as indoor scene understanding, mobility assessment, robot navigation, path planning and surveillance. In this work, we propose a framework for estimating floor regions in cluttered indoor environments. The problem of floor detection and segmentation is challenging in situations where floor and non-floor regions have similar appearances. It is even harder to segment floor regions when clutter, specular reflections, shadows and textured floors are present within the scene. Our framework utilizes a generic classifier trained from appearance cues as well as floor density estimates, both trained from a variety of indoor images. The results of the classifier is then adapted to a specific test image where we integrate appearance, position and geometric cues in an iterative framework. A Markov Random Field framework is used to integrate the cues to segment floor regions. In contrast to previous settings that relied on optical flow, depth sensors or multiple images in a calibrated setup, our method can work on a single image. It is also more flexible as we avoid assumptions like Manhattan world scene or restricting clutter only to wall-floor boundaries. Experimental results on the public MIT Scene dataset as well as a more challenging dataset that we acquired, demonstrate the robustness and efficiency of our framework on the above mentioned complex situations.

Abstract

We consider the problem of automatic identification of faces in videos such as movies, given a dictionary of known faces from a public or an alternate database. This has applications in video indexing, content based search, surveillance, and real time recognition on wearable computers. We propose a two stage approach for this problem. First, we recognize the faces in a video using a sparse representation framework using l1-minimization and select a few key-frames based on a robust confidence measure.We then use transductive learning to propagate the labels from the key-frames to the remaining frames by incorporating constraints simultaneously in temporal and feature spaces. This is in contrast to some of the previous approaches where every test frame/track is identified independently, ignoring the correlation between the faces in video tracks. Having a few key frames belonging to few subjects for label propagation rather than a large dictionary of actors reduces the amount of confusion. We evaluate the performance of our algorithm on Movie Trailer face dataset and five movie clips, and achieve a significant improvement in labeling accuracy compared to previous approaches

Abstract

No Results Found