Abstract

In this research paper, noise suppression ability of associative memories (e.g. Hopfield neural network) is briefly summarized. Motivated by this fact, noise suppression abilities of trained convolutional network is discussed. Introducing the concept of null vectors, noise suppression ability of Single Layer Perceptron, Multi-Layer Perceptron and Extreme Learning Machine are discussed.

Abstract

In the aftermath of 2015 Gorkha Earthquake in Nepal the challenging task was to classify structural damage; for further consideration of the building for living purposes. Manual classification of such a large number of building stock is tedious job in terms of collecting and maintaining data, particularly from the point of view of using the same for further planning purposes. Hence a web based application will make the job simple and easy for the surveyor as they will fill the details from the application and get the damage state of the building and know whether or not the building is safe for living and maintain the record of their decision. The other important feature of this application is it also does the pre-earthquake assessment of the building; thus it evaluates the state of the building before an earthquake through which occupants will know whether it safe or not to live in a particular building and also the improvements required from overall safety point of view. In this paper, description of the web application is given with some case studies. Post-earthquake assessment is done for two types of buildings which is quite common i.e., Reinforced Concrete Building and Brick Masonry Building. Pre-earthquake assessment is done in three parts, first part is Detail Visual Assessment of building based on BMTPC form and second part is Simplified Quantitative Assessment and third one is Detail Qualitative Assessment

Abstract

Mobile image retrieval allows users to identify visual information about their environment by transmitting image queries to an online image database that has associated annotations (e.g. location, product information, etc.) with the images. However, this is reliant on a network connection to transmit the query and retrieve the information. This chapter examines mobile image retrieval for offline use when the data net- work connection is limited or not available. In this scenario, the entire visual search index must reside on the mobile device itself. More specifically, we are interested in ‘instance retrieval’, where the annotations associated with the images (e.g. building’s name, object information) are returned by the query and not the images themselves. Figure 1 shows an example use case where mobile camera photos are used to identify buildings and landmarks without the need for a network connection. While our targeted instance retrieval does not need to store the images, the entire visual index structure needs to fit on a mobile device, ideally within a small footprint (e.g. 16–32 MB). This small memory footprint serves two purposes. First, while mobile phones have up to 16–32 GB of storage, this is mainly in the form of slower flash memory that is an order of magnitude slower than RAM. Having the entire index within tens of MBs makes it possible for use in a resident application on the phone’s RAM. Second, this small size is inline with common practices for mobile applications; e.g. the iPhone average application size is currently 23 MB. Addition- ally, iPhone apps less than 50 MB can be downloaded using 3G/4G, anything larger

Abstract

Building bilateral semantic associations between images and texts is among the fundamental problems in computer vision. In this paper, we study two complementary cross-modal prediction tasks: (i) predicting text(s) given a query image (“Im2Text”), and (ii) predicting image(s) given a piece of text (“Text2Im”). We make no assumption on the specific form of text; i.e., it could be either a set of labels, phrases, or even captions. We pose both these tasks in a retrieval framework. For Im2Text, given a query image, our goal is to retrieve a ranked list of semantically relevant texts from an independent text-corpus (i.e., texts with no corresponding images). Similarly, for Text2Im, given a query text, we aim to retrieve a ranked list of semantically relevant images from a collection of unannotated images (i.e., images without any associated textual meta-data). We propose a novel Structural SVM based unified framework for these two tasks, and show how it can be efficiently trained and tested. Using a variety of loss functions, extensive experiments are conducted on three popular datasets (two medium-scale datasets containing few thousands of samples, and one web-scale dataset containing one million samples). Experiments demonstrate that our framework gives promising results compared to competing baseline cross-modal search techniques, thus confirming its efficacy. Keywords: Image search; Image description; Cross-media analysis

Abstract

Human pose as a query modality is an alternative and rich experience for image and video retrieval. It has interesting retrieval applications in domains such as sports and dance databases. In this work we propose two novel ways for representing the image of a person striking a pose, one looking for parts and other looking at the whole image. These representations are then used for retrieval. Both the representations are obtained using deep learning methods. In the first method, we make the following contributions: (a) We introduce ‘deep poselets’ for pose- sensitive detection of various body parts, built on convolutional neural network (CNN) features. These deep poselets significantly outperform previous instantiations of Berkeley poselets [6], and (b) Using these detec- tor responses, we construct a pose representation that is suitable for pose search, and show that pose retrieval performance is on par with the previous methods. In the second method, we make the follow- ing contributions: (a) We design an optimized neural network which maps the input image to a very low dimensional space where similar poses are close by and dissimilar poses are farther away, and (b) We show that pose retrieval system using these low dimensional representation is on par with the deep poselet representation and is on par with the previous methods. The previous works with which the above two methods are compared include bag of visual words [44], Berkeley poselets [6] and human pose estimation algorithms [52]. All the methods are quantitatively eval- uated on a large dataset of images built from a number of standard benchmarks together with frames from

Abstract

Building robust recognizers for Arabic has always been challenging. We demonstrate the effectiveness of an end-to- end trainable CNN - RNN hybrid architecture in recognizing Arabic text in videos and natural scenes. We outperform previous state- of-the-art on two publicly available video text datasets - ALIF and ACTIV . For the scene text recognition task, we introduce a new Arabic scene text dataset and establish baseline results. For scripts like Arabic, a major challenge in developing robust recognizers is the lack of large quantity of annotated data. We overcome this by synthesizing millions of Arabic text images from a large vocabulary of Arabic words and phrases. Our implementation is built on top of the model introduced here [37] which is proven quite effective for English scene text recognition. The model follows a segmentation-free, sequence to sequence transcription approach. The network transcribes a sequence of convolutional features from the input image to a sequence of target labels. This does away with the need for segmenting input image into constituent characters/glyphs, which is often difficult for Arabic script. Further, the ability of RNN s to model contextual dependencies yields superior recognition results.

Abstract

Inspired by the success of Deep Learning based approaches to English scene text recognition, we pose and bench- mark scene text recognition for three Indic scripts - Devanagari, Telugu and Malayalam. Synthetic word images rendered from Unicode fonts are used for training the recognition system. And the performance is bench-marked on a new - IIIT-ILST dataset comprising of hundreds of real scene images containing text in the above mentioned scripts. We use a segmentation free, hybrid but end-to-end trainable CNN - RNN deep neural network for transcribing the word images to the corresponding texts. The cropped word images need not be segmented into the sub- word units and the error is calculated and backpropagated for the the given word image at once. The network is trained using CTC loss, which is proven quite effective for sequence- to-sequence transcription tasks. The CNN layers in the network learn to extract robust feature representations from word images. The sequence of features learnt by the convolutional block is transcribed to a sequence of labels by the RNN + CTC block. The transcription is not bound by word length or a lexicon and is ideal for Indian languages which are highly inflectional.

Abstract

This paper explores the effectiveness of statistical language model ( SLM ) and dictionary based methods for detec- tion and correction of errors in Indic OCR output. In SLM , we use unicode level ngrams for building the language model. We compare its performance with akshara level ngrams and find that akshara level ngrams perform better in detecting the errors when compared to unicode level ngrams. We experimentally analyze the performance of Indic OCR post-processing using dictionary method, compare the performance with English and analyze the reasons for the under-performance in Indic scripts. We use four major Indian languages for our experiments, namely Hindi, Gurumukhi, Telugu and Malayalam.

Abstract

The power of ConvNets has been demonstrated in a wide variety of vision tasks including pose estimation. But they often produce absurdly erroneous predictions in videos due to unusual poses, challenging illumination, blur, self-occlusions etc. These erroneous predictions can be re- fined by leveraging previous and future predictions as the temporal smoothness constrain in the videos. In this paper, we present a generic approach for pose correction in videos using sequence learning that makes minimal assumptions on the sequence structure. The proposed model is generic, fast and surpasses the state-of-the-art on benchmark datasets. We use a generic pose estimator for initial pose estimates, which are further refined using our method. The proposed architecture uses Long Short-Term Memory ( LSTM ) encoder- decoder model to encode the temporal context and refine the estimations. We show 3.7% gain over the baseline Yang & Ramanan ( YR ) [1] and 2.07% gain over Spatial Fusion Network ( SFN ) [2] on a new challenging YouTube Pose Subset dataset [3]. Keywords-Pose estimation; sequence to sequence learnin

Abstract

We look at the problem of developing a compact and accurate model for gesture recognition from videos in a deep-learning framework. Towards this we propose a joint 3DCNN-LSTM model that is end-to-end trainable and is shown to be better suited to capture the dynamic information in actions. The solution achieves close to state-of-the-art accuracy on the ChaLearn dataset, with only half the model size. We also explore ways to derive a much more compact representation in a knowledge distillation framework followed by model compression. The final model is less than 1 MB in size, which is less than one hundredth of our initial model, with a drop of 7% in accuracy, and is suitable for real-time gesture recognition on mobile devices.