Abstract

Understanding the precise 3D structure of an environment is one of the fundamental goals of computer vision and is challenging due to a variety of factors such as appearance variation, illumination, pose, noise, occlusion and scene clutter. A generic solution to the problem is ill-posed due to the loss of depth information during imaging. In this paper, we consider a specific but common situation, where the scene contains known objects. Given 3D models of a set of known objects and a cluttered scene image, we try to detect these objects in the image, and align 3D models to their images to find their exact pose. We develop an approach that poses this as a 3D-to-2D alignment problem. We also deal with pose estimation of 3D articulated objects in images. We evaluate our proposed method on BigBird dataset and our own tabletop dataset, and present experimental comparisons with state-of-the-art methods. In order to find the pose of an object, we come up with a hierarchical approach whereby we first an initial estimate of the pose and thereby refine it using a robust algorithm. Obtaining the initial estimate is crucial as the refinement is entirely dependant on it. Estimating the object proposals or region proposals from an image is a well-known but difficult task, as the complexity of the problem intensifies due to the presence of object-object interaction and background clutter. We tackle the problem by coming up with a robust Convolutional Neural Network based method which learns object proposals in a supervised manner. As we need region proposals at object level, we solve the problem of instance-level semantic segmentation, where each pixel in the image is classified into one of the known classes. Moreover, two pixels are labelled differently if they belong to two different instances of the same class. We show quantitative and qualitative comparison of our proposed network models with previous approaches, and show our results on the challenging PASCAL VOC dataset.

Abstract

A given piece of textual information produced by any user or agent is said to be inappropriate if the expressed intent may cause anger, annoyance to certain users or exhibits lack of respect, rudeness, discourteousness towards certain individuals/communities or may be capable of inflicting harm to oneself or others. A search engine should regulate its query completion suggestions by detecting and filtering suchqueries as it may hurt the user sentiments or may lead to legal issues thereby tarnishing the brand image. Hence, automatic detection and pruning of such inappropriate queries from completions and related search suggestions is an important problem for most commercial search engines. The problem is rendered difficult due to unique challenges posed by search queries such as lack of sufficient context, natural language ambiguity and presence of spelling mistakes and variations. In this Thesis, we propose a novel deep learning based technique for automatically identifying inap- propriate query suggestions. We propose a novel deep learning architecture called “Convolutional Bi-Directional LSTM (C-BiLSTM)” which combines the strengths of both Convolution Neural Networks (CNN) and Bi-directional LSTMs (BLSTM). Given a query, C-BiLSTM uses a convolutional layer for extracting feature representations for each query word which is then fed as input to the BLSTM layer which captures the various sequential patterns in the entire query and outputs a richer representation encoding them. The query representation thus learnt passes through a deep fully connected network which predicts the target class. C-BiLSTM doesn’t rely on hand-crafted features, is trained end-end as a single model, and effectively captures both local features as well as their global semantics. Evaluating C-BiLSTM on real-world search queries from a commercial search engine reveals that it significantly outperforms both pattern based and other hand-crafted feature based baselines. Moreover, C-BiLSTM also performs better than individual CNN, LSTM and BLSTM models trained for the same task. Rapid growth of chatbots and Interactive Gaming Systems expect large, real time, clean (non-inappropriate) human conversation data to train their models. High demand of social networking sites provide a platform for users to interact and share opinions publicly. Inappropriate conversations and posts in these platforms may also lead to loss of business and damaging company’s image. So we extended our idea to identify inappropriate conversations in chat data. We have applied various techniques to solve this problem and observed that BLSTM performs better than LSTM and Boosted Decision Tree approaches.

Abstract

Understanding the meaning of words is essential for most natural language processing tasks. Word representations are means to mathematically represent the meaning of a word in a way that computers can understand. These representations are often in the form of vectors in which words are represented in a continuous vector-space of fixed dimensionality also referred to as word embeddings. In this thesis, we focus on generating better and reliable word representations for low resource languages. Many languages, though widely spoken, are largely under-represented in this area of research. One of the main reasons for this is the lack of reliable evaluation metrics to compare between different approaches of building these embeddings. Word similarity task is a widely used, computationally efficient method to directly evaluate the quality of word vectors. It relies on finding correlation between human assigned similarities between words, and those between corresponding word vectors. We release word similarity datasets for six low resource languages – Urdu, Telugu, Marathi, Punjabi, Tamil and Gujarati. For the construction of these datasets, our approach relies on translation and re-annotation of word similarity datasets of English. We also present baseline scores for word representation models using state-of-the-art techniques for Urdu, Telugu and Marathi by evaluating them on newly created word similarity datasets. For linguistically similar languages, we show that it is possible to use the better trained word representations of the more resourceful language for the other language in the pair, using a projection learning approach which relies on a mapping between words having similar meaning from the two languages. This cross-lingual vector space transformation results in state of the art results on word similarity test sets of French and German - an increase of 13% in case of French and 19% for German, using English as the source language. We also go on to demonstrate that this approach is better suited for linguistically similar language pairs like Hindi-Urdu (where 60% words are simply transliterations of each other) than English-German or English-French. We go on to see how we modelled prefix-suffix based morphoology using a similar technique.

Abstract

The field of data mining has emerged to extract knowledge hidden in large databases for better decision making. The process of frequent pattern (a set of items represents a pattern (or an itemset)) mining finds interesting information about the association among the items in a transactional database. The notion of support is employed to extract the frequent patterns. A pattern is called a frequent pattern if it satisfies the user-defined threshold on minimum support. An important criterion to assess the interestingness of a frequent pattern is its temporal occurrences in a database. That is, whether a frequent pattern is occurring periodically, irregularly, or mostly at specific time intervals in a database. The class of frequent patterns that are occurring periodically within a database are known as periodic-frequent patterns. Finding these patterns is a significant task with many real-world applications like improving the performance of recommender systems, intrusion detection in computer networks, discovering events in Twitter. Current periodic-frequent pattern models cannot handle datasets in which multiple transactions share a common timestamp or when transactions occur at irregular time intervals. This issue limits the applicability of the model as in many real-world databases like e-Commerce, Twitter, etc., transactions share a common timestamp and uneven time gaps exist in between the consecutive transactions. Most previous models on periodic-frequent pattern mining have focused on finding all patterns in a transactional database that satisfy the user-specified minimum support (minSup) and maximum periodicity (maxP er) constraints. The minSup constraint controls the minimum number of transactions that a pattern must cover in a database. The maxP er constraint controls the maximum duration between the two transactions below which a pattern should reoccur in a database. The usage of a single minSup and maxP er for an entire database leads to the rare item problem, because real-world databases have a non-uniform item distribution, which considers that items have different support and periodicity values. Also, current periodic-frequent pattern models have focused on discovering full periodic-frequent patterns, i.e., finding all patterns that have exhibited complete cyclic repetitions throughout the entire database. These models evaluate the periodic interestingness of a frequent pattern by determining whether all of its inter-arrival times are within the user-specified maxP er threshold. Therefore, the model cannot assess the partial periodic behavior of a frequent pattern in a database. However, partial periodic-frequent patterns are more common due to the imperfect nature of real-world databases. So, to address the above issues, in this thesis, we are proposing two improved approaches that discover periodic-correlated patterns and partial periodic-frequent patterns in non-uniform temporal databases, respectively. In the first approach, we tackle rare item problem by proposing a improved model that discovers periodic-correlated patterns in a non-uniform temporal database. In this thesis, we consider temporal database as a collection of transactions, ordered by their timestamps. Further, a temporal database facilitates multiple transactions to share a common timestamp and allows time-gaps in between consecutive transactions. A temporal database is said to be non-uniform if it contains items with dissimilar support and periodicity. To tackle rare item problem in non-uniform temporal databases, the proposed model considers a pattern as interesting if its support and periodicity are close to that of its individual items. The existing all-confidence measure is used to determine how close is the support of a pattern with respect to the support of its individual items. A new interestingness measure, called periodic-all-confidence, is being proposed to determine how close is the periodicty of a pattern with respect to the periodicity of its individual items. A pattern-growth algorithm has also been discussed to find periodic-correlated patterns. Experimental results show that the proposed model is efficient and tackles rare item problem. We discuss the usefulness of periodic-correlated patterns with a real-world case study on FAA-Accidents database and show that the proposed model may be utilized to discover interesting periodic-correlated patterns involving both frequent and rare items effectively. In the second approach, we have introduced a improved model to discover partial periodic-frequent patterns in non-uniform temporal databases. The proposed model lets the user specify a different maximum inter-arrival time (M IAT ) for each item. An inter-arrival time of a pattern is considered periodic (or cyclic) if it is no more than period. Thus, different patterns may satisfy different period depending on their items’ M IAT values. This solves the rare item problem in partial periodic-frequent pat

Abstract

This research aims to investigate the crossmodal correspondences between pitch and space. Crossmodal correspondences refer to the associations that exist between different physical stimulus attributes or properties in different modalities. In the context of this thesis, we see the association between pitch as one physical attribute and space as another. Research refers to this as pitch-space mapping and has been frequently investigated through various methods. In recent times, however, pitch-space mapping has been investigated by employing the stimulus-response compatibility (SRC) paradigm. SRC refers to the fact that performance is better when stimuli and responses are paired with each other in certain ways i.e. stimulus and response share common features than when they do not. Perception-action coupling is facilitated due to such correspondences resulting in higher accuracy and faster response times (Rts). Previous research shows that performance is better when a high pitch is responded with up or right responses and a low pitch is responded with down or left responses, called the spatial-pitch association of response codes (SPARC) effect. In this thesis, we have investigated the SPARC effect by introducing certain manipulations which are now briefly explained. Our first manipulation stems from the need to employ feedback in an SRC based task, more specifically SPARC effect in this case. Despite the intuitive coupling of perception-action, studies investigating the SPARC effect have, however, used feedback to manipulate the stimulus-response mapping. Feedback contradicts the purpose of intuitive stimulus-response mapping by enabling short-term learning. Our first section primarily investigates the role of feedback on SPARC effect. The second study of the thesis explores the role of loudness in pitch-space mapping. Even though all the recent studies more or less establish the pitch-space mapping, little has been questioned about the dimension of loudness that can interact with the spatial representation of pitch. It has been shown that loudness and pitch interact in the perception of a multidimensional stimulus such as auditory tone. They form an inseparable perceptual couple and therefore variation in one can cause a subjective variation in the other. Hence, pitch perception is confounded with loudness. Moreover, loudness too has its own spatial representation which can interact with the spatial representation of pitch. Our second study, therefore, investigates the role of loudness equalization on the SPARC effect. Finally, the third question addresses the role of blindness in pitch-space mapping. Our motivation for this study comes from the fact that even though visually impaired have an absent modality of vision, they have functional spatial representations of the world around derived through other modalities. They also exhibit similar if not better performance in pitch perception tasks as well. Hence, it was intriguing to see whether a lack of vision has any role on the SPARC effect. We found consistent vertical SPARC effect and horizontal SPARC effect in the first two manipulations. Moreover, both feedback and loudness equalization influenced the horizontal SPARC effect. The third study showed no SPARC effect in visually impaired participants. However, the results from third study are not conclusive due to the limitations and challenges described in the corresponding chapter.

Abstract

Code-mixing is a frequently encountered phenomenon in day-to-day natural language communication in the multilingual and bilingual communities. The phenomenon is so widespread that is often considered a different, emerging variety of the language. Computational modeling of the phenomenon of code-mixing and code-switching assumes particular relevance with the advancement of social media. However, computational studies for both textual and speech processing of code-mixing suffer from a sincere disadvantage: lack of data. In this thesis, we present the development of a Phonetically Balanced read speech corpus of Code-Mixed Hindi-English (the PBCM corpus). A Large Code-Mixed (LCM corpus) has been extracted from selected sections of two widely read Hindi newspapers. These sections (namely: Sports, Technology and Lifestyle) contain frequent English insertions embeddded within the matrix of Hindi sentences. Phonetic balance in the corpus has been established by selecting sentences that contain triphones lower in frequency than a predefined threshold. The assumption with the compulsory inclusion of such rare units is that the high frequency triphones will inevitably be included [66]. Using this metric, the Pearson’s correlation coefficient of the phonetically balanced corpus with a large code-mixed reference corpus was recorded to be 0.996. Statistics on the phone and triphone distribution have been presented, to graphically display the phonetic likeness between the large reference corpus (LCM corpus) and the corpus sampled through our method. A pilot PBCM corpus (PBCM-Phase I) has been recorded by 4 volunteer Hindi speakers. As a second contribution of this thesis, we present a baseline automatic speech recognition (ASR) system for code-mixed read speech in Hindi-English, developed upon the extrapolation of monolingual training resources. Two distinct monolingual acoustic models (Hindi and English) have been implemented to train a neural network and subspace GMM based speech recognition framework. The testing has been performed on PBCM-Phase I corpus. We examine the combination of a code-mixed trigram language model with each of the acoustic models described above. We contrast the combination of the code-mixed language model with a monolingual language model, and see drastic improvements in accuracy. The Hindi monolingual acoustic model, in combination with the code-mixed language model is our best performing experimental setup. (word error rate: 41.22%).

Abstract

Morphology is the branch of linguistics that deals with words, their internal structure, how they are formed, and their relationship to other words in the same language. It involves analyzing the structure of words and parts of words, such as stems, root words, prefixes, and suffixes. It also looks at parts of speech, intonation and stress, and the ways context can change a word’s pronunciation and meaning. In most languages, if not all, many words can be related to other words by rules that collectively describe the grammar for that language. For example, English speakers recognize that the words dog and dogs are closely related, differentiated only by the plurality morpheme “-s”, only found bound to nouns. With recent advancements in computational linguistics, we can now learn one-hot vector represen- tation for each word, also called word representations, from monolingual corpus of a language (training corpus). Word representations have been shown to contain syntactic as well as semantic (morphologi- cal) regularities. These word representations are being widely used to solve problems of various areas of natural language processing. These include but are not limited to dependency parsing, named entity recognition and parsing. One major requirement for learning good word representations (word embeddings) is large enough corpus to train. Size of training corpus directly affects the corresponding quality of word representations our model learns. Many languages, even though widely spoken, suffer from being computationally resource poor, which results in relatively poorer trained word embeddings. On top of it, morphologically rich languages suffer from morphologically induced data sparsity, since there are cases, where one morphological form of a word is common but other is rare in the same training corpus. Hence to learn better word representations for low resourced languages and to better exploit morpho- logical regularities present in distributional word representations, we present a language independent, unsupervised method for building word embeddings using morphological expansion of text by exploit- ing morphological regularities present in distributed word representations. Our model handles the prob- lem of data sparsity and yields improved word embeddings by relying on training word embeddings on artificially generated sentences. We evaluate our method using small sized training sets on eleven test sets for the word similarity task across seven languages. Further, for English, we evaluated the impacts of our approach using a large training set on three standard test sets. Our method improved results across all languages. We also present an unsupervised, language agnostic approach for exploiting morphological regular- ities present in high dimensional vector spaces. We propose a novel method for generating embeddings of words from their morphological variants using morphological transformation operators. We evaluate this approach on MSR word analogy test set with an accuracy of 85% which is 12% higher than the previous best known system.

Abstract

Cultural objects are highly interpretive and complex in nature. Though technological advancements have made it possible to digitize and preserve knowledge of cultural value, capturing details of such objects in-situ has rarely been explored. In this thesis, we develop a platform called Curarium for indepth exploration of cultural collections, within and outside a museum setting. A focus group of 17 researchers gathered cultural collections on a trip to Odisha. The collections contain information pertaining to Pattachitra paintings, Gotipua performances, sustainable crafts, museum objects and processions. The data-sets are unique in nature and content, as they provide for an interplay between different kinds of objects. In this thesis, we examine the cultural value in objects unexplored before - like processions, and what they comprise. To accommodate the different kinds of data, we develop a meta-data structure that would encapsulate all the relevant information. We import the curated data collection into Curarium and utilize the various in-house tools to discover patterns, create annotations, collaborate and visualize the data. Curarium provides a novel approach in delving deep into a cultural object. However, we recognize a gap in cross-referencing objects within its natural environment. In lieu of this, the thesis proposes developing narratives about cultural objects as an approach to capturing information. We develop a platform to generate narratives based on the digital surrogates. We conduct a user study with the focus group to develop narratives on the aforementioned data-sets on a platform called Narratavium. On further research, we recognize these narratives to be of three types - within-object, within-collection, and user generated thematic narratives. Upon recognizing plots within art objects and identifying themes, we account for the human experience in interacting with these cultural objects in their natural environment. This narrows the removed appreciation of the culture by the audience. In this thesis, we utilize sequential art methods to study Pattachitra paintings, Aristotlean narrative theories to study processions and performances, and qualitative analysis to generate themes. By visualizing the narrative information, we gain insights into nuances of the objects and the collections as a whole. We use charts and clusters to extract popular themes in a collection. We develop methods to identify conflict points in a narrative and understand transitions in a story plot. We also study information retention in the process of re-narration, and relevant objects in a collection through frequently told narratives.

Abstract

Dopamine plays a major role in learning and reward. Dopaminergic neurons in the ventral tegmental area (VTA) exhibit the discrepancy between expected and actual rewards known as reward prediction error (RPE). But how these signals are computed in the brain remains unknown. One of the earliest attempts of investigation of animal learning involved pairing an unconditioned stimulus (US) with a cue or conditioned stimulus (CS) and observing that animal starts responding to the CS after some point in time. This is the basis of Pavlovian learning or classical conditioning, a fundamental learning mechanism in animals. The model described here focuses on the mechanism of reward prediction error within Pavlovian learning and does not deal with other conditioning phenomena. To acquire a deeper understanding of the roles of the cerebral structures involved in RPE computation, we need a neuro-computational model implicating them more faithfully in Pavlovian learning. The approach described here is a model that proposes a circuit involving a dissociation between magnitude expectation and timing expectation and explaining more precisely how the computation of the reward prediction error happens inside the VTA. The model described here proposes that the two dimensions involved in computation of reward prediction errors i.e., magnitude and time could be computed separately and later combined unlike the way it is done in traditional reinforcement learning models. The model is built on biological evidences and is able to reproduce various aspects of classical conditioning, namely, the progressive cancellation of the predicted reward, the predictive firing from conditioned stimuli, the twin peaks of dopamine firing and delineation of early rewards by showing more firing for sooner early rewards and less for early rewards that occur with a longer latency in accordance with biological data. Key predictions for experiments are made regarding how neural substrates execute this computation which would validate the model. Potential implications for the reinforcement learning theory are also discussed.

Abstract

The structure of built environment is defined by the collection of fundamental units contained in it. Two of the most basic units in an urban setting are Roads and Buildings. The buildings form most of the origin and destination pairs for intra-city movements and the road-networks aid these movements. The basic aim of this thesis is to identify patterns of arrangements of these elementary units in any urban agglomeration along with deriving a syntactic meaning out of the arrangement geometry and secondly, how the movements patterns and dynamic occupancy of spaces are often affected by the synergy of roads and buildings. We try to identify combinations of arrangement of these two that increase the odds in favor of increasing attractor values of spaces. In past, several studies were carried out to extract patterns in urban environment which were often driven by considering networks only. A few studies paid attention to buildings and their characteristics such as their volumetric capacities and their locations along the network. The approach we have chosen to define symmetries and regularities in structures is hybrid in nature as it considers both buildings and road-networks as well as their intrinsic interactions such that the buildings and roads in closer vicinity inherit the properties of each other. The spatial indices we used for analyzing built environments are based on navigational properties of any location which suggest accessibility of the location depending on its connectivity with neighborhoods. These indices are motivated by a combination of Space Syntax configurational theory that uses topological (Angular & Segment) distances along network and Urban Network Analysis which has metric distances as basis for accessibility computations. The accessibility based on topological as well as metric computations is defined as function of how often a spatial unit lies on the shortest paths while commuting all origin-destination pairs in its neighborhood, how many of surrounding places can be reached from this place, how compactly packed is the neighborhood for a given set of fixed radii, and do the shortest paths between origin-destination pairs resemble straightest distances. These characteristics can be discussed for both buildings and roads. We use spatial autocorrelation for our chosen accessibility indices to analyze significance of occurrence of specific patterns of neighborhoods for city of Hyderabad, India, and thus consider not only spatial units but areas as whole. We discuss the inconsistencies in methodologies used, and propose a weight metrics for urban road network, weights are driven by dimensions of buildings. This enables us to reduce computational complexity by a significant margin using road networks as the only analysis layer for making configurational computations. We verify that it is intrinsic property of spatial organization that more similar areas tend to occur in closer vicinity of each other by using K Means clustering. In the process, we also propose that the clusters formed with varying radii of analysis are suggestive of spatial properties of urban environment. Lastly, we present a data driven methodology to classify regions in terms of amount vehicular traffic. In this technique, Average Annual Daily Counts (AADT) of vehicular traffic are used to represent roads into three categories of high, medium and low which are then superimposed on buildings. The chosen spatial indices form feature vectors for training the model while the vehicular traffic categories superimposed on buildings act as prediction target labels. To get the best accuracy model, we compare performances of Logistic Regression Classification model, Decision Tree Classification model and ensemble based Gradient Boosted Classification model. The ensemble based model using gradient boosting algorithm outperforms all other models in this classification task. While discussing all methodologies to denominate areas in terms of their symmetry and constrained by relative carriage of vehicular traffic, we go from a local measure of trip distances towards global trip distances that implies we go from inspection of nearby regions in close proximity of area under observation towards larger distances. This process enables us to draw comparisons between areas corresponding to their obvious choices of being preferred for various trip distances. For example, an area being preferred for short trip distances might not be a good fit for longer trips and vice versa. The approaches presented in this can draw conclusive reports from the perspective of architectural and infrastructural planning for new city sites and modifying the existing ones.