Abstract

The last two decades have witnessed an exponential rise in web content from a plethora of domains, which has necessitated the use of domain-specific search engines. Diversity of crawled content is one of the crucial aspects of a domain-specific search engine. To a large extent, diversity is governed by the initial set of seed URLs. Most of the existing approaches rely on manual effort for seed selection. In this work we automate this process using URLs posted on Twitter. We propose an algorithm to get a set of diverse seed URLs from a Twitter URL graph. We compare the performance of our approach against the baseline zero similarity seed selection method and find that our approach beats the baseline by a significant margin.

Abstract

The velocity, volume and variety with which Twitter generates text is increasing exponentially. It is critical to determine latent sub-topics from such tweet data at any given point of time for providing better topic-wise search results relevant to users’ informational needs. The two main challenges in mining sub-topics from tweets in real-time are (1) understanding the semantic and the conceptual representation of the tweets, and (2) the ability to determine when a new sub-topic (or cluster) appears in the tweet stream. We address these challenges by proposing two unsupervised clustering approaches. In the first approach, we generate a semantic space representation for each tweet by keyword expansion and keyphrase identification. In the second approach, we transform each tweet into a conceptual space that represents the latent concepts of the tweet. We empirically show that the proposed methods outperform the state-of-the-art methods.

Abstract

Entity Disambiguation is the task of associating entity name mentions in text to the correct referent entities in the knowledge base, with the goal of understanding and extracting useful information from the document. Entity disambiguation is a critical component of systems designed to harness information shared by users on microblogging sites like Twitter. However, noise and lack of context in tweets makes disambiguation a difficult task. In this paper, we describe an Entity Disambiguation system, EDIUM, which uses User interest Models to disambiguate the entities in the user’s tweets. Our system jointly models the user’s interest scores and the context disambiguation scores, thus compensating the sparse context in the tweets for a given user. We evaluated the system’s entity linking capabilities on tweets from multiple users and showed that improvement can be achieved by combining the user models and the context based models.

Abstract

Bilingual dictionaries are the key component of the cross-lingual similarity estimation methods. Usually such dictionary generation is accomplished by manual or automatic means. Automatic generation approaches include to exploit parallel or comparable data to derive dictionary entries. Such approaches require large amount of bilingual data in order to produce good quality dictionary. Many time the language pair does not have large bilingual comparable corpora and in such cases the best automatic dictionary is upper bounded by the quality and coverage of such corpora. In this work we propose a method which exploits continuous quasi-comparable corpora to derive term level associations for enrichment of such limited dictionary. Though we propose our experiments for English and Hindi, our approach can be easily extendable to other languages. We evaluated dictionary by manually computing the precision. In experiments we show our approach is able to derive interesting term level associations across languages.

Abstract

Developers need to understand many Software Engineering (SE) artifacts while making changes to the code. In such cases, developers use cues extensively to establish relevance of an information with the task. Their familiarity with different kind of cues will help them in comprehending a program. But, developers face information overload because (a) there are many cues and (b) they might be unfamiliar with artifacts. So, we propose a novel approach to overcome information overload problem by modeling developer's perceived value of information based on cues. In this preliminary study, we validate one such model for common comprehension tasks. We also apply this model to summarize source code. An evaluation of the generated summaries resulted in 83% similarity with summaries recorded by developers. The promising results encourages us to create a repository of perception models that can later aid complex SE tasks.

Abstract

For privacy reasons, personally identifiable information like age and gender of people is not available publicly. However accurate prediction of such information has important applications in the fields of advertising, forensics and business intelligence. Existing methods for this problem have focused on classifier learning using content based features like word n-grams and style based features like Part of Speech (POS) n-grams. Two major drawbacks of previous approaches are:(1) they do not consider the semantic relation between words, and (2) they do not handle polysemy. We propose a novel method to address these drawbacks by representing the document using Wikipedia concepts and category information. Experimental results show that classifiers learned using such features along with previously used features help us achieve significantly better accuracy compared to the state-of-the-art methods. Indeed, feature selection shows that our novel features are more effective than previously used content based features.

Abstract

A large number of web queries are related to product entities. Studying evolution of product entities can help analysts understand the change in particular attribute values for these products. However, studying the evolution of a product requires us to be able to link various versions of a product together in a temporal order. While it is easy to temporally link recent versions of products in a few domains manually, solving the problem in general is challenging. The ability to temporally order and link various versions of a single product can also improve product search engines. In this paper, we tackle the problem of finding the previous version (predecessor) of a product entity. Given a repository of product entities, we first parse the product names using a CRF model. After identifying entities corresponding to a single product, we solve the problem of finding the previous version of any given particular version of the product. For the second task, we leverage innovative features with a Naïve Bayes classifier. Our methods achieve a precision of 88% in identifying the product version from product entity names, and a precision of 53% in identifying the predecessor.

Abstract

Entities are centric to a large number of real world applications. Wikipedia shows entity infoboxes for a large number of entities. However, not much structured information is available about character entities in books. Automatic discovery of characters from books can help in effective summarization. Such a structured summary which not just introduces characters in the book but also provides a high level relationship between them can be of critical importance for buyers. This task involves the following challenging novel problems: 1. automatic discovery of important characters given a book; 2. automatic social graph construction relating the discovered characters; 3. automatic summarization of text most related to each of the characters; and 4. automatic infobox extraction from such summarized text for each character. As part of this demo, we design mechanisms to address these challenges and experiment with publicly available books.

Abstract

Given a knowledge base, annotating any text with entities in the knowledge base enhances automated understanding of the text. Entities provide extra contextual information for the automated system to understand and interpret the text better. In the special case when the text is in the form of short text queries, automated understanding can be critical in improving the quality of search results and recommendations. Annotation of queries helps semantic retrieval, ensuring diversity of search results including retrieval of relevant news stories. In this paper, we present SIEL@ERD, a system for automated stamping of entity information in short query text. Our system builds from the state-of-the-art TAGME system and is optimized for time and performance efficiency. Our system achieved an F1 measure of 0.53 and the latency of 0.31 seconds on a dataset of 500 queries and a Freebase snapshot provided for the short track in the Entity Recognition and Disambiguation Challenge at SIGIR 2014.

Abstract

Our computing demands have grown so much that we need a robust distributed computing platform to process data. To feed such data hungry systems, we need an equally robust distributed file systems that span across multiple geographically separate locations. Most distributed file systems break any file into a set of blocks or chunks which are spread across the cluster. The major bottleneck is to identify where to place the blocks and where to place the replicas so that cluster is optimized on certain parameters like disk utilization, minimizing network congestion, maximizing throughput, optimal power utilization, etc. This paper proposes assigning a distance measure for each one of the data sources with respect to the other and placing the blocks on the specified disk on the node that minimizes the total distance of the last few requests made for that block. As the request pattern and parameters change, the distances are updated and the blocks are moved dynamically to minimized the distance, in effect optimizing on the required parameters. The distance function is to be modeled based on the cluster and the parameters you wish to optimize on. It can be a function of just bandwidth or bandwidth and latency or other miscellaneous features like disk utilization, processing power, disk speed, power utilization, cooling requirements, temperature, etc. A detailed performance analysis was carried out with disk bandwidth, network bandwidth and disk utilization as the parameters and the performance is far better in comparison to the reference system (10% or more depending on specification differences) which has no understanding of the different type of disks present and the nature of the cluster. Due to the performance aware nature, the system was inherently able to utilize memory for speeding up performance through the use of in-memory partitions.