Abstract

The twin challenges of massive information overload via the web and ubiquitous computers present us with an unavoidable task: developing techniques to handle multilingual information robustly and efficiently, with as high quality performance as possible. Previous research activities on multilingual information access systems have studied cross-language information retrieval (CLIR), information extraction and factoid based question answering tasks in detail. It is believed by the research community and also previously acknowledged in a US-NSF report [Hovy et al., 1999] that a cross-language query focused summarization could play a vital role in multilingual information access, as a bridge between CLIR and machine translation. Surprisingly enough, no detailed studies exist yet on the effects of cross-linguality to query focused summarization. In this paper we study the effects of adding a cross-language dimension to query focused multi-document summarization for the Telugu-English language pair. We use a cross-lingual relevance based language modeling approach to generate extraction based summary. We evaluate the system using DUC1 2005 dataset using ROUGE2 metrics and compare with the mono-lingual baseline which uses relevance based language modeling in mono-lingual setting.

Abstract

Commonly used vocabulary in Indian language documents found on the web contain a number of words that have Sanskrit, Persian or English origin. However, such words may be written in different scripts with slight variations in spelling and morphology. In this paper we explore fuzzy string matching techniques to exploit this situation of relatively large number of cognates among Indian languages, which are higher when compared to an Indian language and a non-Indian language. We present an approach to identify cognates and make use of them for improving dictionary based CLIR when the query and documents both belong to two different Indian languages. We conduct experiments using a Hindi document collection and a set of Telugu queries and report the improvement due to cognate recognition and translation.

Abstract

This chapter describes the role of ontologies and corporate taxonomies in managing the content and knowledge within organizations. Managing content in a reusable and effective manner is becoming increasingly important in knowledge centric organizations as the amount of content generated, both text based and rich media, is growing exponentially. Search, categorization and document characterization, content staging and content delivery are the key technology challenges in knowledge management systems. This chapter describes how corporate taxonomies and ontologies can help in making sense of huge amount of content that gets generated across the locations in different languages and formats Different information silos can be connected and workflow and collaboration can be achieved using ontologies. As the KM solutions are moving from a centralized approach to a distributed approach, a framework where multiple taxonomies and ontologies can co-exist with uniform interfaces is needed.

Abstract

An indexing model is the heart of an Information Retrieval (IR) system. Data structures such as term based inverted indices have proved to be very effective for IR using vector space retrieval models. However, when functional aspects of such models were tested, it was soon felt that better relevance models were required to more accurately compute the relevance of a document towards a query. It was shown that language modeling approaches [1] in monolingual IR tasks improve the quality of search results in comparison with TFIDF [2] algorithm. The disadvantage of language modeling approaches when used in monolingual IR task as suggested in [1] is that they would require both the inverted index (term-todocument) and the forward index (document-to-term) to be able to compute the rank of document for a given query. This calls for an additional space and computation overhead when compared to inverted index models. Such a cost may be acceptable if the quality of search results are significantly improved. In a Cross-lingual IR (CLIR) task, we have previously shown in [3] that using a bilingual dictionary along with term co-occurrence statistics and language modeling approach helps improve the functional IR performance. However, no studies exist on the performance overhead in a CLIR task due to language modeling. In this paper we present an augmented index model which can be used for fast retrieval while having the benefits of language modeling in a CLIR task. The model is capable of retrieval and ranking with or without query expansion techniques using term collocation statistics of the indexed corpus. Finally we conduct performance related experiments on our indexing model to determine the cost overheads on space and time

Abstract

Search engines today often return a large volume of results with possibly a few relevant results. The notion of relevance is subjective and depends on the user and context. Re-ranking of the results to reflect the most relevant results to the user using the relevance feedback has received wide attention in information retrieval in recent years. Also, sharing of information among users having similar interests using collaborative filtering techniques has achieved wide success in recommendation systems. In this paper, we propose a novel approach for re-ranking of the search results using collaborative filtering techniques using relevance feedback of a given user as well as the other users. Our approach is to learn the profiles of the users using machine learning techniques making use of past browsing histories including queries posed and documents found relevant or irrelevant. Re-ranking of the results is done using collaborative filtering techniques. First, the context of the query is inferred from the query category. The user’s community is determined dynamically in the context of the query by using the user profiles. The rank of a document is calculated using the user’s profile as well profiles of the other users in the community

Abstract

In this paper, we have considered a real world information synthesis task, generation of a fixed length multi document summary which satisfies a specific information need. This task was mapped to a topic-oriented, informative multi-document summarization. We also tried to estimate, given the human written reference summaries and the document set, the maximum performance (ROUGE1 scores) that can be achieved by an extraction-based summarization technique. Motivated by the observation that the current approaches are far behind the estimated maximum performance, we have looked at Information Retrieval techniques to improve the relevance scoring of sentences towards information need. Following information theoretic approach we have identified a measure to capture the notion of importance or prior of a sentence. Following a different decomposition of Probability Ranking Principle, the calculated importance/prior is incorporated into the final sentence scoring by weighted linear combination. In order to evaluate the performance, we have explored information sources like WWW and encyclopedia in computing the information measure in a set of different experiments. The t-test analysis of the improvement on DUC2 2005 data set is found to be significant (p∼ 0.05). The same system has outperformed rest of the systems at DUC 2006 challenge in terms of ROUGE scores with a significant margin over the next best system.

Abstract

Software engineering (SE) educators have been advocating the use of non-conventional approaches for SE education since long. In this context, we conducted action-research to compare the effectiveness of a case study approach with conventional lecture based approach. We designed and taught a first course in SE, that involved case study approach as well as the traditional lecture based approach. We recorded and analyzed student's perception of learning over using well defined parameters that reconciled with cognitive, skills and metacognitive goals of SE education. Results corroborated that case study approach is more effective and interesting for learning SE than the lecture based approach. These results indicate that academia and industry should further explore learning-by-doing paradigm, specially the case studies. Benefits of approach include bridging of the industry- academia gap and creation of professionals who are well versed with theory and practice and have experienced the intricacies of real software development even before entering the industry. This paper provides empirical data to support that case study approach is very effective in SE education and hence useful for curriculum designers. This work is useful for SE educators and researchers as it describes methodology for rigorous scientific educational researc

Abstract

This paper presents the experiments of Language Technologies Research Centre (LTRC) 1 as part of their participation in CLEF2 2007 Indian language to English ad-hoc cross language document retrieval task. In this paper we discuss our Hindi and Telugu to English CLIR system and the experiments using CLEF 2007 dataset. We used a variant of TFIDF algorithm in combination with a bilingual lexicon for query translation. We also explored the role of a document summary in fielded queries and two different boolean formulations of query translations. We find that a hybrid boolean formulation using a combination of boolean AND and boolean OR operators improves ranking of documents. We also find that simple disjunctive combination of translated query

Abstract

Today web search engines provide the easiest way to reach information on the web. In this scenario, more than 95% of Indian language content on the web is not searchable due to multiple encodings of web pages. Most of these encodings are proprietary and hence need some kind of standardization for making the content accessible via a search engine. In this paper we present a search engine called WebKhoj which is capable of searching multi-script and multi-encoded Indian language content on the web. We describe a language focused crawler and the transcoding processes involved to achieve accessibility of Indian langauge content. In the end we report some of the experiments that were conducted along with results on Indian language web content.

Abstract

The task in Document Understanding Conferences (DUC1) 2006 is to generate a fixed length, user oriented, multi document summary, which remains same as that of DUC 2005. We have used two features to score the sentences based. The sentences are picked to form the summary based on the calculated score. The first feature is a query dependent scoring of a sentence which is an improvement over the HAL feature. The second feature is based on the observation that sentence importance, which is independent of the query, needs to be captured in the current approaches. We have explored the use of web in scoring the sentences in a query independent manner. Experiments show a performance gain of 6-7% over HAL feature by the inclusion of two new features. Our summarization system was ranked 1st in all automatic evaluations with significant margin from second best system, 5th in responsiveness and 9th in linguistic quality evaluations in DUC 2006. Relatively lower performance in linguistic quality can be attributed to the stripped off sentences at the end of summary, when the summary length is exceeding 250 words.