Abstract

Large language models (LLMs) and generative AI have emerged as the most important areas in the field of natural language processing (NLP). LLMs are considered to be a key component in several NLP tasks, such as summarization, question-answering, sentiment classification, and translation. Newer LLMs, such as Chat- GPT, BLOOMZ, and several such variants, are known to train on multilingual training data and hence are expected to process and generate text in multiple languages. Considering the widespread use of LLMs, evaluating their efficacy in multilingual settings is imperative. In this work, we evaluate the newest generative models (ChatGPT, mT0, and BLOOMZ) in the context of Indic languages. Specifically, we consider natural language generation (NLG) applications such as summarization and questionanswering in monolingual and cross-lingual settings. We observe that current generative models have limited capability for generating text in Indic languages in a zero-shot setting. In contrast, generative models perform consistently better on manual quality-based evaluation in Indic languages and English language generation. Considering limited generation performance, we argue that these LLMs are not intended to use in zero-shot fashion in downstream applications.

Abstract

Sexism, a form of oppression based on one's sex, manifests itself in numerous ways and causes enormous suffering. In view of the growing number of experiences of sexism reported online, automatically classifying these recollections can aid in the battle against sexism by allowing gender studies researchers and government officials involved in policymaking to conduct more effective analyses. This paper investigates the 23-class fine- grained, multi-label classification of accounts (reports) of sexism.

Abstract

Clickbait is the text or a thumbnail image that entices the user to click the accompanying link. Clickbaits employ strategies while deliberately hiding the critical elements of the article and revealing partial information in the title, which arouses sufficient curiosity and motivates the user to click the link. In this work, we identify the kind of spoiler given a clickbait title. We formulate this as a text classification problem. We finetune pretrained transformer models on the title of the post and build models for theclickbait-spoiler classification. We achieve a balanced accuracy of 0.70 which is close to the baselin

Abstract

The Clickbait Challenge targets spoiling the clickbaits using short pieces of information known as spoilers to satisfy the curiosity in- duced by a clickbait post. The large context of the article associated with the clickbait and differences in the spoiler forms, make the task challenging. Hence, to tackle the large con- text, we propose an Information Condensation- based approach, which prunes down the unnec- essary context. Given an article, our filtering module optimised with a contrastive learning objective first selects the parapraphs that are the most relevant to the corresponding clickbait. The resulting condensed article is then fed to the two downstream tasks of spoiler type clas- sification and spoiler generation. We demon- strate and analyze the gains from this approach on both the tasks. Overall, we win the task of spoiler type classification and achieve competi- tive results on spoiler generation

Abstract

This paper describes our system used in the SemEval-2023 Task 11 Learning With Dis- agreements (Le-Wi-Di). This is a subjective task since it deals with detecting hate speech, misogyny and offensive language. Thus, dis- agreement among annotators is expected. We experiment with different settings like loss functions specific for subjective tasks and in- clude anonymized annotator-specific informa- tion to help us understand the level of disagree- ment. We perform an in-depth analysis of the performance discrepancy of these differ- ent modelling choices. Our system achieves a cross-entropy of 0.58, 4.01 and 3.70 on the test sets of HS-Brexit, ArMIS and MD-Agreement, respectively. Our code implementation is pub- licly available.

Abstract

Identifying human values behind arguments is a complex task which requires understanding of premise, stance and conclusion together. We propose a method that uses a pre-trained lan- guage model, DeBERTa, to tokenize and con- catenate the text before feeding it into a fully connected neural network. We also show that leveraging the hierarchy in values improves the performance by .14 F1 score compared to only using level 2 values. Our code is made publicly available here.1

Abstract

This paper describes our approach for SemEval- 2023 Task 10: Explainable Detection of Online Sexism (EDOS). The task deals with identifi- cation and categorization of sexist content into fine-grained categories for explainability in sex- ism classification. The explainable categoriza- tion is proposed through a set of three hierar- chical tasks that constitute a taxonomy of sexist content, each task being more granular than the former for categorization of the content. Our team (iREL) participated in all three hierarchi- cal subtasks. Considering the inter-connected task structure, we study multilevel training to study the transfer learning from coarser to finer tasks. Our experiments based on pretrained transformer architectures also make use of ad- ditional strategies such as domain-adaptive pre- training to adapt our models to the nature of the content dealt with, and use of the focal loss objective for handling class imbalances. Our best-performing systems on the three tasks achieve macro-F1 scores of 85.93, 69.96 and 54.62 on their respective validation sets

Abstract

With the advent of multilingual models like mBART, mT5, IndicBART etc., summarization in low resource Indian languages is getting a lot of attention now a days. But still the number of datasets is low in number. In this work, we (Team HakunaMatata) study how these multilingual models perform on the datasets which have Indian languages as source and target text while performing summarization. We experimented with IndicBART and mT5 models to perform the experiments and report the ROUGE-1, ROUGE-2, ROUGE-3 and ROUGE-4 scores as a performance metric.

Abstract

Lack of encyclopedic text contributors, especially on Wikipedia, makes automated text generation for low resource (LR) languages a critical problem. Existing work on Wikipedia text generation has focused on English only where English reference articles are sum- marized to generate English Wikipedia pages. But, for low-resource languages, the scarcity of reference articles makes monolingual summarization inefective in solving this problem. Hence, in this work, we propose XWikiGen, which is the task of cross-lingual multi-document summarization of text from multiple reference ar- ticles, written in various languages, to generate Wikipedia-style text. Accordingly, we contribute a benchmark dataset, XWikiRef, spanning ∼69K Wikipedia articles covering fve domains and eight languages. We harness this dataset to train a two-stage system where the input is a set of citations and a section title and the output is a section-specifc LR summary. The proposed system is based on a novel idea of neural unsupervised extractive summariza- tion to coarsely identify salient information followed by a neural abstractive model to generate the section-specifc text. Extensive experiments show that multi-domain training is better than the multi-lingual setup on average. We make our code and dataset publicly availableLack of encyclopedic text contributors, especially on Wikipedia, makes automated text generation for low resource (LR) languages a critical problem. Existing work on Wikipedia text generation has focused on English only where English reference articles are sum- marized to generate English Wikipedia pages. But, for low-resource languages, the scarcity of reference articles makes monolingual summarization inefective in solving this problem. Hence, in this work, we propose XWikiGen, which is the task of cross-lingual multi-document summarization of text from multiple reference ar- ticles, written in various languages, to generate Wikipedia-style text. Accordingly, we contribute a benchmark dataset, XWikiRef, spanning ∼69K Wikipedia articles covering fve domains and eight languages. We harness this dataset to train a two-stage system where the input is a set of citations and a section title and the output is a section-specifc LR summary. The proposed system is based on a novel idea of neural unsupervised extractive summariza- tion to coarsely identify salient information followed by a neural abstractive model to generate the section-specifc text. Extensive experiments show that multi-domain training is better than the multi-lingual setup on average. We make our code and dataset publicly available

Abstract

Named Entity Recognition(NER) is a task of recognizing entities at a token level in a sen- tence. This paper focuses on solving NER tasks in a multilingual setting for complex named entities.Our team, LLM-RM partici- pated in the recently organized SemEval 2023 task, Task 2: MultiCoNER II,Multilingual Complex Named Entity Recognition. We approach the problem by leveraging cross- lingual representation provided by fine-tuning XLM-Roberta base model on datasets of all of the 12 languages provided - Bangla, Chinese, English, Farsi, French, German, Hindi, Italian, Portuguese, Spanish, Swedish and Ukrainian