Abstract

Hallucinations in large language models(LLMs) have recently become a significantproblem. A recent effort in this directionis a shared task at Semeval 2024 Task 6,SHROOM, a Shared-task on Hallucinationsand Related Observable Overgeneration Mis-takes. This paper describes our winning so-lution ranked 1st and 2nd in the 2 sub-tasksof model agnostic and model aware tracks re-spectively. We propose a meta-regressor basedensemble of LLMs based on a random forestalgorithm that achieves the highest scores onthe leader board. We also experiment with var-ious transformer based models and black boxmethods like ChatGPT, Vectara, and others. Inaddition, we perform an error analysis com-paring ChatGPT against our best model whichshows the limitations of the former

Abstract

Wikipedia is a highly essential platform because of its informative, dynamic, and easily accessible nature. To identify topics/titles warranting their own Wikipedia article, editors of Wikipedia defined "Notability" guidelines. So far notability is enforced by humans, which makes scalability an issue. There has been no significant work on Notability determination for titles with complex category dependencies. We design a mechanism to identify such titles. We construct a dataset with 9k such titles and propose a category-agnostic approach utilizing Graph neural networks, for their notability determination. Our system outperforms machine learning-based, transformer-based classifiers and entity salience methods. It provides a scalable alternative for notability detection.

Abstract

Given a retail transactional database, the objective of high-utility pattern mining is to discover high-utility itemsets (HUIs), i.e., itemsets that satisfy a user-specified utility threshold. In retail applications, when purchasing a set of items (i.e., itemsets), consumers seek to replace or substitute items with each other to suit their individual preferences (e.g., Coke with Pepsi, tea with coffee). In practice, retailers, too, require substitutes to address operational issues like stockouts, expiration, and other supply chain constraints. The implication is that items that are interchangeably purchased, i.e., substitute goods, are critical to ensuring both user satisfaction and sustained retailer profits. In this regard, this work presents (i) an efficient model to identify HUIs containing substitute goods in place of items that require substitution, (ii) the SubstiTution-based Itemset indeX (STIX) to retrieve HUIs containing substitutes, and (iii) an experimental study to depict the benefits of the proposed approach w.r.t. a baseline method.

Abstract

This study proposed a nonparametric copula hazard framework in the joint risk of river water temperature (RWT) and low flow (LF) events for aquatic ecosystems, specifically ectotherm fish. This nonparametric copula density can adapt to any mutual dependence structure, providing greater flexibility. This can reduce the risk of misspecification if the underlying assumption is violated compared to conventional parametric or semiparametric copula settings. The analysis uses nonparametric copulas densities like Beta kernel copula estimator (BKCE), Bernstein copula estimator (BCE), and Transformation kernel estimator (TKE), conjoined with Gaussian Kernel density estimations (GKDEs) and parametric marginals for joint annual maximum RWT (AMRWT) and LF. The study compares different models for analyzing five Swiss river basins: parametric copulas with best-fitted GKDE and parametric margins, versus nonparametric copula. Six bandwidth selectors are estimated to fit GKDE. BKCE with GKDE margins outperformed most stations, while TKE and BCE density with GKDE margins work best for only one station. However, at one station, BKCE with best-fitted parametric margins is outperformed. All stations except Station 2473 are characterized by temperature and low flows that may be conducive to stress of a number of aquatic species, causing high AMRWT (exceeding 19 °C) and minimum LF or specific discharge, SD) quantiles at low AND-joint return periods (RP). AND (i.e. low flow AND high temperature) hazard events are less likely to occur together than OR hazard events, while univariate RPs happen more often than OR-joint RPs. In addition, the joint RP of AMRWT, given LF at different percentiles, significantly affected AMRWT for various LF conditions. Higher AMRWT with low flow conditions results in lower joint RP (except station 2473), and this stress level would be reduced when conditioned with high LF events at the same AMRWT. Station 2473 has high LF even at low percentiles, and with low AMRWT makes it very less stressful than other station under different LF conditions. Summer river flow maintenance can improve aquatic environments with high RWT. Analyzing joint statistics is crucial to understanding mutual risk in freshwater ecosystems.

Abstract

Background The authors had previously developed AnaVu, a low-resource 3D visualization tool for stereoscopic/ monoscopic projection of 3D models generated from pre-segmented MRI neuroimaging data. However, its utility in neuroanatomical education compared to conventional methods (specifically whether the stereoscopic or mono- scopic mode is more effective) is still unclear. Methods A three-limb randomized controlled trial was designed. A sample (n = 152) from the 2022 cohort of MBBS students at Government Medical College, Thiruvananthapuram (GMCT), was randomly selected from those who gave informed consent. After a one-hour introductory lecture on brainstem anatomy and a dissection session, students were randomized to three groups (S – Stereo; M – Mono and C – Control). S was given a 20-min demonstration on the brainstem lesson module in AnaVu in stereoscopic mode. M was given the same demonstration, but in mono- scopic mode. The C group was taught using white-board drawn diagrams. Pre-intervention and post-intervention tests for four domains (basic recall, analytical, radiological anatomy and diagram-based questions) were conducted before and after the intervention. Cognitive loads were measured using a pre-validated tool. The groups were then swapped -S→ M, M →S and C→S, and they were asked to compare the modes. Results For basic recall questions, there was a statistically significant increase in the pre/post-intervention score difference of the S group when compared to the M group [p = 0.03; post hoc analysis, Bonferroni corrections applied] and the C group [p = 0.001; ANOVA test; post hoc analysis, Bonferroni corrections applied]. For radiological anatomy questions, the difference was significantly higher for S compared to C [p < 0.001; ANOVA test; post hoc analysis, Bon- ferroni corrections applied]. Cognitive load scores showed increased mean germane load for S (33.28 ± 5.35) and M

Abstract

Drug-Drug Interactions (DDI) can trigger unexpected pharmacological consequences, including adverse drug events (ADE). The rise in polypharmacy underscores the importance of understanding how different drug molecules influence each other’s pharmacological activities and necessitates the investigation of potential interactions between newly developed drugs and existing medications. Traditional laboratory methods for DDI detection are time-consuming, making the development of computational prediction methods crucial. This study introduces GraphDDI, a machine learning method utilizing Graph Neural Networks (GNN) to predict DDI accurately. The proposed methodology, trained end-to-end, comprises three stages. (1) Featurization stage: A GNN extracts atomic features from two drugs separately. (2) Interaction stage: an interaction map is calculated between all atom pairs of the drugs. (3) Prediction stage: the model combines the interaction map and drug features to create a unified representation of the drug molecules. Subsequently, the model concatenates these representations and employs a feed-forward neural network to predict the DDI. We demonstrate the efficacy of our proposed model in predicting both the presence of DDI and the specific types of interactions (DDI events). Comparative analysis reveals that our framework surpasses existing models, achieving an F1 score of 0.98 in predicting the existence of drug-drug interactions and 0.90 in categorizing DDI event types. The code is available in our GitHub repository (https://github.com/devalab/GraphDDI).

Abstract

A system and method for establishing a secure communication agreement between an initiating unit 102 and a responding unit 104 through a channel 106 are provided. The method includes generating, using first set of local random numbers, multi-dimensional matrices by an initiating unit. The method includes computing a transcript by the initiating unit based on multi-dimensional matrices and communicating to the responding unit. The method includes determining response by selecting a subset of the transcript to communicate response to the initiating unit. The method includes iteratively generating, communicating a subsequent transcript and waiting for a subsequent response from the responding unit for one or more times. The method includes computing a secret key as a function of the first set of local random numbers, the second set of local numbers, the subsequent transcripts, the subsequent responses

Abstract

Visual Question Answering (VQA) with multiple choice questions enables a vision-centric evaluation of Multimodal Large Language Models (MLLMs). Although it reliably checks the existence of specific visual abilities, it is easier for the model to select an answer from multiple choices (VQA evaluation) than to generate the answer itself. In this work, we offer a novel perspective: we evaluate how well an MLLM understands a specific visual concept by its ability to uniquely describe two extremely similar images that differ only in the targeted visual concept. Specifically, we assess the ability of MLLMs to capture specific points of visual differences using self-retrieval, i.e., by retrieving the target image using its generated caption against the other image in the pair serving as the distractor. We curate 247 highly similar image pairs as part of the D3 benchmark. For each image pair, the model is prompted to: (1) Detect a specific visual difference, and (2) Describe the target image uniquely such that it (3) Discriminates the target image from the distractor. Self-retrieval within D3 enables whitebox evaluation across six different visual patterns, revealing that current models struggle to independently discern fine-grained visual differences, with open-source models failing to outperform random guess.

Abstract

The COVID-19 pandemic wrought havoc across India, particularly during its devastating second and third waves. This study undertakes a crucial epidemiological analysis of these waves, leveraging actual variant count data. Given limited sequencing efforts, variant information is sparse, prompting a novel approach to scaling up with actual case data. Employing a multi-strain variant-level SEIRS model tailored to each Indian state, we modeled the disease’s propagation. We report the estimated parameters of the SEIRS models for the two waves separately. Notably, the transmission coefficients ( ) were estimated to be 0.12 for Kappa, 0.35 for Delta, and 0.38 for Omicron. These coefficients signify the contagiousness of each variant, offering critical information for understanding and managing the pandemic’s dynamics. The findings hold significant implications for public health strategies, emphasizing the urgency of comprehensive variant tracking and proactive measures to mitigate the impact of evolving viral strains.

Abstract

Machine Translation for low-resource languages presents significant challenges, primarily due to limited data availability. We have a baseline model and a primary model. For the baseline model, we first fine-tune the mBART model (mbart-large-50-many-to-many-mmt) for the language pairs English-Khasi, Khasi-English, English-Manipuri, and Manipuri-English. We then augment the dataset by back-translating from Indic languages to English. To enhance data quality, we fine-tune the LaBSE model specifically for Khasi and Manipuri, generating sentence embeddings and applying a cosine similarity threshold of 0.84 to filter out low-quality back-translations. The filtered data is combined with the original training data and used to further fine-tune the mBART model, creating our primary model. The results show that the primary model slightly outperforms the baseline model, with the best performance achieved by the English-to-Khasi (en-kh) primary model, which recorded a BLEU score of 0.0492, a chrF score of 0.3316, and a METEOR score of 0.2589 (on a scale of 0 to 1), with similar results for other language pairs.