Abstract

Modeling and understanding time remains a challenge in contemporary video understanding models. With language emerging as a key driver towards powerful generalization, it is imperative for foundational video-language models to have a sense of time. In this paper, we consider a specific aspect of temporal understanding: consistency of time order as elicited by before/after relations. We establish that six existing video-language models struggle to understand even such simple temporal relations. We then question whether it is feasible to equip these foundational models with temporal awareness without re-training them from scratch. Towards this, we propose a temporal adaptation recipe on top of one such model, VideoCLIP, based on post-pretraining on a small amount of video-text data. We conduct a zero-shot evaluation of the adapted models on six datasets for three downstream tasks which require a varying degree of time awareness. We observe encouraging performance gains especially when the task needs higher time awareness. Our work serves as a first step towards probing and instilling a sense of time in existing video-language models without the need for data and compute-intense training from scratch.

Abstract

We introduce DSAG, a controllable deep neural framework for action-conditioned generation of full body multiactor variable duration actions. To compensate for incompletely detailed finger joints in existing large-scale datasets, we introduce full body dataset variants with detailed finger joints. To overcome shortcomings in existing generative approaches, we introduce dedicated representations for encoding finger joints. We also introduce novel spatiotemporal transformation blocks with multi-head self attention and specialized temporal processing. The design choices enable generations for a large range in body joint counts (24 - 52), frame rates (13 - 50), global body movement (inplace, locomotion) and action categories (12 - 120), across multiple datasets (NTU-120, HumanAct12, UESTC, Human3.6M). Our experimental results demonstrate DSAG’s significant improvements over state-of-the-art, its suitability for action-conditioned generation at scale.

Abstract

We investigate the problem of reducing mistake severity for fine-grained classification. Fine-grained classification can be challenging, mainly due to the requirement of knowledge or domain expertise for accurate annotation. However, humans are particularly adept at performing coarse classification as it requires relatively low levels of expertise. To this end, we present a novel approach for PostHoc Correction called Hierarchical Ensembles (HiE) that utilizes label hierarchy to improve the performance of fine-grained classification at test-time using the coarse-grained predictions. By only requiring the parents of leaf nodes, our method significantly reduces avg. mistake severity while improving top-1 accuracy on the iNaturalist-19 and tieredImageNet-H datasets, achieving a new state-of-the-art on both benchmarks. We also investigate the efficacy of our approach in the semisupervised setting. Our approach brings notable gains in top-1 accuracy while significantly decreasing the severity of mistakes as training data decreases for the fine-grained classes. The simplicity and post-hoc nature of HiE render it practical to be used with any off-the-shelf trained model to improve its predictions further.

Abstract

Moving Target Defense (MTD) has emerged as a key technique in various security applications as it takes away the attacker’s ability to perform reconnaissance for exploiting a system’s vulnerabilities. However, most of the existing research in the field assumes unrealistic access to information about the attacker’s motivations and/or actions when developing MTD strategies. Many of the existing approaches also assume complete knowledge regarding the vulnerabilities of a system and how each of these vulnerabilities can be exploited by an attacker. In this work, we aim to create algorithms that generate effective Moving Target Defense strategies that do not rely on prior knowledge about the attackers. Our work assumes that the only way the defender receives information about its own reward is via interaction with the attacker in a repeated game setting. Depending on the amount of information that can be obtained from the interactions, we devise two different algorithms using multi-armed bandit formulation to identify efficient strategies. We then evaluate our algorithms using data mined from the National Vulnerability Database to showcase that they match the performance of the state-of-the-art techniques, despite using a lot less amount of information.

Abstract

Objective: Gestational diabetes mellitus (GDM) is a common glucose metabolism disease occurs in pregnancy that affects both maternal and neonatal health. Recently, increasing studies have attached importance to the relationship between growth differentiation factor 15 (GDF-15) and GDM, but the results were inconclusive. Therefore, we conducted a meta-analysis to examine the association between GDF-15 and GDM. Materials and methods: A systematical search was performed in Gene Expression Omnibus (GEO), PubMed and Google Scholar till Oct 27, 2022. We first calculated the mean and standard deviation of GDF-15 expression levels from the included eligible datasets and articles. Then, a meta-analysis was conducted to depict the difference in GDF-15 mRNA or GDF-15 protein expression between case and control groups by using conservative random effect model. Moreover, the potential publication …

Abstract

In recent years, the gamification research community has widely and frequently questioned the effectiveness of one-size-fits-all gamification schemes. In consequence, personalization seems to be an important part of any successful gamification design. Personalization can be improved by understanding user behavior and Hexad player/user type. This paper comes with an original research idea: It investigates whether users’ game-related data (collected via various gamer-archetype surveys) can be used to predict their behavioral characteristics and Hexad user types in non-game (but gamified) contexts. The affinity that exists between the concepts of gamification and gaming provided us with the impetus for running this exploratory research. We conducted an initial survey study with 67 Stack Exchange users (as a case study). We discovered that users’ gameplay information could reveal valuable and helpful information about their behavioral characteristics and Hexad user types in a non-gaming (but gamified) environment. The results of testing three gamer archetypes (i.e., Bartle , Big Five, and BrainHex ) show that they can all help predict users’ most dominant Stack Exchange behavioral characteristics and Hexad user type better than a random labeler’s baseline. That said, of all the gamer archetypes analyzed in this paper, BrainHex performs the best. In the end, we introduce a research agenda for future work. Keywords: Game · Gamification · archetypes · Bartle · Big Five · BrainHex · Prediction · User behavior · CQA · Hexad.

Abstract

In crowdsourcing systems, requesters publish tasks, and interested workers provide answers to get rewards. Worker anonymity motivates participation since it protects their privacy. Anonymity with unlinkability is an enhanced version of anonymity because it makes it impossible to “link” workers across the tasks they participate in. Another core feature of crowdsourcing systems is worker quality which expresses a worker’s trustworthiness and quantifies their historical performance. Notably, worker quality depends on the participation history, revealing information about it, while unlinkability aims to disassociate the workers’ identities from their past activity. In this work, we present AVeCQ, the first crowdsourcing system that reconciles these properties, achieving enhanced anonymity and verifiable worker quality updates. AVeCQ relies on a suite of cryptographic tools, such as zeroknowledge proofs, to (i) guarantee workers’ privacy, (ii) prove the correctness of worker quality scores and task answers, and (iii) commensurate payments. AVeCQ is developed modularly, where the requesters and workers communicate over a platform that supports pseudonymity, information logging, and payments. In order to compare AVeCQ with the state-of-theart, we prototype it over Ethereum. AVeCQ outperforms the state-of-the-art in three popular crowdsourcing tasks (image annotation, average review, and Gallup polls). For instance, for an Average Review task with 5 choices and 128 participating workers AVeCQ is 40% faster (including overhead to compute and verify the necessary proofs and blockchain transaction processing time) with the task’s requester consuming 87% fewer gas units.

Abstract

Autonomous systems are often deployed in the open world where it is hard to obtain complete specifications of objectives and constraints. Operating based on an incomplete model can produce negative side effects (NSEs), which affect the safety and reliability of the system. We focus on mitigating NSEs in environments modeled as Markov decision processes (MDPs). First, we learn a model of NSEs using observed data that contains state-action trajectories and the severity of the associated NSEs. Unlike previous works that associate NSEs with state-action pairs, our framework associates NSEs with entire trajectories, which is more general and captures non-Markovian dependence on states and actions. Second, we learn finite state controllers (FSCs) that predict the NSE severity for a given trajectory and generalize well to unseen data. Finally, we develop a constrained MDP model that uses information from both the underlying MDP and the learned FSC for planning while avoiding NSEs. Our empirical evaluation demonstrates the effectiveness of our approach in learning and mitigating Markovian and non-Markovian NSEs.

Abstract

Multilingual Automatic Speech Recognition (ASR) for Indian lan- guages is an obvious technique for leveraging their similarities. We present a detailed analysis of how phonological similarities and differences between languages affect Time Delay Neural Net- work (TDNN) and End-to-End (E2E) ASR. To study this, we select genealogically similar pairs from five Indian languages and train bilingual acoustic models. We compare these against corresponding monolingual acoustic models and find similar phoneme distribu- tions within speech to be the primary factor for improving model performance, with phoneme overlap being secondary. The influ- ence of phonological properties on performance is visible in both cases. Word Error Rate (WER) of E2E decreased by a median of 2.35%, and upto 8.5% when the phonological similarity was great- est. WER of TDNN increased by 11.69% when the similarity was lowest. Thus, it is clear that the choice of supplementary language is important for model performance. Index Terms— Phonetics and Phonology, Analysis, Automatic Speech Recognition, Low-Resource Languages, Multilingual Sys-

Abstract

Open-domain question qnswering (OpenQA) involves a retriever for selecting relevant passages from large text corpora (e.g. Wikipedia) and a reading comprehension (RC) model for extracting answers from these retrieved passages. The retrieved passages are often noisy. Since OpenQA relies heavily on efficient passages for better answer prediction, many passage ranker models have been pro- posed to filter out noisy passages. However, their performance is limited because their ranker model scores each passage separately by modelling only the relationship between query and passage. Thus, they could not capture local context information. Their ranker model also ignored the rich initial rank of passages ranked by a search engine. This paper presents a Passage Ranker model that captures local-context information through cross-passage interac- tion. Our ranker model integrates initial ranking and uses modified attention in the cross-passage interaction to compute a better confi- dence score for each passage. Moreover, we integrate SRL into our passage reader and train it on proposed sampled data. Our semantic reader can absorb contextual semantics. Experimental results on four public OpenQA datasets show that o