Abstract

This paper describes our system submission for SemEval-2023 Task 12 AfriSenti-SemEval: Sentiment Analysis for African Languages. We propose an XGBoost-based ensemble model trained on emoticon frequency-based features and the predictions of several statistical models such as SVMs, Logistic Regression, Random Forests, and BERT-based pre-trained language models such as AfriBERTa and AfroXLMR. We also report results from additional experi- ments not in the system. Our system achieves a mixed bag of results, achieving a best rank of 7th in three of the languages - Igbo, Twi, and Yoruba

Abstract

A system for generating high-resolution de-warped omni-directional stereo image from captured omni-directional stereo image by correcting optical distortions using projection patterns is provided. The system includes a projection pattern capturing arrangement, a projector or a display, and a de-warping server. The projection pattern capturing arrangement includes one or more omnidirectional cameras to capture projection patterns from the captured omni-directional stereo image from each omni-directional stereo camera. The projector or the display displays the projection patterns. The de-warping server obtain the projection patterns and processes the projection patterns to generate high resolution de-warped omni-directional stereo image by correcting optical distortions in the captured omni-directional stereo image and mapping the captured omni-directional stereo image and the high resolution de-warped omni-directional stereo image.

Abstract

Text and signs around roads provide crucial information for drivers, vital for safe navigation and situational awareness. Scene text recognition in motion is a challenging problem, while textual cues typically appear for a short time span, and early detection at a distance is necessary. Systems that exploit such information to assist the driver should not only extract and incorporate visual and textual cues from the video stream but also reason over time. To address this issue, we introduce RoadTextVQA, a new dataset for the task of video question answering (VideoQA) in the context of driver assistance. RoadTextVQA consists of 3,222 driving videos collected from multiple countries, annotated with 10,500 questions, all based on text or road signs present in the driving videos. We assess the performance of state-of-the-art video question answering models on our RoadTextVQA dataset, highlighting the significant potential for improvement in this domain and the usefulness of the dataset in advancing research on in-vehicle support systems and text-aware multimodal question answering.

Abstract

Movie story analysis requires understanding characters’ emotions and mental states. Towards this goal, we formulate emotion understanding as predicting a diverse and multi-label set of emotions at the level of a movie scene and for each character. We propose EmoTx, a multimodal Transformer-based architecture that ingests videos, multiple characters, and dialog utterances to make joint predictions. By leveraging annotations from the MovieGraphs dataset [72], we aim to predict classic emotions (e.g. happy, angry) and other mental states (e.g. honest, helpful). We conduct experiments on the most frequently occurring 10 and 25 labels, and a mapping that clusters 181 labels to 26. Ablation studies and comparison against adapted stateof-the-art emotion recognition approaches shows the effectiveness of EmoTx. Analyzing EmoTx’s self-attention scores reveals that expressive emotions often look at character tokens while other mental states rely on video and dialog cues.

Abstract

Efficiently and accurately rendering hair accounting for multiple scattering is a challenging open problem. Path tracing in hair takes long to converge while other techniques are either too approximate while still being computationally expensive or make assumptions about the scene. We present a technique to infer the higher order scattering in hair in constant time within the path tracing framework, while achieving better computational efficiency. Our method makes no assumptions about the scene and provides control over the renderer’s bias & speedup. We achieve this by training a small multilayer perceptron (MLP) to learn the higher-order radiance online, while rendering progresses. We describe how to robustly train this network and thoroughly analyze our resulting renderer’s characteristics. We evaluate our method on various hairstyles and lighting conditions. We also compare our method against a recent learning based & a traditional real-time hair rendering method and demonstrate better quantitative & qualitative results. Our method achieves a significant improvement in speed with respect to path tracing, achieving a run-time reduction of 40% −70% while only introducing a small amount of bias.

Abstract

Significant progress has been made in speaker dependent Lip-to-Speech synthesis, which aims to generate speech from silent videos of talking faces. Current state-of-the-art approaches primarily employ non-autoregressive sequence-to-sequence architectures to directly predict mel-spectrograms or audio waveforms from lip representations. We hypothesize that the direct mel-prediction hampers training/model efficiency due to the entanglement of speech content with ambient information and speaker characteristics. To this end, we propose RobustL2S, a modularized framework for Lip-to-Speech synthesis. First, a non-autoregressive sequence-to-sequence model maps self-supervised visual features to a representation of disentangled speech content. A vocoder then converts the speech features into raw waveforms. Extensive evaluations confirm the effectiveness of our setup, achieving state-of-the-art performance on the unconstrained Lip2Wav dataset and the constrained GRID and TCD-TIMIT datasets. Speech samples from RobustL2S

Abstract

With the advent of multilingual models like mBART, mT5, IndicBART etc., summarization in low resource Indian languages is getting a lot of attention now a days. But still the number of datasets is low in number. In this work, we (Team HakunaMatata) study how these multilingual models perform on the datasets which have Indian languages as source and target text while performing summarization. We experimented with IndicBART and mT5 models to perform the experiments and report the ROUGE-1, ROUGE-2, ROUGE-3 and ROUGE-4 scores as a performance metric.

Abstract

Traffic flow prediction has been regarded as a critical problem in intelligent transportation systems. An accurate prediction can help mitigate congestion and other societal problems while facilitating safer, cost and time-efficient travel. However, this requires the prediction algorithm to consider several complex characteristics of traffic flow data. These complex characteristics are an amalgamation of the spatial, temporal and periodic features exhibited by traffic flow data. To extract and leverage these features for traffic flow prediction, several hybrid deep learning models have been developed recently; however, there are still some challenges to determine the optimal architecture considering both spatial and temporal features. In this work, we perform an extensive comparison of hybrid deep learning models with and without periodicity to understand the prediction accuracy of these popular models. We propose an optimization framework that unifies genetic algorithm (GA) embedded optimization with prediction models in order to derive optimized deep learning architectures for hybrid traffic flow prediction exploring 2D spatial and temporal information. The framework enables the improvement of prediction performance and eliminates the handtuning process. An improved temporal convolutional network (TCN) architecture is derived using the GA driven optimization, which achieves superior traffic flow prediction accuracy compared to all other existing hybrid deep learning models on the freeway and urban traffic data from the PeMS traffic data set. We also evaluate the performance of the derived hybrid deep learning algorithms on the Raspberry PI embedded platform. Index Terms—Traffic flow prediction, Hybrid deep learning models, Temporal convolutional networks

Abstract

Unmanned Aerial Vehicles (UAVs), particularly low-cost UAVs, have become increasingly important due to their wide range of applications and ease of use. However, with the rapid growth of the UAV market, the rising security concerns pose a greater risk. One such primary concern is location spoofing attacks which can compromise UAV's navigation system, making it crucial to analyze various location-based attacks. In this paper, we identify 16 such GPS spoofing scenarios based on environmental conditions, attack type, and spoof signal propagation path. We evaluate these scenarios based on various GPS parameters like Horizontal Dilution Of Precision (HDOP), Vertical Dilution Of Precision (VDOP), GPS satellite count in view, and avg signal-to-noise power density (CN0). We then analyze the variations in GPS parameters for various such attack scenarios. Further, we analyze the impact of distance on average CN0 and the effect of satellite count on effective spoofable distance. We also discuss several critical insights which are empirically observed during our experimental trials. Our experiments revealed that the natural conditions within indoor and outdoor scenarios can vary considerably, and effective spoofable distance can be up to 100 meters when the satellite count is less than 10.

Abstract

This study considers simple acceleration sensitive Non-structural elements (NSEs) with fundamental period TS ≤ 0.07seconds, modelled as a rigid cantilever with lumped mass at the top (stick model). Numerical modelling of building and NSE together (complete model) and nonlinear time history analyses using 11 natural ground motions are performed in commercial software PERFORM 3D. Parametric studies of NSEs with lumped mass of 1-10 percent total seismic weight of building (in increments of 1 percent) for each fundamental period of NSE (in increments of 0.01s) is carried out, using spectrally scaled ground motions. Variations in: (a) lateral force demand on NSE, (b) Component Amplification Factor (CAF), and (c) Floor Amplification Factor (FAF), are monitored. Results are compared with the values of above parameters estimated as per EAK (2000), Eurocode 8 (1998 (1):2004), ASCE 07 (2016), NZS 4219 (2009), and NZS 1170 (5) (2004). It is observed that FAF varies nonlinearly across the floors (and significantly in the lower floors), as against linear variation stipulated in the above codes. The average CAF for the ground motions considered ranged between 1 and 2 up to Ts ≤ 0.06s. However, it increased significantly for Ts = 0.07s, thereby complementing ASCE 07 stipulation of 0.06s limit for rigid NSEs. The value of 2 is consistent with NZS 1170(5):2004 and 1 with ASCE 07 and EAK (2000). Further, as the NSE weight increased, the fundamental period of the building increased, and the average CAF marginally decreased. And, seismic force demand imposed on all NSEs during nonlinear time history analysis exceeded code recommended seismic design force under most ground motions.