Abstract
                                                                        The exponential rise of online social media has enabled the creation,  distribution, and consumption of information at an unprecedented  rate. However, it has also led to the burgeoning of various forms  of online abuse. Increasing cases of online antisemitism have be-  come one of the major concerns because of its socio-political con-  sequences. Unlike other major forms of online abuse like racism,  sexism, etc., online antisemitism has not been studied much from  a machine learning perspective. To the best of our knowledge, we  present the first work in the direction of automated multimodal de-  tection of online antisemitism. The task poses multiple challenges  that include extracting signals across multiple modalities, contex-  tual references, and handling multiple aspects of antisemitism. Un-  fortunately, there does not exist any publicly available benchmark  corpus for this critical task. Hence, we collect and label two datasets  with 3,102 and 3,509 social media posts from Twitter and Gab re-  spectively. Further, we present a multimodal deep learning system  that detects the presence of antisemitic content and its specific anti-  semitism category using text and images from posts. We perform an  extensive set of experiments on the two datasets to evaluate the ef-  ficacy of the proposed system. Finally, we also present a qualitative  analysis of our study.