Abstract

Prior works have addressed the problem of driver intention prediction (DIP) by identifying maneuvers after their onset. On the other hand, early anticipation is equally important in scenarios that demand a preemptive response before a maneuver begins. However, there is no prior work aimed at addressing the problem of driver action anticipation before the onset of the maneuver, limiting the ability of the advanced driver assistance system (ADAS) for early maneuver anticipation. In this work, we introduce Anticipating Driving Maneuvers (ADM), a new task that enables driver action anticipation before the onset of the maneuver. To initiate research in ADM task, we curate Driving Action Anticipation Dataset, DAAD, that is multi-view: in- and out-cabin views in dense and heterogeneous scenarios, and multimodal: egocentric view and gaze information. The dataset captures sequences both before the initiation and during the execution of a maneuver. During dataset collection, we also ensure to capture wide diversity in traffic scenarios, weather and illumination, and driveway conditions. Next, we propose a strong baseline based on a transformer architecture to effectively model multiple views and modalities over longer video lengths. We benchmark the existing DIP methods on DAAD and related datasets. Finally, we perform an ablation study showing the effectiveness of multiple views and modalities in maneuver anticipation

Abstract

This paper presents the competition report on Reading Documents through Aria Glasses (ICDAR 2024 RDTAG) held at the 18th International Conference on Document Analysis and Recognition (ICDAR 2024). From a mixed reality perspective, understanding the text in the world is of paramount importance. However, all day long, always on, machine perception devices like Aria Glasses pose a unique primary challenge of lower resolution due to their power and sensor constraints. Moreover, diverse everyday scenes like variations in the lighting conditions and reading positions further complicate the reading tasks. To address this, we propose a new dataset and a challenge. Specifically, we propose three novel tasks: Isolated Word Recognition in Low Resolution (Task A), Prediction of Reading Order (Task B), and Page Level Recognition (Task C). We provide new training and test sets consisting of document images captured by Aria Glasses while reading diverse documents in English under various everyday scenarios. Our aim is to engage researchers with prior experience in English language OCR, and to establish benchmarks contributing to the academic literature in this field. A total of thirty-three different teams from around the world registered for this competition, and twelve teams submitted their results along with algorithm details. The winning team, SRCB, achieved a 97.23% Character Recognition Rate (CRR) and a 90.45% Word Recognition Rate (WRR) for Task A: Isolated Word Recognition in Low Resolution. Team Gang-of-N won Task B: Prediction of Reading Order with a BLEU score of 0.0939. Team SRCB also won Task C: Page Level Recognition and Reading with a 77.44% average Page Level Character Recognition Rate (PCRR) and a 50.55% average Page Level Word Recognition Rate (PWRR).

Abstract

This paper presents a competition report on the Recognition and Visual Question Answer on Handwritten Documents towards deeper understanding of handwritten multilingual documents (ICDAR 2024-HWD) held at the 18th International Conference on Document Analysis and Recognition (ICDAR 2024). Documents are in English or Indian Languages. Earlier editions related to recognition of Indian handwriting were held in conjunction with ICFHR 2022 and ICDAR 2023. A related DocVQA task was held in DAS 2020. This edition proposes three main tasks: Isolated Word Recognition (Task A), Page Level Recognition and Reading (Task B), and Visual Question Answers on Handwritten Documents (Task A). While Task A was already part of our previous competitions, we bring in new data as part of this edition. Task B and Task C are novel additions for this year. By attracting researchers with experience in printed and handwritten documents, we aim to establish benchmarks that significantly contribute to the academic literature in this field. A total of thirty-two teams around the world registered for this competition. Among them, only ten teams submitted their results along with algorithm details. The winning team, TSNUK, achieved an average 98.00% Character Recognition Rate (CRR) and 94.26% Word Recognition Rate (WRR) across four languages for Task A: Isolated Word Recognition. IndependentOCR excelled in Task B: Page Level Recognition and Reading, with 76.32% average Page Level Character Recognition Rate (PCRR) and 62.57% average Page Level Word Recognition Rate. The team PA_VCG won Task C: Visual Question Answering on Handwritten Documents with a 0.643 ANLS score.

Abstract

Extensive research and the development of benchmark datasets have primarily focused on Scene Text Recognition (STR) in Latin languages. However, the scenario differs for Indian languages, where the complexities in syntax and semantics have posed many challenges, resulting in limited datasets and comparatively less research in this domain. Overcoming these challenges is crucial for advancing scene text recognition in Indian languages. Although a few works have touched upon this issue, they are constrained in the size and scale of the data as far as we know. To bridge this gap, this paper introduces a large scale, diverse dataset, named as IIIT-IndicSTR-Word for Indic scene text. Comprising a total of 250K word level images in ten different languages—Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Odia, Punjabi, Tamil, and Telugu, these images are extracted from roadside scenes captured by a GoPro camera. The dataset encompasses a wide array of realistic adversarial conditions, including blur, changes in illumination, occlusion, non-iconic texts, low resolution, and perspective text. We establish a baseline for the proposed dataset, facilitating evaluation and benchmarking with a specific focus on STR tasks. Our findings indicate that our dataset is a practical training source to enhance performance on respective datasets

Abstract

The practical applications of Handwritten Text Recognition (HTR) have flourished with many successful commercial APIs, solutions, and diverse use cases. Despite the availability of numerous industrial solutions, academic research in HTR, particularly for English, has been hindered by the scarcity of publicly accessible data. To bridge this gap, this paper introduces IIIT-HW-English-Word, a large and diverse collection of offline handwritten English documents. This dataset comprises unconstrained camera-captured images featuring 20,800 handwritten documents crafted by 1,215 writers. Within this dataset, covering 757,830 words, we identify 174,701 unique words encompassing a variety of content types, such as alphabetic, numeric, and stop-words. We also establish a baseline for the proposed dataset, facilitating evaluation and benchmarking, explicitly focusing on word recognition tasks. Our findings suggest that our dataset can effectively serve as a training source to enhance performance on respective datasets

Abstract

Charts are tools for data communication used in a wide range of documents. Recently, the pattern recognition community has shown interest in developing methods for automatically processing charts found in the wild. Following previous efforts on ICPR’s CHART-Infographics competitions, here we propose a newer, larger dataset and benchmark for analyzing and recognizing charts. Inspired by the steps required to make sense of a chart image, the benchmark is divided into 7 different tasks: chart image classification, chart text detection and recognition, text role classification, axis analysis, legend analysis, data extraction, and end-toend data extraction. We also show the performance of different baselines for the first five tasks. We expect that the increased scale of the proposed dataset will enable the development of better chart recognition systems.

Abstract

We present Ego-Exo4D, a diverse, large-scale multimodal multiview video dataset and benchmark challenge. Ego-Exo4D centers around simultaneously-captured egocentric and exocentric video of skilled human activities (e.g., sports, music, dance, bike repair). 740 participants from 13 cities worldwide performed these activities in 123 different natural scene contexts, yielding long-form captures from 1 to 42 minutes each and 1,286 hours of video combined. The multimodal nature of the dataset is unprecedented: the video is accompanied by multichannel audio, eye gaze, 3D point clouds, camera poses, IMU, and multiple paired language descriptions—including a novel “expert commentary” done by coaches and teachers and tailored to the skilled-activity domain. To push the frontier of first-person video understanding of skilled human activity, we also present a suite of benchmark tasks and their annotations, including fine-grained activity understanding, proficiency estimation, cross-view translation, and 3D hand/body pose. All resources are open sourced to fuel new research in the community.

Abstract

The efective management of brain tumors relies on precise typing, subtyping, and grading. We present the IPD-Brain Dataset, a crucial resource for the neuropathological community, comprising 547 high-resolution H&E stained slides from 367 patients for the study of glioma subtypes and immunohistochemical biomarkers. Scanned at 40x magnifcation, this dataset is one of the largest in Asia, specifcally focusing on the Indian demographics. It encompasses detailed clinical annotations, including patient age, sex, radiological fndings, diagnosis, CNS WHO grade, and IHC biomarker status (IDH1R132H, ATRX and TP53 along with proliferation index, Ki67), providing a rich foundation for research. The dataset is open for public access and is designed for various applications, from machine learning model training to the exploration of regional and ethnic disease variations. Preliminary validations utilizing Multiple Instance Learning for tasks such as glioma subtype classifcation and IHC biomarker identifcation underscore its potential to signifcantly contribute to global collaboration in brain tumor research, enhancing diagnostic precision and understanding of glioma variability across diferent populations.

Abstract

India’s linguistic diversity is a testament to the cultural richness of the subcontinent, with 22 officially recognized languages, each possessing its unique script and identity. In this context, the need for efficient Optical Character Recognition (OCR) tools tailored to the complexities of these languages is paramount. IndicOCR is introduced as an innovative OCR pipeline designed to address this challenge. This paper aims to shed light on the capabilities of IndicOCR, underlining its role in bridging the gap between India’s linguistic heritage and the ever-expanding digital world. Finally, IndicOCR1 is a powerful tool for inclusivity, preservation, and progress in a linguistically diverse society.

Abstract

With the recent advancements, organizations have brought data to the forefront of their digital transformation journeys. Financial services industry is also moving towards adopting data-driven strategies for improved and faster decision making and providing enhanced customer experience. While advances generally in Artificial Intelligence (AI) and specifically in Machine Learning (ML) have fueled a lot of analytics, it is largely restricted to structured data as it is well organized and is easy to work with. This tutorial presents the opportunities to unlock the potential in unstructured documents in financial domain. These forms of data are more challenging to interpret, but can deliver a more comprehensive and holistic understanding of the bigger picture. While there are challenges around processing such document, ability to quickly make decisions by leveraging such data can provide differentiated value propositions and competitive benefits. This tutorial start with select problems & challenges in Document AI space and use cases involving such documents, show the business opportunities present and describe the technical challenges involved. Subsequently, we discuss techniques and algorithms for several document processing requirements and real world applications. References