Abstract

This paper addresses a key challenge in Handwritten Text Recognition (HTR): the dependence on large volumes of labeled data. To overcome this, we propose a self-supervised learning (SSL) framework, LoGo-HTR, that minimizes labeling requirements while achieving strong recognition performance. We introduce a large-scale dataset, SSL-HWD of 10 million word-level handwritten images from diverse scanned documents, partitioned into a small labeled subset and a much larger unlabeled subset. LoGo-HTR combines a local contrastive loss for spatial consistency and a global decorrelation loss to enhance feature diversity. This dual objective enables robust, invariant, and spatially discriminative feature learning. After self-supervised pretraining, we fine-tune a transformer-based decoder using limited labeled data. Extensive experiments on standard HTR benchmarks, which include multilingual and historical data, demonstrate that, after SSL pretraining on our unlabeled dataset, our method consistently outperforms state-of-the-art approaches, even when fine-tuned using only 80% and 20% of the available labeled training data from the respective benchmarks. Ablation studies highlight the effectiveness of our dual loss design and demonstrate the potential of scalable, label-efficient handwritten text recognition. The SSL-HWD dataset and LoGo-HTR model with code are publicly available at https://logo-ssl.github.io/.

Abstract

Sign Language Production (SLP) systems can significantly improve accessibility for Deaf communities by translating spoken or written language into sign language videos. For millions of Indian Sign Language (ISL) users, such systems could bridge persistent communication gaps in education, healthcare, and public services. However, SLP in ISL is hindered by the scarcity of annotated datasets and the expressive complexity of the language. Existing ISL datasets, such as iSign cite{joshi2024isign} and ISL-CSLRT cite{elakkiya2021islcsltr}, are designed for recognition and lack gloss-levelfootnote{Gloss is word representation of sign.} annotations, forcing full-sentence modeling that struggles to generalize to new constructions.

We propose an interpolation-based approach that stitches together individual gloss-level signs using modern text-to-motion frameworks like FramePack cite{zhang2025framepack}. This produces contextually accurate and grammatically consistent sign sequences while retaining fine-grained articulation. To support this pipeline, we curate a 1,000-sentence ISL dataset, translated by a native interpreter and annotated with precise gloss boundaries. Results show that with high-quality gloss-level supervision, interpolation-based synthesis offers a practical, scalable path for inclusive SLP in low-resource sign languages like ISL. The project page can be found here url{https://cvit.iiit.ac.in/research/projects/cvit-projects/towards_ssp}.

Abstract

Automated table structure recognition from input images faces significant visual and linguistic challenges, including layout variances, table densities, and the presence of empty or multi-line cells. This problem is crucial due to its extensive applications in business and scientific domains. For this purpose, we present , our end-to-end trainable model using a two-step process for table parsing: Top Down, decomposing the table image into cells, and Bottom Up, associating cells to define the complete structure by assigning row and column spans. This method leverages constraints inspired by human cognition, such as cell alignment, continuity, and non-overlapping characteristics, optimizing the objective function to preserve these attributes. We introduce a vision transformer backbone augmented with local attention tailored for high-resolution input images. We also enhance inference speed for row and column adjacency prediction by replacing sampling-based graph neural networks with a single head self attention layer. Furthermore, we propose an effective split-and-merge strategy to handle highly dense tables without altering the network architecture. Additionally, we present problem-specific training strategies that improve performance across diverse visual characteristics and support cross-domain adaptations. Finally, through our robust protocol for comprehensive dataset analysis and result interpretation, we offer clear directions for future work in automated table structure recognition, highlighting its potential for substantial advancements in practical applications.

Abstract

Financial statements are critical for corporate analysis, fraud detection, and regulatory compliance, yet their complex tabular layouts, multi-year data structures, and hierarchical relationships pose significant challenges for automated reasoning. Existing Visual Question Answering (VQA) models struggle to deliver audit-ready responses due to their inability to explicitly localize evidence and reason over structured financial data. To bridge this gap, we introduce EviFiVQA, a large-scale benchmark dataset for evidence-based financial VQA, built on real-world financial statements. Leveraging FinTabNet, our dataset contains over 1.5 million question-answer pairs spanning five reasoning categories, each meticulously annotated with bounding boxes for relevant table cells to ensure traceable and interpretable responses. EviFiVQA introduces new challenges for Large Language Models (LLMs) and Vision-Language Models (VLMs) by requiring multi-step numerical reasoning, explicit evidence localization, and hierarchical financial aggregation--all within complex, irregular table layouts. We benchmark state-of-the-art models on EviFiVQA, revealing their limitations in evidence-grounded understanding of financial statements. We release EviFiVQA to drive progress in audit-ready financial AI, enhancing transparency and reliability in AI-driven financial analytics. The dataset and code are available at https://github.com/sachinraja13/EviFiVQA

Abstract

Handwritten Text Recognition (HTR) remains a challenging task,
especially in Indic languages, due to varied handwriting styles, complex character structures, and the scarcity of annotated data. While deep learning models have shown promising results on curated benchmarks, their performance often degrades in real-world scenarios with high variability in writing. To address this, we propose a semi-supervised approach, SemiHastakshar, that leverages large-scale unlabeled word-level images collected from diverse sources across the Internet. We employ a high-confidence pseudo-labeling strategy to train on unlabeled samples iteratively. It allows the model to learn from a wider distribution of handwriting styles and improve generalization across writers. Our method demonstrates that combining a small amount of labeled data with a large unlabeled corpus leads to more robust HTR models for Indic scripts, advancing scalable recognition in low-resource settings. The code, model, and data will be publicly available for research.

Abstract

Optical Character Recognition (OCR) for Indic scripts faces severe performance drops when models are deployed in domains with unseen fonts, degraded print quality, or complex background common in historical and low-resource archives. Annotating new data for each domain is costly and often impractical.
We present a label-free adaptation framework for Indic printed OCR requiring zero manual annotations. Our method uses an iterative semi-supervised training pipeline, starting from a pre-trained OCR model and refining it with high-confidence pseudo-labels from large unlabeled word-image corpora. A confidence-based selection mechanism filters noisy predictions, enabling effective domain adaptation even under heavy image degradation. Our adapted OCR demonstrates superior performance on degraded, historically significant documents featuring diverse printed fonts and multiple Indic languages. The proposed approach enables scalable and efficient OCR adaptation across varied domains and archive conditions by eliminating the need for manual annotations. It is particularly suited for large-scale digitization projects in multilingual, resourceconstrained settings. All code and models will be publicly released to foster further research and real-world adoption.

Abstract

Breast cancer, the most common malignancy among women, requires precise detection and classification for effective treatment. Among immunohistochemistry (IHC) biomarkers, HER2 plays a critical role in guiding therapy decisions. In particular, a recent clinical trial has shown that 3-way classification of HER2 (0, low, and high) using IHC is essential for identifying patients with HER2 low expression who may benefit from new targeted therapies. However, traditional IHC classification relies on the expertise of pathologists, making it labor-intensive and prone to significant inter-observer variability. To address these challenges, this study introduces the India Pathology Breast Cancer Dataset, comprising HER2 IHC slides from 500 patients, with a primary focus on automating 3-way HER2 classification. Evaluation of multiple deep learning models revealed that an end-to-end ConvNeXt network using low-resolution IHC images achieved an F1 score of 83.52%, representing an improvement of 5.35% over patch-based methods. Class-wise F1 scores were 75.6% for HER2-0, 82.4% for HER2-low, and 91.5% for HER2-high, indicating the challenge in distinguishing HER2-0 and HER2-low cases. This study highlights the potential of simple yet effective deep learning techniques to significantly improve accuracy and reproducibility in breast cancer classification, supporting their integration into clinical workflows for better patient outcomes.

Abstract

Generative AI models, particularly Text-to-Video (T2V) systems, offer a promising avenue for transforming science education by automating the creation of engaging and intuitive visual explanations. In this work, we take a first step toward evaluating their potential in physics education by introducing a dedicated benchmark for explanatory video generation. The benchmark is designed to assess how well T2V models can convey core physics concepts through visual illustrations. Each physics concept in our benchmark is decomposed into granular teaching points, with each point accompanied by a carefully crafted prompt intended for visual explanation of the teaching point. T2V models are evaluated on their ability to generate accurate videos in response to these prompts. Our aim is to systematically explore the feasibility of using T2V models to generate high-quality, curriculum-aligned educational content--paving the way toward scalable, accessible, and personalized learning experiences powered by AI. Our evaluation reveals that current models produce visually coherent videos with smooth motion and minimal flickering, yet their conceptual accuracy is less reliable. Performance in areas such as mechanics, fluids, and optics is encouraging, but models struggle with electromagnetism and thermodynamics, where abstract interactions are harder to depict. These findings underscore the gap between visual quality and conceptual correctness in educational video generation. We hope this benchmark helps the community close that gap and move toward T2V systems that can deliver accurate, curriculum-aligned physics content at scale. The benchmark and accompanying codebase are publicly available https://github.com/meghamariamkm/PhyEduVideo.

Abstract

Imagine sitting in a presentation, trying to follow the speaker while simultaneously scanning the slides for relevant information. While the entire slide is visible, identifying the relevant regions can be challenging. As you focus on one part of the slide, the speaker moves on to a new sentence, leaving you scrambling to catch up visually. This constant back-and-forth creates a disconnect between what's being said and the most important visual elements, making it hard to absorb key details--especially in fast-paced or content-heavy presentations, as in a conference talk. This will require an understanding of slides, especially text, graphics, and layout. We introduce a method that automatically identifies and highlights the most relevant slide regions based on the speaker's narrative. By analyzing spoken content and matching it with textual or graphical elements in the slides, our approach ensures better synchronization between what you hear and what you need to attend to. We explore different ways of solving this problem and assess their success and failure cases. Analyzing multimedia documents is emerging as a key requirement for seamless understanding of content-rich videos, such as educational videos and conference talks, by reducing cognitive strain and improving comprehension. Code and dataset available at: https://github.com/meghamariamkm2002/Slide_Highlight

Abstract

Handwritten Text Recognition (HTR) presents a significant challenge in computer vision due to various factors such as individual writing styles, noise, blur, and other imperfections in the text. This challenge is further exacerbated when dealing with languages using Indian scripts, which are characterized by complex character structures, extensive character inventories, and specific cultural nuances. In this study, we address these challenges by focusing on enhancing handwritten text recognition for ten Indic languages: Hindi, Bengali, Telugu, Tamil, Gujarati, Gurumukhi, Oriya, Kannada, Malayalam, and Urdu. We aim to improve recognition accuracy by leveraging the Permuted Autoregressive Sequence Model (PARSeq), an extension of the transformer-based model. Our results demonstrate the superiority of the PARSeq model over existing approaches, particularly in achieving state-of-the-art performance across most languages. Additionally, we investigate the efficacy of transfer learning from printed text to handwritten text, revealing its potential to enhance recognition performance. The trained models and code are publicly available at https://github.com/LalithaEvani/Indic-HTR-CVIP-2024.