Abstract

Sign Language Production (SLP) systems can significantly improve accessibility for Deaf communities by translating spoken or written language into sign language videos. For millions of Indian Sign Language (ISL) users, such systems could bridge persistent communication gaps in education, healthcare, and public services. However, SLP in ISL is hindered by the scarcity of annotated datasets and the expressive complexity of the language. Existing ISL datasets, such as iSign cite{joshi2024isign} and ISL-CSLRT cite{elakkiya2021islcsltr}, are designed for recognition and lack gloss-levelfootnote{Gloss is word representation of sign.} annotations, forcing full-sentence modeling that struggles to generalize to new constructions.

We propose an interpolation-based approach that stitches together individual gloss-level signs using modern text-to-motion frameworks like FramePack cite{zhang2025framepack}. This produces contextually accurate and grammatically consistent sign sequences while retaining fine-grained articulation. To support this pipeline, we curate a 1,000-sentence ISL dataset, translated by a native interpreter and annotated with precise gloss boundaries. Results show that with high-quality gloss-level supervision, interpolation-based synthesis offers a practical, scalable path for inclusive SLP in low-resource sign languages like ISL. The project page can be found here url{https://cvit.iiit.ac.in/research/projects/cvit-projects/towards_ssp}.

Abstract

Automated table structure recognition from input images faces significant visual and linguistic challenges, including layout variances, table densities, and the presence of empty or multi-line cells. This problem is crucial due to its extensive applications in business and scientific domains. For this purpose, we present , our end-to-end trainable model using a two-step process for table parsing: Top Down, decomposing the table image into cells, and Bottom Up, associating cells to define the complete structure by assigning row and column spans. This method leverages constraints inspired by human cognition, such as cell alignment, continuity, and non-overlapping characteristics, optimizing the objective function to preserve these attributes. We introduce a vision transformer backbone augmented with local attention tailored for high-resolution input images. We also enhance inference speed for row and column adjacency prediction by replacing sampling-based graph neural networks with a single head self attention layer. Furthermore, we propose an effective split-and-merge strategy to handle highly dense tables without altering the network architecture. Additionally, we present problem-specific training strategies that improve performance across diverse visual characteristics and support cross-domain adaptations. Finally, through our robust protocol for comprehensive dataset analysis and result interpretation, we offer clear directions for future work in automated table structure recognition, highlighting its potential for substantial advancements in practical applications.

Abstract

Financial statements are critical for corporate analysis, fraud detection, and regulatory compliance, yet their complex tabular layouts, multi-year data structures, and hierarchical relationships pose significant challenges for automated reasoning. Existing Visual Question Answering (VQA) models struggle to deliver audit-ready responses due to their inability to explicitly localize evidence and reason over structured financial data. To bridge this gap, we introduce EviFiVQA, a large-scale benchmark dataset for evidence-based financial VQA, built on real-world financial statements. Leveraging FinTabNet, our dataset contains over 1.5 million question-answer pairs spanning five reasoning categories, each meticulously annotated with bounding boxes for relevant table cells to ensure traceable and interpretable responses. EviFiVQA introduces new challenges for Large Language Models (LLMs) and Vision-Language Models (VLMs) by requiring multi-step numerical reasoning, explicit evidence localization, and hierarchical financial aggregation--all within complex, irregular table layouts. We benchmark state-of-the-art models on EviFiVQA, revealing their limitations in evidence-grounded understanding of financial statements. We release EviFiVQA to drive progress in audit-ready financial AI, enhancing transparency and reliability in AI-driven financial analytics. The dataset and code are available at https://github.com/sachinraja13/EviFiVQA

Abstract

Handwritten Text Recognition (HTR) remains a challenging task,
especially in Indic languages, due to varied handwriting styles, complex character structures, and the scarcity of annotated data. While deep learning models have shown promising results on curated benchmarks, their performance often degrades in real-world scenarios with high variability in writing. To address this, we propose a semi-supervised approach, SemiHastakshar, that leverages large-scale unlabeled word-level images collected from diverse sources across the Internet. We employ a high-confidence pseudo-labeling strategy to train on unlabeled samples iteratively. It allows the model to learn from a wider distribution of handwriting styles and improve generalization across writers. Our method demonstrates that combining a small amount of labeled data with a large unlabeled corpus leads to more robust HTR models for Indic scripts, advancing scalable recognition in low-resource settings. The code, model, and data will be publicly available for research.

Abstract

Optical Character Recognition (OCR) for Indic scripts faces severe performance drops when models are deployed in domains with unseen fonts, degraded print quality, or complex background common in historical and low-resource archives. Annotating new data for each domain is costly and often impractical.
We present a label-free adaptation framework for Indic printed OCR requiring zero manual annotations. Our method uses an iterative semi-supervised training pipeline, starting from a pre-trained OCR model and refining it with high-confidence pseudo-labels from large unlabeled word-image corpora. A confidence-based selection mechanism filters noisy predictions, enabling effective domain adaptation even under heavy image degradation. Our adapted OCR demonstrates superior performance on degraded, historically significant documents featuring diverse printed fonts and multiple Indic languages. The proposed approach enables scalable and efficient OCR adaptation across varied domains and archive conditions by eliminating the need for manual annotations. It is particularly suited for large-scale digitization projects in multilingual, resourceconstrained settings. All code and models will be publicly released to foster further research and real-world adoption.

Abstract

Breast cancer, the most common malignancy among women, requires precise detection and classification for effective treatment. Among immunohistochemistry (IHC) biomarkers, HER2 plays a critical role in guiding therapy decisions. In particular, a recent clinical trial has shown that 3-way classification of HER2 (0, low, and high) using IHC is essential for identifying patients with HER2 low expression who may benefit from new targeted therapies. However, traditional IHC classification relies on the expertise of pathologists, making it labor-intensive and prone to significant inter-observer variability. To address these challenges, this study introduces the India Pathology Breast Cancer Dataset, comprising HER2 IHC slides from 500 patients, with a primary focus on automating 3-way HER2 classification. Evaluation of multiple deep learning models revealed that an end-to-end ConvNeXt network using low-resolution IHC images achieved an F1 score of 83.52%, representing an improvement of 5.35% over patch-based methods. Class-wise F1 scores were 75.6% for HER2-0, 82.4% for HER2-low, and 91.5% for HER2-high, indicating the challenge in distinguishing HER2-0 and HER2-low cases. This study highlights the potential of simple yet effective deep learning techniques to significantly improve accuracy and reproducibility in breast cancer classification, supporting their integration into clinical workflows for better patient outcomes.

Abstract

Generative AI models, particularly Text-to-Video (T2V) systems, offer a promising avenue for transforming science education by automating the creation of engaging and intuitive visual explanations. In this work, we take a first step toward evaluating their potential in physics education by introducing a dedicated benchmark for explanatory video generation. The benchmark is designed to assess how well T2V models can convey core physics concepts through visual illustrations. Each physics concept in our benchmark is decomposed into granular teaching points, with each point accompanied by a carefully crafted prompt intended for visual explanation of the teaching point. T2V models are evaluated on their ability to generate accurate videos in response to these prompts. Our aim is to systematically explore the feasibility of using T2V models to generate high-quality, curriculum-aligned educational content--paving the way toward scalable, accessible, and personalized learning experiences powered by AI. Our evaluation reveals that current models produce visually coherent videos with smooth motion and minimal flickering, yet their conceptual accuracy is less reliable. Performance in areas such as mechanics, fluids, and optics is encouraging, but models struggle with electromagnetism and thermodynamics, where abstract interactions are harder to depict. These findings underscore the gap between visual quality and conceptual correctness in educational video generation. We hope this benchmark helps the community close that gap and move toward T2V systems that can deliver accurate, curriculum-aligned physics content at scale. The benchmark and accompanying codebase are publicly available https://github.com/meghamariamkm/PhyEduVideo.

Abstract

Imagine sitting in a presentation, trying to follow the speaker while simultaneously scanning the slides for relevant information. While the entire slide is visible, identifying the relevant regions can be challenging. As you focus on one part of the slide, the speaker moves on to a new sentence, leaving you scrambling to catch up visually. This constant back-and-forth creates a disconnect between what's being said and the most important visual elements, making it hard to absorb key details--especially in fast-paced or content-heavy presentations, as in a conference talk. This will require an understanding of slides, especially text, graphics, and layout. We introduce a method that automatically identifies and highlights the most relevant slide regions based on the speaker's narrative. By analyzing spoken content and matching it with textual or graphical elements in the slides, our approach ensures better synchronization between what you hear and what you need to attend to. We explore different ways of solving this problem and assess their success and failure cases. Analyzing multimedia documents is emerging as a key requirement for seamless understanding of content-rich videos, such as educational videos and conference talks, by reducing cognitive strain and improving comprehension. Code and dataset available at: https://github.com/meghamariamkm2002/Slide_Highlight

Abstract

Handwritten Text Recognition (HTR) presents a significant challenge in computer vision due to various factors such as individual writing styles, noise, blur, and other imperfections in the text. This challenge is further exacerbated when dealing with languages using Indian scripts, which are characterized by complex character structures, extensive character inventories, and specific cultural nuances. In this study, we address these challenges by focusing on enhancing handwritten text recognition for ten Indic languages: Hindi, Bengali, Telugu, Tamil, Gujarati, Gurumukhi, Oriya, Kannada, Malayalam, and Urdu. We aim to improve recognition accuracy by leveraging the Permuted Autoregressive Sequence Model (PARSeq), an extension of the transformer-based model. Our results demonstrate the superiority of the PARSeq model over existing approaches, particularly in achieving state-of-the-art performance across most languages. Additionally, we investigate the efficacy of transfer learning from printed text to handwritten text, revealing its potential to enhance recognition performance. The trained models and code are publicly available at https://github.com/LalithaEvani/Indic-HTR-CVIP-2024.

Abstract

Optical Character Recognition (OCR) for Indian languages is challenging due to diverse scripts, complex characters, and varied writing styles. This work presents HindiOCR-VLM, a Vision-Language Model adapted for Hindi OCR using Low-Rank Adaptation (LoRA) on a model pre-trained for Chinese and English. We explore two adaptation strategies: (i) a textit{single-stage model that directly predicts text at the page level --- bypassing intermediate word or line detection --- focused on printed documents, and (ii) a two-stage model for multi-domain scenarios (printed, handwritten, and scene text), which first detects words and then recognizes them individually. Inspired by human learning processes, we propose a progressive learning approach --- a training strategy to the textit{single-stage} model to enhance language acquisition and accelerate convergence. Leveraging the vision encoder's rich representations, our method enables effective multi-domain training. Experimental results and ablation studies show that textit{HindiOCR-VLM} handles the complexities of the Devanagari script well and outperforms domain-specific models, offering a unified and robust solution for Hindi OCR. Code and resources are available at: https://hindiocr-vlm.github.io/.