Abstract

The performance for many document-based problems (OCR, Document Layout Segmentation, etc.) is typically studied in terms of a single aggregate performance measure (Intersection-Over-Union, Character Error Rate, etc.). While useful, the aggregation is a trade-off between instance-level analysis of predictions which may shed better light on a particular approach’s biases and performance characteristics. To enable a systematic understanding of instance-level predictions, we introduce DocVisor - a web-based multi-purpose visualization tool for analyzing the data and predictions related to various document image understanding problems. DocVisor provides support for visualizing data sorted using custom-specified performance metrics and display styles. It also supports the visualization of intermediate outputs (e.g., attention maps, coarse predictions) of the processing pipelines. This paper describes the appealing features of DocVisor and showcases its multi-purpose nature and general utility. We illustrate DocVisor’s functionality for four popular document understanding tasks – document region layout segmentation, tabular data detection, weakly-supervised document region segmentation and optical character recognition. DocVisor is available as a documented public repository for use by the communit

Abstract

Reservoirs are essential infrastructures in human life. It provides water supply, flood control, hydroelectric power supply, navigations, irrigation, recreation, and other functionalities. To provide these services and resources from the reservoir, it’s necessary to know the reservoir system's inflow. The Machine Learning (ML) techniques are widely acknowledged to forecast the inflow into the reservoir system. In this paper, the popular ML technique, Gradient Boosting Regressor (GBR), is used to predict the reservoir system's inflow. This technique has been applied to the Bhadra reservoir of India at a daily time scale. In this study, the effect and complex relationship of climate phenomenon indices with inflow has been considered. The considered climate phenomenon indices are (1) Arctic Oscillation (AO), (2) East Pacific/North Pacific Oscillation (EPO), (3) North Atlantic Oscillation (NAO), (4) Extreme Eastern Tropical Pacific SST (NINO1+2), (5) Eastern Tropical Pacific SST (NINO3), (6) Central Tropical Pacific SST (NINO4), (7) East Central Tropical pacific SST (NINO34), (8) Pacific North American Index (PNA), (9) Southern Oscillation Index (SOI), (10) Western Pacific Index (WP), (11) Seasonality. In this paper, different parameter settings have been discussed on the models’ performances. The analysis of the GBR method for the Bhadra reservoir includes the number of estimators, maximum depth. The results indicate that the GBR model can capture the inflow's peaks and droughts into the reservoir systems. The study demonstrates how ML methods can be used to generate accurate reservoir inflow predictions.

Abstract

Datasets for training crowd counting deep networks are typically heavy-tailed in count distribution and exhibit discontinuities across the count range. As a result, the de facto statistical measures (MSE, MAE) exhibit large variance and tend to be unreliable indicators of performance across the count range. To address these concerns in a holistic manner, we revise processes at various stages of the standard crowd counting pipeline. To enable principled and balanced

Abstract

Precise boundary annotations of image regions can be crucial for downstream applications which rely on region-class semantics. Some document collections contain densely laid out, highly irregular and overlapping multi-class region instances with large range in aspect ratio. Fully automatic boundary estimation approaches tend to be data intensive, cannot handle variable-sized images and produce sub-optimal results for aforementioned images. To address these issues, we propose BoundaryNet, a novel resizing-free approach for high-precision semi-automatic layout annotation. The variable-sized user selected region of interest is first processed by an attention-guided skip network. The network optimization is guided via Fast Marching distance maps to obtain a good quality initial boundary estimate and an associated feature representation. These outputs are processed by a Residual Graph Convolution Network optimized using Hausdorff loss to obtain the final region boundary. Results on a challenging image manuscript dataset demonstrate that BoundaryNet outperforms strong baselines and produces high-quality semantic region boundaries. Qualitatively, our approach generalizes across multiple document image datasets containing different script systems and layouts, all without additional fine-tuning. We integrate BoundaryNet into a document annotation system and show that it provides high annotation throughput compared to manual and fully automatic alternatives.

Abstract

Handwritten documents are often characterized by dense and uneven layout. Despite advances, standard deep network based approaches for semantic layout segmentation are not robust to complex deformations seen across semantic regions. This phenomenon is especially pronounced for the low-resource Indic palm-leaf manuscript domain. To address the issue, we first introduce Indiscapes2, a new large-scale diverse dataset of Indic manuscripts with semantic layout annotations. Indiscapes2 contains documents from four different historical collections and is 150% larger than its predecessor, Indiscapes. We also propose a novel deep network Palmira for robust, deformation-aware instance segmentation of regions in handwritten manuscripts. We also report Hausdorff distance and its variants as a boundary-aware performance measure. Our experiments demonstrate that Palmira provides robust layouts, outperforms strong baseline approaches and ablative variants. We also include qualitative results on Arabic, South-East Asian and Hebrew historical manuscripts to showcase the generalization capability of Palmira .

Abstract

We introduce SynSE, a novel syntactically guided generative approach for Zero-Shot Learning (ZSL). Our end-to-end approach learns progressively refined generative embedding spaces constrained within and across the involved modalities (visual, language). The inter-modal constraints are defined between action sequence embedding and embeddings of Parts of Speech (PoS) tagged words in the corresponding action description. We deploy SynSE for the task of skeleton-based action sequence recognition. Our design choices enable SynSE to generalize compositionally, i.e., recognize sequences whose action descriptions contain words not encountered during training. We also extend our approach to the more challenging Generalized Zero-Shot Learning (GZSL) problem via a confidence-based gating mechanism. We are the first to present zero-shot skeleton action recognition results on the largescale NTU-60 and NTU-120 skeleton action datasets with multiple splits. Our results demonstrate SynSE’s state of the art performance in both ZSL and GZSL settings compared to strong baselines on the NTU-60 and NTU-120 datasets.

Abstract

In many works, the problem of extractive summarisation has been framed as the problem of extracting the best summary from a given document. Many popular recent works aim to solve this by employing neural networks. And many of them are trained using a very limited scope, for example, a vast majority of the neural models are trained only using the best summary. Some also consider pruning useless summaries using other models. In this work, we show the problems that can arise when training neural models using such methods. We analyse those problems in some major milestones in Neural Extractive Summarisation. We also show and demonstrate ways to overcome them experimentally. Keywords: Extractive Summarisation · training bias

Abstract

Machine learning (ML) has been increasingly adopted due to its ability to model complex and nonlinearities between river water temperature (RWT) and its predictors (e.g., Air Temperature, AT). Most of these ML approaches have been applied using average AT without any detailed sensitivity analysis of other forms of AT (e.g., maximum and minimum). The present study demonstrates how new ML approaches, such as ridge regression (RR), K-nearest neighbors (KNN) regressor, random forest (RF) regressor, and support vector regression (SVR), can be coupled with Sobol’ global sensitivity analysis (GSA) to predict accurate RWT estimates with the most appropriate form of AT. Furthermore, the proposed ML approaches have been combined with the Ensemble Kalman Filter (EnKF), a data assimilation (DA) technique to improve the predicted values based on the measured data. The proposed modelling framework’s effectiveness is demonstrated with a tropical river system of India, Tunga-Bhadra River, as a case study. The SVR has been noted as the most robust ML model to predict RWT at a monthly time scale compared with daily and seasonal. The study demonstrates how ML methods can be coupled with a global sensitivity algorithm and DA techniques to generate accurate RWT predictions in river water quality modelling.

Abstract

A structure-based drug design pipeline involves the development of potential drug molecules or ligands that form stable complexes with a given receptor at its binding site. A prerequisite to this is finding druggable and functionally relevant binding sites on the 3D structure of the protein. Although several methods for detecting binding sites have been developed beforehand, a majority of them surprisingly fail in the identification and ranking of binding sites accurately. The rapid adoption and success of deep learning algorithms in various sections of structural biology beckons the usage of such algorithms for accurate binding site detection. As a combination of geometry based software and deep learning, we report a novel framework, DeepPocket that utilises 3D convolutional neural networks for the rescoring of pockets identified by Fpocket and further segments these identified cavities on the protein surface. Apart from this, we also propose another dataset SC6K containing protein structures submitted in the Protein Data Bank (PDB) from 1st January, 2018 till 28th February, 2020 for ligand binding site (LBS) detection. DeepPocket’s results on various binding site datasets and SC6K highlights its better

Abstract

Protein-drug interactions play important roles in many biological processes and therapeutics. Prediction of the active binding site of a protein helps discover and optimise these interactions leading to the design of better ligand molecules. The tertiary structure of a protein determines the binding sites available to the drug molecule. A quick and accurate prediction of the binding site from sequence alone without utilising the three-dimensional structure is challenging. Deep Learning has been used in a variety of biochemical tasks and has been hugely successful. In this paper, a Residual Neural Network (leveraging skip connections) is implemented to predict a protein's most active binding site. An Annotated Database of Druggable Binding Sites from the Protein DataBank, sc-PDB, is used for training the network. Features extracted from the Multiple Sequence Alignments (MSAs) of the protein generated using DeepMSA, such as Position-Specic Scoring Matrix (PSSM), Secondary Structure (SS3), and Relative Solvent Accessibility (RSA), are provided as input to the network. A weighted binary cross-entropy loss function is used to counter the substantial imbalance in the