Abstract

We introduce GPNN-CAM, a novel method for CNN explanation, that bridges two distinct areas of computer vision: Image Attribution, which aims to explain a predictor by highlighting image regions it finds important, and Single Image Generation (SIG), that focuses on learning how to generate variations of a single sample.GPNN-CAM leverages samples generated by Generative Patch Nearest Neighbors (GPNN) into a Class Activation Map (CAM) flavored attribution scheme. Our findings reveal that the incorporation of these samples yields remarkably effective results, enabling GPNN-CAM to demonstrate superior performance across multiple classifier architectures, and datasets.

Abstract

Air quality estimation through sensor-based methods is widely used. Nevertheless, their frequent failures and maintenance challenges constrain the scalability of air pollution monitoring efforts. Recently, it has been demonstrated that air quality estimation can be done using image-based methods. These methods offer several advantages including ease of use, scalability, and low cost. However, the accuracy of these methods hinges significantly on the diversity and magnitude of the dataset utilized. The advancement of air quality estimation through image analysis has been limited due to the lack of available datasets. Addressing this gap, we present TRAQID - Traffic-Related Air Quality Image Dataset, a novel dataset capturing 26,678 front and rear images of traffic alongside co-located weather parameters, multiple levels of Particulate Matters (PM) and Air Quality Index (AQI) values. Spanning over multiple seasons, with over 70 hours of data collection in the twin cities of Hyderabad and Secunderabad, India, the TRAQID offers diverse day and night imagery amid unstructured traffic conditions, encompassing six AQI categories ranging from “Good” to “Severe”. State-of-the-art air quality estimation techniques, which were trained on a smaller and less-diverse dataset, showed poor results on the dataset presented in this paper. TRAQID models various uncertainty types, including seasonal changes, unstructured traffic patterns, and lighting conditions. The information from the two views (front and rear) of the traffic can be combined to improve the estimation performance in such challenging conditions. As such, the TRAQID serves as a benchmark for image-based air quality estimation tasks and AQI prediction, given its diversity and magnitude.

Abstract

Forensic sketch-to-mugshot matching is a challenging task in face recognition, primarily hindered by the scarcity of annotated forensic sketches and the modality gap between sketches and photographs. To address this, we propose CLIP4Sketch, a novel approach that leverages diffusion models to generate a large and diverse set of sketch images, which helps in enhancing the performance of face recognition systems in sketch-to-mugshot matching. Our method utilizes Denoising Diffusion Probabilistic Models (DDPMs) to generate sketches with explicit control over identity and style. We combine CLIP and Adaface embeddings of a reference mugshot, along with textual descriptions of style, as the conditions to the diffusion model. We demonstrate the efficacy of our approach by generating a comprehensive dataset of sketches corresponding to mugshots and training a face recognition model on our synthetic data. Our results show significant improvements in sketch-to-mugshot matching accuracy over training on an existing, limited amount of real face sketch data, validating the potential of diffusion models in enhancing the performance of face recognition systems across modalities. We also compare our dataset with datasets generated using GAN-based methods to show its superiority.

Abstract

Portrait sketching involves capturing identity specific attributes of a real face with abstract lines and shades. Unlike photo-realistic images, a good portrait sketch generation method needs selective attention to detail, making the problem challenging. This paper introduces textbf{Portrait Sketching StyleGAN (PS-StyleGAN)}, a style transfer approach tailored for portrait sketch synthesis. We leverage the semantic $W+$ latent space of StyleGAN to generate portrait sketches, allowing us to make meaningful edits, like pose and expression alterations, without compromising identity. To achieve this, we propose the use of Attentive Affine transform blocks in our architecture, and a training strategy that allows us to change StyleGAN's output without finetuning it. These blocks learn to modify style latent code by paying attention to both content and style latent features, allowing us to adapt the outputs of StyleGAN in an inversion-consistent manner. Our approach uses only a few paired examples ($sim 100$) to model a style and has a short training time. We demonstrate PS-StyleGAN's superiority over the current state-of-the-art methods on various datasets, qualitatively and quantitatively.

Abstract

In the field of biometric security, the quality assessment of fingerprint images is paramount for boosting the accuracy of fingerprint recognition systems. These systems are fundamental for the secure and efficient authentication and identification of individuals. Our research presents FRBQ (Fingerprint Recognition-Based Quality), an innovative quality metric designed to navigate the limitations of the NFIQ2 model. FRBQ exploits deep learning algorithms in a weakly supervised setting and utilizes matching scores from DeepPrint, a Fixed-Length Fingerprint Representation Model. Each score is paired with labels indicating the robustness of fingerprint image matches. However, in a fully referenced setting, these labels can be subjective, lacking a clear definition of what "image quality" inherently means. This weakly labeled approach strives to capture diverse perspectives on image quality, potentially making it a more encompassing metric. In comparison to NFIQ2, our research showcases the superior performance of the FRBQ model. It not only correlates better with recognition scores but also effectively evaluates challenging images that NFIQ2 struggles with. Validated by the esteemed FVC 2004 dataset, FRBQ proves its efficacy in fingerprint image quality assessment. This study underscores the transformative potential of AI in biometrics, emphasizing its capability to capture details that traditional methods might overlook. Our work stresses the critical role of precise quality assessment in the evolution of fingerprint recognition systems.

Abstract

Fingerprint recognition stands as a pivotal component of biometric technology, with diverse applications from identity verification to advanced search tools. In this paper, we propose a unique method for deriving robust fingerprint representations by leveraging enhancement-based pre-training. Building on the achievements of U- Net-based fingerprint enhancement, our method employs a specialized encoder to derive representations from fingerprint images in a self-supervised manner. We further refine these representations, aiming to enhance the verification capabilities. Our experimental results, tested on publicly available fingerprint datasets, reveal a marked improvement in verification performance against established self-supervised training techniques. Our findings not only highlight the effectiveness of our method but also pave the way for potential advancements. Crucially, our research indicates that it s feasible to extract meaningful fingerprint representations from de- graded images without relying on enhanced samples.

Abstract

The increasing prominence of fingerprint recognition as a biomet ric identifier has made it more vulnerable to presentation attacks, specifically non-distal attacks that exploit ridge and minutiae pat terns foundinnon-distalphalanges.Inthisstudy,wepresentpresen tation attacks through non-distal/toe prints and a state-of-the-art lightweight inverted residual network that excels at differentiating between distal and non-distal prints, providing unrivaled perfor mance in terms of accuracy, inference time, and false negative rate (FNR). Our proposed model surpasses other statistical machine learning methods, such as variable-margin SVM, and lightweight models like MobileNet v2, MobileNet v3, and ResNet18. We meticu lously evaluate our model using a diverse array of datasets, includ ing the NIST dataset, an in-house collected dataset, a toe dataset, a synthetic dataset generated by VeriFinger software, and a six-class dataset. To assess performance when only minutiae points are avail able, we develop analgorithmthat converts fingerprints to minutiae points and subsequently reconstructs fingerprints. Furthermore, we examine the ridge density of distal and non-distal prints across datasets, emphasizing their similarities and underscoring the need for advanced detection techniques. To the best of our knowledge, this study represents the first endeavor to propose a solution for presentation attack detection in non-distal phalanges. Our research demonstrate various challenges of presentation attacks, the effectiveness of our approach, which holds the potential to significantly influence the domain of finger print recognition and security. By sharing our dataset, model, and experimental details with the research community, we aim to foster further advancements in this crucial area. Upon publication, we will make our dataset and experimental details available alongside the paper

Abstract

Deployment of Face Recognition systems on the edge has seen significant growth due to advancements in hardware design and efficient neural architectures. However, tailoring SOTA Face Recognition solutions to a specific edge device is still not easy and is vastly unexplored. Although, benchmark data is available for some combinations of model, device, and framework, it is neither comprehensive nor scalable. We propose an approximation to determine the relationship between a model and its inference time in an edge deployment scenario. Using a small number of data points, we are able to predict the throughput of custom models in an explainable manner. The prediction errors are small enough to be considered noise in observations. We also analyze which approaches are most efficient and make better use of hardware in terms of accuracy and error rates to gain a better understanding of their behaviour. Related & necessary modules such as Face Anti-Spoofing are also analyzed. To the best of our knowledge, we are the first to tackle this issue directly. The data and code along with future updates to the models and hardware will be made available at https://github.com/AyanBiswas19/Resource_Efficient_FR.

Abstract

Biometric signal consisting of irrelevant or non-distinctive features can contain useful correlational properties that privacy-preserving verification schemes can exploit. While an efficient protocol for iris verification using noise has been presented, it is not applicable to other widely used modalities, i.e., face and fingerprint, since the methods of noise extraction and comparison are different. In this work, we design a verification protocol for secure dot product computation and also propose noise extraction mechanisms for face and fingerprint modalities. We evaluate the performance of the protocol on CFP, LFW, CelebA, FVC 2004 DB1A, DB2A, DB3A, and SOCOFing datasets. While the protocol exhibits a slight degradation in accuracy, it provides information-theoretic security with a practical computational complexity.

Abstract

For decades, fingerprint recognition has been prevalent for security, forensics, and other biometric applications. However, the availability of good-quality fingerprints is challenging, making recognition difficult. Fingerprint images might be degraded with a poor ridge structure and noisy or less contrasting backgrounds. Hence, fingerprint enhancement plays a vital role in the early stages of the fingerprint recognition/verification pipeline. In this paper, we investigate and improvise the encoder-decoder style architecture and suggest intuitive modifications to U-Net to enhance low-quality fingerprints effectively. We investigate the use of Discrete Wavelet Transform (DWT) for fingerprint enhancement and use a wavelet attention module instead of max pooling which proves advantageous for our task. Moreover, we replace regular convolutions with depthwise separable convolutions, which significantly reduces the memory footprint of the model without degrading the performance. We also demonstrate that incorporating domain knowledge with fingerprint minutiae prediction task can improve fingerprint reconstruction through multi-task learning. Furthermore, we also integrate the orientation estimation task to propagate the knowledge of ridge orientations to enhance the performance further. We present the experimental results and evaluate our model on FVC 2002 and NIST SD302 databases to show the effectiveness of our approach compared to previous works.