Abstract

Unlike traditional optoelectronic satellite imaging, Synthetic Aperture Radar (SAR) allows remote sensing applications to operate under all weather conditions. This makes it uniquely valuable for detecting ships/vessels involved in illegal, unreported, and unregulated (IUU) fishing. While recent work has shown significant improvement in this domain, detecting small objects using noisy point annotations remains an unexplored area. In order to meet the unique challenges of this problem, we propose a progressive training methodology that utilizes two different spatial sampling strategies. Firstly, we use stochastic sampling of background points to reduce the impact of class imbalance and missing labels, and secondly, during the refinement stage, we use hard negative sampling to improve the model. Experimental results on the challenging xView3 dataset show that our method outperforms conventional small object localization methods in a large, noisy dataset of SAR images.Source code for our method can be found at: https://github.com/manupillai308/DeepSAR

Abstract

Recently, diamond color defect based quantum sensing applications such as nitrogen-vacancy (NV) center magnetometry have emerged in CMOS technology, which use optically detected magnetic resonance (ODMR) for sensing magnetic field strengths (|B~|) from different environmental physical quantities. For ODMR based sensing, CMOS quantum sensors seek an onchip 2.87 GHz microwave (MW) signal generator. Moreover, in order to sense smaller |B~|, these CMOS quantum sensors also require that MW signal should be swept with sufficiently small step-size near 2.87 GHz. In this work, we present a fractional-N synthesizer based 2.87 GHz MW-generator (MWG) with an extremely small programmable sweep step-size for improved sensitivity of |B~| measurements in CMOS NV magnetometry. The proposed MWG is implemented in 180 nm CMOS technology and simulations were done to validate the proposed design. Post-layout simulation results show that the proposed MWG achieves a minimum sweep-step size of 50 kHz, which can be used to sense |B~|<0.9μT and exhibits a phase noise of −114.5 dBc/Hz at an offset of 1 MHz near 2.87 GHz center frequency.

Abstract

In vision-and-language navigation (VLN), an embodied agent is required to navigate in realistic 3D environments following natural language instructions. One major bottleneck for existing VLN approaches is the lack of sufficient training data, resulting in unsatisfactory generalization to unseen environments. While VLN data is typically collected manually, such an approach is expensive and prevents scalability. In this work, we address the data scarcity issue by proposing to automatically create a large-scale VLN dataset from 900 unlabeled 3D buildings from HM3D. We generate a navigation graph for each building and transfer object predictions from 2D to generate pseudo 3D object labels by cross-view consistency. We then fine-tune a pretrained language model using pseudo object labels as prompts to alleviate the cross-modal gap in instruction generation. Our resulting HM3D-AutoVLN dataset is an order of magnitude larger than existing VLN datasets in terms of navigation environments and instructions. We experimentally demonstrate that HM3D-AutoVLN significantly increases the generalization ability of resulting VLN models. On the SPL metric, our approach improves over state of the art by 7.1% and 8.1% on the unseen validation splits of REVERIE and SOON datasets respectively.

Abstract

In this work we outline an approach to mine insights from calls between delivery partners (DP) and customers involved in hyperlocal food delivery in India. Incorrect addresses/ locations or other impediments prompt the DPs to call customers leading to suboptimal experiences like breaches in the promised arrival-time, cancellation, fraud, etc. We demonstrate an end-to-end system that utilizes a multi-modal approach where we combine data across speech, text and geospatial domains to extract the intent behind these calls. To transcribe calls to text, we develop an Automatic Speech Recognition (ASR) engine that works in the Indian context where the calls are typically highly code-mixed (in our case Hindi and English) along with variations in dialects and pronunciations. Additionally in the hyperlocal delivery space, the calls are also corrupted by high levels of background noise due to the nature of the business. Starting with Wav2Vec2.0 as the base we carried out a series of data and model based experiments to progressively reduce the WER from 85.30% to 31.17%. The transcripts from the ASR engine are encoded into embeddings by adapting an IndicBERT based model. Features extracted from the geospatial markers of calls are concatenated with the embeddings and passed through an XGBoost classification head to classify calls into one of three intents. Through ablation studies we show incremental improvements attributable to signals from different modalities. The winning multi-modal model has a macro average precision of 68.33% which is a 29.3pp lift over the baseline not utilizing all the modalities

Abstract

Epilepsy is a common neural abnormality of the nervous system, which is constituted by recurring abnormalities, and humans with their class often suffered from disgrace and inequity. Therefore seizure detection has much importance in the medical domain for the treatment of epilepsy-affected persons. Electroencephalography signals are the most commonly used data modality for epilepsy identification. Yet, it can be challenging to proclaim the understated but severe modulation present in electroencephalogram signals. Here discrete wavelet transform is used for feature extraction, and the extracted features are classified by using an artificial neural network machine learning classifier. The 12 features are classified, and the accuracy among them is compared. To estimate the efficiency of the proposed method, different classifiers and feature extracting techniques are considered in this work. The obtained outcomes have proved that the new system obtained a promising identification accuracy of 97.24%, that is, capable of being an important approach for realistic applications curing epileptics.

Abstract

Complementary fashion recommendation aims at identifying items from different categories (e.g. shirt, footwear, etc.) that "go well together" as an outfit. Most existing approaches learn representation for this task using labeled outfit datasets containing manually curated compatible item combinations. In this work, we propose to learn representations for compatibility prediction from in-the-wild street fashion images through self-supervised learning by leveraging the fact that people often wear compatible outfits. Our pretext task is formulated such that the representations of different items worn by the same person are closer compared to those worn by other people. Additionally, to reduce the domain gap between in-the-wild and catalog images during inference, we introduce an adversarial loss that minimizes the difference in feature distribution between the two domains. We conduct our experiments on two popular fashion compatibility benchmarks - Polyvore and Polyvore-Disjoint outfits, and outperform existing self-supervised approaches, particularly significant in cross-dataset setting where training and testing images are from different sources.

Abstract

The fidelity of a pose transfer system depends on its ability to generate realistic images of a person under novel poses while preserving the desired human attributes (like face, hairstyle, and clothes). However, the visual fidelity is often compromised as the existing methods fail to extract rich appearance and pose features since they propagate the pose and the appearance information through the same pathway. Also, the repeated downsampling in these pathways leads to the loss of finer details, thus producing blurry results. Further, these methods use vanilla convolution that treats all the pixels as important and fail to focus primarily on significant regions needed for the desired transformation. This work proposes an appearance-consistent human pose transfer framework that progressively transforms the person in the source image to the desired target pose using the information from three pathways: an image pathway, a pose pathway, and an appearance pathway. We propose the use of gated convolution to dynamically extract features relevant for generating the transformed image. The appearance pathway generates an appearance code to produce an image consistent in appearance with that of the source image. We establish the efficacy of the proposed framework through an extensive set of experiments on DeepFashion, Market-1501, and the Action Class dataset. We also generate coherent action sequences through a given set of desired poses from the action class dataset that contains humans in three actions: golf,

Abstract

Due to hardware constraints, standard off-the-shelf digital cameras suffers from low dynamic range (LDR) and low frame per second (FPS) outputs. Previous works in high dynamic range (HDR) video reconstruction uses sequence of alternating exposure LDR frames as input, and align the neighbouring frames using optical flow based networks. However, these methods often result in motion artifacts in challenging situations. This is because, the alternate exposure frames have to be exposure matched in order to apply alignment using optical flow. Hence, over-saturation and noise in the LDR frames results in inaccurate alignment. To this end, we propose to align the input LDR frames using a pre-trained video frame interpolation network. This results in better alignment of LDR frames, since we circumvent the error-prone exposure matching step, and directly generate intermediate missing frames from the same exposure inputs. Furthermore, it allows us to generate high FPS HDR videos by recursively interpolating the intermediate frames. Through this work, we propose to use video frame interpolation for HDR video reconstruction, and present the first method to generate high FPS HDR videos. Experimental results demonstrate the efficacy of the proposed framework against optical flow based alignment methods, with an absolute improvement of 2.4 PSNR value on standard HDR video datasets [1], [2] and further benchmark our method for high FPS HDR video generation.

Abstract

The channel output entropy of a transmitted word is the entropy of the possible channel outputs and similarly the input entropy of a received word is the entropy of all possible transmitted words. The goal of this work is to study these entropy values for the k-deletion, k-insertion channel, where exactly k symbols are deleted, inserted in the transmitted word, respectively. If all possible words are transmitted with the same probability then studying the input and output entropies is equivalent. For both the 1-insertion and 1-deletion channels, it is proved that among all words with a fixed number of runs, the input entropy is minimized for words with a skewed distribution of their run lengths and it is maximized for words with a balanced distribution of their run lengths. Among our results, we establish a conjecture by Atashpendar et al. which claims that for the binary 1-deletion, the input entropy is maximized for the alternating words. For the 2-deletion channel, it is proved that constant words with a single run minimize the input entropy.

Abstract

The current approach to marking attendance in colleges is tedious and time consuming. I propose AttenFace, a standalone system to analyze, track and grant attendance in real time using face recognition. Using snapshots of class from live camera feed, the system identifies students and marks them as present in a class based on their presence in multiple snapshots taken throughout the class duration. Face recognition for each class is performed independently and in parallel, ensuring that the system scales with number of concurrent classes. Further, the separation of the face recognition server from the backend server for attendance calculation allows the face recognition module to be integrated with existing attendance tracking software like Moodle. The face recognition algorithm runs at 10 minute intervals on classroom snapshots, significantly reducing computation compared to direct processing of live camera feed. This method also provides students the flexibility to leave class for a short duration (such as for a phone call) without losing attendance for that class. Attendance is granted to a student if he remains in class for a number of snapshots above a certain threshold. The system is fully automatic and requires no professor intervention or any form of manual attendance or even camera set-up, since the back-end directly interfaces with inclass cameras. AttenFace is a first-of-its-kind one-stop solution for face-recognition-enabled attendance in educational institution