Abstract

The development of ultra-low power LSIs is a promising area of research in microelectronics. Such LSIs would be suitable for use in power-aware LSI applications such as portable mobile devices, implantable medical devices, and smart sensor networks. From compact sensors to advanced multi-core smartphones, various IoT devices hold precision and power efficiency as critical factors. Efficient management of switch on-off time acts as a key to minimize the power consumption. Therefore, maintaining accurate time is crucial for such type of devices. This is embraced by an on-chip RTC to retain reliable information of event occurrences. In addition, RTC’s interrupt management to alert the micro-controller host evaluates the robustness of the device. An RTC maintains a precise track of time by counting the oscillator cycles. Ultimately, the oscillation frequency (32.768KHz) is bounded by a speculation of time resolution and low power. Although the stable and precise nature of crystal oscillators (XOs) serve the purpose, they have few downsides. For being bulky, expensive and having high power consumption, the replacement of XOs is highly recommended. Focused research has been carried out for years on CMOS based oscillator architectures with sophisticated compensation techniques for reducing PVT variations. However, boosting the stability at low frequencies with low complexity, area and power have been a major challenge for next-generation IoT applications. This work presents a novel PVT-invariant CMOS relaxation oscillator for Real-Time Clock (RTC) applications. The proposed design works in a core voltage domain of SoC. The introduced self-adaptive compensation loop dynamically modifies the time constant of the oscillator core on sensing the process variation. Moreover, the design is accompanied by a digital calibration unit for further process compensation. The replacement of the resistor with a voltage-biased MOSFET has greatly enhanced the area efficiency. The required voltage and current bias in the circuit are generated through a novel current mixing technique which plays a crucial role in enhancing the performance of the system. Besides, the introduced supply independent bias circuit has greatly improved the supply regulation of the oscillator frequency

Abstract

In the last few decades, we can see our day-to-day life being dependent on software programs. The digital revolution in the software industry has been progressing at a staggering rate. Thousands of software applications like web browsers, search engines, simulators (aviation), etc. are being developed by the different enterprises in various domains that are changing every facet of our lives. Most of these software programs are written in fast, low-level and memory-unsafe languages like C/C++. These memory unsafe languages rely on the programmer’s expertise for memory management of objects created during runtime and hence, leads to some memory-corruption bugs like buffer-overflow [1], dangling pointers [2], etc. Later, these bugs have the potential to be exploited by some malicious users compromising the software security. These bugs can cause serious security problems, such as confidential information leakage, system crashes, user permission changes, etc. These bugs (when exploited) have an adverse effect on both consumers and enterprises in terms of money, time and even human lives. In such a scenario, it becomes extremely important to catch these bugs apriori and mitigate before any exploitation. This thesis helps in identifying and reporting a class of these bugs, which could breach some confidential information out of the software (aka information leaks).Fuzzing is arguably the most effective security testing technique to find memory-corruption bugs in applications. However, fuzzing is effective in finding specific bugs that result in some observable effects, like crash. Consequently, information leak bugs often remain undetected by fuzzing, in spite of their serious effect on the security of the application. In order to help fuzzing detect such bugs, address sanitiser (LLVM/GCC Asan option) based approaches are employed. However, such an option is available only when we have the source code of the application available. Other binary-only solutions, like Valgrind, have limited ability to detect information leaks (e.g., stack variables are not covered). In this thesis, we investigate the application of dynamic taint flow analysis(DTA) to detect information leak in binary executables. In this approach, we leverage DTA to track data and pointer taintedness and device a set of rules to infer when there could be a possible information leak. Our method is suitable for typical desktop applications (for example, image and document viewers) that parse and display the information contained in the provided input. In order to show the effectiveness of the proposed technique, we build a dynamic binary analysis (DBA) tool - TailFinder (Taint Assisted Info Leak Finder) in C++ using PIN dynamic binary instrumentation framework and evaluate it on real-world applications. On such applications, we find information leaks that have known CVEs1 , thereby showing the applicability of the proposed technique. We also present, how we can integrate these tools with fuzzing to find possible information leaks apriori.

Abstract

Graph centralities play a crucial role in graph/network analytics to help us determine the importance of nodes in real-world graphs. Betweenness centrality is one such metric and is indeed the most popularly used centrality measure. Algorithms to calculate Betweenness centrality have been studied in the literature for a long time, the latest and most profound one being the Brandes Algorithm. Although many optimizations of the Brandes algorithm exist, very few of them address the problem of finding centrality in a continually evolving graph i.e, graphs that change due to the addition/deletion of nodes/edges. The majority of the graphs where these metrics are required are dynamic graphs. In such scenarios, having techniques to update centrality quickly and efficiently becomes paramount. There have been works on updating centrality on the addition/deletion of a single node/edge. However, the techniques become computationally expensive when updates happen in bulk. We build upon the work done on single edge betweenness centrality update and present a novel parallel algorithm to process a batch of edges at once. We also introduce some new optimization techniques along the way that exploits specific structural properties of graphs. We implement our algorithm on both multi-threaded CPUs and GPU and provide detailed analysis of our algorithm across CPU, GPU for varied batch edge updates on a variety of graph datasets spanning major application areas.

Abstract

With the growing use of computer vision tools in wide-ranging applications, it becomes imperative to understand and resolve issues in computer vision models when they are used in production settings for various applications. In particular, it is essential to understand that the model can be wrong quite frequently during deployment. Developing a better understanding of the mistakes made by a model can help mitigate and handle them without catastrophic consequences. To investigate the severity of mistakes, we first explore this in a simple classification setting. Even in this setting, understanding the severity of mistakes of difficult to quantify, especially since manually defining pairwise costs does not scale well for large-scale classification datasets. Therefore most works have used class taxonomies/hierarchies, which allow pairwise costs to be defined using graph distances. There has been increasing interest in building deep hierarchy-aware classifiers, aiming to quantify and reduce the severity of mistakes and not just count the number of errors. However, most of these works require the hierarchy to be available during training and cannot adapt to new hierarchies or even small modifications to the existing hierarchy without having to re-train the model. We explore a different direction for hierarchy-aware classification – amending mistakes by making post-hoc corrections by resorting to the classical Conditional Risk Minimization(CRM). Surprisingly, we find that this method is a far more suitable alternative than the works on deep hierarchy-aware classification; CRM preserves the base model’s top-1 accuracy and brings the most likely predictions of the model closer to the ground truth and is able to provide reliable probability estimates, unlike hierarchy-aware classifiers. We firmly believe that this serves as a very strong and useful baseline for future exploration in this direction. We turn our attention to a crucial problem in many video processing pipelines: visual(single) object tracking. In particular, we explore the long-term tracking scenario where given a target in the first frame of the video; the goal is to track the object throughout a (long) video during which the object may undergo occlusion, vary in appearance, or go out-of-view. The temporal aspect of videos also makes it an ideal scenario to understand the accumulation of errors that would not be otherwise seen if every image is independent. We hypothesize that there are three crucial abilities that a tracker must possess to be effective in the long-term tracking scenario, namely Re-Detection, Recovery and, Reliability. The tracker must be able to re-detect the target when the target goes out of the scene, and returns must recover from failure and track an object contiguously to be of practical utility. We propose a set of novel and comprehensive experiments to understand each of these aspects which give a thorough understanding of the strengths and limitations of various state-of-the-art tracking algorithms. We finally visit the problem of multi-object tracking. Unlike the problem of single-object tracking where the target is initialized in the first frame, the goal here is to track all objects of a particular category(such as pedestrians, vehicles, animals etc.). Since this problem does not require user-initialization, it has found use in wide-ranging real-time applications such as autonomous driving. The typical multiobject tracking pipeline follows the tracking-by-detection paradigm, i.e. an object detector is first used to detect all the objects in the scene. These detections are linked together to form the final trajectories using a combination of Spatio-temporal features and appearance/Re-Identification(ReID) features. The appearance features are extracted using a Convolutional Neural Network(CNN) trained on a corpus of labelled videos. Our central insight is that only the appearance model requires labelled videos in the entire pipeline, while the rest of the pipeline can be trained with just image-level supervision. Inspired by the recent successes in unsupervised contrastive learning which enforces the similarity in feature space between an image and its augmented version, we resort to a simple method that leverages the spatio-temporal consistency in videos to generate “natural” augmentations which are then used as pseudo-labels to train the appearance model. When integrated into the overall tracking pipeline, we find that this unsupervised appearance model can match the performance of its supervised counterparts in reducing the identity switches present in the trajectories, thereby saving costly video annotations that are impractical to scale up without sacrificing performance.

Abstract

Scene analysis has been a topic of great interest in computer vision. Humans are the most important and most complex subject involved in scene analysis. Humans exhibit different forms of expressions and behaviour with its environment. These interactions with its environment have been in study for a long time and to interpret these interactions various challenges and tasks have been identified. We focus on two tasks in particular: Head Pose estimation and Emotion recognition. Head poses are an important mean of non-verbal human communication and thus a crucial element in understanding human interaction with its environment. Head pose estimation allows a robot to estimate the region of focus of attention for an individual. Head pose estimation requires learning a model that computes the intrinsic Euler angles for pose (yaw, pitch, roll) from an input image of the human face. Annotating ground truth head pose angles for images in the wild is difficult and requires ad-hoc fitting procedures (which provides only coarse and approximate annotations). This highlights the need for approaches which can train on data captured in a controlled environment and generalize on the images in the wild (with varying appearance and illumination of the face). Most present day deep learning approaches which learn a regression function directly on the input images fail to do so. To this end, we propose to use a higher level representation to regress the head pose while using deep learning architectures. More specifically, we use the uncertainty maps in the form of 2D soft localization heatmap images over five facial keypoints, namely left ear, right ear, left eye, right eye and nose, and pass them through a convolutional neural network to regress the head-pose. We show head pose estimation results on two challenging benchmarks BIWI and AFLW and our approach surpasses the state of the art on both the datasets. We also propose a synthetically generated dataset for head pose estimation. Emotions are fundamental to human lives and decision-making. Human emotion detection can be helpful in understanding human mood, intent or choice of action. Recognizing emotions from images or video accurately is not easy for humans themselves and for machines it is even more challenging as humans express their emotions in different forms and there is a lack of temporal boundaries among emotions. Facial Expression Recognition has remained a challenging and interesting problem in computer vision. Despite efforts made in developing various methods for facial expression recognition, existing approaches lack generalizability when applied to unseen images or those that are captured in wild setting (i.e. the results are not significant). We propose use of facial action unit’s soft localization heatmap images for facial expression recognition. To account for lack of large well labelled dataset we propose a method for automated spectrogram annotation where we use two modalities(visual and textual) used in expression of emotion by humans to label one other modality(speech) for emotion recognition.

Abstract

The advent of modern machine learning and deep learning methods have revolutionized various fields such as computer vision, natural language processing, speech recognition, etc. In the the past decade these methods have made a profound impact in the field of natural science. Particularly, the computations involved in the field of molecular science have been greatly transformed with the advent of machine learning. These changes have resulted in tremendous reduction of time and cost involved in tasks such as molecular generation and property prediction, which are two crucial steps involved in the process of molecular design. In this thesis, two studies on in each of these is reported. First part of the thesis presents a deep learning based inorganic material generator (DING) framework consisting of a generator module and a predictor module. The generator module is developed based upon conditional variational autoencoders (CVAE) and the predictor module consists of three deep neural networks trained for predicting enthalpy of formation, volume per atom and energy per atom chosen to demonstrate the proposed method. The predictor and generator modules have been developed using a one-hot key representation of the material composition. A series of tests were done to examine the robustness of the predictor models, to demonstrate the continuity of the latent material space, and its ability to generate materials exhibiting target property values. The DING architecture proposed in this paper can be extended to other properties based on which the chemical space can be efficiently explored for interesting materials/molecules. Second part of the thesis presents an interpretable deep learning method based on graph networks that accurately predict solvation free energies of small organic molecules. The proposed model CIGIN, comprising of three phases, namely message passing, interaction and prediction is able to predict solvation free energies in any generic organic solvent with a mean absolute error of 0.16 kcal/mol. In terms of the accuracy, the CIGIN model outperforms all the proposed machine learning based models so far. The atomic interactions predicted in an unsupervised manner is able to explain the trends of free energies consistent with chemical wisdom. Further, the robustness of the machine learning based model has been tested thoroughly and its capability to interpret the predictions has been verified with several examples.

Abstract

Centrality plays an essential role in identifying the essential nodes in social networks. This thesis proposes a new flavor of centrality, where we focus on farness and closeness centrality, which identify closer nodes in a network.Since closeness-centrality is a fundamental problem in graph theory, several algorithms exist for this problem for both static and dynamic environments. Recent algorithms for computing centrality have relied heavily on breadth-first search (BFS) traversals of the input graph. Usually, a significant portionof the algorithm’s overall run-time is spent on BFS operations. We have developed a framework (called BRICS) based on the graph’s structural properties, which reduces the time taken for a single BFS traver-sal and reduces the overall number of traversals. Since finding the exact values of centrality requiresconsiderable computational time, we have devised an estimation algorithm with a reasonable trade-offbetween accuracy and speedup.Additionally, we have created a GPU algorithm to update closeness-centrality in a dynamic settingwhen a batch of edges is added or deleted by extending Sarıy ̈uce et al. (CLUSTER 2013), an incremen-tal algorithm (single edge update) to the batch update.We have implemented our algorithm in Nvidia Tesla V100 GPU parallel architecture with 80 SMXs.We experimented on five different types of real-world graphs and varying the optimization techniques to achieve the max speed up. Our experimental results show that we were able to achieve a maximumof 12.54x speedup on computing fairness centrality and 53x on updating closeness-centrality in the dynamic setting.

Abstract

Terrain, representing features of an earth surface, plays a crucial role in many applications such as simulations, hazard prevention and mitigation planning, route planning, analysis of surface dynamics, computer graphics-based games, entertainment, films, to name a few. With recent advancements in digital technology, these applications demand the presence of high-resolution details in the terrain. However, currently available public datasets, providing terrain scans in the form of Digital Elevation Models (DEMs) have low resolution compared with the terrain information available in other modalities like aerial images. Publicly available DEM datasets for most parts of the world have a resolution of 30 m whereas the aerial images or satellite images are available at a resolution of 50 cm. The cost involved in capturing of such high-resolution DEMs (HRDEMs) turns out to be a major hurdle for making such high-resolution available in the public domain. This motivates us to provide a software solution for generating high-resolution DEM from the existing low-resolution DEMs (LRDEMs). In natural image domain, super-resolution has set up higher benchmarks by incorporating deep learning based solutions. Despite such tremendous success in image super-resolution task using deep learning solutions, there are very few works that have used these powerful systems on DEMs to generate HRDEMs. A few of them used additional modalities as aerial images or satellite images, temporal sequence of DEMs etc., to generate high-resolution terrains. However, the applicability of these methods is highly subject to the available input formats. In this research effort, we explore a new direction in DEM super-resolution by using feedback neural networks. Availing the capability of feedback neural networks to redefine the features learned by shallow layers of the network, we design DSRFB, a DEM super-resolution architecture that generates high-resolution DEM with a super-resolution factor of 8X with minimal input. Our experiments on Pyrenees and Tyrol mountain range datasets show that DSRFB can perform near to the state-of-the-art without using information from any additional modalities like aerial images. Further, by understanding the limitations of DSRFB, which primarily occur in case of highly degraded low-resolution input. In such cases, the major structures are entirely lost and the reconstruction becomes challenging. In such cases, to avail the elevation cues from alternate sources of information becomes necessary. To utilize such information from other modalities, we inherit the attention mechanism from natural language processing (NLP) domain. We integrate the attention mechanism into the feedback network to present Attentional Feedback Module (AFM). Our proposed network, Attentional Feedback vivii Network (AFN) with AFM as a backbone, outperforms the state-of-the-art methods with the best margin of 7.2%. We also emphasize on the reconstruction of the structures across patch boundaries. While generating HRDEM by splitting large DEM tiles into patches, we propose to use overlapped tiles and generate an aggregated response to dilute the artefacts due to structural discontinuities. To summarize, in this research, we propose two methods DSRFB and AFN to generate a high- resolution DEM from existing low-resolution DEM. While DSRFB achieves near to the state-of-the-art performance, coupling DSRFB with attentional mechanism (i.e., AFN) outperforms state-of-the-art methods.

Abstract

This work is an attempt to study polarization on social media data, in the Indian context. We focuson multiple controversial and talked about events and topics in the context of India, namely 1) theSabarimala Temple (located in Kerala, India) incident which became a nationwide controversy whentwo women under the age of 50 secretly entered the temple breaking a long standing temple rule thatdisallowed women of menstruating age (10-50) to enter the temple and 2) the Indian government’s moveto demonetise all existing 500 and 1000 denomination banknotes, comprising of 86% of the currencyin circulation, in November 2016 3) The Pulwama-Balakot incidents and its effect on popular sentimenttowards India’s Prime Minister Narendra Modi 4) Social media activity on events surrounding multiplelynching incidents from 2017 to 2019 and 5) the Indian Economy from 2013 to 2019 and popularresponse towards it on social media. We gather tweets from news media, and from common people onTwitter around these events in various time periods. Then, we pre-process and annotate them with theirsentiment polarity and emotional category, and analyse trends to help us understand changing polarityover time around controversial events. While this study tries to find trends and patterns to decode theaspect of polarization in India’s social fabric, it’s also a case study on all these events and how Indiareacted to each one of them. The tweets collected are in English, Hindi and code-mixed Hindi-English.

Abstract

Over the last decade, a tremendous emphasis has been laid on collection and digitization of a vast number of books leading to the creation of so-called ‘Digital Libraries’. Projects like Google Book and Project Gutenberg have made significant progress in digitizing over millions of books and making it available to the public. Efforts have also been made from the perspective of Indic languages where the task to identify and recognize books from several Indian languages has been undertaken by the National Digital Library of India. Advantages of digital libraries can be manifold. Digitization of ancient manuscripts ensures the preservation of knowledge and promotes research. Books in digital libraries are indexed which facilitates easy search and retrieval. They are easy to store and do not take as much effort in maintenance as their physical counterparts. One of the most important steps in the digitization effort is the recognition and conversion of physical pages into editable text using an OCR. There are commercial OCRs available like Tesseract and Abby fine reader, however, the ability of an OCR to recognize text without committing too many errors depends very much on the print quality of the pages as well as font style of the type-written text. A pre-trained OCR will invariably make errors across pages whose distribution is different in terms of fonts and print quality from the pages on which it was trained. If the domain gap is too large then the number of error words will be too high which will result in investing significant effort in the correction. Since the books need to be indexed, one cannot afford to have too many word errors in the OCR recognized pages. Thus, a major effort must be spent on correcting the error words, misclassified by the OCR. Manually correcting each isolated error word will incur a huge cost and is infeasible. In this thesis, we look at methods to improve OCR accuracy with minimum human involvement. To this effect, we propose two approaches. In the first approach, we strive to improve the OCR performance via an efficient post- processing technique where we aim to group similar erroneous words and correct them simultaneously. We argue that since a book has a common underlying theme, it will contain many word repetitions. These word co-occurrences can be taken advantage of by grouping similar error words and correcting them in batches. We propose a novel clustering scheme which combines features from both images as well as its text transcription to group error word predictions. The grouped error predictions can then be corrected either automatically or with the help of a human annotator. We show via experimental verification that automatic correction of error word batches might not be the most efficient way to correct the error words and employing a human annotator to verify the error word clusters will be a more systematic way to address the issue. Next, we look at the problem of adapting an OCR to a new dataset without requiring too many annotated pages. Traditional norm dictates finetuning the existing OCR on a portion of target data. However, even annotating a portion of data to create image-label pairs can be a costly affair. For this, we employ a self-training approach where the OCR is finetuned on its own predictions from the target dataset. To curtail the effects of noise present in the predictions, we include only those samples in the training set on which the model is sufficiently confident. We also show that by employing various regularization strategies we can outperform the traditional finetuning method without the need for any additional labelled data. We further show that by combining self-training with finetuning we can achieve a maximum gain in terms of OCR accuracy across all the datasets. We furnish thorough empirical evidence to support all our claims.