Abstract

Drug discovery involves the process of designing molecules that interact well with a given target protein structure in order to modulate to its function. With large scale availability of 3D structures of proteins, structure based drug design has been developed as a rational approach towards de novo drug design. In its core, it is about developing drug molecules that bind well with the receptor at its binding site and also portray ideal drug-like properties for Absorption, Distribution, Metabolism and Excretion (ADME). A prerequisite to this is finding druggable and functionally relevant binding sites on the 3D structure of the protein. Once the binding site is identified large libraries of molecules are screened against it to identify top candidate molecules, a process more commonly known as virtual screening. However, these chemical libraries represent only a minute portion of the larger chemical space where more ideal molecules may reside. To access this larger chemical space generative model need to be designed that can learn a distribution over this space for efficient sampling. Such a situation specifically beckons for inverse molecule design wherein generative models sample molecules conditioned to have specific properties and chemical structures. The rapid adoption and success of deep learning algorithms in various sections of structural biology motivates the usage of such algorithms for important tasks in drug design. In this thesis, we present three studies, namely DeepPocket, Apobind and MolGPT, that address important tasks in drug design such as binding site detection, molecular generation, and optimization using deep learning. DeepPocket uses 3D convolutional neural networks to identify and segment out druggable binding sites on the surface of the protein. However, DeepPocket was initially trained on only annotated ligand-bound (holo) protein structures since only such datasets were available for binding site detection. Therefore, Apobind a dataset of annotated ligand-unbound (Apo) structures is mined to circumvent this limitation and also act as an useful resource for other tasks like binding affinity prediction and molecular docking. Finally, MolGPT, a novel molecular generative model based on the attention-based transformer architecture is discussed while and its efficacy in molecular optimization and inverse molecule design is demonstrated.

Abstract

Emotions are an essential part of daily communication and play a critical role in conveying a person’s mental state. Understanding, categorization, and controllable emotions generation are widely researched in NLP and social sciences. Different work in social sciences aimed to define emotions and identify different emotions. Ekman’s initial work identified six primary emotions: anger, disgust, fear, joy, sadness, and surprise. Plutchik identified eight emotions and classified them into four polar opposite pairs. With the advent of computer science, different computational approaches have been developed to identify and categorize emotions and generate emotional text. In this thesis, we develop data-driven, psychologically grounded computational approaches to solve two broad problems relating to emotions in text. The first problem is identifying emotions at a paragraph level. Existing work detects emotion at a sentence level having a limited context. We aim to detect emotions at a paragraph level consisting of multiple sentences. Thus, the model needs to consider the emotional cues present in an individual sentence and paragraph as a whole unit. We also explore the classification problem in scientific documents when the context is large but cannot be broken down into individual sentences. The second problem that we explore is changing the emotion of a sentence while controlling its intensity. In the process, the meaning of the sentence should be preserved. This task of changing specific attributes of a sentence while preserving its overall meaning is defined as style transfer. For both the tasks, the proposed system achieved state-of-the-art. Pretrained language models are current de facto in Natural Language Processing (NLP). We develop an ensemble solution consisting of three finetuned language models for the first task. The language models are RoBERTa, BART, and RoBERTa model that is first finetuned on sentence-level emotion detection tasks. RoBERTa model is used due to its remarkable performance in many natural language understanding tasks. BART has shown good performance for tasks involving multi-sentence level context like summarization, thus is used here. RoBERTa model that is first finetuned on sentence-level emotion detection tasks is used to identify strong sentence-level cues. Our proposed system was ranked one globally for the Empathy Detection and Emotion Classification shared task at the WASSA workshop at ACL 2022. We also explore the power of the pretrained language model for the classification of scientific documents when the context is large but cannot be divided into sentences. Our proposed system of using highly contextualized embeddings of science BERT achieved a new state of the art and was ranked one globally for 3C Citation Context Classification shared task at the Scholarly Document Processing workshop at NAACL 2021.

Abstract

Acetone sensors find applications in healthcare, environmental, food, agriculture, and industrial fields. Semiconducting metal oxide (SMO)-based acetone sensors are costeffective, stable, and could be user-friendly and portable. In SMO-based acetone sensors, interferents like alcohol, which is structurally and chemically similar to acetone, impair the usability of the sensor. In this experimental study, we have investigated how Ag nanoparticles loading onto nanostructured ZnO materials improves the sensor performance, including the selectivity towards acetone gas over ethanol. The sensor involves facile chemical synthesis methods for nanostructured ZnO preparation and preparation/loading Ag nanomaterials onto ZnO. We have varied the Ag nanoparticle loading to see its effect on the acetone sensing performance. Ag is the cheaper noble metal and has been widely used in optoelectronics fields. UV-visible spectrophotometry, Raman scattering spectroscopy and field emission scanning electron microscopy (FESEM) have been used to characterize the sensor materials. It is found that an optimal Ag nanoparticle loading onto nanostructured ZnO shows remarkable acetone selectivity with high sensor response, fast response and recovery at relatively low working temperature compared to pristine ZnO. The nanostructured ZnO/Ag sensor material does not exhibit any interference from water vapor, a significant constituent of the air and exhaled breath. Furthermore, the sensor response and logarithmic acetone concentration yield a good linearity relationship which suggests suitable nanostructured ZnO/Ag material has immense potential as a sensitive and selective low-cost acetone sensor.

Abstract

This research aims to study Indian folk paintings in terms of style, icons, and their interrelations. The thesis is an attempt to do the above through presenting a formal theory of paintings, built upon, and developed from pre-existing formal theories in India and the west. We have selected Indian folk paintings as a case to demonstrate our formal theory for the following reasons: first, Indian folk paintings have not been studied in terms of their formal aspects; second, their unique compositional sense of style stability and iconic similarity have remained under-researched. The thesis begins by explaining the nature of space of the painting. It gradually develops the following five core chapters: (1) Concept of style (2) Concept of icon (3) Blending of style and icons in paintings (4) Case study of Indian folk paintings and (5) Formal analysis of folk paintings of India. The idea developed in the above core chapters is to study Indian folk paintings in terms of their artistic qualities. The styles and patterns of the Indian folk paintings are as diverse as the nation. However, Indian folk paintings show diversity in style and similarity in iconic representations. For example, one can see similar iconic representations of snake, elephant, tiger and even humans being repeated across various Indian folk painting styles. The purpose of our research is to qualify these differences in style and study the artistically and contextually informed aspects influencing an artist’s craft. We will describe and analyse folk paintings in terms of their formal aspects, which helps to develop and shape our research process. Finally, to validate our research efforts in qualifying style in Indian folk paintings, with undertake an evaluation employing existing arthistorical theories and modules.From the above study, this thesis will attempt to derive and explain formal features guiding the style of an artist and her art, in this case folk paintings from India. Our thesis leads further to the following inquiry- how can we define the concept of style in painting? What are the definitions of icons? What is the nature of space of a painting which allows blending of the style and icon? What is the role of memory and imagination in creating icon and understanding a pictorial style? “If the icon is what an artist wants to expose, the style is the medium for this exposure” (Merleau-Ponty, 1993). Painting reflects the ‘way’ (style/action) and ‘what’ (icon) in a concrete form. In painting, style/action can be understood only through strokes. “These strokes are traces of the artist's moving hand. The hand movements are guided by the icon, which is etched in the artist's mind. Strokes are nothing but the parts of that icon” (MerleauPonty, 1993). For example, if the intention is to draw a tiger, one starts by drawing its parts like head, tail, body. Here the strokes are syntactic, which reveal the semantics of the icon to a spectator. Only the strokes and patches are visible in a painting. “The primary sine qua non to create or to understand icons is imagination” (Feldman, 1967). The artist using her imagination to create artwork is understandable. However, it is indeed intriguing how the viewer comprehends the artist's mind by using her imaginative prowess. How does the viewer get access to the world of the artist's imagination? The artist makes her art accessible through her artworks. The strokes and colour patches in the painting become paths through which the viewer enters the world of the artist's imagination. This research is about the role of style and icon in creating these paths and the journey that the artist undergoes while creating and, the viewer, while seeing. Thisstudy involves an analysis of five Indian folk painting styles. They belong to five different Indian states: Telangana, Odisha, West Bengal, Bihar, and Rajasthan. We tried to trace the changes and developments within a particular style and clarify the core elements in each of the region’s (mentioned above) art school style and their compositional principles. For this, we have made an exhaustive and elaborate database of thousands of images of paintings from each style. The images belong to varying time periods and are created by several artists. We adopted qualitative methodologies to interact and interview artists to know and get a clear perception of their painting styles. We have videotaped the interviews to aid in capturing the process in the making of a painting from the perspective o the artist.

Abstract

Crowd counting is an important task in security, surveillance and monitoring. There are many competitive benchmark datasets available in this domain. The data distribution in the crowd counting datasets show a heavy-tailed and discontinuous nature. This nature of the dataset is majorly ignored while building solutions to this problem. However, the skew in datasets contradicts few assumptions made by the stages of the training pipeline. As a consequence of the skew in the dataset, unacceptably large standard deviation wrt to the customarily used performance measures (MAE, MSE) is observed. To address these issues, this thesis provides modifications that incorporate the dataset skew in training and evaluation pipelines. In the training pipeline, to enable principled and balanced minibatch sampling, a novel smoothed Bayesian binning approach is presented that stratifies the entire count range. Further, these strata are sampled to construct uniform minibatches. The optimization is upgraded with a novel strata-aware cost function that can be readily incorporated into the existing crowd counting deep networks. In the evaluation pipeline, as an alternative to the customary evaluation MAE, this thesis provides three alternative evaluation measures. Firstly, a strata-level performance in terms of mean and standard deviation gives range specific insights. Secondly, relative error perspective is brought in by using a novel Thresholded Percentage Error Ratio (TPER). Lastly, a localization included counting error metric Grid Average Mean absolute Error (GAME) is used to evaluate the different networks. In this thesis, it is shown that proposed binning-based modifications retain their superiority wrt the novel strata-level performance measure. Overall, this thesis contributes a practically useful training pipeline and detail-oriented characterization of performance for crowd counting approaches.

Abstract

This thesis proposes solutions to two major problems (highlighted) faced when developing a selfdriving car using traditional modular approaches. That is to sense, percieve, localize/map, predict and plan/control. The first one is doing real-time Simultaneous Localization and Mapping SLAM at low sampling rates and/or at high linear or angular velocities. The keynote of the work showcases SROM’s ability to maintain localization at low sampling rates or at high linear or angular velocities where most popular LiDAR-based localization approaches get degraded fast. We also demonstrate SROM to be computationally efficient and capable of handling high-speed maneuvers. It also achieves low drifts without the need for any other sensors like IMU and/or GPS. Our method has a two-layer structure wherein first, an approximate estimate of the rotation angle and translation parameters are calculated using a Phase Only Correlation (POC) method. Next, we use this estimate as an initialization for a point-to-plane ICP algorithm to obtain fine matching and registration. Another key feature of the proposed algorithm is the removal of dynamic objects before matching the scans. This improves the performance of our system as the dynamic objects can corrupt the matching scheme and derail localization. Our SLAM system can build reliable maps at the same time generating high-quality odometry. We exhaustively evaluated the proposed method in many challenging highways/country/urban sequences from the KITTI dataset and the results demonstrate better accuracy in comparisons to other state-of-the-art methods with reduced computational expense aiding in real-time realizations. We have also integrated our SROM system with our in-house autonomous vehicle and compared it with the state-of-the-art methods like LOAM and LeGO-LOAM. The second one is the problem of an agent/robot with non-holonomic kinematics avoiding dynamic and static obstacles whose state and velocity noise are non-parametric. Additionally, there may be bounds/constraints on the configurational space of the robot in the form of lane/corridor boundaries. In this work we consider the problem of an agent/robot with non-holonomic kinematics avoiding dynamic and static obstacles. Additionally there may be bounds/constraints on the configurational space of the robot in the form of lane/corridor boundaries. The robot’s state and velocity noise, the lanes, the obstacles, and the robot’s control noise are modelled as non-parametric distributions as Gaussian assumptions of noise models are violated in real-world scenarios. Under these assumptions, we formulate a robust MPC that samples robotic controls effectively in a manner that aligns the robot to the goal state while avoiding obstacles and staying within the lane bounds under the duress of such non-parametric noise. In particular, the MPC incorporates a distribution matching cost that effectively aligns the distribution of the current collision cone to a certain desired distribution whose samples are collision-free. This cost is posed as a distance function in the Hilbert Space, whose minimization typically results in the collision cone samples becoming collision-free. We show tangible performance gains compared to methods that model the collision cone distribution by linearizing the Gaussian approximations of the original nonparametric state and obstacle distributions. We also show superior performance to methods that pose a chance constraint formulation of the Gaussian approximations of non-parametric noise without subjecting such approximations to further linearizations. The performance gain is shown both in terms of trajectory length and control costs that vindicates the efficacy of the proposed method. Finally, we show the proposed method being used to navigate with a non holonomic differential drive robot in real-time in a realistic setting in Gazebo with dynamic and static obstacles. To the best of our knowledge, this is the first presentation of non-holonomic collision avoidance of stationary obstacles, moving obstacles and lane constraints in the presence of non-parametric state, velocity, actuator and lane boundary noise models.

Abstract

The last few decades have witnessed tremendous growth in the sensors industry, spurred by advancements in electronics, nanoscience, biotechnology and computer science. They are employed for a wide variety of applications and they possess the potential to play a pivotal role in biochemical applications. Among the various sensing modalities being explored, we have Radio Frequency based sensing. It studies the interaction between EM waves and matter and uses this interaction for qualitative and quantitative analysis. RF sensors offer a reduction in process complexity, speed, cost, portability and ease of integration with existing technology. In this work, Radio Frequency (RF) based sensing technique is applied for different biochemical applications. In particular, emphasis is given to combating two significant challenges of RF sensing, namely, specificity and reusability. In addition to that, the use of RF sensing in new sub-domains is explored, which would enable the development of a portable biochemical sensing platform. The sensors put into use in this work are Interdigitated Capacitor based sensor and Microstrip Patch based sensor. This sensor is reusable and can be put into use for diverse applications. Reusability of the sensor implies that it can not only be used several times, but the same sensor can detect different sets of analytes with equal efficiency. We have performed both computational and experimental studies on surfactants, nanoparticles and biomolecules using RF sensing. Surfactants are essentially surface-active agents that are amphiphilic. Above a specific concentration called the Critical Micelle Concentration (CMC), they self assemble to form micelles. The estimation of CMC is vital for all the varied applications of surfactants. The first application develops a fast, facile, efficient method for CMC detection that works well for all classes of surfactants. With their plethora of inherent advantages, Nanoparticles play vital roles in the biochemical and biomedical domains. Their study is essential to utilize them for diverse applications efficiently. There is a growing need to develop a methodology that performs preliminary characterization of nanoparticles in the absence of sophisticated laboratory facilities. In the second application, we study nanoparticles of various morphology and arrangements and characterize them. Our third application deals with developing a biomolecule detection platform that is rapid, portable, efficient and reusable. Gold nanoparticles are functionalized to improve the reusability and specificity aspects of conventional RF sensors. Further, the Biotin-Streptavidin binding is chosen to demonstrate the methodology

Abstract

The popularity of egocentric cameras and their always-on nature has lead to the abundance of daylong first-person videos. Because of the extreme shake and highly redundant nature, these videos are difficult to watch from beginning to end and often require summarization tools for their efficient consumption. However, traditional summarization techniques developed for static surveillance videos, or highly curated sports videos and movies are, either, not suitable or simply do not scale for such hours long videos in the wild. On the other hand, specialized summarization techniques developed for egocentric videos limit their focus to important objects and people. In this work, we present a novel unsupervised reinforcement learning technique to generate video summaries from day long egocentric videos. Our approach can be adapted to generate summaries of various lengths making it possible to view even one minute summaries of one’s entire day. The technique can also be adapted to various rewards, such as distinctiveness, indicativeness of the summary. When using the facial saliency-based reward, we show that our approach generates summaries focusing on social interactions, similar to the current state of the art. Quantitative comparison on the benchmark Disney dataset shows that our method achieves significant improvement in Relaxed F-Score (RFS) (32.56 vs. 19.21) and BLEU score (12.12 vs. 10.64). Finally, we show that our technique can be applied for summarizing traditional, short, hand-held videos as well, where we improve the state of the art F-score on benchmark SumMe and TVSum datasets from 41.4 to 45.6 and 57.6 to 59.1 respectively

Abstract

Random Number Generators (RNGs) are the solution for cryptographic applications to enhance hardware security. These RNGs should have three specific properties unpredictability, aperiodic, and good statistical criteria. This brief presents a True Random Number Generator (TRNG) based on Ring oscillators' jitter with MLFSR. The MLFSR is augmented with a set of prime primitive polynomials, Boolean, and nonlinear functions to attain a non-linear, unpredictable, and extended sequence period. Paper mainly focused on achieving high randomness by integrating the TRNG with a new promising crypto engine Keccak as a post-processing block, leading to more extensive security in data transfer, encryption keys, data authenticity of ICs, and IoT based applications. The TRNG design is coded in Verilog HDL and implemented on the FPGA Zed board. Hashing is performed with a throughput of 2.4Gbps at 100MHz either with RNG data or SHA data. The HRNG (Hardware Random Number Generator) is an integrated block of TRNG and hashing along with the IP cores of memory, clock Generator, and Integrated Logic Analyzer, implemented on the FPGA hardware. Evaluated the randomness of the generated non-deterministic bit streams (10Mb) using the NIST 800-22 & Diehard test suite and successfully passed.

Abstract

The launch of level 5 autonomous driving is imminent which would result in removal of controls like steering wheel, acceleration and brakes. This warrants the need for a better system to direct the self-driving agent to perform maneuvers if the passenger wishes to do so. Hence, this thesis proposes a set of solutions tackle the problem of vision and language navigation in an autonomous driving setting which has been broken down into multiple sub problems. The first problem we attempt to solve, we present a simple baseline for visual grounding for autonomous driving which outperforms the state of the art methods, while retaining minimal design choices. The framework minimizes the cross-entropy loss over the cosine distance between multiple image ROI features with a text embedding (representing the given sentence/phrase). We use pre-trained networks for obtaining the initial embeddings and learn a transformation layer on top of the text embedding.We perform experiments on the Talk2Car dataset and achieve 68.7% AP50 accuracy, improving upon the previous state of the art by 8.6%. Humans have a natural ability to effortlessly comprehend linguistic commands such as “park next to the yellow sedan” and instinctively know which region of the road the vehicle should navigate. Extending this ability to autonomous vehicles is the next step towards creating fully autonomous agents that respond and act according to human commands where knowing only the location of the object of interest in the scene is not enough. To this end, a novel task of Referring Navigable Regions (RNR), i.e., grounding regions of interest for navigation based on the linguistic command is proposed. RNR is different from Referring Image Segmentation (RIS), which focuses on grounding an object referred to by the natural language expression instead of grounding a navigable region. For example, for a command “park next to the yellow sedan,” RIS will aim to segment the referred sedan, and RNR aims to segment the suggested parking region on the road. We introduce a new dataset, Talk2Car-RegSeg, which extends the existing Talk2car dataset with segmentation masks for the regions described by the linguistic commands. We benchmark the proposed dataset using a novel transformer-based architecture. We present extensive ablations and show superior performance over baselines on multiple evaluation metrics. A downstream path planner generating trajectories based on RNR outputs confirms the efficacy of the proposed framework.