Abstract

Abstract International Relations deals with the interaction of political entities in the international arena. Iden- tifying and analyzing these interactions in near real-time is important to predict upcoming political alignments and fathom upcoming conflicts. This effort is widespread as part of governmental diplo- macy. Our approach presents a computational notion of this same phenomenon. This requires analysis and data which was scarce and highly secured. Technology and computation, was limited and expen- sive. Classified documents regarding the interactions weren’t public. With increased digitization and penetration of internet across domains, this research has renewed impetus. One of the major tasks of analyzing actor interactions is to identify events involving actors from the news. Event is one of the most important temporal phenomena. It is that which has a beginning and an end. Automatic coding or classification of events happening in international relations is an important part of social science data ecosystem. The last few decades have witnessed significant work in detecting political events in the international arena. Most of the current work involves classification of news based on pattern matching from a large set of verb patterns, political actors, compound nouns, compound verb phrases, reference to pronouns and deep parsing of sentences in news articles. We introduce a methodology for extracting international relations events using news media data. This method involves a graph based unsupervised learning method for extracting topics in the news articles and identifying events based on various entities: actors and other spatial and temporal features. Since the same event might be reported by multiple news media across the world, multiple events corresponding to the same real-world event would be detected. This is referred as Event Coreference in NLP. We work on the problem of Event Coreference resolution which consists of finding clusters of event mentions that refer to the same event. Since events are complex, they co-refer fully as well as partially. We extract full co-reference, partial co-reference (sub-event and member-of relations) for the events extracted based on rule-based approach and obtain substantially good results. International relations events require an ontology for the classification of events to analyze the in- teractions among actors. We propose a novel mediation ontology for the categorization of international relations events and also introduce a novel method of classifying events through the mediation ontol- ogy using Beth Levin Verbs Classification, word2vec and Universal dependencies. The selected feature space is a result of mapping language entities to ontological entities. Hereafter, we explore various applications of international relations events. We demonstrate the relevance of our work in IREvent2Story. Event detection is a key aspect of story development which is itself composed of multiple narrative layers. Most of the narratives are template-based and follow a narration theory. We demonstrate a narrative from events detected in the international relations which presents interactions of international actors over various topics and other interactive visualizations. Another application we demonstrate is IREvent2Perspectives. Media is a classic example of the freedom to expression which may incorporate polar sentiments on the reportage. We create a system for readers with various perspectives on the same international relations event from various news sources across the world. Our study opens newer avenues for journalists and policy makers to exploit the opportunities for data driven diplomacy. It is hoped, that this research finds base especially with national and international organizations that the inexact and amorphous nature of world diplomacy is handled on hard facts and real analysis.

Abstract

Last few years have seen a tremendous increase in interest for autonomous driving. As advances have been made in this field, autonomous driving has faced various challenges. In this thesis, we present vision based solutions to curb and intersection detection problems faced by self-driving cars. In the first part of the thesis, we present a discriminative approach to the problem of curb detection under diverse road conditions. Curb detection is a critical component of driver assistance and autonomous driving systems. We define curbs as the intersection of drivable and non-drivable area which are classified using dense Conditional random fields(CRF). In our method, we fuse output of a neural network used for pixel-wise semantic segmentation with depth and color information from stereo cameras. CRF fuses the output of a deep model and height information available in stereo data and provides improved segmentation. Further, we introduce temporal smoothness using a weighted average of Segnet output and output from a probabilistic voxel grid as our unary potential. Finally, we show improvements over the current state of the art neural networks. Our proposed method shows accurate results over a large range of variations in curb curvature and appearance, without the need of retraining the model for the specific dataset. While it is important to detect road boundaries, another interesting problem is to know how the boundaries fork and lead to an intersection. In the second part of the thesis, we focus on the problem of road intersection detection. Long-short term memory networks(LSTM) models have shown considerable performance on a variety of problems dealing with sequential data. In this thesis, we propose a variant of Long-Term Recurrent Convolutional Network(LRCN) to detect road intersection. We call this network as IntersectNet. We pose road intersection detection as binary classification task over a sequence of frames. The model combines deep hierarchical visual feature extractor with recurrent sequence model. The model is end to end trainable with a capability of capturing the temporal dynamics of the system. We exploit this capability to identify road intersection in a sequence of temporally consistent images. The model has been rigorously trained and tested on various different datasets. We think that our findings could be useful to model behavior of autonomous agent in the real world.

Abstract

The thesis brings out technology impact on the two key political processes viz. news reporting and public protests. In the first part theoretical models are developed for the attention seeking ability of the news headlines online, based on the theories of human curiosity termed as the information seeking behaviour without utilitarian motive. A classifier is built to identify the clickbaity headlines indicating high degree of curiosity they generated in reader’s mind. The methodology is also used to study the headline dataset scraped and collated from the digital newspaper start-ups in India for their ability to attract reader views and attention. News headline gets spotted by the reader for the novelty of the topic, it creates surprise in readers’ mind by the choice of word arrangements used. Headline stoking optimal reader anticipation attracts a click pointing to its autonomous significance independent of the narrative. Finally headline’s structural, syntactic and semantic features create information gap in reader’s mind, by elevating the information reference point and the headline which is ”self-important”, salient, and promises to close the information gap piques reader curiosity and triggers clicking behaviour. The work also computationally validates a century old postulate about the relationship between curiosity and stimulus levels by computing novelty and surprise from Facebook reader response to news headlines. We have plotted the curious responses of reactions, shares and comments against the stimuli causing novelty and surprise to check their impact on the reader curiosity. The result of inverted ”U” curve, is as per the psychological modelling done during late 19th century by Wilhelm Wundt hypothesizing intermediate arousal potential. The second part of the thesis brings out the biography of public outrage online by computationally analyzing two recent incidents viz. Jyoti Singh incident(2012) at Delhi and murder of noted journalist Gauri Lankesh(2017). This study has tried to prise-open the manner in which online outrage builds up and evolves into politically meaningful public conversations. It suggests that outrage by itself has a short life, is intense in its expression but dies quickly. However, it appears that where it builds on a certain robust mapping of emotions and incidents to each other, it has a good chance to develop an afterlife as a larger public conversation, and thus impacts politics.

Abstract

Inverse Kinematics (IK) problem deals with finding an appropriate actuator configuration that makes the end effector Cartesian coordinates and pose match the given coordinates and pose. In most of the cases, it is enough if we can solve IK for the given coordinates whatever may be the pose. Real time calculation of inverse kinematics with dynamically stable configuration is of high necessity in humanoid robots as they are highly susceptible to lose balance. General Inverse Kinematic solvers may not guarantee real-time control of the end-effectors in external coordinates along with maintaining stability. This work proposes a methodology to generate joint-space trajectories of stable configurations for solving inverse kinematics using Deep Reinforcement Learning (RL). Our approach is based on the idea of exploring the entire configuration space of the robot and learning the best possible solutions using an actor-critic based policy learning in continuous action spaces, Deep Deterministic Policy Gradient (DDPG). The proposed strategy was evaluated on the highly articulated upper body of a humanoid model with 27 degree of freedom (DoF). The trained model was able to solve inverse kinematics for the end effectors with 90% accuracy while maintaining the balance in double support phase. Following the success of learning a general IK solver for goal reachability tasks of a single hand of humanoid, a more challenging problem of solving IK for both hands of the humanoid simultaneously is taken up. For addressing this problem, the proposed methodology was extended with significant changes in the state vector and reward function modelling. The extended strategy was then evaluated on the highly articulated upper body of the given humanoid for learning multi-goal reachability tasks of both hands along with maintaining stability in double support phase. Results show that the trained model was able to solve inverse kinematics for both the hands, where the articulated torso contributed to both the tasks. However it was observed that DDPG was unstable and in some cases even though it was stable initially, the cumulative reward started degrading later. Consequently, this works moves on to address these instability issues by using multi-task reinforcement learning. However, most reinforcement learning algorithms are inefficient for learning multiple tasks in complex robotic systems, where actions between different tasks are not fully separable and share a set of actions. In case of tasks where actions are fully separable, each task policy can be learnt independently using different policy networks. However when the actions are not fully separable, the policies cannot be learnt independently. In such environments a compound policy may be learnt with shared neural network parameters, which performs multiple tasks concurrently. However such compound policy may get biased towards a task or the gradients from different tasks negate each other, making the learning unstable and sometimes less data efficient. In this work, we propose a new approach for simultaneous learning of multiple tasks sharing a set of common actions in continuous action spaces, which we call as DiGrad (Differential Policy Gradient). The proposed framework is based on differential policy gradients and can accommodate multi-task learning in a single actor-critic network. We also propose a simple heuristic in the differential policy gradient update in case of partially separable actions to further improve learning. In order to show the efficiency of framework, the proposed architecture was tested on the given humanoid for learning multi-goal reachability tasks (each end effector has a different goal) and the results were compared with that of DDPG. Training results show that DiGrad converges faster than DDPG and removes the instability in training. Finally a comparative analysis with different settings in DiGrad along with DDPG was shown.

Abstract

The cellular users’ demand has increased drastically with the emergence of mobile internet technologies. In order to satisfy the demand, cellular service providers have improved each generation of cellular networks over its predecessors. However, the fifth generation (5G) cellular technologies would revolutionize the network architecture and would go beyond the trend of previous generations. Device-to-Device (D2D) communication is one such revolutionary technology in which the mobile users communicate with each other directly without traversing the base station. D2D communication is actively studied with special focus on public safety in 3GPP. D2D communication with network control has many other use-cases. Also, the 5G cellular technology includes millimeter wave (mmWave) for communication. Along with massive bandwidth availability at mmWave frequencies, the advantage of extensive frequency reuse and directive propagation is gained. The integration of device-to-device (D2D) and mmWave technology would improve the system performance and spectral efficiency to a great extent. Moreover, with the emerging trends in Internet-of-Things (IoT), massive device connectivity is required with high connection speed. Hence, D2D communication technology plays a vital role in maintaining Quality of Service (QoS) in IoT applications. The thesis studies D2D communication in inband mode where the resources of the licensed spectrum are used for communication. The work emphasizes on the system performance improvement because of the integration of mmWave technology and D2D communication. Underlay mode D2D communication, where the devices reuse the resources of a cellular user, is analyzed. The problems of resource allocation and interference avoidance are addressed. Then, the importance of fairness in resource allocation and the power allocation are studied. An approach for resource allocation and power allocation is proposed which is adaptive to traffic demand. Results show that the proposed scheme can significantly enhance the system performance in terms of fairness and spectral efficiency. Inband D2D communication with selective operation in underlay and overlay modes aiming to maximize QoS is explored. Spectrum underutilization is the key challenge in overlay mode which is addressed in this work. In areas with high call traffic density, D2D communication can operate in overlay mode because of its low complexity as resources are dedicated and the absence of interference with cellular users. In order to maximize spectrum utility, resources are dynamically assigned to overlay mode based on traffic demand. System performance in overlay mode is analyzed. A dynamic mode selection scenario is considered while addressing resource allocation, power control and interference avoidance. Thus, formulated problem is solved using a single heuristic algorithm for resource management in both the modes. Performance of the proposed algorithm is evaluated through simulations. The proposed scheme has optimal performance for a dense 5G network in controlled underlay and overlay operations and overcomes the problem of spectrum wastage in overlay mode by its selective operation.

Abstract

Abstract LiDAR has been emerging as one of the most sought after technologies with it’s novelty to perceive 3D environments accurately. One of the popular application of the lidar data has been object segmen- tation of different objects from a given scene. Lidar as a data format can record the 3D structure of the buildings in urban areas with precise object information. Such platforms to record lidar are mostly clas- sified into three types that is terrestrial, aerial and mobile based lidar systems. The data being recorded from respective platforms varies a lot in terms of range, coverage and point density. Our work presents new approaches to segment buildings from a given lidar scene with flexibilty to work with lidar data from multiple different platforms. The challenge of using LiDAR datasets is the very high resolution of datasets, with high computa- tional requirements in visualizing and processing the datasets. The current methods of object segmen- tation and extraction and classification of aerial LiDAR data are manual and tedious. Our work tries at modelling this problem into a generic pipeline which can be used to segment different varieties of lidar datasets. The pipeline constructed is a result of multiple geometric approaches taken to segment the buildings. The massiveness of data to process, the accuracy of groundline estimation, detection of connected roofplanes and occlusion and overlap of multiple objects in a given lidar scene poses a range of challenges while processing and segmenting such data. Our work tries to overcome some of the challenges and shortcomings of the existing works by proposing two new methods of segmenting objects. Objects in this thesis refer to only buildings. The justification of segmenting only buildings and not focusing on any other type of objects is primarily due to the uniformity in shape of the building and its adherance to geometric properties. In case of other objects, such advantages are not present making their segmentation more challenging and difficult to formulate. The scope of the thesis is limited to only aerial LiDAR because of it’s larger coverage and more processing friendly density compared to terrestrial scans. Terrestrial LiDAR can not capture the roof planes of the buildings while in case of aerial LiDAR we can capture the roof planes of the buildings, which are later used to segment buildings from the given scene. Existing methods of building segmentation from a LiDAR scene mostly involves the usage of image processing based techniques like morphological operations and fitting done in a plane through RANSAC algorithm. These methods often are highly dependent on the resolution of the input data being fed into it. Some methods also take the help of other data to aid in segmentation. Our work tries to tackle the shortcomings of the above works and present a new and novel method to segment buildings from a given scene. We propose two different approaches in segmenting buildings from a given scene. The commonality of the different approaches is that both of them focus on using the facade planes of the buildings to segment them. The algorithm has been tested in multiple datasets ranging from synthetic and real-world datasets. A bottom-up geometric rule-based approach was used initially to devise a way to segment buildings out of the LiDAR datasets. Multiple lidar datasets were used to check the accuracy and performance of the algorithm. Preliminary results show successful segmentation of the buildings objects from a given scene in case of synthetic datasets and promissory results in case of real-world data with an accuracy of 68%. Our work tries to solve a part of the puzzle by segmenting only the buildings in a given lidar scene automatically with limited tuning, in an intuitive manner. The further advantage of the proposed work is its non-dependence on any other form of data required except LiDAR. It is an unsupervised method of building segmentation, thus requires no model training as seen in supervised techniques. It focuses on extracting the walls of the buildings to construct the footprint, rather than focusing on the roof. The focus on extracting the wall to reconstruct the buildings

Abstract

The thesis presents immensely pipelined parallel FPGA based architectures for Image segmentation by a graph-based partitioning approach and by state-of-the-art CNN’s approach. Efficient and real-time segmentation of color images has a variety of importance in many fields of computer vision such as image compression, medical imaging, mapping and autonomous navigation. Being one of the most computationally expensive operations, it is usually done through software implementation using high-performance processors. In robotic systems, however, with the constrained platform dimensions and the need for portability, low power consumption and simultaneously the need for real-time image segmentation, we envision hardware parallelism as the way forward to achieve higher acceleration. Field-programmable gate arrays (FPGAs) are attractive alternatives for this task because of their reconfigurability, high per-watt performance in a small physical area. They exceed the computing speed of various software-based implementations by breaking the paradigm of sequential execution and accomplishing more per clock cycle operations by enabling hardware level parallelization at an architectural level. The first contribution proposes three novel architectures for a well known Efficient Graph-based Image Segmentation algorithm. These proposed implementations optimize time and power consumption when compared to software implementations. The hybrid design proposed, has notable furtherance of acceleration capabilities delivering at least 2X speed gain over other implementations, which henceforth allows real-time image segmentation that can be deployed on Mobile Robotic systems. The second contribution focuses on FPGA architectures designed for state-of-the-art approach CNN’s for image segmentation and classification. Convolutional Neural Networks (CNN’s) are rapidly gaining popularity in varied fields such as computer vision, information retrieval, and mobile robotics. Due to their increasingly deep structures, modern CNN’s are continuously becoming computationally and memory intensive. Although this enhances accuracy, it becomes difficult to deploy them on energy-constrained mobile devices such as drones. As a consequence, hardware accelerators such as FPGAs with their inherent hardware parallelism, have come up with an attractive alternative. The major bottleneck while implementing huge networks on FPGA is meeting high memory throughput requirement of CNN’s with limited on-chip memory. Hence, this work proposes a high-performance FPGA based architecture - Depth Concatenation and Inter-Layer Fusion based ConvNet accelerator-DeCoILFNet which exploits the intra-layer parallelism of CNN’s by flattening across the depth and combining it with the inter-layer fusion. DeCoILFNet perfectly pipelines the data flow across convolution layers which significantly reduces off-chip memory accesses and maximizes the throughput by using multiple line buffers. To validate our approach, we demonstrate results for first five convolution layers of the VGG-16 network implemented on Xilinx Virtex7 VC709 FPGA board in Verilog .

Abstract

Deep neural networks have yielded immense success in speech recognition, computer vision and natural language processing. However, the exploration of deep neural networks for news recommendation systems has received a relatively less amount of attention. Also, different recommendation scenarios have their own issues which creates the need for different approaches for recommendation. With news stories coming from a variety of sources, it is crucial for news aggregators to present interesting articles to the user to maximize their engagement. This creates the need to have a recommendation system which accounts for both the content of the articles and the user preferences. Methods such as Collaborative Filtering, which are well known for general recommendations, are not suitable for news because of the short life span of articles and because of the large amount of articles published each day. Typically, it is desirable that a news recommendation system be able to discriminate between and select articles from a pool to recommend to a user as soon as they are published. Apart from this, such methods do not harness the information present in the sequence in which the articles are read by the user and hence are unable to account for the various interests of the user which may keep changing with time. Alternatively, the other class of models based on Content Filtering, can handle cold start problems but in the long run tend to suffer from the problem of over specialization. In this thesis, we address these issues in a step-by-step manner. We start off with a problem that is commonly associated with deep learning based methods i.e lack of sufficient data. In order to tackle this issue we come up with an item-based collaborative filtering approach which utilizes Markov Decision Process (MDP). We also come up with a novel semantic similarity measure which we incorporate as a reward for the MDP. This helps us gain insights about the various interests that users may have by only having prior knowledge of a few articles read by them in the past. We then move on to the exploration of various deep learning based methods to tackle our problem. We give importance to utilizing the content of the articles, and taking into account the historical reading data of a user. We attempt to solve the cold-start problem, and make effective recommendations for users who have had little to no interaction with items. We design the model to keep learning parameters based on implicit feedback from the users. A description of how we go about it is described as follows. We first come up with a user profiling based approach and use it in combination with a Deep Semantic Structured Model (DSSM). We then later expand the model and utilize a recurrent neural network with an attention mechanism. Such a mechanism helps us discriminate between the various interests of the user. We then come up with the Recurrent Attentive Recommendation Engine (RARE). RARE consistsof two components and utilizes the distributed representations of news articles. The first component is used to model the user’s sequential behaviour of news reading in order to understand her general interests i.e to get a summary of her interests. The second component utilizes an article level attention mechanism to understand her particular preferences. We feed the information obtained from both the components to a Siamese Network in order to make predictions which pertain to the user’s generic as well as specific interests. We carry out extensive experiments to establish the effectiveness of our methods. We also perform experiments to prove the efficacy of our model on solving the item cold-start cases as well as making effective recommendations for users who have had very little interaction with articles. Finally, we experiment with a novel 3D Convolutional Neural Network based model in an attempt to solve similar problems as the ones we tried to address earlier. Applying 3D convolution helped us identify both spatial (features of a particular article) as well as temporal information (features present in the sequence of articles read by the user) which are pertinent to a user’s interest. The initial results of this experimentation are also presented in this thesis.

Abstract

Glaucoma is an eye disease characterized by weakening of nerve cells often resulting in a permanent loss of vision. Glaucoma progression can occur without any physical indication to patients. Hence, early diagnosis of Glaucoma is recommended for preventing the permanent damage to vision. Early Glaucoma is often characterized by thinning of the Retinal Nerve Fiber Layer which is commonly called as RNFL defect (RNFLD). Computer-aided diagnosis (CAD) of eye diseases is popular and is based on automated analysis of fundus images. CAD solutions for diabetic retinopathy have reached more maturity than for glaucoma as the latter is more difficult to detect from fundus images. This is due to the fact that nerve damage appears in the form of subtle change in the background around optic disc. SD-OCT (Spectral Domain - Optical Coherence Tomography), a recently introduced modality, helps to capture 3D information of retina. Hence, it is more reliable for detecting nerve damage in retina compared with fundus imaging. However, a wide usage of OCT is limited due to cost per scan, time and ease of acquisition. This thesis focuses on integrating information from multiple modalities (OCT and fundus) for improving retinal nerve fibre layer defect or detection of RNFLD from fundus images. We examine two key problems in the context of CAD development: i) spatial alignment or registration of two modalities of imaging, namely, 2D fundus and 3D OCT volume images. This can pave way to integrate information across the modalities. Multimodal registration is challenging because of the varied Field of View and noise levels across the modalities. We propose a computationally efficient registration algorithm which is capable of handling complementary nature of modalities. Extensive qualitative and quantitative evaluations are performed to show the robustness of proposed method. ii) Detection of RNFLD from fundus images with good accuracy. We propose a novel CAD solution which utilises information from the 2 modalities for learning a model and uses it to predict the presence of RNFLD from fundus images. The problem is posed as learning from 2 modalities (fundus, OCT images) and predicting from only one (fundus images) with the other (OCT) as missing data. Our solution consists of a deep neural network architecture which learn modality independent representations. In the final part of the thesis we explore the scope of a new imaging modality angiography-Optical Coherence Tomography (A-OCT) in diagnosing Glaucoma. Two case studies are reported which help in understanding the progression of Retinal Nerve Fiber Layer thickness, Capillary Density in normaland glaucoma effected patients. The experiments on new modality has shown potential for considering it as a reliable biomarker along with existing modalities.

Abstract

This study aims to investigate different aspects of sequencing. Sequencing and timing the elements of a sequence are fundamental to animal behaviour and in fact, even basic animal sustenance. Sequencing refers to a set of related events rolling out in time. Animals need to adopt mechanisms to execute several facets of sequencing in their day-to-day life, including the perception of, recognition of and response to sequences. In this thesis, we specifically explore the aspects of duration and order. To investigate duration estimation in humans, we conduct a behavioural study, using the paradigm of the temporal bisection task, and assess the influence of a neuromodulator, caffeine. The task requires candidates to form memory templates of two reference durations, and then, judge which standard a particular stimulus duration is closer to. When tested on a between-subject setting, caffeine administration was observed to lead to an accelerated perception of time. A given probe duration was perceived to be longer by subjects, under the influence of caffeine, than by participants in the control group. We argue that caffeine plausibly modulates memory representation of time, as well as, the internal pacemaker, on the basis of the scalar expectancy theory. We, additionally, conducted a computational investigation to explore the effect of caffeine on duration judgement. We constructed two models, one based on the Gaussian representation of time and one anchored in reinforcement learning. While one model gives us a quantitative account of the empirical data obtained above, the other offers a qualitative understanding of the circuitry involved in the perception of duration. In the second part of the thesis, we investigate the order aspect of timing using the paradigm of vocal learning in songbirds. Song is a sequence of syllables, i.e. motor commands. We attempt to model the HVc, i.e. an avian brain region involved in forming a representation of the conspecific song that the bird tries to imitate, using a biologically realistic adaptation of the pyramidical neurons found in the neocortex. In addition, electrophysiology experiments show that lesions to the basal ganglia homologue in birds, Area X, fatally affects vocal learning, while lesions made after crystallisation of song, has a negligible effect. Such experiments motivated us to set out to explore the circuitry underlying the learning and issuing of the motor commands responsible for song synthesis. We build a biologically realistic insightful model of vocal learning via the anterior frontal pathway, consisting of the Area X, and the eventual consolidation of this skill in the song motor pathway. The thesis concludes with a summary of the work, and a discussion on its limitations and future directions.