Abstract

In this thesis we mainly focus on three major aspects of computational humour recognition. We start with categorizing humour based on the classical theories of humour along with features like theme, emotions and topics. We then look at the problem of recognizing humour in conversations and broadcasted speeches which are more complex and large than short jokes. Finally, we try to differentiate between different types of off-color humour and try to detect insulting remarks from off-colour humour in which dark humour is often misclassified as insulting humour. Most scholarly works in the field of computational detection of humour derive their inspiration from the incongruity theory. Incongruity is an indispensable facet in drawing a line between humorous and non-humorous occurrences but is immensely inadequate in shedding light on what actually made the particular occurrence a funny one. Classical theories like Script based Semantic Theory of Humour(SSTH) and General Verbal Theory of Humour(GVTH) try and achieve this feat to an adequate extent. We adhere to a more holistic approach towards classification of humour based on these classical theories with a few improvements and revisions. Through experiments based on our linear approach and performed on large data-sets of short jokes, we are able to demonstrate the adaptability and show componentizability of our model, and that a host of classification techniques can be used to overcome the challenging problem of distinguishing between various categories and sub-categories of jokes. Almost all the studies done in the field ofcomputational humour recognition has been done on datasets consisting of short jokes, tweets and puns. We try to detect humour in conversations and broadcasted speeches as they are complex and contains more contextual information when compared to short jokes. For the purpose of automatic humour detection in monologues we built a corpus containing humorous utterances of TED talks and for dialogues we analysed data from a popular TV-sitcom Friends whose canned laughter gives an indication of when the audience would react. We classified dialogues/monologues into humorous and non-humorous by using multiple deep learning methods. Our experiments on the data show that such deep learning methods outperform the baseline by 21 accuracy points respectively on the TED Talk dataset. Off colour humour is a category of humour which is considered by many to be in poor taste or overly vulgar. Most commonly, off-colour humour contains remarks on particular ethnic group or gender, violence, domestic abuse, acts concerned with sex, excessive swearing or profanity. Blue humour, black humour and insult humour are types of off-colour humour. Blue and black humour unlike insult humour are not outrightly insulting in nature but are often misclassified because of the presence of insults and harmful speech. We then provide an original data-set consisting of nearly 15,000 instances and a novel approach towards resolving the problem of separating black and blue humour from offensive humour which is essential so that free speech on the internet is not curtailed. Our experiments show that deep learning methods outperforms other n-grams based approaches like SVMs, Naive Bayes and Logistic Regression by a large margin.

Abstract

Autonomously navigating to a goal pose is central to several robotic applications such as inspection, surgery, object manipulation, transportation and more. Often navigation frameworks require the goal pose to be specified in Cartesian space and a localization module that estimates the current pose of the robot. However, in the absence of a metric-scale map and an accurate position estimation module, it is difficult to attain the correct goal pose. Visual servoing approaches, therefore represent the goal pose in the form of an image, which is more intuitive for humans and easier to obtain as compared to the 6-DoF coordinates. The image feedback from a robot's camera is further employed by visual servoing approaches for localizing the robot in image space and navigating it to the goal pose. Existing visual servoing approaches control the robot such that the overlap between the current image and the goal image is iteratively maximized. Therefore, existing approaches do not generalize while servoing to different objects or even instances from the same category. We introduce a novel problem of instance invariant visual servoing, where the robot is required to servo a given instance in the desired pose represented by a goal image of a different instance from the same category. This problem is more relevant in practical scenarios since the same robot can handle multiple object instances without changing the goal image. We initially tackle the problem of instance invariant visual servoing using a geometric approach. We propose a novel visual servoing control law based on a linear combination of 3D models representing the given object category. Our approach is capable of accommodating the deformation in shapes among object instances and the resulting controller is able to smoothly navigate the robot to the desired pose even though the instances are different. To tackle the large variation in appearance across object instances, we further propose part-based visual features that uniquely capture the locations of an object's parts in images. Advantages of using such part-aware semantics are two-fold. (i) It conceals the illumination and textural variations from the visual servoing algorithm. (ii) Semantic keypoints enable us to accurately match descriptors across instances. To circumvent the requirement of 3D models by the above-mentioned geometric solution, we propose a discriminative learning based framework for visual servoing across object instances. This approach is able to learn the desired pose from previously seen examples of goal images and is able to navigate the robot despite appearance and shape variations. Specifically, we learn a binary classifier that discriminates the goal image from images captured from other viewpoints for that object category. The classification error is then used to navigate the robot towards the desired pose. Furthermore, we design controllers for linear, kernel and exemplar Support Vector Machine (SVM) and empirically discuss their performance in the visual servoing context. To address large intra-category variation in appearance, we introduce Principal Oriented Glyph (POG) features that are easier to obtain as compared to part-based features. We next consider the problem of navigating a robot in unstructured environments using deep learning techniques. Motivated by recent breakthroughs in performance of data driven methods on recognition and detection tasks, we aim to learn visual feature representations suitable for servoing tasks in unstructured and unknown environments. In contrast to existing visual servoing approaches that require the knowledge of a scene geometry especially depth and camera parameters, we present an end-to-end learning based approach for visual servoing in diverse scenes that is agnostic to the knowledge of a scene's geometry and camera parameters. This is achieved by training a Convolutional Neural Network (CNN) over color images with synchronized camera poses. After assessing the capability of CNNs in classical visual servoing tasks. We next apply them to our instance invariant visual servoing task. However, catering to shape, appearance and viewpoints variations all together is a difficult task for a CNN. Thus, instead of directly regressing to the desired pose, we use CNNs to predict locations of part based keypoints in images so that appearance variations are decoupled. To tackle geometrical and viewpoint variations, we further present a pose induction strategy based on the part-based keypoints predicted by our network. Specifically, we reconstruct the locations of parts of the given instance in 3D, from a sequence of 2D keypoints estimated by the network and align this reconstruction with a template instance. This aligned reconstruction is then used to localize the robot and navigate it the desired pose. Finally, we employ our framework for the autonomous inspection of vehicles using Micro Aerial Vehicles (MAVs), which is vital

Abstract

Homo sapiens have evolved over centuries owing to their curiosity. We have travelled millions of miles and spent countless years in quest for answers. Fast forward to today, the invention of the internet has democratised access to enormous amounts of information at the click of a button. For the past two decades, search engines like Google have been the primary way of finding information online. However, search engines break down when a user presents complex information expressed as natural language questions. Additionally, as more people access the web from mobile devices on the move, the need for software that can interpret and answer questions becomes more important. Recently, voice assistants aim to provide us with natural and declarative access to information. The proliferation of the Internet in newer demographics has led to many new phenomena. One such phenomenon called “code-mixing” has been frequently observed in online social media and has attracted a lot of research and interest from sociolinguists. Code-mixing (CM) is the mixing of linguistic units, such as phrases, words or morphemes, from multiple languages. It is prevalent in countries where people speak multiple languages with native proficiency such as India. It helps in speeding-up communication and allows a wider variety of expression due to which it has become an accessible mode of expression on the web. Hence, there is a need for the development of software to allow these communities to communicate naturally and ask questions in code-mixed. In this thesis, we study the design of an effective Question Answering (QA) framework for QA over Knowledge Bases (KB). A given question may have multiple candidate answers amongst which we have to pick the correct one. We adopt a two-step approach: candidate (answer) generation and candidate (answer) re-ranking to answer questions. We propose a Triplet-Siamese-Hybrid CNN (TSHCNN) to re-rank candidate answers. We show experiments on the SimpleQuestions dataset which uses Freebase as the underlying KB to answer questions. We demonstrate the effectiveness of our approach on monolingual question answering task for English, beating previous benchmarks. We also study how existing QA resources for English and bilingual embeddings can be effectively used in a resource-constrained setting such as for monolingual Hindi and CM QA. We discuss the creation of our QA dataset for CM questions, study the effectiveness of our QA framework and do an in-depth analysis of approaches that work for CM questions. We show that our system that works with CM questions in its original form fares better compared to translating them and using a QA system trained on monolingual English to obtain answers.

Abstract

Scene understanding is a very crucial task in computer vision. With the recent advances in research of autonomous driving, road scene segmentation has become a very important problem to solve. Few exclusive datasets were also proposed to address this research problem. Advances in other methods of deep learning gave rise to deeper and computationally heavy models and were subsequently adapted to solve the problem of semantic segmentation. In general, larger datasets and heavier models are a trademark of segmentation problems. This thesis is a direction to propose methods to reduce annotation effort of datasets and computational efforts of models in semantic segmentation. In the first part of the thesis, we introduce the general sub-domain of semantic segmentation and explain the challenges of dataset annotation effort. We also explain a sub-field of machine learning called active learning which relies on a smart selection of data to learn from, thereby performing better with lesser training. Since obtaining labeled data for semantic segmentation is extremely expensive, active learning can play a huge role in reducing the effort of obtaining that labeled data. We also discuss the computational complexity of the training methods and give an introduction to a method called knowledge distillation which relies on transferring the knowledge from complex objective functions of deeper and heavier models to improve the performance of a simpler model. Thus, we will be able to deploy better performing simpler models on computationally restrained systems such as self-driving cars or mobile phones. The second part is a detailed work on active learning in which we propose a Region-Based Active Learning method to efficiently obtain labelled annotations for semantic segmentation. The advantages of this method include: (a) using lesser labeled data to train a semantic segmentation model (b) proposing efficient models to obtaining the labeled data thereby reducing the annotation effort. (c) transferring model trained on one dataset to another unlabeled dataset with minimal annotation cost which significantly improves the accuracy. Studies on Cityscapes and Mapillary datasets show that this method is effective to reduce the dataset annotation cost. The third part is the work on knowledge distillation and we propose a method to improve the performance of faster segmentation models, which usually suffer from low accuracies due to model compression. This method is based on distilling the knowledge during the training phase from models with high performance to models with low performance through a teacher-student type of architecture. The knowledge distillation allows the student to mimic the performance of the teacher network, thereby improving its performance during inference. We quantitatively and qualitatively validate the results on three different networks for Cityscapes dataset. Overall, in this thesis, we propose methods to reduce the bottlenecks of semantic segmentation tasks, specifically related to road scene understanding: the high cost of obtaining labeled data and the high computational requirement for better performance. We show that smart annotation of partial regions in data can reduce the effort of obtaining labels. We also show that extra supervision from a better performing model can improve the accuracies for computationally faster models which are useful for real-time deployment.

Abstract

The problem of estimating scene-depth from a single image has seen great progress recently. It is one of the foundational problems in computer vision and hence been studied from various angles. Since the advent of deep learning, most of the approaches are data driven. These methods train high-capacity models with large amount of data in an end-to-end fashion. They rely on ground-truth depth, which is hard to capture and process. Recently self-supervised methods have been proposed which rely on view supervision as an alternative. These methods minimize photometric reconstruction error in order to learn depth. In this work,we propose a geometry-aware generative adversarial network to generate multiple novel views from a single image. Novel views are generated by learning depth as an intermediate step. The synthesized views are discerned from real images using discriminative learning. We show the gains of using the ad- versarial framework over previous methods. Furthermore, we present a structured adversarial training routine to train the network, going from easy examples to difficult ones. The combination of adversarial framework, multi-view learning, and structured training produces state-of-the-art performance on unsupervised depth estimation for monocular images. We also compare our method with human depth perception by conducting a series of experiments. We investigate the existence of monocular depth cues like relative size, occlusion and height in the visual field in artificial vision systems. With quantitative and qualitative experiments, we highlight the shortcomings of artificial depth perception and propose future avenues for research.

Abstract

Sentiment is an important feature of any text. Feelings and opinions are purely subjective, unlike facts. Analyzing these opinions accurately is a challenging task. However, using the sentiment, we are looking to understand the attitude of a writer with regard to a specific topic in a piece of text. Sentiment analysis has been an important topic of research since a very long time. Simply put, the task involves a system that would be predicting (classifying) the sentiment of a given input sentence as either positive, neutral or negative. Sometimes we have more fine-grained classes and sometimes we even evict the neutral class based on what we need to use the sentiment for. The capability to automatically compute the sentiment for things such as user reviews, critique on movies, e-commerce etc is of tremendous importance. It helps organizations and businesses take multiple large scale important decisions which are based strongly on user satisfaction. Movies are one of the most prominent means of entertainment. The favourite pastime of many people would be watching movies. They provide a break from the hectic schedule a person goes throughout the day. People often prefer to express their views online in English as compared to other local languages because of many factors such as non availability of applications which makes typing in local languages easy whereas typing in English is very easy. Even if one manages to write a review in a local language there are very less number of platforms which accept such reviews. This leaves us with a very little amount of data in languages apart from English to work on. The widespread use of the Internet in recent times has led to large volumes of data related to movies being generated and shared online. People watch movies, write reviews and give ratings online. This method of broadcasting opinions has gained a lot of popularity ever since. However, this led to a decrease in the quality of opinions that were shared. Due to this, people find it challenging and difficult to browse through all the opinions. This issue of bogus and random opinions is witnessed in a lot of cases where the user can provide feedback in a quick manner such as multiple choice options, checkboxes etc. Movie ratings and genres fall in this category. Movie ratings and genres play an important role in tasks such as user movie recommendations, verifying the relationship between user-submitted reviews and ratings etc. The ability to predict the correct rating/genre of a movie would be useful considering these aspects. In this thesis, we attempt to solve problems we face in real life with the help of sentiment. One problem is encountering invalid information(wrong ratings) by using the sentiment of the movie reviews. We propose methods to predict the movie rating based on its summary. We then set out to use priors that are generally available with movie summaries in order to improve the accuracy. In order to achieve this, we consider the associated movie reviews as well while predicting the rating and provide insights on why this helps our models perform better. We use sentiment of reviews along with the summary in order to predict the rating more accurately since the sentiment captures a lot of essential information that can aid rating prediction. The majority of methods used to study NLP problems employed shallow machine learning models and time-consuming, hand-crafted features for a very long time. Many problems were encountered due to this. One of the problem is the curse of dimensionality as linguistic information was represented with sparse representations. However, with the recent popularity and success of word embeddings, sentence embeddings, neural network based models have achieved better results on various language-related tasks as compared to traditional machine learning models like SVM or logistic regression. Deep learning methods are starting to out-compete the statistical methods on some challenging natural language processing problems with singular and simpler models. We propose deep learning models to solve our problem at hand and compare the results with traditional methods. The other problem is preserving sentiment during translation. An important aspect of translation is ensuring that the complete meaning (including the sentiment/opinion) of the source text is translated appropriately. Consider product reviews, retaining the sentiment is an important aspect of translating these reviews. We cannot afford to have phrases such as “not at all good” get translated to “not good” since there is a significant difference in the sentiment of the two phrases. Most of the metrics that evaluate machine translators only consider n-grams, number of overlaps, etc in order to produce translation scores. This does not account for the preservation of sentiment. In order to solve this other problem, we propose a new metric that considers the sentiment of a sentence along with the existing me

Abstract

The Discrete Fourier transform (DFT) is one of the most prominent algorithms of signal processing. The algorithm for computing the DFT effectively is known as the Fast Fourier Transform (FFT) algorithm. FFT core is the key component in many of the digital signal processing, Audio, and video signal and telecommunication systems applications and most of the advanced communication systems need an FFT core to process the Orthogonal Frequency Division Multiplexing (OFDM) function. Cooley-Tukey radix-2 FFT algorithm is one of the most popular FFT algorithms which tries to reduce the computational complexity of DFT. In recent times, the use of FFT processors is even more significant and it is rated among the top 10 algorithms with greatest influence on the practice and development of engineering. Since an FFT core is the basic module in all of these applications, the performance and accuracy of the FFT core are of prime importance. Hence, we use the IEEE 754 half precision format to represent the floating point numbers. The increasing demands for speed and performance in some real-time DSP applications make sequential systems inadequate and led to the design of special-purpose parallel FFT processors that include several arithmetic units operating in parallel. In this work, we try to implement the new Modified Radix-2 decimation-in-time (DIT) FFT algorithm with reduced arithmetic complexity which is in the form of Tangent FFT algorithm. As implementation of the Tangent FFT forms is relatively unexplored even though these FFTs give a lot of advantage by reducing the number of computations. This algorithm is implemented using a design which is similar to that of a perfect shuffle architecture as they possess high computation rates by parallel processing and low latencies. Additionally, we proposed a new bit slicing scheme for the IEEE 754 half precision floating representation format to represent the twiddle factors in the proposed architecture to obtain a better implementation of the Modified Radix-2 DIT FFT algorithm. Further, we tried to implement the FFT algorithm using optimal Canonical Signed Digit (CSD) based complex constant multipliers in an attempt to reduce the area occupied by the complex multipliers and increase the precision/SNR of the output. The entire proposed system has been designed using Verilog Hardware description language. All the simulations are performed and verified using Xilinx Vivado tools and synthesis are done on Cadence Encounter RTL compiler with a UMC CMOS 180nm technology library

Abstract

Multiple access schemes play a crucial role in determining the performance of wireless networks. Already established access schemes which were part of the first generation (1G) to fourth generation (4G) cellular networks are time division multiple access (TDMA), frequency division multiple access (FDMA), code division multiple access (CDMA), and orthogonal frequency division multiple access (OFDMA). Though these techniques provide good service with simple receivers, they lack the ability to tackle emerging challenges arising due to monumental rise in the number of devices accessing wireless channel. In this background, non-orthogonal multiple access (NOMA) is seen as a suitable candidate not only to handle the connectivity challenges but also to improve spectral efficiency significantly. NOMA has been in the research literature for long, but it has garnered a lot of interest recently and extensive research is underway. Major hurdles in making NOMA a reality is the complexity involved in realizing successive interference cancellation (SIC) and imperfections in the channel state information (CSI). Hence to make NOMA a reality, effective algorithms that addresses these issues are needed. In this thesis, we focus on system design with optimal resource allocation and relay precoder design in a cooperative relay network, which employs NOMA. The proposed optimal design aims at maximizing the weighted sum-rate. The design problems turn out to be a non-convex optimization problem. We employ minorization-maximization (MM) technique to tackle the non-convexity of the optimization problems. Then, we address the problem of maximization of weighted sum-rate (WSR) while making the system performance robust to errors in the CSI. To solve this problem, we establish a relationship between WSR and weighted mean square error (WMSE) and solve the latter problem which is simpler to obtain a solution for the former. Cognitive radio (CR) is another technology that can help in solving the spectrum scarcity problem. NOMA and CR in combination is expected to improve spectral efficiency significantly. But very few works have explored the two technologies together. So, we consider the design of secondary system working on principles of NOMA in a cognitive radio network by ensuring interference to a primary node is below the permissible threshold. Then, we modify the algorithm to take into account the errors in the CSI and thus ensuring the system performance is robust. Apart from the theoretical analyses, extensive numerical simulations have been performed in order to verify the effectiveness of our proposed designs in terms of sum-rate, weighted mean square error, convergence, etc.

Abstract

The construction in hilly regions is generally constrained by local topography resulting in vertical as well as horizontal irregularities in the buildings. These irregularities cause irregular distributions of mass and stiffness of the structures. As a result, the dynamic properties such as natural period and mode shape of such buildings vary significantly when compared to those constructed on the flat terrain. During past earthquakes like 2005 Kashmir earthquake, 2011 Sikkim earthquake, 2015 Nepal earthquake, severe damages were observed in such constructions on hill slopes. The damage attributed to these buildings was observed to be disproportionately higher when compared to those on a flat land, despite smaller intensity of shaking. This damage is attributed to several reasons viz.,a. Landslide issues, Natural period, Shear distribution in columns and Torsional effects. Therefore, there is an urgent need to understand the seismic behaviour of such buildings standing on hill slopes and include corresponding changes in codal provisions. In the current study, two important seismic parameters, Natural Period and Base Shear of the structure have been studied. Fundamental Natural Period of a building is one such important dynamic property which has its significant use especially during the design phase of the building i.e., while calculation of base shear. Hence, to understand the same the natural period study has been divided into two parts i.e., in first part, natural period of buildings constructed on hill slopes have been obtained and in the second part, parametric study was done to obtain the best fit curve. In the first part, Natural Period of nine buildings constructed on the hill slopes of Mussoorie were obtained using Ambient Vibration testing. From the recorded data, it has been observed that the existing code provisions for estimating the approximate natural period of a building are not efficient enough in the case of such buildings with a standard deviation of errors of 0.1. Since the available data is too less to derive any expression, in the second part, the data has been extrapolated by modelling 270 similar buildings in SAP 2000 by varying different parameters like angle of inclination (5o to 45o), base dimension (3x3 to 5x5) and height (3 to 7 stories). Regression Analysis has been performed on the above data to derive an Empirical Relation to estimate Fundamental Natural Period of buildings constructed on slopes. It has been observed that the standard deviation of errors of the proposed expression with respect to the recorded data has reduced to 0.05. Another major issue with the step back buildings constructed on hill slopes is the distribution of the total Base Shear acting on the structure among the respective columns. The irregular configurations of a step back building causes stiffness differences in the structure. This results in short column/uphill column attracting the maximum base shear causing severe damages to the corresponding short columns and hence the whole structure. For example, in a 3x3 bays building standing on 30o slope, it has been observed that all the short columns were attracting 118% of the total base shear acting on the structure whereas the largest ones were attracting only 1.5%. To understand and reduce these stiffness differences, different building configurations using strut elements were studied. And the configuration of struts which supports optimum distribution of stiffness differences has been identified. The short column of proposed model attracts only 38% of total base shear acting on the structure and hence reducing the design cost as well as damage vulnerability of the structure.

Abstract

Energy consumed in the building sector accounts for 20.1% of the total delivered energy consumed worldwide. In 2016, the residential and commercial sectors in the US accounted for around 40% of the total US energy consumption, which largely contributes to green house gas (GHG) emissions. Immediate action in the building sector is essential to avoid hazardous climate change and also to incorporate energy efficiency in the building design. The early design stage offers more opportunities for use of energy simulation in achieving energy efficiency because there are fewer constraints from other design decisions. Although there are a large number of building simulation tools available, these tools are mostly limited to the design development stage and not the early design stage. In the early design stage, there are a large number of variables involved, and it is difficult for design teams to consider all the solutions and take decisions because this might lead to missing some parameter combinations. In addition, in the early design stage, there are very few users of building energy simulation tools because: • Performing energy simulation requires sufficient knowledge of building physics and expertise • Simulating a large number of combinations takes significant time and computation power • After simulating a large number of combinations, extracting relevant information from output data can be tedious and difficult to understand To fill this gap, an early design methodology has been developed for energy efficient building design. The methodology includes: 1. Getting inputs from the user in a simple user interface 2. Generating input files for the simulations 3. Performing energy simulations efficiently by using parallel simulations and/or using machine learning techniques to speedup simulations 4. Identifying clusters in output data to communicate the strategies to users in the form of humanly describable rules. The contributions of this thesis are as follows: (i) Machine learning regression techniques to speed up the parametric simulations. There were two case studies performed for two different cities in which the use of two algorithms (kNN and SVR) were shown to increase speed of parametric energy simulations by approximately 40 times. (ii) Axis Aligned Hyper Rectangle (AAHR) clustering method for building energy simulation data was used to produce humanly describable or easy to understand rules for design strategies. An energy model was developed and simulated for five cities in different climates and findings from these are discussed in the thesis. (iii) Development of a tool called eDOT to demonstrate the working of early design methodology. eDOT considers five variable parameters, which are orientation, aspect ratio, window to wall ratio (WWR), overhang profile angle, and glass type. Once the user enters these parameters, backend simulations are performed in parallel on the cloud. The user is provided with interactive visualisation of output data. An algorithm is applied to find strategies in output data in humanly understandable rules. To use this tool, the user does not need not to be an expert at using any energy simulation tools. (iv) Development of the Surrogate City Finder (SCF) tool to find similar weather data based on location, latitude range, altitude range, temperature range and distance range. (v) An approach was developed using EnergyPlus simulations to calculate the equivalent SHGC for various fixed/dynamic shading configurations. For the fenestration, the prescriptive method provides maximum allowable U-value and Solar Heat Gain Coefficient (SHGC). Some modifications are allowed to SHGC if fixed horizontal overhangs and/or vertical fins are installed on the fenestration. The modified SHGC with shades is called the equivalent SHGC for code compliance requirements. In this approach, equivalent SHGC is calculated using energy simulations for a reference case (with fixed/dynamic shades) and other parametric cases with varying SHGC (without shades).