Abstract

High-level synthesis (HLS) has received significant attention in recent years for improving programmability of FPGAs. One could raise the level of abstraction further by using domain-specific languages (DSLs), improving productivity and performance simultaneously. PolyMage is a domainspecific language and compiler for image processing pipelines. Its PolyMage-HLS backend translates an input expressed as a DAG of image processing stages through the DSL into an equivalent circuit that can be synthesized on FPGAs, while leveraging an HLS suite. The power and area savings while performing arithmetic operations on fixed-point data type are well known to be significant over using floating-point data type. PolyMage-HLS stores data at each stage of a pipeline using a fixed-point data type (α, β) where α and β denote the number of integral and fractional bits. The integral bitwidth (α) requirement at a pipeline stage can be inferred from its range. In this work, we first propose an interval-arithmetic based range analysis algorithm to estimate the number of bits required to store the integral part of the data at each stage of an image processing pipeline. The analysis algorithm uses the homogeneity of pixel signals at each stage to cluster them and perform a combined range analysis. Secondly, we propose a software architecture for easily deploying any kind of interval/affine arithmetic based range analyses in the DSL compiler. Thirdly, we show that interval/affine arithmetic based techniques fail to take into account correlated computations across stages and hence could lead to poor range estimates. These errors in range estimates accumulate across stages, especially for iterative programs, such as Horn-Schunck Optical Flow, resulting in estimates nearly unusable in practice. Then, we propose a new range analysis technique using Satisfiability Modulo Theory (SMT) solvers, and show that the range estimates obtained through it are very close to the lower bounds obtained through profile-driven analysis. Finally, for estimating fractional bitwidth (β) requirement at each stage of the pipeline, we propose a simple and practical heuristic search algorithm, which makes very few profile passes, as opposed to techniques such as simulated annealing used in prior work. The analysis algorithm attempts to minimize the number of fractional bits required at each stage while preserving an application-specific quality metric. We evaluated our bitwidth analysis algorithms on four image processing benchmarks listed in the order of increasing complexity: Unsharp Mask, Down-Up Sampling, Harris Corner Detection and Horn-Schunck Optical Flow. The performance metrics considered are quality, power and area. For example, on Optical Flow, the interval analysis based approach showed an 1.4× and 1.14× improvement on area and power metrics over floatingpoint representation respectively; whereas the SMT solver based approach showed 2.49× and 1.58× improvement on area and power metrics when compared to interval analysis.

Abstract

One of the major challenges for information theorists is to build such communication schemes which allow participants to communicate efficiently . Index Coding , which was first introduced by Birk and Kol in 1998 [1] as Coding on Demand by an Informed Source is a communication scheme dealing with a broadcast channel in which receivers have prior side information about the messages to be broadcast. Exploiting the knowledge about the side information, the sender may significantly reduce the number of required transmissions , in turn improve the efficiency of the communication over this type of broadcast channels. The index coding problem provides a simple yet rich model for several important engineering problems in network communication, such as content broadcasting, peerto-peer communication, distributed caching, device-to-device relaying, interference management and oppoturnistic wireless networks. It also has close relationships to network coding, distributed storage and guessing games. The Index Coding problem is to identify the minimum number of transmissions (optimal length) to be made so that all receivers can decode their wanted messages using the transmitted symbols and their respective prior information and also the codes with optimal length. Optimal Index codes can be constructed using graph theoretic methods, algebraic methods, matrix completion, linear programming, interference alignment, etc. It is known that the index coding problem can be represented using a side-information graph G (with number of vertices n equal to the number of source symbols). The size of the maximum acyclic induced subgraph of side information graph, denoted by MAIS is a lower bound on the broadcast rate. For index coding problems with MAIS = n − 1 and MAIS = n − 2, prior work has shown that binary (over F2) linear index codes achieve the MAIS lower bound for the broadcast rate and thus are optimal. In this thesis, we use the the duality relationship between network coding and index coding to show that for a class of index coding problems with MAIS = n − 3, binary linear index codes achieve the MAIS lower bound on the broadcast rate. In contrast, it is known that there exists index coding problems with MAIS = n−3 and optimal broadcast rate strictly greater than MAIS. Relevant figures has been included for clear understating of reader.

Abstract

The problem of source coding is one of the central areas of information theory and coding. Designing source codes for the broadcast applications, where the receivers may have some prior information about the source, before it was sent is the key idea in this thesis. The challenge is to come with encoding schemes that exploit the side-information from the receivers in order to reduce the length of the code. This problem of source coding with side-information is called as index coding, first introduced by Birk and Kol [1] as coding on demand by an informed source. The index coding problem provides a simple yet rich model for several important engineering problems in network communication, such as content broadcasting, peer-to-peer communication, distributed caching, device-to-device relaying, interference management and opportunistic wireless networks. It also has close relationships to network coding, distributed storage, and guessing games.The Index Coding problem is to identify the minimum number of transmissions (optimal length) to be made so that all receivers can decode their wanted messages using the transmitted symbols and their respective prior information and also the codes with optimal length. Optimal Index codes can be constructed using different methods which include graph theoretic methods, algebraic methods, matrix completion, linear programming, interference alignment,etc. An index coding problem can be represented using a demand graph (with number of edges equal to the demands at receivers and number of vertices n equal to the number of source symbols). Various graph theoretic bounds were well studied in the literature. Uniprior index coding problem is studied in this thesis. For single uniprior index coding problem , in which each receiver has a single unique side-information symbol, a polynomial complexity construction of an optimal index code is known from prior work. In this thesis we model the uniprior index coding problem as a supergraph G S , and focus on a class of uniprior problems defined on special supergraphs known as generalized cycles G gc , in which the sizes of demand set and the side-information set are equal at every receiver. For such problems, we prove upper and lower bounds on the optimal broadcast rate. Using a connection with Eulerian directed graphs, we also show that the upper and lower bounds are equal for a subclass of uniprior problems. We show the NP-hardness of finding the lower bound for uniprior problems on generalized cycles, hence contrasting such uniprior problems with single uniprior problems. Finally we look at a simple extension of the generalized cycle uniprior class for which we give bounds on the optimal rate and show an explicit scheme which achieves the upper bound.

Abstract

Human body tracking typically requires specialized capture set-ups. Although pose tracking is available in consumer devices like Microsoft Kinect, it is restricted to stick figures visualizing body part detection. In this thesis, we propose a method for full 3D human body shape and motion capture of arbitrary movements from the depth channel of a single Kinect, when the subject wears casual clothes. We do not use the RGB channel or an initialization procedure that requires the subject to move around in front of the camera. This makes our method applicable for arbitrary clothing textures and lighting environments, with minimal subject intervention. Our method consists of 3D surface feature detection and articulated motion tracking, which is regularized by a statistical human body model [40]. We also propose the idea of a Consensus Mesh (CMesh) which is the 3D template of a person created from a single view point. We demonstrate tracking results on challenging poses and argue that using CMesh along with statistical body models can improve tracking accuracies. Quantitative evaluation of our dense body tracking shows that our method has very little drift which is improved by the usage of CMesh We explore the possibility of improving the quality of CMesh using RGB images in a post processing step. For this we propose a pipeline involving Generative Adversarial Networks. We show that CMesh can be improved from RGB images of the original person by learning corresponding relative normal maps ( N R map ). These N R map have the potential to encode the nuances in the CMesh with respect to ground truth object. We explore such method in a synthetic setting for static human like objects. We demonstrate quantitatively that details which are learned from such a pipeline are invariant to lighting and texture changes. In future the generated N R map can be used to improve the quality of CMesh

Abstract

Keywords: Speech signal analysis; analysis of production dynamics; zero time windowing; heavily decaying window; Hilbert envelope; group delay spectrum; dominant resonant frequencies; glottal activity; open quotient; vowel nasalization; nasal segments; oral nasal tract coupling; Speech signal is the output of a dynamic production mechanism. The articulators involved in the production process move continuously at different rates, giving rise to a time–varying vocal tract transfer function. The process of speech production is dictated by the linguistic and para–linguistic information being conveyed by the speaker. The resulting speech signal embodies the characteristics of this time–varying acoustic system, which is easily perceived by humans. Algorithmic processing of the speech signal attempts to study these characteristics, to uncover the behavior of the acoustic response of the vocal tract system during the production of different sounds. Analysis of the dynamics of the production mechanism from the speech signal is a difficult task due to rapid movement of articulators and complex interaction among different elements of production. The elements of the speech production comprise of static and dynamic articulators. Movement of the articulators can be voluntary, driven by the motor functions (such as the movement of lips, tongue, jaw etc.), to produce a desired sequence of sounds. There are some involuntary or semi– voluntary movements of these articulators that take place due to factors such as the air pressure and muscular tension. Glottal opening and closing, and the velar opening and closing are two such examples of semi–voluntary movements, which are caused during the production of voiced and nasal sounds, respectively. Changes in the acoustic response of the vocal tract system during these events are rapid, and tracking the dynamics of such responses is difficult. Assumption of stationarity or quasi–stationarity behavior of the system limits our comprehension of the production viii ix process. A detailed study of the dynamic nature of the acoustic transfer function is therefore necessary to gain deeper insight into the speech production process. Traditional methods to study the speech production mechanism assume stationarity of the vocal tract acoustic system over a short duration of 10–30 ms. The spectral characteristics for these segments are extracted using methods based on short time analysis of speech. The spectral envelope obtained from the magnitude of the short time Fourier transform (STFT) is interpreted as the frequency response of the acoustic system. The fine structure of the magnitude is attributed mostly to the excitation component of the production system. The gross and fine structure of the spectrum is exploited to develop models for the speech production system. The source–system modeling is a paradigm based on this principle. Analysis using bank of bandpass filters is another method to study the spectral characteristics. The output energy of each filter within a short (10–30 ms) segment is attributed to the spectral energy in the band corresponding to the filter. There is an implicit quasi–stationarity assumption in the implementation of these methods. These methods do not bring out the underlying production mechanism well enough to explain dynamics of the articulatory phenomena. The objective of this study is to examine changes in the vocal tract system to bring out the dynamic nature of the system during production of different sounds. The glottal activity in voiced speech involves opening and closing of the vibrating vocal folds, which results in coupling and decoupling of the subglottal and supraglottal tracts, respectively. The production of nasal consonants and nasalized vowels also involves coupling and decoupling of the nasal tract due to velar movement. These changes occur continuously over a short interval of time, and hence are difficult to track using analysis based on quasi–stationarity assumption. A method is proposed to track the dominant resonance characteristics of the vocal tract acoustic system continuously as a function of time. The method is useful to study the dynamics of the production system caused by the movement of vocal folds, velum, and other articulators. In order to study the dynamic characteristics of the vocal tract system during speech production, a speech analysis method, called the zero time windowing (ZTW) is used. The ZTW method gives spectral characteristics of the time varying system at each instant of time, reflecting the instantaneous behavior of the production system. The spectral characteristics are derived using the Hilbert envelope of the numerator of the group delay (HNGD) function. The behavior of the x time varying system response is captured using the dominant resonance frequency (DRF) obtained from the HNGD spectrum. Changes in the production system due to co

Abstract

Sports video data is recorded for nearly every major tournament but remains archived and inaccessi-ble to large scale data mining and analytics. However, Sports videos have a inherent temporal structure,due to the nature of sports themselves. For instance, tennis, comprises of points, games and sets played between the two players/teams. Recent attempts in sports analytics are not fully automatic for finer details or have a human in the loop for high level understanding of the game and, therefore, have limited practical applications to large scale data analysis. Many of these applications depend on specialized camera setups and wearable devices which are costly and unwieldy for players and coaches, specially in a resource constrained environments like India. Utilizing very simple and non-intrusive sensor(s) (like a single camera) along with computer vision models is necessary to build indexing and analytics systems. Such systems can be used to sort through huge swathes of data, help coaches look at interesting video segments quickly, mine player data and even generate automatic reports and insights for a coach to monitor. Firstly, we demonstrate a score based indexing approach for broadcast video data. Given a broadcast sport video, we index all the video segments with their scores to create a navigable and searchable match. Even though our method is extensible to any sport with scores, we evaluate our approach on broadcast tennis videos. Our approach temporally segments the rallies in the video and then recognizes the scores from each of the segments, before refining the scores using the knowledge of the tennis scoring system. We finally build an interface to effortlessly retrieve and view the relevant video segments by also automatically tagging the segmented rallies with human accessible tags such as ‘fault’ and ‘deuce’. The efficiency of our approach is demonstrated on broadcast tennis videos from two major tennis tournaments. Secondly, we propose an end-to-end framework for automatic attributes tagging and analysis of broadcast sport videos. We use commonly available broadcast videos of badminton matches and, un- like previous approaches, we do not rely on special camera setups or additional sensors. We propose a method to analyze a large corpus of broadcast videos by segmenting the points played, tracking and recognizing the players in each point and annotating their respective strokes. We evaluate the performance on 10 Olympic badminton matches with 20 players and achieved 95.44% point segmentation accuracy, 97.38% player detection score (mAP@0.5), 97.98% player identification accuracy, and stroke segmentation edit scores of 80.48%. We further show that the automatically annotated videos alone could enable the gameplay analysis and inference by computing understandable metrics such as player’s reaction time, speed, and footwork around the court, etc. Lastly, we adapt our proposed framework for tennis games to mine spatiotemporal and event data from large set of broadcast videos. Our broadcast videos include all Grand Slam matches played between Roger Federer, Rafael Nadal and Novac Djokovic. Using this data, we demonstrate that we can infer the playing styles and strategies of tennis players. Specifically, we study the evolution of famous rivalries of Federer, Nadal, and Djokovic across time. We compare and validate our inferences with expert opinions of their playing styles.

Abstract

Forecasting energy consumption enables users to plan their resource utilization optimally. For this,it is essential to establish a reliable forecast of consumption profile in real-time, which is very challenging and still emerging. In this study, we focus on forecasting HVAC cooling energy consumption at a sub-hourly forecasting horizon which enables us to analyze and control the demand in real-time. With the use of energy meters and BACnet technology in the target building (KCIS), we have collected the cooling energy consumption data and HVAC systems usage at a higher resolution in the target building. In this methodology, weather data from Weather Underground of Gachibowli, Hyderabad location has been used. These recorded weather parameters are being used as explanatory variables. Along with the weather parameters, many other feature-engineered explanatory variables which are discussed in the thesis have been used to capture the trend and variation of energy consumption. Required pre-processing and data analysis is performed on the raw data to obtain meaningful information for modeling. We have implemented a one-step-ahead static forecasting using regression and deep learning techniques like Lin- ear Regression, Support Vector Machine (SVM) Regression, K-Nearest Neighbour (KNN) Regression, Gradient Boosting Regression Trees (GBRT), Long Short-Term Memory (LSTM). By comparing the results obtained from using the aforementioned forecast learning algorithms for various combinations of input parameters/explanatory variables, we have carried out a critical model evaluation to attain an ex-emplary forecasting model. All the methods were observed to fit comparatively well. However, GBRT was observed to provide better results marginally and consistently. Our results show that we could improve forecast accuracy by almost 45% by including feature-engineered variables. We have achieved a Mean Absolute Percentage Error (MAPE) of 14.7% with GBRT for the best combination of input features. We observe that at higher values of energy consumption, we are able to forecast with higher accuracy. For energy values greater than 10kWh, the GBRT model was observed to produce precise results with an accuracy of 9.6% MAPE. The model evaluation was done for a data-set ranging from the months of April to August in 2018, as the data of the real building is available during this period. We further evaluate the performance of the model throughout the year based on the data generated from the EnergyPlus simulation software. As calibrating the target building is an onerous and massive task, we design a basic model with realistic set-point, occupancy and internal load schedules. We have achieved a MAPE of 6.7% for the simulated data. Based on recent studies in peak demand reduction, simple measures like increasing cooling set-point temperatures in the case of HVAC systems load management, serves as an effective demand response. Majority of the past studies in demand response focus on developing strategies that reduce peak demand. To practically adapt these strategies, there is a necessity that the user is capable of understanding the impact of these strategies and aware of how to respond, especially during peak demand. Research in estimating the potential of DR programs is gaining momentum these days and a lot of research has to be done to develop reliable applications that can be used in real-time. In the next part of the thesis, we have therefore focused on developing a model that predicts the impact of changing the HVAC set-point temperatures on the cooling energy demand. The model was thus evaluated on the basis of the afore-mentioned simulated data. The model performance was evaluated by changing set-point temperatures for a period of time and comparing them with the results obtained from simulation. An analysis was performed to test the model performance at any given time in a day. We have achieved a MAPE of 7%. In the next part of the thesis, we have demonstrated the application of the prediction model for the demand response. This is a simple and user friendly application using which the decision makers or the building energy engineers can know how to respond during a peak demand. After a DR event when the set-points are back to the normal schedule, we have observed peaks while the energy is ramping up. For buildings which have a prescribed demand limit, these peaks cause demand penalty. Addressing this issue, we propose a strategy to enable stable ramping up process

Abstract

The use of Internet and Social Media has increased exponentially during the last few years globally giving the opportunity to interact with other people and share ideas, thoughts, opinions etc. Large-scale data are shared every day through social media platforms with enormous speed and reach an incredibly huge number of people. Additionally, the possibility of writing anonymous posts and comments makes it even more easy to express and spread hate speech. Social media platforms, in order to improve the experience of their users, are trying to eliminate comments expressing hatred. In this thesis, we mainly focus on developing automated techniques to identify hate-speech. We start with Traditional techniques used for hate-speech detection, followed by advanced techniques for identifying hate-speech on social media. Lastly we attempt to identify cases where these automated systems can fail miserable and propose technique to make these systems robust. Traditionally, various social media platforms, such as Wikipedia, Facebook, YouTube etc, employ hundreds of staff members as part of their human review team which manually read every reported post and decide if it is inappropriate for the users. Nowadays, having administrators detecting which comments are offensive or relying on user reports are ineffective methods not only due to the large-scale of data produces through social media and also due to the fact that during the last years an increase of hatred is noticed in the modern societies. As of 2018, there are about 317,000 status updates on Facebook every 60 seconds making it harder than finding a needle in a haystack every 60 seconds! We commence our research work with the problem of hate-speech detection on social media. Popular techniques like handcrafted feature-based Machine Learning and rule-based approaches have their own disadvantages. The former method is hard to scale and generalize, while the latter fails to provide good results making it unreliable for use in production level systems. Motivated by the improvements in the computational resources and availability of large annotated datasets, we work towards building deep learning based approaches for hate-speech detection. We propose composite models which utilizes the capability of deep models for generating concise representations along with the power of traditional machine learning classifiers. Every word has at least one meaning when it stands alone. But the meaning can change depending on context, or even over time. A sentence full of neutral words can be hostile (“Only whites should have rights”), and a sentence packed with potentially hostile words (“F**k what, f**k whatever y’all been wearing”) can be neutral when you recognize it as a Kanye West lyric. Motivated by this idea, we then move towards building techniques that utilize the contextual information instead of using just the content for identifying hate-speech. We propose a Context-based neural network architecture that utilizes the relationship between the content & context for identifying hate-speech text. Modeling relationship between the content & the neighboring text (known as context), helps in resolving disambiguity in cases where its very difficult to identify if the text is abusive or not just from the content. Finally we identify issues with the automated hate-speech detection systems and propose novel techniques to build robust models. Use of social-media data for training automated systems poses risk of it learning biases which are abundant on social media. Almost every forum is biased in some form, either women, gender, race, right-wing groups, gay/lesbians, religion etc. Learning powerful systems that can change the user experience requires the system to be fair towards every religion, group or gender. The artificial intelligence can not yet detect/understand these lingual nuances. We propose techniques that identify a type of bias, stereotypical bias, in an arbitrary model and then propose techniques on we can learn unbiased-artificially intelligent systems by observations from the biased social media. We carry out extensive experiments to establish the effectiveness of our methods. We also perform experiments to prove the efficacy of our model on a real-world API, the Perspective API, and show its effectiveness in identifying biased systems.

Abstract

In this thesis, we propose a novel planning technique for monocular camera based Simultaneous Localization and Mapping(VSLAM) and fast Frontier detection method for autonomous explorartion in an indoor evvironment. In VSLAM, the objective is to estimate the trajectory of camera and simultaneously identify key 3D feature points and build a map, using camera as a depth sensor. Unlike a laser range finder(LRF) based SLAM, VSLAM is known to be erroneous when camera motion includes an in-place rotation or feature displacement is large for successive frames. We propose a motion planning framework which combines motion primitives based planning and trajectory optimization approach to generate trajectories which exactly connects an initial and final state and also ensures that the change in camera’s field of view between subsequent instances is less than some specified threshold. As a consequence of this motion planning framework we are able to automate SLAM and generate automated monocular SLAM maps of an indoor lab area. We also show when the robot follows the path of a generic planner, PTAM trajectory breaks more often than when it executes the path computed by the proposed planner. This performance improvement is further utilised to develop an autonomous vision based exploration system. Frontier detection is a critical component in autonomous exploration, wherein the robot decides the next best location to move in order to continue its mapping process. The existing frontier detection methods require dense reconstruction which is difficult to attain in a poorly textured indoor environment using a monocular camera. In this effort, we present an alternate method of detecting frontiers during the course of robot motion that circumvents the requirement of dense mapping. Based on the observation that frontiers typically occur around areas with sudden change in texture (zero-crossings), we propose a novel linear chain Conditional Random Field(CRF) formulation that is able to detect the presence or absence of frontier regions around such areas. We use cues like spread of 3D points and scene change around these areas as an observation to CRF. We demonstrate that this method gives us more relevant frontiers compared to other monocular camera based methods in the literature. Finally, we present results in an indoor environment, wherein frontiers are reliably detected around walls leading to new corridors, doors leading to new rooms or corridors and tables and other objects that open up to a new space in rooms.

Abstract

Sequence Variants are the most common variants observed in human genome, with single nucleotide variations (SNVs) occurring at a frequency of 1 in every 300bp and small insertions and deletions (INDELs) at a slightly lower frequency. When these variations occur in the coding regions, they can have a deleterious effect on the protein product and function, leading to disease situations. Reducing costs of next generation sequencing technologies has made it possible to take genetic profiling to the clinical setting. This requires not just fast and accurate algorithms for variant detection, but also a knowledge-base for variant annotation and prioritization to facilitate tailored therapeutics based on an individual’s genetic profile. With this objective, we have developed a pipeline, Sequence Variants Identification and Annotation (SeqVItA) that provides an end-to-end solution for small sequence variants detection, annotation and prioritization on a single platform. Parallelization of the variant detection step and incorporation of numerous resources to infer functional impact, clinical relevance and drug-variant associations, SeqVItA will benefit the clinical and research communities alike. Its open-source platform and modular framework allows for easy customization of the workflow depending on the data type (single, paired or pooled samples), variant type (germline and somatic), and variant annotation and prioritization. Performance comparison of SeqVItA on simulated data, real genomic data, and detection, interpretation and analysis of somatic variants on 24 liver cancer patients is carried out. In this study we demonstrate the efficacy of annotation module in facilitating personalized medicine based on patient’s mutational landscape.