Abstract

The footprint of partial or fully autonomous vehicles is increasing gradually with time. The existenceand availability of the necessary modern infrastructure are crucial for the widespread use of autonomousnavigation. One of the most critical efforts in this direction is to build and maintain HD maps efficientlyand accurately. The information in HD maps is organized in various levels 1) Geometric layer, 2)Semantic layer and 3) Map prior’s layer. The conventional approaches to capturing and extractinginformation at different HD map levels rely heavily on huge sensor networks and manual annotation.This is not scalable to create HD maps for massive road networks. We propose two novel solutionsto address the mentioned problems in this work. The first solution deals with the generation of thegeometric layer with parametric information of the road scene and other one to update information onroad infrastructure and traffic violations in the semantic layer.Firstly, the creation of the geometric layer of the HD map requires understanding the road layout interms of structure, number of lanes, lane width, curvature, etc. Prediction of these attributes as part ofa generalizable parametric model with which road layout can be rendered would suite the creation of ageometric layer. Many previous works that tried to solve this problem rely only on ground imagery andare limited by the narrow field of view of the camera, occlusions, and perspective shortening. This workdemonstrates the effectiveness of using aerial imagery as an additional modality to overcome the abovechallenges. We propose a novel architecture, Unified, that combines aerial and ground imagery featuresto infer scene attributes. We quantitatively evaluate on the KITTI dataset and show that our Unifiedmodel outperforms prior works. Since this dataset is limited to road scenes close to the vehicle, we sup-plement the publicly available Argoverse dataset with scene attribute annotations and evaluate far-awayscenes. We quantitatively and qualitatively show the importance of aerial imagery in understanding roadscenes, especially in regions farther away from the ego-vehicle.Finally, we also propose a simple mobile imaging setup to address and audit several common prob-lems in urban mobility and road safety, which can enrich the information in a semantic layer of HDmaps. Recent computer vision techniques are used to identify street irregularities (including missinglane markings and potholes), absence of street lights, and defective traffic signs using videos obtainedfrom a moving camera-mounted vehicle. Beyond the inspection of static road infrastructure, we alsodemonstrate the applicability of mobile imaging solutions to spot traffic violations. We validate ourproposal on the long stretches of unconstrained road scenes covering over 2000Km and discuss practi-cal challenges in applying computer vision techniques at such a scale. Exhaustive evaluation is carried

Abstract

In recent years there has been a lot of research work regarding media synthesis and generation using artificial intelligence. Artificial intelligence programs like Dall-e can generate an image from a phrase. StyleGAN from Nvidia can create images of the faces and bodies of people who do not exist. Video manipulation using deep learning can change one person’s likeness to another using deep learning and neural networks. Programs like GPT-3 can generate text of such quality that it is hard to determine whether a human wrote it on not. There has also been much progress in generating music using artificial intelligence. MuseNet by Openai, a deep neural network that can produce 4-minute musical pieces that have different instruments and styles. Jukebox by Openai, a neural network that produces music and simple singing as raw audio for a few genres and artistic styles. WaveNet from Deepmind, a generative model that generates audio waveforms, can generate background/Ambient music using deep learning. There exists a lot of research work that takes as its subject western popular music and classical music to generate new performances using old performances by an artist, but there exists no research work for the generation of improvised sequences for North Indian classical music. This thesis presents three methods to generate improvised sequence generation in North Indian classical music. The first method we propose uses context-free grammar, which can be used to generate these sequences. We also discuss how to generate these sequences and problems with this method. We illustrate this method by preparing context-free grammar for yaman raga ¯ . In our second method to this problem, We propose a bi-gram language model to generate improvised sequences. We propose a bi-gram model to evaluate this approach. To assess this model, we introduce our Pt. Ajoy Chakraborty Bhoopali Compositions Data Set. All compositions in this data set are by a single artist, Pandit Ajoy Chakraborty. We evaluate the bi-gram model on this data-set to achieve a TOP1 accuracy of 46.63 percent and a TOP-3 accuracy of 83.50 percent on this data set. We discuss how to qualify and generate new improvised sequences using this model. We also discuss the disadvantages of this model. In our third method to the problem, we propose a LSTM-RNN based model to generate improvised sequences. We discuss the motivation for this method. We also discuss model architecture and training settings. We evaluate our first LSTM-RNN based model on Pt. Ajoy Chakraborty Bhoopali Compositions Dataset. We achieve TOP-1 accuracy of 45.56 percent and TOP-3 accuracy of 80.42 percent. We discuss how to qualify and generate a new improvised sequence using this model. Further, we also discuss the disadvantages of this method.

Abstract

Centrality measures are used to quantify various notions of importance of nodes in a network. These measures find application in social network analysis, the study of disease spread, and the like. Since the networks of interest are usually large, parallelism has become a powerful tool to compute these measures. We study parallel algorithms for the percolation centrality measure which extends the betweennesscentrality measure by incorporating a time dependent state variable with every node. We present parallel algorithms that compute the source-based and source-destination variants of the percolation centrality values of nodes in a network. Our algorithms extend the algorithm of Brandes, introduce optimizations aimed at exploiting the structural properties of graphs, and extend the algorithmic techniques introduced by Sariyuce et al. [38] in the context of centrality computation. However, for some of these applications, it is important to study how these centrality values change when there are changes to the underlying network. As a result, parallel algorithms that study how to update these values due to changes to the graph are gaining research interest. We present parallel algorithms to handle both changes to the percolation value of a batch of vertices and the addition and deletion of a batch of edges to the graph. We implement and benchmark our dynamic algorithms on an Nvidia Tesla V100 GPU. Our experiments on a collection of real-world graphs indicate that our algorithms achieve speedups of 1.59× for edge updates, and 63.02× for vertex percolation updates over state-of-the-art static algorithms for a batch of 100 edges and vertices respectively.

Abstract

The study of quantum entanglement is significant for the development of Quantum Information science. Hence its detection becomes very vital. Detection of quantum entanglement is an NP-hard problem, and this is why we don’t have a generalized solution for the detection of quantum entanglement. There are many ways to detect quantum entanglement, but our study will mainly focus on detecting entanglement using the mathematical theory of positive but not completely positive maps. Positive but not completely positive maps applied to a subsystem of a bipartite quantum state are a central tool in characterizing entanglement in a bipartite system. In this study, we utilize a particular type of non-Markovianity known as the eternal non-Markovianity to construct a non-complete positive map. In the multipartite case, the direct application of a positive but not completely positive map does not work. It fails to capture the distinction among different classes of multipartite entanglement. We thus generalise this bipartite concept to the multipartite setting by introducing a special non-positive maps that are positive on the subset of bi-separable states, but can map to a non-positive element if applied to a genuine multipartite entangled state. We further propose a witness operator to detect genuinely multipartite entangled states experimentally based on this theory. Our study sheds light on a hitherto unexplored connection between entanglement theory and quantum non-Markovianity.

Abstract

The 3D reconstruction of a human from a monocular RGB image is an interesting yet very challenging research problem in the field of computer vision. It has various applications in the movie industry, the gaming industry, and AR/VR applications. Hence it is important to recover detailed geometric 3D reconstruction of humans in order to enhance the realism. The problem is ill-posed in nature owing to high freedom of human pose, self-occlusions, loose clothing, camera viewpoint, and illumination. Though it is possible to generate accurate geometric reconstructions using structured light scanners or multi-view cameras these are cost expensive and require a specific setup e.g., numerous cameras, controlled illumination, etc. With the current advancement of deep learning techniques, the focus of the community shifted to the 3D reconstruction of people from monocular RGB images. The goal of the thesis is toward 3D Human Digitization of people in loose clothing with accurate person-specific details. The problem of reconstruction of people in loose clothing is difficult as the topology of clothing is different from the human body. A multi-layered shape representation called PeeledHuman was able to deal with loose clothing and occlusions but it suffered from discontinuity or distorted body parts in the occluded regions. To overcome this we propose peeled semantic segmentation maps which provide the semantic information of the body parts across multiple peeled layers. These Peeled semantic maps help the network in predicting consistent depth for the pixels belonging to the same body part across different peeled maps. Our proposed Peeled Segmentation maps help in improving reconstruction both quantitatively and qualitatively. Additionally, the 3D semantic segmentation labels have various applications, for example, can be used to extract the clothing from the reconstructed output. The face plays an important role in 3D Human digitization, it gives a person their identity and the realism enhances with high-frequency facial details. However, the face appears in a smaller region of the image which captures the complete body and makes the task of high-fidelity face reconstruction along with the body even more challenging. We reconstruct person-specific facial geometry along with the complete body by incorporating a facial prior and refining it further using our proposed framework. Another common challenge faced by the existing methods is the surface noise i.e, false geometric edges generated because of the textural edges present in the image space. We address this problem by incorporating a wrinkle map prior that distinguishes the geometrical from the textural edges coming from image space. In summary, in this thesis, we address the problem of 3D human digitization from monocular images. We evaluate our proposed solution on various existing 3D human datasets and demonstrate that our proposed solutions outperform the existing state-of-the-art methods. The limitations present in the proposed methods were mentioned and potential solutions on how to address them have been briefly discussed. In the end, some future directions that can be potentially explored based on the proposed solutions have been discussed. Finally, we proposed efficient and robust methods to recover accurate and personalized 3D humans from images.

Abstract

Semiconductor Industry has undergone rapid miniaturization in recent decades due to advances in manufacturing technology and the demand for more functionality per chip area. However, scaling down metal-oxide-semiconductor field-effect transistors (MOSFETs) offer many advantages in terms of functionality and speed; it also poses many challenges like increased leakage power. A shrink in device size leads to lowering the threshold voltage to meet the functionality metrics. Due to the inverse relationship of leakage power with the threshold voltage, a rapid increase in leakage power is encountered. In deep sub-micron technologies, static power is dominant over dynamic power, leading to an urgent need for leakage reduction techniques. The primary outcome of scaling down transistors is a reduction in propagation delay. This is achieved by the lowering of threshold voltage and oxide thickness. However, as this comes with the higher cost of increased sub-threshold leakage current, scaling down of Vt is restricted. Hence, the need for delay optimization techniques. Another major issue associated with the miniaturization of MOSFETs is the variation of process parameters. Due to scaled-down devices, slight variation in process parameters impacts significantly on performance and yield of the devices. In addition, external environmental changes like temperature and supply voltage variations also affect the performance parameters like delay and leakage by affecting device densities and switching activities of various blocks. Hence, Process variations, Voltage and Temperature (PVT) are the key factors to be considered while optimizing a circuit. Furthermore, aging due to Negative-bias temperature instability (NBTI) affects the delay of the circuit. So, NBTI aging should be considered in delay optimization, for manufacturing more reliable and long-life devices. The above factors call for the indispensable need for leakage-delay optimization techniques. Hence, in this research, we propose a multi-objective algorithm-based leakage-delay optimization approach. We used various intuitive multi-objective algorithms to model device parameters such as the Width (W) and Length (L) of a MOSFET. Considering PVT variations for low-power optimization and Process, Supply Voltage, Temperature and Aging (PVTA) variations for high-performance applications, we have applied various swarm-based and genetic algorithms on basic and complex benchmark circuits and obtained up to 70% optimization of Leakage power with PVT and 30% optimization of propagation delays with PVTA conditions. Further, we developed an automated framework of algorithms to analyze the characteristics of a given circuit, choose the appropriate algorithm and load-based sizing to replace in a given circuit for obtaining the required performance metrics. In addition, we have proposed Backward traversal replacement and Partitioning large cells methods for enhanced optimisation of complex cells. The next step to further optimize this optimization process of the circuits is to eliminate the inherent extensive run time taken by the SPICE tool. The algorithm optimization process involves the calculation of circuit performances like leakage and propagation delay for huge samples. moreover, with the scaled down technology nodes, statistical variation aware analysis has gained importance, which also involves large number of circuit performance calculations. SPICE tool is widely used to calculate the circuit performances and computing results of an extensive sample set with the SPICE tool is time complex. Hence, we proposed a statistical variation aware estimation models based on machine learning techniques like Polynomial Regression and Artificial Neural Networks (ANN), which replaces the SPICE tool with <1% Error and 10,000 times reduction in time complexity.

Abstract

There is no doubt that videos are today’s most popular content consumption method. With the rise of the streaming giants such as YouTube, Netflix, etc., video content is accessible to more people. Naturally, video content creation has also increased to cater to the rising demand. In order to reach out to a wider audience, the creators dub their content. An important aspect of dubbing is not only changing the speech but also lip synchronizing the speaker in the video. Talking-face video generation works have achieved state-of-the-art results in synthesizing videos with accurate lip synchronization. However, most of the previous works deal with low-resolution talking-face videos (up to 256 × 256 pixels), thus, generating extremely high-resolution videos still remains a challenge. Also, with advancements in internet and camera tech more and more number of people are able to create video content and that too in ultra high resolution such as 4K (3840 × 2160). In this thesis, we take a giant leap and propose a novel method to synthesize talking-face videos at resolutions as high as 4K! Our task presents several key challenges: (i) Scaling the existing methods to such high resolutions is resource-constrained, both in terms of compute and the availability of very high-resolution datasets, (ii) The synthesized videos need to be spatially and temporally coherent. The sheer number of pixels that the model needs to generate while maintaining the temporal consistency at the video level makes this task non-trivial and has never been attempted in literature. We propose to train the lip-sync generator in a compact Vector Quantized (VQ) space for the first time to address these issues. Our core idea to encode the faces in a compact 16 × 16 representation allows us to model high-resolution videos. In our framework, we learn the lip movements in the quantized space on the newly collected 4K Talking Faces (4KTF) dataset. Our approach is speaker agnostic and can handle various languages and voices. We benchmark our technique against several competitive works and show that we can achieve a remarkable 64-times more pixels than the current state-of-the-art! Now, how to edit videos using the above algorithm or any other deep learning algorithm? To do so, the person has to download the source code of the required method and run the code manually. How amazing would it be if people could use the deep learning techniques in video editors with a click of a single button? In this thesis, we also propose a video editor based on OpenShot with several state-of-theart facial video editing algorithms as added functionalities. Our editor provides an easy-to-use interface to apply modern lip-syncing algorithms interactively. Apart from lip-syncing, the editor also uses audio and facial re-enactment to generate expressive talking faces. The manual control improves the overall experience of video editing without missing out on the benefits of modern synthetic video generation algorithms. This control enables us to lip-sync complex dubbed movie scenes, interviews, television shows, and other visual content. Furthermore, our editor provides features that automatically translate lectures from spoken content, lip-sync of the professor, and background content like slides. While doing so, we also tackle the critical aspect of synchronizing background content with the translated speech. We qualitatively evaluate the usefulness of the proposed editor by conducting human evaluations. Our evaluations show a clear improvement in the efficiency of using human editors and an improved video generation quality

Abstract

The latest digital revolution has caused an outburst of data. With growing data in different online platforms, it is crucial to process and streamline the data into valuable information. Extensive research is being carried out to achieve the same goal, and for achieving this, we understand that Natural Language Understanding (NLU) plays a vital role. It is considered a core component of many applications like Dialogue Systems, Question Answering, Text Classification, Automatic Text Generation, etc. However, as of today, a lot of the research in this direction is limited to the English language only. Our main aim is to extend these to other resource-poor languages like Telugu. Hence we propose a Dialogue System for Telugu that internally does Question Classification and Named Entity Recognition (NER). NER is one of the essential parts of NLU. It helps us in extracting essential phrases present in the sentence. Once we identify the essential phrases, we can classify them as various entity types and then use this information for decision-making. Our attempt is one step closer to bridging the gap between user and computer interaction in Telugu by proposing a sustainable method for creating domain-specific dialogue systems with limited data. We also improve the availability of various methods and tools that facilitate Natural Language Understanding and research of NLU in Telugu by building the state-of-the-art Named Entity Recognition for Telugu and providing sufficient annotated datasets. As discussed above, we first propose a dialogue system for the Hospital domain in Telugu. The main aim of the dialogue system is communication with the user in spoken or written form, and they have been gaining popularity over the past decade since they are found useful and helpful for the users. The dialogue system for the Hospital domain handles various hospital and doctor-related queries. However, the idea is to present an approach for modeling a dialogue system in a resource-poor language by combining linguistic and domain knowledge. Focusing on the question-answering aspect of the dialogue system, we identified Question Classification and Query Processing as the most vital parts of this dialogue system. Our method combines deep learning techniques for question classification and rule-based computational analysis for query processing. Human evaluation of the system has been performed as there is no automated evaluation tool for dialogue systems in Telugu. The results show that our system achieves a high overall rating and a significantly accurate context-capturing method. On the other hand, Named Entity Recognition (NER), which facilitates applications like dialogue systems, is about identifying entities in a sentence. It is a successful and well-researched problem in English due to the availability of resources. The transformer models, specifically the masked-language models (MLM), have shown remarkable performance in NER during recent times. However, with grow ing data and applications in all languages, there is a need for NER in other languages too. But, NER remains to be explored in Indian languages due to the lack of resources and tools. Our contributions include (i) Three annotated NER datasets for the Telugu language in multiple domains: Newswire Dataset (ND), Medical Dataset (MD), and Combined Dataset (CD). (ii) Comparison of the finetuned Telugu pretrained transformer models (BERT-Te, RoBERTa-Te, and ELECTRA-Te) with other baseline models (CRF, LSTM-CRF, and BiLSTM-CRF). (iii) Further investigation of the performance of Telugu pretrained transformer models against the multilingual models mBERT [20], XLM-R [15], and IndicBERT [45]. We find that pretrained Telugu language models (BERT-Te and RoBERTa) outperform the existing pretrained multilingual and baseline models in NER. On a large dataset (CD) of 38,363 sentences, the BERT-Te achieves a high F1-score of 0.80 (entity-level) and 0.75 (token-level). Further, these pretrained Telugu models have shown state-of-the-art performance on various existing Telugu NER datasets. We open-source our dataset, pretrained models, and code

Abstract

Globally, due to the changes in climate, rapid urbanization, land use changes, and increased water demands, water quantity-quality management has become a significant sustainability problem. Maintaining a complete natural river flow is usually impossible due to the development of the water resources and variations in land and soil usage in the catchment. These developed resources can create differences in the balance of the ecosystem and socioeconomic activities. Stream flow is a prime factor affecting the river flow health and quality in such watersheds. Admitting many issues associated with the water quality of the rivers, stream flow plays a vital role as the changes in the stream flow can directly transmit the changes in water quality. The variations in the flow regimes over time relatively affect the riverine ecosystems of the watersheds. Environmental Flows (EFs) assessment, which depends on the flow regime, illustrates the quantity and quality of water flow required for the sustainability of freshwater, estuarine ecosystem, and human livelihood. In this context, Environmental Flow Allocations (EFA) of reservoir-river systems do not adequately address the connections between reservoir inflows, releases, and corresponding downstream river water quality, rendering water resources management difficult. Thus, this study aims to develop an integrated, holistic modeling approach for Eflow estimation by combining a hydrological model, reservoir release analysis, Global Environmental Flow Calculator (GEFC), and river water quality model. In this present study, the GEFC method is used to estimate the ecological flows by using the flow duration curves (FDC) generated from the given monthly discharge data of the river. The FDC in this system contains 17 fixed percentile points concerning the discharge. In the current study, we analyze the Tunga-Bhadra River basin's Environmental flows (Eflows) by considering different discharge stations. The stations are Balehonuur, Haralahalli, Hosaritti, Shivamogga, Honalli, Rattihalli, and Tunga-Bhadra Dam, with a mean annual flow (MAF) of 36%, 24.8%, 27.2%, 16.2%, 23.3%, 21.1%, and 12.2% respectively to maintain the ecological conditions of the river. The monthly discharge data from 1995 to 2017 for those stations are obtained from the Advance Centre for Integrated Water Resource (ACIWR) in Bengaluru, India. River flow health is a study that helps understand the environmental variables that affect the river's habitat structure, flow regime, water quality, and biological conditions. To estimate the flow health of the Tunga-Bhadra River, we used a tool called Flow Health which uses nine indicators to represent the Flow Health (FH) score for the stations Balehonnur, Haralahalli, Hosaritti, Shivamogga, Honalli, Rattihalli, and Tunga-Bhadra. This tool uses the gauge discharge data in the form of reference (1995-2005) and test periods (2006-2017), with Flow Health score of 0.72, 0.4, 0.72, 0.70, 0.58, 0.73, 0.71 and 0.72, 0.63, 0.63, 0.7, 0.66, 0.67, 0.66 for test and reference period with respect to stations. The study noted that most of the discharge stations along the Tunga-Bhadra River show moderate to low flow variations for the reference and test periods. Overall, Tunga-Bhadra river health, measured by the flow indices, declined from 1995-2005 to 2006-2017. A hydrological model, Soil & Water Assessment Tool (SWAT), is selected to simulate the inflows of the Bhadra reservoir. The calibration and validation of the SWAT estimated inflows are done using the SWAT-CUP tool, which shows satisfactory results. The SWAT findings were utilized in reservoir release analysis to determine the reservoir releases. Environmental Flow Management Classes (EMCs) for estimating and assessing the minimum flow to be maintained in a river were derived using the GEFC for the Bhadra watershed. The river water quality-modeling tool, QUAL2K, was chosen to simulate the Dissolved Oxygen (DO) corresponding to the EMCs developed from the GEFC for the Bhadra Reservoir. The study considered various plausible scenarios of inflows (10 to 20% reduction), pollution scenarios of Biological Oxygen Demand (BOD) (0 to 100 % treatment) for arriving drains by altering headwaters DO (4 to 8 mg/l). The study revealed that by maintaining the headwater DO (about 7 mg/l) in conjunction with BOD treatment of drains (25%) and by maintaining sufficient Eflow allocation (class D with 1269 Million Cubic meters (MCM)); downstream river water quality could be improved (6.5 mg/l of DO at Holehonnur monitoring point of Bhadra reservoir-river system, India). From all these performed analyses, we provided an insight into the reservoir supply management and maintaining the riverine ecosystem by combining the water quantity and quality models corresponding to the environmental aspects.

Abstract

Navigation of Unmanned Aerial Vehicles (UAVs) amongst high-rises in urban environments be-comes inevitable due to increasing demand for various applications in Urban Air Mobility (UAM).Moreover, these city scale urban environments are populated with tall buildings and skyscrapers withinherent piecewise planar structures. Leveraging this inherent structural information for navigating aUAV equipped with a single camera and an IMU is the primary focus of this thesis. We make threecontributions that leverage these geometric cues to detect and map these planar structures for obstacleavoidance and path planning.Our first contribution,Multi-View Planarity Constraints for skyline estimation from UAV im-ages in city scale urban environments, is a three-stage pipeline that brings together several modulescombining a data-driven monocular plane segmentation network and geometric constraints from 3D vi-sion to showcase a quick reconstruction of planar facades within a few views. We evaluate the efficacyof our pipeline with various constraints and errors from multi-view geometry using ablation studies. Wethen retrieve the skyline of the buildings in synthetic as well as real-world scenes.In our second contribution,A new geometric approach for three view line reconstruction andmotion estimation in Manhattan Scenes, we propose a novel method of pose estimation using linefeatures from three views of a Manhattan Scene. We leverage the vanishing point directions to estimatethe relative rotations as well as to fix the 3D line direction. In consequence we build a constraints matrix,which has the relative translations and 3D line depth as its null space. We then perform 1-parameter lineBA using factor graph based cost function. We compare the efficacy of our method with standard linetriangulation in synthetic as well as real-world scenes.Finally, we proposeUrbanFly: Uncertainty-Aware Planning for Navigation Amongst High-Rises with Monocular Visual-Inertial SLAM Maps, a sequential convex program based novel trajec-tory planner tailored for outdoor urban scenes. It uses the sparse point clouds generated by a MonocularVisual Inertial SLAM (VINS) backend to build a cuboid representation of the environment through adata-driven monocular plane segmentation network. Our chosen world model provides faster distancequeries than the more common voxel-grid representation. The trajectory optimizer is initialized based onsafety estimates on a set of randomly drawn trajectories. We perform extensive simulations in a tightlyintegrated perception-control loop to validate its efficacy. UrbanFly outperforms competing baselinesin metrics such as collision rate, trajectory length, etc., on a high fidelity AirSim simulator augmentedwith synthetic and real-world dataset scenes.