Abstract

We leverage the modern advancements in talking head generation to propose an end-to-end system for talking head video compression. Our algorithm transmits pivot frames intermittently while the rest of the talking head video is generated by animating them. We use a state-of-the-art face reenactment network to detect key points in the non-pivot frames and transmit them to the receiver. A dense flow is then calculated to warp a pivot frame to reconstruct the non-pivot ones. Transmitting key points instead of full frames leads to significant compression. We propose a novel algorithm to adaptively select the best-suited pivot frames at regular intervals to provide a smooth experience. We also propose a frame-interpolater at the receiver's end to improve the compression levels further. Finally, a face enhancement network improves reconstruction quality, significantly improving several aspects like the sharpness of the generations. We evaluate our method both qualitatively and quantitatively on benchmark datasets and compare it with multiple compression techniques. We release a demo video and additional information at https://cvit.iiit.ac.in/research/projects/cvit-projects/talking-video-compression.

Abstract

In this work, we propose IndoLayout, a novel real-time approach for generating high-quality occupancy maps from an RGB image for indoor scenes. Such occupancy maps are often crucial for path-planning and mapping in indoor environments but are often built using only information contained in the ego view. In contrast, our approach also predicts occupancy values beyond immediately visible regions from just a monocular image, leveraging learnt priors from indoor scenes. Hence, our proposed network can produce a hallucinated, amodal scene layout that includes areas occluded in the RGB image, such as a navigable floor behind a desk. Specifically, we propose a novel architecture that uses self-attention and adversarial learning to vastly improve the quality of the predicted layout. We evaluate our model on several photorealistic indoor datasets and outperform previous relevant work on all metrics that measure layout quality, including newly adopted ones. Finally, we demonstrate the effectiveness of our method by showing significant improvements on the PointNav task over similar approaches using IndoLayout. For more details, please refer to the project page: https://indolayout.github.io/.

Abstract

Modern autonomous vehicles (AVs) often rely on vision, LIDAR, and even radar-based simultaneous localization and mapping (SLAM) frameworks for precise localization and navigation. However, modern SLAM frameworks often lead to unacceptably high levels of drift (i.e., localization error) when AVs observe few visually distinct features or encounter occlusions due to dynamic obstacles. This paper argues that minimizing drift must be a key desiderata in AV motion planning, which requires an AV to take active control decisions to move towards feature-rich regions while also minimizing conventional control cost. To do so, we first introduce a novel data-driven perception module that observes LIDAR point clouds and estimates which features/regions an AV must navigate towards for drift minimization. Then, we introduce an interpretable model predictive controller (MPC) that moves an AV toward such feature-rich regions while avoiding visual occlusions and gracefully trading off drift and control cost. Our experiments on challenging, dynamic scenarios in the state-ofthe-art CARLA simulator indicate our method reduces drift up to 76.76% compared to benchmark approaches.

Abstract

When the Indian government declared the frst lockdown on 25 March 2020 to control the increasing number of COVID-19 cases, people were forced to stay and work from home. The aim of this study is to quantify the impact of stay-at-home orders on residential Air Conditioning (AC) energy and household electricity consumption (excluding AC energy). This was done using monitored data from 380 homes in a group of fve buildings in Hyderabad, India. We gathered AC energy and household electricity consumption data at a 30-min interval for each home individually in April 2019 and April 2020. Descriptive and inferential statistical analysis was done on this data. To ofset the diference in temperatures for the month of April in 2019 and 2020, only those weekdays were selected where the average temperature in 2019 was same as the average temperature in 2020. The study establishes that the average number of hours the AC was used per day in each home increased in the range 4.90–7.45% depending on the temperature for the year 2020. Correspondingly, the overall AC consumption increased in the range 3.60–4.5%, however the daytime (8:00 AM to 8:00 PM) AC energy consumption increased in the range 22–26% and nighttime (8:00 PM to 8:00 AM) AC energy consumption decreased by 5–7% in the year 2020. The study showed a rise in household electricity consumption of about 15% for the entire day in the year 2020. The household electricity consumption increased during daytime by 22- 27.50% and 1.90- 6.6% during the nighttime. It was observed that the morning household electricity peak demand shifted from 7:00 AM in 2019 to 9:00 AM in 2020. Conversely, the evening peak demand shifted from 9:00 PM in 2019 to 7:00 PM in 2020. An additional peak was observed during afternoon hours in the lockdown.

Abstract

We propose a novel flow synthesis based visual servoing framework enabling long-range obstacle avoidance for Micro Air Vehicles (MAV) flying amongst tall skyscrapers. Recent deep learning based frameworks use optical flow to do high-precision visual servoing. In this paper, we explore the question: can we design a surrogate flow for these high-precision visual-servoing methods, which leads to obstacle avoidance? We revisit the concept of saliency for identifying high-rise structures in/close to the line of attack amongst other competing skyscrapers and buildings as a collision obstacle. A synthesised flow is used to displace the salient object segmentation mask. This flow is so computed that the visual servoing controller maneuvers the MAV safely around the obstacle. In this approach, we use a multi-step Cross-Entropy Method (CEM) based servo control to achieve flow convergence, resulting in obstacle avoidance. We use this novel pipeline to successfully and persistently maneuver high-rises and reach the goal in simulated and photo-realistic real-world scenes. We conduct extensive experimentation and compare our approach with optical flow and short-range depth-based obstacle avoidance methods to demonstrate the proposed framework’s merit. Additional Visualisation can be found at https://sites.google.com/view/monocular-obstacle/home

Abstract

A tool Smart Glazing Simulator (SGLSim), has been developed to perform parametric simulation analysis of different window systems with several window-to-wall ratios and orientations to compute and compare the annual energy performance. The net annual energy performance of the building is based on the electricity consumption in heating, cooling, interior lighting, and appliances, along with the electricity generation by the photovoltaic (PV) glazing, which is used to evaluate the energy performance of smart glazing. Performing parametric energy simulations and calculating the net annual electricity consumption of different combinations requires building modeling and energy simulation expertise. A web-based parametric tool can assist the user in carrying out the desired studies without requiring extensive technical knowledge. A case study is prepared for India’s warm and humid climatic zone. This study examines the benefits of double pane semi-transparent photovoltaics (STPV) glazing, STPV glaz-ing with dynamic internal blind, and electrochromic (EC) glazing over other traditional glazing systems. The study shows that the optimal net annual electricity consumption in the case of STPV windows is 10–12% less than the optimal value obtained in a sim-ple glazing case. Additionally, the result suggested that glare-controlled interior blinds in the STPV window further reduce the net annual electricity consumption by up to 15% compared to conventional glazing. Similarly, installing the EC glazing reduces the yearly electricity consumption by up to 5% compared to standard glazing.

Abstract

We present a method to control the emotional prosody of Text to Speech (TTS) systems by us- ing phoneme-level intermediate features (pitch, energy, and duration) as levers. As a key idea, we propose Differential Scaling (DS) to dis- entangle features relating to affective prosody from those arising due to acoustics conditions and speaker identity. With thorough exper- imental studies, we show that the proposed method improves over the prior art in accu- rately emulating the desired emotions while retaining the naturalness of speech. We ex- tend the traditional evaluation of using indi- vidual sentences for a more complete evalua- tion of HCI systems. We present a novel ex- perimental setup by replacing an actor with a TTS system in offline and live conversa- tions. The emotion to be rendered is either predicted or manually assigned. The results show that the proposed method is strongly pre- ferred over the state-of-the-art TTS system and adds the much-coveted “human touch” in ma- chine dialogue. Audio samples for our exper- iments and the code are available at: https: //emtts.github.io/tts-demo/

Abstract

The present study compared various empirical and data-driven algorithms to predict Actual Evapotranspiration (AET) using various hydro climatic variables. The AET over semi-arid climatic conditions of Hyderabad, Telangana, India, and Waipara (New Zealand) was estimated using different empirical methods-based PET using Budyko and Turc models. Modelled PET from five data-driven algorithms, such as Long short-term memory neural networks (LSTM), Artificial Neural Network (ANN), Gradient Boosting Regressor, Random Forest, and Support Vector Regression were trained to predict AET using meteorological variables. The results show simple empirical-based AET models, Budyko and Turc, can estimate AET very well. The results indicated that 99% accuracy could be achieved with all climatic input, whereas accuracy drops to 86% with limited data. Both LSTM and ANN models based on PET have been noted as the most robust models for estimating AET with minimal climate data. It was observed that the meteorological variables of temperature and solar radiation have more significant contributions than other variables in the estimation of AET. In addition, the effects of the meteorological variables were found to be essential and effective in the estimation of AET. The research findings of the study reveal that under limited data availability, the best input combinations were identified as temperature and wind speed for estimating PET; temperature, wind speed, and precipitation for estimating AET for semi-arid climatology.

Abstract

The simulation of appropriate ground motion is the main objective in earthquake-resistant designing to mitigate the impact of landslides and structural damages. The seismic hazard studies serve as the basis for quantifying the hazard associated within the particular area over a given time period. In this study, the detailed site specific hazard analysis of Tindharia site as well as Darjeeling Sikkim Himalaya (DSH) region has been carried out to assure the safety of human and infrastructure against seismic risk. The study area is mountainous region on north western side of India, falls in seismic zone IV and V, and is one of the most seismically active regions with vulnerable earthquakes along with fatal landslides. The future large earthquakes are likely to occur in the region due to constant movement of Indian plate towards Eurasian plate.Therefore, the heightened seismic risk increases the demand for seismic studies near the Himalayan belt. Thus, the assessment of associated seismic hazard in the study area has been carried out by deterministic and probabilistic seismic hazard approaches. From the results, the seismo-tectonic map of a 300km radial distance of the study area is developed. The deterministic hazard distribution in terms of peak ground acceleration (PGA) and the PGA and peak spectral acceleration (PSA) hazard maps for 2% and 10% probability of exceedance in 50 years at different time periods have been generated. The updated hazard maps developed in this study play pivotal roles in mitigation measures with better pre-disaster prevention by earthquake resistant design and in preparedness and post disaster rescue in the study area

Abstract

Kathakali is one of the major forms of Classical Indian Dance. The dance form is distinguished by the elaborately colourful makeup, costumes and face masks. In this work, we present (a) a framework to analyze the facial expressions of the actors and (b) novel visualization techniques for the same. Due to extensive makeup, costumes and masks, the general face analysis techniques fail on Kathakali videos. We present a dataset with manually annotated Kathakali sequences for four downstream tasks, i.e. face detection, background subtraction, landmark detection and face segmentation. We rely on transfer learning and fine-tune deep learning models and present qualitative and quantitative results for these tasks. Finally, we present a novel application of style-transfer of Kathakali video onto a cartoonized face. The comprehensive framework presented in the paper paves the way for better understanding, analysis, pedagogy and visualization of Kathakali videos.