Abstract

Precomputed Radiance Transfer (PRT) is widely used for real-time photorealistic effects. PRT disentangles the rendering equation into transfer and lighting, enabling their precomputation. Transfer accounts for the cosine-weighted visibility of points in the scene, while Lighting is usually a distant emitted lighting, e.g., environment. Transfer computation involves tracing several rays into the scene from every point on the surface. For every ray, the binary visibility is calculated, and a spherical function is obtained. The spherical function is projected into Spherical Harmonic(SH) domain. SH is a band-limited representation of spherical functions, and the order of SH decides the representation capacity of the SH (the higher the SH order better the approximation of a spherical function). The SH domain also facilitates fast and efficient integral computation by simplifying the integral into simple dot products and convolutions. The original formulation of PRT by Sloan et al. 2002 provides different storage requirements for the transfer—vectors in the case of diffuse materials and matrices in the case of glossy materials. Using matrices for Transfer representation makes it infeasible as the SH orders increase. The work of Triple Product Formulation by Ng et al. in 2004 extended the formulation to allow simple vector-based Transfer storage even for the case of glossy materials. Prior art stored precomputed transfer in a tabulated manner in vertex space. These values are fetched with interpolation at each point for shading. Since the barycentric interpolation is finally employed to calculate the final color across the geometry apart from the vertex locations, the vertex space methods require densely tessellated mesh vertices to obtain accurate radiance. Sometimes high-density(tessellated) meshes adversely affect runtimes and memory requirements. This is mainly observed in simple geometries with no additional detailing but still demanding higher triangle counts (e.g., planes, walls, etc.). The first work provides a solution by leveraging Texture space, which is more continuous than the Vertex space. We also added additional functionality to obtain inter-reflection effects in the texture space. While Texture space methods provide faithful results in meshes, they require non-overlapping, areapreserving UV mapping, and a high-resolution texture to avoid artifacts. In the subsequent work, we propose a compact transfer representation that is learnt directly on scene geometry points. Specifically, we train a small multi-layer perceptron (MLP) to predict the transfer at sampled surface points. Our approach is most beneficial where inherent mesh storage structure and natural UV mapping are unavailable, such as Implicit Surfaces, as it learns the transfer values directly on the surface. Using our approach, we demonstrate real-time, photorealistic renderings of diffuse and glossy materials on SDF geometries with PRT.

Abstract

Pseudowords are a part of language that are not translatable to another, as they have no meaning attached to them while also having the constraint of sounding like a phonologically valid sequence under the desired language’s native phonotactics. This thesis thus explores automated language-agnostic pseudoword generation, evaluation of them, and use of them outside psycholinguistics research and clinical use. As the thesis progresses, we highlight current research, draw inspiration from close topics of study, build a pipeline to generate pseudowords and generate Hindi and English pseudoword candidates for further experiemntation. We make this reusable pipeline available on a public repository, as one of the deliverables of this work. Then we show how the current evaluation work in this field is very scarce and sew an evaluation framework with reproducible details on how to design and analyse a human-inthe-loop experiment for something as tricky as pseudoword judgement, conducted for a layman native speaker. After showing various ways to prod a pseudoword set for quality, we compare notes against past sets in English and present observations summarising how comparable they are. However as there is no Hindi pseudoword dataset yet, we add in psycholinguistic features on top of results of evaluation metrics per Hindi pseudoword and release “Soodkosh” another fully public and usable for research resource. Finally, we conduct two separate studies involving pseudowords to show the application, impact, and importance of them across fields. The first study uses pseudowords to establish gradient between high-frequency words, low-frequency words, and non-sensical sequences of alphanumerics used as passwords. The aim of this study is to find correlation and its strength between the perceived security and memorability of a password/phrase. The other part of this chapter is an exploration into language models’ performance on Aphasia classification and if replacing pseudowords can help them. This is as pseudowords like neologisms, mis-pronunciations, and other novel forms generated by Aphasic speakers are largely out-of-vocabulary to a standard languge model that functions off of a pile of mostly well-formed and coherent data. As these are not directly helpful to the field of Aphasia, this work replaces one possible hurdle to see if it is a feasible solution. However the results show that pseudowords are passively used as features and cannot be replaced directly

Abstract

Smart cities leverage IoT to improve citizens’ quality of life by providing better infrastructure, enhanced transportation systems, and efficient public services. With IoT-enabled smart city applications receiving traction, mechanisms must be implemented to cater to the diverse requirements of the use cases. The increasing number of connected devices increases the risk of cyber attacks and breaches. Further, smart cities rely heavily on integrating critical infrastructure such as power grids and water treatment plants. Securing these systems is crucial to protect citizens and institutions from potential harm caused by attacks. However, securing the IoT ecosystem is a challenging task. There are constraints on processing, memory, and energy consumption in addition to interoperability challenges, cost, and complexity. Designing and implementing the appropriate security controls for the devices and services requires on-ground testing and analysis. This thesis explores and analyzes the security of IoT-enabled smart cities. The work consists of two main contributions. First, IoT security requirements, potential threats, and vulnerabilities to the various layers of IoT are analyzed. Vulnerability assessment and modeling of threats are performed on the air quality monitoring vertical of the smart city deployment of IIIT-H to gain visibility into the baseline security requirements. The recommendations and state-of-the-art solutions are extensible to applications that require Wi-Fi and mobile network-based security for large-scale IoT deployments. Second, the entire smart city deployment was examined, covering applications such as air quality monitoring, water quantity management, weather monitoring, and energy monitoring. With each application operating using different hardware components, communication technologies, and software dependencies, requirements such as interoperability, scalability, resiliency, and security are essential. This study focused on the provisions of the oneM2M standard in catering to these requirements and its role in smart cities. Experiments were conducted on the OM2M platform, an implementation of oneM2M, used in the smart city deployment of IIIT-H. The recommendations are a foundation for the security of oneM2Mbased smart city applications.

Abstract

In today’s digital era, biometric authentication has become increasingly widespread for verifying a user across a range of applications, from unlocking a smartphone to securing high-end systems. Various biometric modalities such as fingerprint, face, and iris offer a distinct way to recognize a person automatically. Fingerprints are one of the most prevalent biometric modalities. They are widely utilized in security systems owing to their remarkable reliability, distinctiveness, invariance over time and user convenience. Nowadays, automatic fingerprint recognition systems have become a prime target for attackers. Attackers fabricate fingerprints using materials like playdoh and gelatin, making it hard to distinguish them from live fingerprints. This way of circumventing biometric systems is called a presentation attack (PA). To identify such attacks, a PA detector is added to these systems. Deep learning-based PA detectors require large amounts of data to distinguish PA fingerprints from live ones. However, there exists significantly less training data with novel sensors and materials. Due to this, PA detectors do not generalize well on introducing unknown sensors or materials. It is incredibly challenging to physically fabricate an extensive train dataset of high-quality counterfeit fingerprints generated with novel materials captured across multiple sensors. Existing fingerprint presentation attack detection (FPAD) solutions improve cross-sensor and cross-material generalization by utilizing styletransfer-based augmentation wrappers over a two-class PA classifier. These solutions generate large artificial datasets for training by using style transfer which learns the style properties from a few samples obtained from the attacker. They synthesize data by learning the style as a single entity, containing both sensor and material characteristics. However, these strategies necessitate learning the entire style upon adding a new sensor for an already known material or vice versa. This thesis proposes a decomposition-based approach to improve cross-sensor and cross-material FPAD generalization. We model presentation attacks as a combination of two underlying components, i.e., material and sensor, rather than the entire style. By utilizing this approach, our method can generate synthetic patches upon introducing either a new sensor, a new material, or both. We perform two different methods of fingerprint factorization - traditional and deep-learning based. Traditional factorization of fingerprints into sensor and material representations using tensor decomposition establishes a baseline using machine learning for our hypothesis. The deep-learning method uses a decompositionbased augmentation wrapper for disentangling fingerprint style. The wrapper improves cross-sensor and cross-material FPAD, utilizing one fingerprint image of the target sensor and material. We also reduce vi vii computational complexity by generating compact representations and utilizing lesser combinations of sensors and materials to produce several styles. Our approach enables us to generate a large variety of samples using a limited amount of data, which helps improve generalization

Abstract

The issue of ambiguity in natural language poses a significant challenge to computational linguistics and natural language processing. Ambiguity arises when words or phrases can have multiple meanings, depending on the context in which they are used. In natural language processing, addressing the challenge of ambiguity is crucial for building more accurate and effective language models that can better reflect the complexity of human communication. In this thesis, we investigate two specific forms of linguistic ambiguities - polysemy, which is the multiplicity of meanings for a specific word, and tautology, which are seemingly uninformative and ambiguous phrases used in conversations. Both phenomena are widely-known manifestations of linguistic ambiguity - at the lexical and pragmatic level, respectively. The first part of the thesis focuses on addressing this challenge by proposing a new method for quantifying the degree of polysemy in words, which refers to the number of distinct meanings that a word can have. The proposed approach is a novel, unsupervised framework to compute and estimate polysemy scores for words in multiple languages, infusing syntactic knowledge in the form of dependency structures. The framework adopts a graph-based approach by computing the discrete Ollivier Ricci curvature on a graph of the contextual nearest neighbors. The effectiveness of the framework is demonstrated by significant correlations of the quantification with expert human-annotated language resources like WordNet. The proposed framework is tested on curated datasets controlling for different sense distributions of words in three typologically diverse languages - English, French, and Spanish. The framework leverages contextual language models and syntactic structures to empirically support the widely held theoretical linguistic notion that syntax is intricately linked to ambiguity/polysemy. The second part of the thesis explores how language models handle colloquial tautologies, a type of redundancy commonly used in conversational speech. Colloquial tautologies pose an additional challenge to language processing, as they involve the repetition of words or phrases that may appear redundant, but convey a specific meaning in a given context. We first present a dataset of colloquial tautologies and evaluate several state-of-the-art language models on this dataset using perplexity scores. We conduct probing experiments while controlling for the noun type, context and form of tautologies. The results reveal that BERT and GPT2 perform better with modal forms and human nouns, which aligns with previous literature and human intuition. We hope this work bolsters further research on ambiguity in language models. Our contributions have important implications for the development of more accurate and reliable natural language processing systems.

Abstract

In this modern world, due to the increased mobility of humans, encountering a foreign language has become a common challenge for many people around the world. This causes a language barrier in their regular lives, which makes communication quite difficult. This makes machine translation a facility that helps people to overcome this language barrier. Research on machine translation has been going on for many decades, and there are many MT models that give good-quality translations, but even the best of these models fail to produce quality output when a domain-specific input is given to them. These models are trained on large general domain data, which makes their domain-specific translations not up to par. This brings up the issue of domain adaptation for different areas. For Indian languages, the issue arises with the lack of domain-specific data and good baseline models. This thesis will try to put forward an approach to improving the scores of domain-specific translations with efficient use of domain data. Before getting into domain adaptation, this thesis will try to understand how a domain is defined and how domain information is stored in these documents or sentences. For this study, we will discuss two tasks that will help us to understand the importance of domain terminologies. The first one discussed fine-grained domain classification as a task. It tries to get information out of similar domains and what makes those domains different by classifying an unknown document into a similar set of domains. The other task helps to find domain terms in a document in an unsupervised manner. It used an improved TextRank approach, where n-grams are used to get the most important terms in a document. Both of these approaches helped in understanding domain terms and their importance in defining a domain. After understanding domains, we detail different approaches done for domain adaptation and give a comparative analysis of them. We started with a very basic domain adaptation approach that gave us a good result but proved to be an inefficient task for multiple domains. All the approaches were for the English-Hindi language pairs, but the basic domain adaptation of individual domains was also done for the English-Telugu and English-Bengali language pairs. After multiple experiments, we show in this thesis how we can get better performances in an efficient manner when we use the domain knowledge of different domains in the task of domain adaptation. Different approaches will be discussed for all the domains to create different translation models for our task of domain adaptation.

Abstract

The electroencephalography (EEG) technique measures scalp electrical activity with high temporal resolution; required to map the neuronal global functional brain networks supporting motor movements. The time series of the change in electric potential across electrodes is analysed using frequency (and/or) time domain methods. An alternate approach is to represent a dynamic system with states and transitions between them defined using microstates. These are quasi-stable (60-120 ms) scalp potential configurations that represent spontaneous EEG as a sequence of a small number of scalp potential field maps and provide insight into brain health exhibiting sub-second coherent activations of brain. The present study investigates microstates extracted from EEG signals of motor-tasks. We investigate the extracted microstate parameters (occurrence, duration, coverage and no. of microstate) for Tip-pinch movement (index finger & thumb) and Wrist flexion & extension performed by both hands independently. These movements are selected due to its relevance in benchmarking rehabilitation of paretic arm post-stroke. The data was extracted from 17 electrodes (of a 32 channel EEG system) covering the pre-frontal, frontal, central and motor regions. Twenty-four right-handed and five left-handed healthy male subjects (age = 18-30 years; mean = 24.25 years; SD = 3.96 years) participate in this study. The approach of analysing each dynamic variable (occurrence, duration, coverage, and number of microstate) values during pre-event, event, and post-event conditions with reported resting state microstates highlights the similarities or differences as a function of motor action. We identified new microstate topographies specific to each motor task. The number of microstates for both motor activities is independent of the handedness of the participant (right-handed and left-handed). Task dependent variation in the topography of the microstates is observed, deviating from the standard maps reported for resting state EEG. The tip-pinch movement for event condition demonstrates higher microstate class stability than Wrist flexion & extension, indicative of lower disruption in the processing for the former action. The identification of disruptions or novel topologies for a task indicates the potential to function as a physiological marker, even considering the arguments on microstate’s state change being discrete or continuous. The findings will serve as a template for future studies analysing data from stroke patients.

Abstract

Data-Driven approaches are becoming increasingly popular for robot control problems. In this thesis, we present two approaches that make use of deep learning frameworks to solve classic robot control problems. In the first part(chapter 3), we propose a lightweight visual servoing MPC framework that generates optimal control near real-time at a frequency of 10.52 Hz. This work utilizes the differential crossentropy sampling method for quick and effective control generation along with a lightweight neural network, significantly improving the servoing frequency. We also propose a flow depth normalization layer that ameliorates the issue of inferior predictions of two view depths from the flow network. We conduct extensive experimentation on the Habitat simulator and show a notable decrease in servoing time compared to other approaches that optimize over a time horizon. We achieve the right balance between time and performance for visual servoing in six degrees of freedom (6DoF), while retaining the advantageous MPC formulation. In the second part(chapter 4), we present a real-time algorithm for computing the optimal sequence and motion trajectories for a fixed-base manipulator to pick and place a set of given objects. The optimality is defined in terms of the total execution time of the sequence or its proxy, the arc length in the joint space. The fundamental complexity stems from the fact that the optimality metric depends on the joint motion, but the task specification is in the end-effector space. Moreover, mapping between a pair of end-effector positions to the shortest arc-length joint trajectory is not analytic; instead, it entails solving a complex trajectory optimization problem. Existing works ignore this complex mapping and use the Euclidean distance in the end-effector space to compute the sequence. In this chapter, we overcome the reliance on the Euclidean distance heuristic by introducing a novel data-driven technique to estimate the optimal arc-length cost in joint space (a.k.a the value function) between two given endeffector positions. We parametrize the value function as a Neural Network and motivate a niche choice for its architecture, inspired by the works on metric learning. The learned value function is then used as an edge cost in a capacitated vehicle routing problem (CVRP) set-up to compute the optimal visitation sequence. Finally, we optimize over the input space of the learned value function network to propose a novel Inverse Kinematics (IK) algorithm that produces substantially shorter joint arc-length trajectories than existing approaches while executing the computed optimal sequence. We show that our sequence planner, in combination with our proposed IK contoller, offers a substantial improvement in joint arc length over existing state-of-the-art while maintaining scalability to a large number of objects

Abstract

Natural language generation has gained tremendous popularity in recent times primarily due to the advent of large pretrained language models trained on vast amount of data. However, most of this progress has only been limited to few high resource languages like English. Almost all low resource(LR) languages still suffer from the lack of sufficient training data and hence the lack of usable generative models. Furthermore, multiple business scenarios also require an automated generation of descriptive human-readable long text from structured input data, where the source is typically a high-resource language and the target is a low or medium resource language. In this work, we present systems and approaches which can be utilised to ultimately enrich the structured and unstructured content available for low resource languages in the encyclopedic domain. In order to do so, we introduce cross lingual techniques which efficiently utilise the abundant structured data available in high resource languages. We also introduce systems to further enrich this structured data using the information present in the form of natural language text in low resource languages. Firstly we propose novel problem of cross lingual fact to text alignment in order to construct the XAlign dataset for the purpose of cross lingual fact to text generation and fact extraction. We explore several methods to automatically align English facts from Wikidata to sentences from native language Wikipedia. We experiment with approaches accounting for syntactic and semantic matches between the fact and the sentence and propose a two stage pipeline for automated alignment and evaluate it on a manually annotated high quality test set. We also experiment with distant supervision and transfer learning based techniques in order to achieve quality alignment. We use the best approach to create the XAlign dataset which consists of more than half a million aligned (sentence, facts) pairs across 12 Indian languages. Following the construction of the dataset we propose the problem of Cross Lingual Fact Extraction (CLFE). Recent approaches concentrate on automatically enriching large knowledge graphs like Wikidata and DBPedia from text. However a lot of information present in the form of natural text in low resource languages is often missed out. Furthermore, considering the potential use case of utilising structured data for generating content in various LR languages, the CLFE task aims at extracting factual information in the form of English triples from LR Indian Language text. Despite its massive potential, progress made on this task is lagging when compared to Monolingual Information Extraction. We propose strong baselines and an end-to-end generative approach for the CLFE task which achieves an overall F1 score of 77.46. We then introduce and explore the problem of cross lingual fact to text generation (XF2T). We extensively explore multiple approaches for the task and analyse different components of the pipeline. Starting from the choice of pretrained transformer model used, we explore the impact of different continued pretraining strategies. We also show that building cross lingual systems results in better performance than translation based approaches or multiple bi-lingual modes, thus validating the necessity of the proposed problem. We introduce novel techniques like fact-aware embeddings to further improve the generation quality. We demonstrate that these methods produce coherent and precise sentences. Our experiments with the XF2T task lead to the observation that these generative models suffer from hallucination and due to the training setup, are only limited to generating a single sentence at a time. In order to mitigate these limitations, we extend the XF2T task to the problem of Cross-Lingual Fact to Long Text Generation (XFLT). The task involves generating descriptive and human readable long text in a target language from structured input data (such as fact triples) in a source language. XFLT is challenging because of (a) hallucinatory nature of the state-of-the-art NLG models, (b) lack of good quality training data, and (c) lack of a suitable cross-lingual NLG metric. Unfortunately previous work focuses on different related problem settings like monolingual graph to text and has made no specific efforts to handle hallucinations. Hence, we propose a novel solution to the XFLT task which addresses these challenges by training multilingual Transformer-based encoder-decoder models with coverage prompts and grounded decoding. Further, it improves on the XFLT quality by defining taskspecific reward functions and training on them using reinforcement learning. On a dataset with over 64,000 paragraphs across 12 different languages, we compare this novel solution with several strong baselines using a new metric, cross-lingual PARENT. Overall, we work on multiple related tasks aimed at automating the gene

Abstract

Artificial time delay controller was conceptualised for nonlinear systems to reduce dependency on precise system modelling unlike the conventional adaptive and robust control strategies. In this approach unknown dynamics is compensated by using input and state measurements collected at immediate past time instant (i.e., artificially delayed). The advantage of this kind of approach lies in its simplicity and ease of implementation. However, the applications of artificial time delay controllers in robotics, which are also robust against unknown statedependent uncertainty, are still missing at large. This thesis presents the study of this control approach toward two important classes of robotic systems, namely a fully actuated bipedal walking robot and an underactuated quadrotor system. In the first work, we explore the idea of a unified control design instead of multiple controllers for different walking phases in adaptive bipedal walking control (i.e. control taking care of unknown robot parameters) while bypassing computing constraint forces, since they often lead to complex designs. State-of-the art attempts to design a single controller for all walking phases either ignored or oversimplified the state-dependent constraint forces which may lead to conservative performance or even instability. This work proposes an innovative artificial time delay based adaptive control method, which covers the entire bipedal walking phase and provides robustness against state-dependent unmodelled dynamics such as constraint forces and external impulsive forces arising during walking. Studies using a high fidelity simulator under various forms of disturbances show the effectiveness of the proposed design over the state of the art. The second work focuses on quadrotors employed for applications such as payload delivery, inspection and search-and-rescue. Such operations pose considerable control challenges, especially when various (a priori unbounded) state-dependent unknown dynamics arises from payload variations, aerodynamic effects and from reaction forces while operating close to the ground or in a confined space. The existing adaptive control strategies for quadrotors, unfortunately, are suitable to handle unknown state-dependent uncertainties. We address such unsolved control challenge in this work via a novel adaptive artificial time delay controller. The effectiveness of this controller is validated using experimental results.