Abstract

The hand is the most commonly used body part for interacting with our three-dimensional world. While it may seem ordinary, replicating hand movements with robots or in virtual/augmented reality is highly complex. Research on how hands interact with objects is crucial for advancing robotics, virtual reality, and human-computer interaction. Understanding hand movements and manipulation is critical to creating more intuitive and responsive technologies, which can significantly improve accuracy, efficiency, and scalability in various industries. Despite extensive research, programming robots to mimic human-hand interactions remains a challenging goal. One of the biggest challenges is collecting accurate 3D data for hand-object grasping. This process is complicated because of the hand�s flexibility and how hands and objects occlude in grasping poses. Collecting such data often requires expensive and sophisticated setups. However, recently, neural fields [1] have emerged, which can model 3D scenes using only multi-view images or videos. Neural fields use a continuous neural function to represent 3D scenes without needing 3D ground truth data, relying instead on differentiable rendering and multi-view photometric loss. With growing interest, these methods are becoming faster, more efficient, and better at modeling complex scenes. This thesis explores how neural fields can address two specific subproblems in hand-object interaction research. The first problem is generating novel grasps, which means predicting the final grasp pose of a hand based on its initial position and the object�s shape and location. The challenge is creating a generative model that can predict accurate grasp poses using only multi-view videos without 3D ground truth data. To solve this, we developed RealGrasper, a generative model that learns to predict grasp poses from multi-view data using photometric loss and other regularizations. The second problem is accurately capturing grasp poses and extracting contact points from multi-view videos. Current methods use the MANO model [2], which approximates hand shapes but lacks the details for precise contacts. Additionally, there is no easy way to get ground truth data for evaluating contact quality. To address this, we propose MANUS, a method for markerless grasp capture using articulated 3D Gaussians that reconstructs high-fidelity hand models from multi-view videos. We also created a large dataset, MANUS-Grasps, which includes multi-view videos of three subjects grasping over 30 objects. Furthermore, we developed a new way to capture and evaluate contacts, providing a contact metric for better assessment. We thoroughly evaluated our methods through detailed experiments, ablations, and comparisons, demonstrating that our approach outperforms existing state-of-the-art methods. We also summarize our contributions and discuss potential future directions in this field. We believe this thesis will help advance the research community further.

Abstract

Visual servoing been gaining popularity in various real-world vision-centric robotic applications, offering enhanced end-effector control through visual feedback. In the realm of autonomous robotic grasping, where environments are often unseen and unstructured, visual servoing has demonstrated its ability to provide valuable guidance. However, traditional servoing-aided grasping methods encounter challenges when faced with dynamic environments, particularly those involving moving objects. In the first part of the thesis (Chapter 3), we introduce DynGraspVS, a novel Visual Servoing-aided Grasping approach that models the motion of moving objects in its interaction matrix. Leveraging a single-step rollout strategy, our approach achieves a remarkable increase in success rate, while converging faster and achieving a smoother trajectory, while maintaining precise alignments in six degrees of freedom (6 DoF). By integrating the velocity information into the interaction matrix, our method is able to successfully complete the challenging task of robotic grasping in the case of dynamic objects, while outperforming existing deep Model Predictive Control (MPC) based methods in the PyBullet simulation environment. We test it with a range of objects in the YCB dataset with varying range of shapes, sizes, and material properties. We show the effectiveness of our approach by reporting against various evaluation metrics such as photometric error, success rate, time taken, and trajectory length. In addition to introducing DynGraspVS, this thesis in the second half (Chapter 4) explores the integration and implementation of Image-Based Visual Servoing (IBVS) mechanisms on the XARM7 robotic platform. Through successful integration, our work demonstrates the feasibility and practical applicability of IBVS in real-world robotic systems. Furthermore, a comprehensive analysis of the Recurrent Task-Visual Servoing (RTVS) framework�s performance in diverse real-world scenarios sheds light on its robustness and versatility. Additionally, the introduction of Imagine2Servo, a conditional diffusion model for generating target images, enhances the capabilities of IBVS for complex tasks. Through a combination of experimental validation and rigorous testing, this thesis provides valuable insights into the effectiveness and potential applications of IBVS in real-world robotic systems, setting the stage for future advancements in visual servoing technology

Abstract

Voice disorders are caused due to abnormality in the laryngeal system. The signs and symptoms of voice disorder may include: abnormal pitch (too high pitch, too low pitch, pitch breaks), reduction in loudness, degradation of individual�s voice quality (breathy, rough, and strained voice quality), loss of voice and so on. Instrumental assessment, auditory-perceptual assessment and objective assessment are most widely used methods for diagnosing the voice disorders. Instrumental assessment methods often involve the use of laryngoscopes and stroboscopes, but these procedures can be expensive and painful. Auditory-perceptual methods used by Speech-Language Pathologists (SLPs) is considered as a gold standard for detecting voice disorder. The decisions taken in the subjective intelligibility test vary with experience of SLPs, type of scale used, and also depend on the examiner�s experience. To address these limitations, objective or automatic assessment methods have been extensively explored in the literature. These approaches extract acoustic features from speech signals, offering reliable, costeffective, and repeatable assessments. Objective assessment methods have potential to be used as a pre-diagnostic measure for voice disorder assessment by SLPs. This thesis primarily focuses on the objective or automatic assessment methods of voice disorders. Various objective assessment methods for the automatic detection of voice disorders have been explored in the literature. These methods aim to detect the presence or absence of voice disorders, as well as assess their severity ratings. However, clinical assessment of voice disorders relies on considering the underlying etiological diagnosis. Therefore, this study proposes a clinical approach to assess voice disorders. Along with the detection which was explored in the literature, this thesis explored an objective assessment method which can automatically identify the cause of voice disorders based on the acoustic features extracted from the speech signal. The resulting speech samples are categorized into four distinct categories: structural, neurogenic, functional, and psychogenic. To conduct a comprehensive clinical analysis, a multi-level classification approach is employed. This approach involves training four binary classifiers on acoustic features to achieve a thorough assessment from a clinical perspective. Voice disorders are characterised by irregularities in the vocal fold vibration, incomplete glottal closure and opening, variation in the amplitude of consecutive opening and closing of the vocal folds. Hence the parameters, which can capture these disturbances in a better way will be able to discriminate the voice disorders from healthy samples. From the source-filter model of speech production these features can be captured in a better way from excitation source signals. Glottal flow waveform, zero frequency filtered (ZFF) signal and linear prediction (LP) residual signals are some evidence of excitation source signal. Features derived from these evidences were used to capture the characteristics of voice disorders. First study explores perturbation (jitter, shimmer, noise to harmonic ratios etc.) and cepstral features derived from the excitation source evidence for detection and identification of voice disorders. In this regard state-of-art speech signal processing techniques, such as quasi-closed-phase (QCP) analysis, LP analysis and ZFF techniques, have been explored in this thesis in order to capture the excitation source information. From this study, it was concluded that perturbation parameters can capture voice disorder information in a better way. In addition it was also found that excitation source based features can discriminate between the organic voice disorder from non-organic voice disorder, as well as structural voice disorders from the neurogenic voice disorder category. However, distinguishing functional voice disorders from psychogenic voice disorders proved to be challenging in the study. From the first study, it was found that excitation source based features are able to differentiate the various categories of voice disorders. Computation of these features involves the detection of epoch locations from speech. Therefore, accurate estimation of epoch locations is important for computing these features for the automatic detection and identification of voice disorders. Second study aimed to compare the various algorithms for detecting epoch locations from the speech associated with voice disorders. In this regard, nine state-of-the-art epoch extraction algorithms were considered, and their performance for different categories of voice disorders was evaluated. From the results it can be concluded that most of the epoch extraction methods showed better performance for healthy speech; however, their performance was degraded for speech associated with voice disorders. Furthermore, the performance of epo

Abstract

Deep Reinforcement Learning (DRL) has emerged as a prominent method for enhancing the training of autonomous Unmanned Aerial Vehicles (UAVs). At the heart of this advancement lies the critical and complex challenge of obstacle avoidance, which is essential for ensuring that UAVs can navigate safely and efficiently. The incorporation of DRL into UAV training protocols enhances their ability to autonomously navigate through and adapt to diverse environments. This, in turn, is pivotal for expanding the operational capabilities of UAVs, enabling them to tackle a broader array of applications across various challenging and complex scenarios. In our research, we undertake a multifaceted examination of the impact of measurement uncertainty on the performance of DRL-based waypoint navigation and obstacle avoidance for UAVs covering both static and dynamic obstacles environment scenario. The challenge increases when the obstacles changes from static to dynamic as now these obstacles can change position in time and affect the navigation. The measurement uncertainty primarily stems from sensor noise, which impacts localization accuracy and obstacle detection capabilities. We model this uncertainty as adhering to a Gaussian probability distribution, characterized by an unknown mean and variance. Our research unfolds by assessing the performance of a DRL agent, meticulously trained using the Proximal Policy Optimization (PPO) algorithm, operating within an environment characterized by continuous state and action spaces. This investigation takes place in unseen randomized environments, each exposed to varying degrees of state-space noise, effectively emulating the effects of noisy sensor readings. Our primary objective is to pinpoint the threshold at which the policy becomes susceptible to noise, ultimately paving the way for us to explore diverse filtering techniques to mitigate these detrimental effects. Our findings reveal that the DRL agent exhibits a remarkable degree of inherent robustness against specific types of noise. We leverage this inherent robustness to bolster its performance in scenarios where it would otherwise succumb to the deleterious influence of state-space noise. To empirically validate our research, we undertake extensive training and testing of the DRL agent across a spectrum of UAV navigation scenarios within the PyBullet physics simulator. In a significant stride towards practical applicability, we port the policy distilled through simulation directly onto a real UAV without any additional modifications, and subsequently, we meticulously verify its performance in a real-world operational setting. This transformative approach holds the potential to inform the selection of sensors with varying biases and variances, and intriguingly, it suggests that artifcially injecting noise into measurements can yield performance enhancements in certain scenarios. The substantiation of this proposition is achieved through rigorous testing on a real UAV, thereby solidifying the practicality and real-world utility of our approach. In summation, our research contributes to the burgeoning field of UAV autonomy by shedding light on the intricate interplay between measurement uncertainty and Deep Reinforcement Learning-based navigation and obstacle avoidance. The insights garnered from our study not only advance our understanding of UAV operation but also provide actionable guidance for sensor selection and noise injection strategies to bolster real-world performance.

Abstract

In this thesis, we work towards improving the representation of low-resource languages in the digital world by easing the access and participation of these communities to reliable information hubs like Wikipedia. Although the internet has brought in an information age, it is disproportionately distributed amongst language communities since content and tools for low-resource languages are less readily available. Recognizing the importance of Wikipedia as the primary source of reliable, unbiased information, we seek to improve the information available by automatically generating Wikipedia articles in low-resource languages to improve the quality and quantity of articles available. Our work begins with XWikiGen, a cross-lingual multi-document summarization task that aims to generate Wikipedia articles using reference texts and article outlines. We propose the XWikiRef dataset to facilitate this, which spans eight languages and five distinct domains, laying the groundwork for our experimentation. We observe that existing Wikipedia text generation tools rely on Wikipedia outlines to provide a structure for the article. Hence, we also propose Multilingual Outlinegen, a task focused on generating Wikipedia article outlines with minimal input in low-resource languages. To support this task, we introduce another novel dataset, WikiOutlines, which encompasses ten languages over eight domains, further enriching available multilingual tools for further research work. An important question with text generation is the reliability of the generated information. For this, we propose the task of Cross-lingual Fact Verification (FactVer). In this task, we aim to verify the facts in the source articles against their references, addressing the growing concern over hallucinations in Language Models. We manually annotate the FactVer dataset for this task to benchmark our results against it. By exploring these three tasks, we highlight the disparity in content and tools available in low-resource languages, underscore the importance of multilingual and cross-lingual tools in global participation and propose innovative solutions to enhance Wikipedia�s accessibility and reliability for low-resource languages. Overall, we contribute multiple novel datasets and methodologies to automatic text generation and highlight the importance of inclusivity in the Internet age. By tackling the challenges of article generation, outline generation and fact verification, we pave the way for future advancements that promise to improve the quality and quantity of information available to low-resource language communities of the world.

Abstract

In analog and radio frequency (RF) circuit design, innovation is a cornerstone driving progress amid escalating demands for energy efficiency, reliability, and performance optimization. This thesis, titled �Innovative Design Approaches with Self-Adaptive Methods for Analog and RF Circuits,� presents a comprehensive exploration of cutting-edge methodologies to revolutionize circuit design paradigms. Divided into two distinct yet interconnected sections, the thesis embarks on a journey through the intricate landscape of analog and RF circuitry. The first section delves into the intricate art of low-power analog and RF design, driven by the relentless pursuit of energy efficiency. Here, novel architectures, circuit topologies, and optimization techniques have been explored to minimize power consumption while preserving signal integrity and performance metrics. The journey begins with a meticulous investigation into two low-supply 0.5V current/voltage reference designs tailored for ultra-low power IoT and biomedical applications. The first one, designed in CMOS 90nm technology, proposes a current and voltage reference that achieves a typical accuracy of 34.6ppm/?C (29.68ppm/ ?C) over a wide temperature range of -55?C to 75?C with typical value 63.32pA(0.35V). We observe excellent line sensitivities of 0.0318%/V and 0.0576%/V for voltage and current references in a supply range of 0.5V - 2.3V. The second reference focuses on reducing the power to half compared to the previous one. The design generates reference values of 90.7pA and 288mV, giving an excellent temperature coefficient of 15.2ppm/�C and 36.8ppm/�C for compensated current and voltage values, respectively, at a nominal supply of 0.5V for a wide temperature range of -55�C to 100�C. The circuit works for a wide voltage range of 0.5V - 2.6V with a supply sensitivity of 0.028%/V and 0.154%/V for current and voltage reference, respectively. The thesis then introduces a new nW range gate-leakage-based Sub-Bandgap Voltage Reference (SUB-BGR) for low-power, high-temperature IoT applications. Designed in TSMC 65nm technology, the proposed architecture achieves an accuracy of 94ppm/?C. It achieves an excellent line sensitivity of 0.0066%/V for a supply range of 0.7V to 4V and PSRR of 89dB at DC 1V supply. As the spectrum broadens, the focus shifts to the high-frequency domain, where the work explores the intricacies of on-chip Vector Network Analyzers (VNAs). It introduces a fully integrated low-power, low-area on-chip single-port CMOS VNA, designed explicitly for bio-molecule detection, operating in a tunable frequency range of 0.5GHz to 2.5GHz. This design incorporates innovative features such as an IDC sensor and a high-linearity Low Noise Amplifier (LNA), demonstrating superior performance metrics within a compact footprint. The process, voltage, and temperature (PVT) variations immensely affect the circuit performance of the Analog and RF circuits. The second section of the thesis delves into self-adaptive methodologies tailored to mitigate variations across PVT, addressing the challenge of ensuring consistent circuit performance. Leveraging adaptive control and feedback mechanisms, self-adaptive loops are developed and implemented in simulation, dynamically adjusting circuit parameters in real-time. For efficient self-adaptation, the information of P, V, and T is necessary to provide accurate real-time feedback, enabling dynamic parameter adjustments in analog and RF circuits. This thesis significantly contributes to the advancement of analog and RF circuit design methodologies through a synthesis of theoretical analyses, simulation studies, and practical implementations. By embracing innovation and harnessing the power of self-adaptive methods, this research paves the way for a new era of agile, efficient, and resilient analog and RF circuits.

Abstract

Multi-agent systems (MAS) are distributed systems composed of multiple autonomous agents interacting to achieve a common or conflicting goal. MAS tackles complex and dynamic problems that a single agent cannot solve, resulting in better problem-solving skills, enhanced reliability, and improved scalability. This thesis explores the challenges facing MAS, particularly related to their game-theoretic, fairness, security, and privacy guarantees. A game-theoretically sound MAS is one where the agent interaction can be modeled as a game and analyzed using game-theoretic concepts. This leads to a more stable and efficient system, as agents are incentivized to make decisions that align with the system goals. This thesis focuses on civic crowdfunding, a method for raising funds through voluntary contributions for public projects (e.g., public parks). Our work enriches the existing literature by designing more inclusive mechanisms and providing fairer rewards and efficiency over the blockchain. Fairness is also an essential aspect of MAS as it ensures that the actions and outcomes of agents are equitable and just, resulting in MAS�s long-term stability and sustainability. This thesis looks at fair incentives in Transaction Fee Mechanisms (TFM). Blockchains employ TFMs to include transactions from the set of outstanding transactions in a block. We argue that existing TFMs� incentives are misaligned for a cryptocurrency�s greater market adoption. We propose TFMs that provide fairer rewards to the transaction creators and minimize the surplus collected to the creators Last, security and privacy are crucial aspects of MAS, as the autonomy and decentralization of agents in MAS can lead to the exposure of sensitive information. In this thesis, we specifically focus on privacy guarantees for MAS like (i) auctions, (ii) voting, and (iii) distributed constraint optimization (DCOPs). We propose privacy-preserving applications that preserve agents� sensitive information while proving the computation�s verifiability

Abstract

Embodied AI, where artificial agents interact with their environment through sensors and actuators, holds immense potential for real-world applications such as robotic assistance. Efficient object navigation and locating strategies are crucial for robotic assistance in real-world environments. However, existing methodologies often encounter challenges in adapting to dynamic environments and incorporating human-like reasoning for optimal decision-making. This thesis aims to bridge these gaps by addressing two fundamental challenges in embodied AI: Multi-Object Navigation (MultiON) and optimal object location within household environments. First, we tackle MultiON, where a robot is tasked with localizing multiple instances of diverse object classes in dynamic environments. This is a fundamental task for an assistive robot in a home or a factory. Existing methods for this task have viewed this as a direct extension of Object Navigation (ON), the task of localising a single instance of an object class, and are pre-sequenced, i.e., the sequence in which the object classes are to be explored is provided in advance. This is a strong limitation in practical applications characterized by dynamic changes. We present Sequence-Agnostic MultiON (SAM), which is the task of locating an instance each of multiple objects in a household environment in no pre-defined order. We present a deep reinforcement learning framework for an actor-critic architecture and a reward specification. It exploits past experiences and seeks to reward progress towards individual as well as multiple target object classes. We use photo-realistic scenes from Gibson benchmark dataset in the AI Habitat 3D simulation environment to experimentally show that our method performs better than a pre-sequenced approach and a state-of-the-art ON method extended to MultiON. Next, we present CLIPGraphs, a novel method for determining the best room to place or find objects within home environments. Existing approaches predominantly rely on large language models (LLMs) or reinforcement learning (RL) policies, neglecting commonsense domain knowledge. CLIPGraphs effectively integrates domain knowledge, data-driven methods, and multimodal learning to ascertain object-room affinities. Specifically, it (a) encodes a knowledge graph of prior human preferences about the room location of different objects in home environments, (b) incorporates vision-language features to support multimodal queries based on images or text, and (c) uses a graph network to learn object-room affinities based on embeddings of the prior knowledge and the vision-language features. We demonstrate that our approach provides better estimates of the most appropriate location of objects from a benchmark set of object categories in comparison with state-of-the-art baselines.

Abstract

Metabolism is an integral part of cellular physiology, with the liver as the central organ for a wide range of metabolic functions and homeostasis. The liver, unlike other organs, has a remarkable capacity to regenerate after partial loss of its mass, thus maintaining a constant liver-to-body weight ratio to preserve homeostasis. In an injury-free liver, the turnover of regeneration is very slow, but in the presence of an injury or perturbation, the regenerative response is triggered as a reparative strategy. Perturbations interfering with liver functions and disturbing the homeostatic state have serious repercussions leading to metabolic disorders like hepatic steatosis, non-alcoholic fatty liver disease (NAFLD), non-alcoholic steatohepatitis (NASH) and hepatocellular carcinoma (HCC). Ageing is one risk factor that increases the susceptibility to these diseases. The regenerative ability of the liver has been used to treat these diseases in the form of liver transplantation and partial liver resection, but recurrence of the disease is often observed after a few years. Understanding the molecular network that controls liver function and its regenerative ability in health and disease is crucial for developing clinical applications. The emergence of high throughput omics technologies provides a scope to develop a systems-level understanding of the disease. In this direction, we attempt to understand the pathophysiology of the liver by adopting systems biology approach for omics data interpretation. Tissue homeostasis and regeneration depend on the reversible transitions between quiescence (G0) and cell proliferation. During regeneration, the liver needs to maintain the essential metabolic tasks along with the fulfilment of metabolic requirements for hepatocyte growth and division. To understand the regulatory mechanisms involved in balancing the liver function and proliferation demand after injury or resection, we analyzed RNA sequencing temporal data of liver regeneration after two-thirds partial hepatectomy (PH) using network inference and mathematical modelling approaches. The reconstruction of the dynamic regulatory network revealed the overall temporal coordination of metabolism, RNA splicing and cell cycle during liver regeneration. A temporal shift in the gene expression pattern corresponding to increased hepatocyte proliferation and decreased hepatocyte function is observed with HNF4A as a key transcriptional activator. Based on these key observations, we developed a mathematical model of the HNF4A regulatory circuit, which showed the emergence of different states corresponding to compensatory metabolism, proliferation, and epithelial-to-mesenchymal transition. We showed that a mutually exclusive behaviour emerges due to the bistable inactivation of HNF4A, which controls the initiation and termination of liver regeneration and different population-level behaviour. Through our approach of modelling a regulatory circuit from high-throughput gene expression data, we proposed a mechanistic explanation of different states observed in single-cell RNA sequencing data of liver regeneration. The functional impairment of the liver with ageing reduces its regenerative capability and predisposes it to NAFLD and HCC. Mapping the molecular network of the liver encompassing these physiological (ageing) and pathological conditions may help to understand the crosstalk of ageing with different liver diseases. We performed networklevel analyses by integrating mouse transcriptomic data with protein-protein interaction (PPI) network. A sample-wise analysis using network entropy measure was performed, which showed an increasing trend with ageing and helped to identify ageing genes based on local entropy changes. To gain further insights, we also integrated the differentially expressed genes (DEGs) between young and different age groups with the PPI network and identified core modules and nodes associated with ageing. Finally, we computed the network proximity of the ageing network with different networks of liver diseases and regeneration to quantify the effect of ageing. Our analysis revealed the complex interplay of immune, cancer signalling, and metabolic genes in the ageing liver. We found significant network proximities between ageing and NAFLD, HCC, liver damage conditions, and the early phase of liver regeneration with common nodes, including NLRP12, TRP53, GSK3B, CTNNB1, MAT1 and FASN. A common theme involving pathways in cancers connects ageing, regeneration and liver diseases. Overall, our study maps the network-level changes of ageing and their interconnections with the physiology and pathology of the liver. While the regulated process of liver regeneration is crucial for damage-induced repair, its dysregulation may lead to HCC, the most common type of liver cancer. Understanding the molecular pathogenesis of HCC sequentially from precancerous state to cancer

Abstract

Cognitive and psychological studies on human reading indicate that the effort required to read and understand text increases with sentence complexity (Klein and Kurkowski, 1974). Modern natural language processing (NLP) applications face similar challenges. Processing complex sentences with high accuracy remains difficult in computational linguistics, necessitating automatic sentence simplification techniques (Chandrasekar et al., 1996). Sentence complexity can be classified into �lexical complexity� and �syntactic complexity�. Lexical complexity can be managed by utilizing resources like lexicons, dictionaries, and thesauruses to replace infrequent words with common ones. To address syntactic complexity, analyzing sentence structure and applying simplification operations is essential. There are many applications of sentence simplification in NLP. Machine translation systems struggle with long, complex sentences, especially when dealing with divergent language pairs. For parsing, (McDonald and Nivre, 2007) showed that syntactic parsing of long sentences and identifying long-distance dependencies remain challenging. Simplifying sentences can aid parsing and machine translation by breaking down sentences into smaller parts. In automatic summarization, simplification can improve accuracy by extracting smaller units of information. In this work, we studied and analyzed existing sentence simplification methods for Hindi. We developed new approaches, both rule-based and statistical, to design a better system that overcomes the limitations of existing systems while improving quality and readability. As an application, we examined the effects of sentence simplification on Hindi to English machine translation systems.