Abstract

Speaker recognition (SR) involves automatic identification of individual speakers based on their voices, often representing acoustic traits as fixed-dimensional vectors through speaker embedding. A standard speaker recognition system (SRS) consists of three key phases: training, enrollment, and recognition. In each stage, acoustic features are extracted from raw speech signals using an acoustic feature extraction module, resulting in the acquisition of essential acoustic characteristics. Commonly used acoustic features include speech spectrogram, filter bank, and Mel-frequency cepstral coefficients. During the training stage, a background model is trained to establish a mapping from training voices to embeddings. The traditional background model employs a Gaussian Mixture Model (GMM) to generate identity-vector (ivector) embeddings. In contrast, more recent and promising background models leverage deep neural networks (DNNs) to generate deep embeddings, like xvector. In the enrollment stage, a voice spoken by an individual undergoing enrollment is mapped to an enrollment embedding using the previously trained background model. In the recognition stage, the process begins by retrieving the testing embedding of a given voice from the background model. Subsequently, the scoring module is engaged to measure the similarity between the enrollment and testing embeddings. The scoring module evaluates the similarity between the speaker and recorded embedding. Following the assessment, the scoring and decision module makes a decision based on the similarity score. A decision threshold is established, which serves as a criterion to determine whether the claimed identity of the speaker is accepted or rejected. The concept of voiceprint is rapidly gaining prominence as one of the emerging biometrics, primarily owing to its seamless integration with natural and human-centered Voice User Interface (VUI). The fast progress of Speaker Recognition Systems (SRSs) is intricately linked to the evolution of Neural Networks (NNs), with a particular emphasis on Deep Neural Networks (DNNs). With strides made in deep learning, Speaker Recognition (SR) has also benefitted and found extensive applications across hardware and software platforms. However, it has been shown that NNs are vulnerable to adversarial attacks, highlighting a challenge that needs to be addressed. Thus, even though users have the convenience of authentication with Speaker Recognition services, it has become evident that these solutions are vulnerable to adversarial attacks. This vulnerability highlights that Speaker Recognition (SR) is encountering security threats, raising significant concerns about user privacy. Adversarial attack was initially implemented with images, where an image classification model was successfully deceived using adversarial examples. Drawing inspiration from the progress made in adversarial attacks within the image domain, there is a growing interest in extending these techniques to the audio field. With emerging trends, convolutional neural networks have demonstrated instability to artificially crafted perturbations that remain undetectable to the human eye. Virtually every type of model, ranging from CNN to graphical neural network (GNN), has shown vulnerability to adversarial examples, particularly in the domain of image classification. Deep learning models typically get audio input by converting the audio into a spectrogram for further processing. A spectrogram serves as a condensed representation of an audio input. Given its image-like nature, the audio spectrogram is frequently used as input data for deep learning models, especially Convolutional Neural Networks (CNNs) adapted for audio tasks. CNN-based architectures were initially designed for image processing. This thesis contributes to the assessment of Convolutional Neural Networks (CNNs) for their resilience against adversarial attacks, a domain that is yet to be extensively investigated concerning endto-end trained CNNs for speaker recognition. This examination is essential for sustaining the integrity and security of speaker recognition systems. Our study fills this gap by exploring the variations of iterative Fast Gradient Sign Method (FGSM) to carry out adversarial attacks. We note that using a vanilla iterative FGSM technique can alter the identity of each speaker sample to any other speaker within the LibriSpeech dataset. Additionally, we introduce adversarial attacks specific to Mel spectrogram features by (a) constraining the number of manipulated pixels, (b) confining alterations to certain frequency bands, (c) limiting changes to particular time segments, and (d) employing a substitute model to generate the adversarial sample. Through comprehensive qualitative and quantitative analyses, we illustrate the vulnerability and counterintuitive behavior of existing CNN-based speaker recognition systems, wherein the predicted speak

Abstract

Imagine a robot navigating a complex and unfamiliar environment - underwater, subterranean, or even a dilapidated building. Current Visual Place Recognition (VPR) techniques often struggle with such diverse scenarios, requiring re-training for each new environment. This limitation hinders the development of truly autonomous robots. This work introduces AnyLoc, a novel VPR system taking a significant step towards universality. AnyLoc leverages feature representations learned by powerful foundation models, eliminating the need for VPR-specific training. Furthermore, by combining these features with unsupervised aggregation techniques, AnyLoc can uncover unique visual characteristics that define different environments. Our experiments demonstrate AnyLoc�s potential to function across multiple environments (outdoor, indoor, aerial, underwater, subterranean, and dilapidated) without retraining, laying the groundwork for VPR solutions that could be deployed anywhere, anytime, and across anyview.

Abstract

Localization and tracking play a critical role in various fields, ranging from wireless communications and robotics to surveillance and autonomous vehicles. In the context of signal processing, localization typically involves measuring signals received from multiple sensors to estimate the source�s position. Tracking, on the other hand, extends localization to a temporal context. In this thesis, we explore the data-driven approaches to enhance the direction of arrival (DOA) trajectory estimates. DOA estimation methods involve processing multi-channel sensor array data to determine the directions from which signals originate. Classical methods have two key limitations: first, they rely on analytical properties of observed signals which do not generalize well to non-ideal conditions; second they employ block-level processing, assuming static DOA within each block. In this thesis, the first challenge is resolved with a data-driven approachs, while the second is addressed by operating in DOA-trajectory space instead of DOA space. So, instead of estimating individual DOA, the approach focuses on estimating DOA trajectories. We proposed two data-driven approaches: one grid-based and the other grid-free. In the grid-based approach, we develop a fully convolutional neural network (FCNN) inspired by the computer vison techniques of image translation. The FCNN processes the 2D low resolution spectrum as input and outputs a refined 2D spectrum. The input spectrum typically has broad peaks, while the transformed spectrum has sharp peaks, which allows precise identification of DOA trajectory parameters. In another study, we develop a data-driven approach that is both grid-free and re-useable. This addresses two issues: first, the limitation of parameter estimation on predefined grids, which affects resolution; and second, the problem of estimating all sources at once, which can make traditional methods dependent on the number of sources. A deep complex network is proposed that directly processes the complex sensor array data, with outputs comprising of complex signal amplitude (per snapshot) and trajectory parameters for a single source. We obtain a residual by removing contribution of the identified source from the input data and this residual is again fed back into the network to identify the next source making it re-useble and independent of the number of sources to be estimated. These proposed methods are rigorously evaluated through extensive experiments and analysis, demonstrating their advantages over existing approaches. These findings have the potential to impact a variety of applications, with room for further improvement.

Abstract

This thesis explores the critical challenges and provides solutions associated with automatic spoken data validation in the complex multilingual and multicultural context of India, which is crucial for developing efficient human-computer interaction (HCI) systems such as automatic speech recognition (ASR) and Text-to-speech synthesis (TTS). The diversity in linguistic backgrounds and the prevalence of non-native language speakers create unique challenges in speech communication. These challenges are exacerbated by the frequent mismatches between recorded speech and its reference text, referred to as misspoken utterances. To tackle some of these challenges, this work introduces novel unsupervised techniques for detecting spoken content mismatches. The developed methods leverage state-of-the-art self-supervised speech representation models such as Wav2Vec-2.0 and HuBERT, integrating them with Dynamic Time Warping (DTW) as well as its variants such as Phone level cost maximised DTW approach (Ph-DTW), and Phone level cost maximised weighted DTW approach (Ph-WDTW) along with cross-attention mechanisms. This work develops and tests the techniques on specially curated datasets such as IIITH MM2 Speech-Text and Indic TIMIT, which include a wide variety of phonetic and linguistic features reflective of India�s language diversity. The methodologies proposed are rigorously evaluated for their effectiveness in improving the accuracy and efficiency of spoken data validation in an unsupervised manner. The results demonstrate significant advancements in the automatic detection of mismatches, thereby enhancing the reliability of speech data for training sophisticated HCI systems. By reducing the reliance on labour-intensive manual validation processes, these approaches significantly contribute to the scalability of speech data processing. Overall, this thesis not only addresses a significant gap in the technological handling of spoken data validation but also sets a foundation for future research and development in speech technology applications within diverse linguistic landscapes. The implications of this work are broad, offering potential improvements in various data-intensive speech applications such as ASR, TTS, and Computer-aided language learning systems (CALL) to name a few. This would be achieved by ensuring a readily accessible clean training, testing, and validation set for the development of target models for the aforementioned use-cases, thus addressing the reliable data scarcity to a great extent

Abstract

The popularity of High-Frequency Trading (HFT) or algorithmic trading has surged in the last ten years, largely due to the exponential increase in computing power. Despite the evolution and optimization of software solutions tailored for HFT, challenges persist, notably due to inefficiencies associated with network stack overheads and the separation between kernel and user spaces. A pivotal element in any HFT framework is the construction of the order book, which serves as a critical snapshot of the market, underpinning trading strategies and executions. In this thesis, we introduce an innovative, simple linear data structure designed for efficient order book tracking, coupled with a hybrid binary-linear search algorithm. This algorithm is specifically engineered to dynamically maintain the top bid and ask offers, which is crucial for reflecting the market depth on Field-Programmable Gate Arrays (FPGAs). This methodology is premised on the understanding that a significant portion of trading activities concentrates at the top levels of the bid and ask prices. Through design, analysis and experimentation, we demonstrate that our simple approach is scalable and practical, outperforming previous methods.

Abstract

The Internet of Things (IoT) has witnessed exponential growth, enabling a plethora of smart applications ranging from wearable devices to smart cities. However, the widespread adoption of IoT devices hinges on their ability to operate with minimal power consumption. However, one of the critical challenges in IoT design is the need for ultra-low power consumption to enable long battery life and energy harvesting. Moreover, systems powered by energy harvesting sources need to operate effectively with lower supply voltages. Similarly, systems relying on miniaturized batteries must be capable of functioning over a broad range of supply voltages, eliminating the necessity for voltage regulators. Low power, high precision and low supply voltages are the primary requirement of an IoT based design. Voltage and current references are the basic blocks used in IoT based system. This thesis focuses on the designing of Voltage and current references circuit with low power consumption, low supply voltage and high precesion without using any external calibration circuit. Firstly. the thesis introduces a All-in-one low-power, voltage and current reference design into a single block without using any external calibration. The design works with a supply of 1V with a power consumption of 37nW shows temeprature invariancy through a wide tempertaure range of -40 to 100�C. This design specification shows the best results among the previous state-of-art works. Furthermore, in response to the growing demand of low power high precsion circuits we designed an ultra-low power current reference circuit which is independent of temperature, voltage and process without using any extra trimming circuitry. The design works with a supply of 0.8V and generates a refrence current of 543pA by consuming a power of 3.3nW. Lastly, we come up with a novel design of voltage reference. The proposed work is designed in 180nm with a supply voltage of 0.6V with an excellent Line sensitivity of 0.0012/V without using an external calibration circuit. In summary, this thesis encompases the circuit designing of low power, low voltgae and high accuracy voltage and current references without using resistors, amplifiers and external calibration circuit used in IoT applications.

Abstract

The hand is the most commonly used body part for interacting with our three-dimensional world. While it may seem ordinary, replicating hand movements with robots or in virtual/augmented reality is highly complex. Research on how hands interact with objects is crucial for advancing robotics, virtual reality, and human-computer interaction. Understanding hand movements and manipulation is critical to creating more intuitive and responsive technologies, which can significantly improve accuracy, efficiency, and scalability in various industries. Despite extensive research, programming robots to mimic human-hand interactions remains a challenging goal. One of the biggest challenges is collecting accurate 3D data for hand-object grasping. This process is complicated because of the hand�s flexibility and how hands and objects occlude in grasping poses. Collecting such data often requires expensive and sophisticated setups. However, recently, neural fields [1] have emerged, which can model 3D scenes using only multi-view images or videos. Neural fields use a continuous neural function to represent 3D scenes without needing 3D ground truth data, relying instead on differentiable rendering and multi-view photometric loss. With growing interest, these methods are becoming faster, more efficient, and better at modeling complex scenes. This thesis explores how neural fields can address two specific subproblems in hand-object interaction research. The first problem is generating novel grasps, which means predicting the final grasp pose of a hand based on its initial position and the object�s shape and location. The challenge is creating a generative model that can predict accurate grasp poses using only multi-view videos without 3D ground truth data. To solve this, we developed RealGrasper, a generative model that learns to predict grasp poses from multi-view data using photometric loss and other regularizations. The second problem is accurately capturing grasp poses and extracting contact points from multi-view videos. Current methods use the MANO model [2], which approximates hand shapes but lacks the details for precise contacts. Additionally, there is no easy way to get ground truth data for evaluating contact quality. To address this, we propose MANUS, a method for markerless grasp capture using articulated 3D Gaussians that reconstructs high-fidelity hand models from multi-view videos. We also created a large dataset, MANUS-Grasps, which includes multi-view videos of three subjects grasping over 30 objects. Furthermore, we developed a new way to capture and evaluate contacts, providing a contact metric for better assessment. We thoroughly evaluated our methods through detailed experiments, ablations, and comparisons, demonstrating that our approach outperforms existing state-of-the-art methods. We also summarize our contributions and discuss potential future directions in this field. We believe this thesis will help advance the research community further.

Abstract

Visual servoing been gaining popularity in various real-world vision-centric robotic applications, offering enhanced end-effector control through visual feedback. In the realm of autonomous robotic grasping, where environments are often unseen and unstructured, visual servoing has demonstrated its ability to provide valuable guidance. However, traditional servoing-aided grasping methods encounter challenges when faced with dynamic environments, particularly those involving moving objects. In the first part of the thesis (Chapter 3), we introduce DynGraspVS, a novel Visual Servoing-aided Grasping approach that models the motion of moving objects in its interaction matrix. Leveraging a single-step rollout strategy, our approach achieves a remarkable increase in success rate, while converging faster and achieving a smoother trajectory, while maintaining precise alignments in six degrees of freedom (6 DoF). By integrating the velocity information into the interaction matrix, our method is able to successfully complete the challenging task of robotic grasping in the case of dynamic objects, while outperforming existing deep Model Predictive Control (MPC) based methods in the PyBullet simulation environment. We test it with a range of objects in the YCB dataset with varying range of shapes, sizes, and material properties. We show the effectiveness of our approach by reporting against various evaluation metrics such as photometric error, success rate, time taken, and trajectory length. In addition to introducing DynGraspVS, this thesis in the second half (Chapter 4) explores the integration and implementation of Image-Based Visual Servoing (IBVS) mechanisms on the XARM7 robotic platform. Through successful integration, our work demonstrates the feasibility and practical applicability of IBVS in real-world robotic systems. Furthermore, a comprehensive analysis of the Recurrent Task-Visual Servoing (RTVS) framework�s performance in diverse real-world scenarios sheds light on its robustness and versatility. Additionally, the introduction of Imagine2Servo, a conditional diffusion model for generating target images, enhances the capabilities of IBVS for complex tasks. Through a combination of experimental validation and rigorous testing, this thesis provides valuable insights into the effectiveness and potential applications of IBVS in real-world robotic systems, setting the stage for future advancements in visual servoing technology

Abstract

Voice disorders are caused due to abnormality in the laryngeal system. The signs and symptoms of voice disorder may include: abnormal pitch (too high pitch, too low pitch, pitch breaks), reduction in loudness, degradation of individual�s voice quality (breathy, rough, and strained voice quality), loss of voice and so on. Instrumental assessment, auditory-perceptual assessment and objective assessment are most widely used methods for diagnosing the voice disorders. Instrumental assessment methods often involve the use of laryngoscopes and stroboscopes, but these procedures can be expensive and painful. Auditory-perceptual methods used by Speech-Language Pathologists (SLPs) is considered as a gold standard for detecting voice disorder. The decisions taken in the subjective intelligibility test vary with experience of SLPs, type of scale used, and also depend on the examiner�s experience. To address these limitations, objective or automatic assessment methods have been extensively explored in the literature. These approaches extract acoustic features from speech signals, offering reliable, costeffective, and repeatable assessments. Objective assessment methods have potential to be used as a pre-diagnostic measure for voice disorder assessment by SLPs. This thesis primarily focuses on the objective or automatic assessment methods of voice disorders. Various objective assessment methods for the automatic detection of voice disorders have been explored in the literature. These methods aim to detect the presence or absence of voice disorders, as well as assess their severity ratings. However, clinical assessment of voice disorders relies on considering the underlying etiological diagnosis. Therefore, this study proposes a clinical approach to assess voice disorders. Along with the detection which was explored in the literature, this thesis explored an objective assessment method which can automatically identify the cause of voice disorders based on the acoustic features extracted from the speech signal. The resulting speech samples are categorized into four distinct categories: structural, neurogenic, functional, and psychogenic. To conduct a comprehensive clinical analysis, a multi-level classification approach is employed. This approach involves training four binary classifiers on acoustic features to achieve a thorough assessment from a clinical perspective. Voice disorders are characterised by irregularities in the vocal fold vibration, incomplete glottal closure and opening, variation in the amplitude of consecutive opening and closing of the vocal folds. Hence the parameters, which can capture these disturbances in a better way will be able to discriminate the voice disorders from healthy samples. From the source-filter model of speech production these features can be captured in a better way from excitation source signals. Glottal flow waveform, zero frequency filtered (ZFF) signal and linear prediction (LP) residual signals are some evidence of excitation source signal. Features derived from these evidences were used to capture the characteristics of voice disorders. First study explores perturbation (jitter, shimmer, noise to harmonic ratios etc.) and cepstral features derived from the excitation source evidence for detection and identification of voice disorders. In this regard state-of-art speech signal processing techniques, such as quasi-closed-phase (QCP) analysis, LP analysis and ZFF techniques, have been explored in this thesis in order to capture the excitation source information. From this study, it was concluded that perturbation parameters can capture voice disorder information in a better way. In addition it was also found that excitation source based features can discriminate between the organic voice disorder from non-organic voice disorder, as well as structural voice disorders from the neurogenic voice disorder category. However, distinguishing functional voice disorders from psychogenic voice disorders proved to be challenging in the study. From the first study, it was found that excitation source based features are able to differentiate the various categories of voice disorders. Computation of these features involves the detection of epoch locations from speech. Therefore, accurate estimation of epoch locations is important for computing these features for the automatic detection and identification of voice disorders. Second study aimed to compare the various algorithms for detecting epoch locations from the speech associated with voice disorders. In this regard, nine state-of-the-art epoch extraction algorithms were considered, and their performance for different categories of voice disorders was evaluated. From the results it can be concluded that most of the epoch extraction methods showed better performance for healthy speech; however, their performance was degraded for speech associated with voice disorders. Furthermore, the performance of epo

Abstract

Deep Reinforcement Learning (DRL) has emerged as a prominent method for enhancing the training of autonomous Unmanned Aerial Vehicles (UAVs). At the heart of this advancement lies the critical and complex challenge of obstacle avoidance, which is essential for ensuring that UAVs can navigate safely and efficiently. The incorporation of DRL into UAV training protocols enhances their ability to autonomously navigate through and adapt to diverse environments. This, in turn, is pivotal for expanding the operational capabilities of UAVs, enabling them to tackle a broader array of applications across various challenging and complex scenarios. In our research, we undertake a multifaceted examination of the impact of measurement uncertainty on the performance of DRL-based waypoint navigation and obstacle avoidance for UAVs covering both static and dynamic obstacles environment scenario. The challenge increases when the obstacles changes from static to dynamic as now these obstacles can change position in time and affect the navigation. The measurement uncertainty primarily stems from sensor noise, which impacts localization accuracy and obstacle detection capabilities. We model this uncertainty as adhering to a Gaussian probability distribution, characterized by an unknown mean and variance. Our research unfolds by assessing the performance of a DRL agent, meticulously trained using the Proximal Policy Optimization (PPO) algorithm, operating within an environment characterized by continuous state and action spaces. This investigation takes place in unseen randomized environments, each exposed to varying degrees of state-space noise, effectively emulating the effects of noisy sensor readings. Our primary objective is to pinpoint the threshold at which the policy becomes susceptible to noise, ultimately paving the way for us to explore diverse filtering techniques to mitigate these detrimental effects. Our findings reveal that the DRL agent exhibits a remarkable degree of inherent robustness against specific types of noise. We leverage this inherent robustness to bolster its performance in scenarios where it would otherwise succumb to the deleterious influence of state-space noise. To empirically validate our research, we undertake extensive training and testing of the DRL agent across a spectrum of UAV navigation scenarios within the PyBullet physics simulator. In a significant stride towards practical applicability, we port the policy distilled through simulation directly onto a real UAV without any additional modifications, and subsequently, we meticulously verify its performance in a real-world operational setting. This transformative approach holds the potential to inform the selection of sensors with varying biases and variances, and intriguingly, it suggests that artifcially injecting noise into measurements can yield performance enhancements in certain scenarios. The substantiation of this proposition is achieved through rigorous testing on a real UAV, thereby solidifying the practicality and real-world utility of our approach. In summation, our research contributes to the burgeoning field of UAV autonomy by shedding light on the intricate interplay between measurement uncertainty and Deep Reinforcement Learning-based navigation and obstacle avoidance. The insights garnered from our study not only advance our understanding of UAV operation but also provide actionable guidance for sensor selection and noise injection strategies to bolster real-world performance.