Abstract

This thesis addresses the challenge of real-time facial emotion recognition through the lens of Deep Convolutional Neural Networks (DCNNs). While DCNNs have proven to be state-ofthe-art for image classification, their efficacy is contingent upon abundant, accurately labeled data. Emotion recognition, a complex and subjective task, suffers from a scarcity of comprehensive datasets. Existing facial expression databases lack the depth required to train deep neural networks effectively, particularly those that have excelled in object recognition tasks. The limitations are further compounded by high inter-subject variations, including age, gender, ethnic backgrounds, and individual expressiveness levels. Facial images are also subject to occlusions, pose variations, and wearable accessories, intensifying the complexity of the emotion recognition problem. Consequently, the choice of a benchmark dataset plays a pivotal role in the development and evaluation of an effective model. In this research, we explore the nuances of real-time facial emotion recognition, delving into the challenges posed by limited and diverse datasets. Our approach involves a deep investigation into the intricacies of emotion representation in facial images, leveraging the power of DCNNs. By addressing the dataset limitations and exploring techniques to mitigate the impact of diverse personal attributes, we aim to contribute to the advancement of facial emotion recognition models. The ultimate goal is the development of a real-time facial emotion recognition web application, demonstrating the practical application of our findings in a user-friendly and accessible manner

Abstract

The thesis entitled �Design of Low-Power Electronic Circuits & Systems for Point of Care Healthcare Applications� focuses on the development of a cutting-edge point of care device tailored for deployment in remote and resource-constrained settings. The impetus for this project stems from the urgent need to extend quality healthcare services in areas where conventional, cumbersome, and expensive medical equipment is untenable. Engineered for efficiency and ease of use, the device is characterized by its low power consumption and portability, making it well-suited for environments with sparse access to electrical power and established medical facilities. The core technology utilizes Electrical Impedance Spectroscopy (EIS) integrated with vital signs monitoring to detect physiological changes indicative of health conditions that require immediate medical attention. This dual functionality not only enables the early detection of critical health issues but also promotes timely medical interventions, significantly enhancing patient outcomes. This research further encompasses the innovative design of an active high pass filter within the analog baseband chain of the device�s integrated circuit. These components are specifically tailored for use in Frequency Modulated Continuous Wave (FMCW) radar systems, which form part of the receiver design to accurately detect heart and respiratory rates. Comprehensive evaluation of the device throughout the research confirms its effectiveness in meeting the healthcare delivery needs of isolated regions. The findings highlight the transformative potential of such devices in improving healthcare accessibility and efficacy across underserved populations. This device emerges as a vital instrument in the global initiative to enhance healthcare delivery, demonstrating significant implications for the future of remote medical practices and technology-driven health interventions.

Abstract

The rapid advancements in natural language processing have primarily focused on high-resource languages, leaving low-resource languages, such as those in the Dravidian family, underrepresented in terms of research and resources. To bridge this gap and enable more effective processing of Dravidian languages, this thesis explores the utilization of context beyond the sentence level, delving into linguistic structures at the paragraph and document level, and leverages multilingual training. This is done particularly for the tasks of neural machine translation and multi-class sentence classification. CoPara, the first publicly available paragraph-level n-way aligned corpus for Dravidian languages (Kannada, Malayalam, Tamil, Telugu) and English, was created. This dataset fills a critical gap in multilingual resources for low-resource Dravidian languages and enables research on the impact of paragraph-level information on NMT tasks. The corpus contains aligned paragraphs across English and four Dravidian languages, providing a rich resource for studying cross-lingual phenomena and improving machine translation quality. To demonstrate the utility of CoPara, neural machine translation experiments were conducted by fine-tuning a pre-trained multilingual sequence-to-sequence model on the dataset. The results show significant improvements in translation quality for paragraphs across all language pairs over models trained using sentence-level data, highlighting the potential of leveraging paragraph-level information and multilingual training for enhancing machine translation systems in low-resource settings. An annotated dataset of SEBI legal case files, along with its Indic adaptation, was created. The dataset consists of sentences classified into legally applicable categories and is the first of its kind for Indian legal adjudication orders concerning insider trading. This dataset facilitated the development of a novel system for aspect-based semantic segmentation of legal case files, tailored to different stakeholders like investors, defense lawyers, and adjudicating officers. The system leverages document-level context to improve sentence classification performance, and multilingual training using the Indic adaptation further enhances the results. The results highlight the importance of incorporating context beyond the sentence level and leveraging multilingual training to advance Dravidian language processing. In summary, the thesis provides a comprehensive exploration of utilizing paragraph and document-level context for NMT and sentence classification tasks. The datasets and methodologies introduced pave the way for future research on long-text processing and context-aware NLP tasks in low-resource languages, with potential implications for other language families and domains.

Abstract

Autonomous Underwater Vehicles (AUVs) play a vital role in exploring and mapping underwater environments. However, the presence of nadir gaps, or blind zones, in commercial AUVs can lead to unexplored areas during mission execution, limiting their effectiveness. We refer to the area under inspection not sensed by the AUV as nadir gap. We study the problem of underwater exploration of a rectangular region focusing on single and multiple AUVs with nadir gaps. Complete exploration of a given area using such a AUV requires careful planning such that no area is left uncovered due to nadir gaps. Initially unexplored areas can be covered later using extra trips, but that may cause some areas to be covered multiple times leading to redundant overlap and ineffective resource usage. Hence, we devise strategies that not only cover the entire input area but also aim to minimize mission completion time and the total number of turns taken by the AUV(s). There has been extensive work on developing strategies which explore the entire area of interest but all of them are limited to exploration with a single AUV when nadir gap is a constraint. When multiple AUVs are available for deployment, the challenge is how to facilitate efficient collaboration among them while effectively compensating for the nadir gap and avoiding collision. A comprehensive study on efficient exploration of the entire region using an arbitrary number of AUVs each with nadir gaps is lacking in the literature. Our thesis aims to fill this gap by proposing scalable path planning strategies that offer complete coverage while minimizing either the mission completion time or the total number of turns performed. Another challenge with the availability of multiple AUVs can be their unexpected failures. We take the first step towards addressing this challenge with two AUVs specifically and develop robust strategies which can handle the failure of a single AUV by effectively re-planning the path for the other (alive) AUV to obtain complete coverage. On the empirical front, we consider diverse input configurations based on real-world instances and provide extensive simulation results to validate the key properties of our strategies developed for single AUV, two AUV and multiple AUV scenarios. Further, we also show the maximal number of AUVs required that can be utilized fruitfully to achieve complete coverage for a given input (increasing the number of AUVs after this threshold doesn�t decrease the overall completion time) and how to compute it efficiently. We end with a discussion of our most effective strategies from which a practitioner can select the best strategy based on the use case.

Abstract

For about fifty years, hardware designers have been relying on different types of semiconductor scaling laws, like Moore�s Law and Dennard scaling, to achieve gains in performance. The industry became used to processor performance per watt, doubling approximately every 18 months. Over the past decade, we have seen the breakdown of these scaling predictions. With the old certainties of scaling silicon geometries gone forever, the industry is already changing. The number of cores has increased in a single die. SoCs like mobile phone processors are combining application-specific co-processors, GPUs, and DSPs in different configurations to maintain the performance scaling trend. However, in a postDennard, post-Moore world, further processor specialization will be needed to achieve performance improvements. Emerging applications such as artificial intelligence and vision are demanding heavy computational performance that cannot be met by conventional architectures. This inevitably leads to creating special purpose, domain-specific accelerators. A domain-specific accelerator (DSA) is a processor or set of processors that are optimized to perform a narrow range of computations. They are tailored to meet the needs of the algorithms required for their domain. For example, an AI accelerator might have an array of elements, including multiplyaccumulate functionality, to efficiently undertake matrix operations. Google�s Tensor Processing Unit (TPU), Neural Engine in Apple�s M1 processor, and Xilinx Vitis-AI Engine are popular ASIC-based DSAs. ASIC bases DSAs provide significant gains in performance and power efficiency. However, due to the long design cycles and high engineering costs, they may not cope with the ever-evolving computation landscape. Field-Programmable Gate Arrays (FPGAs) offer advantages over ApplicationSpecific Integrated Circuits (ASICs) in certain scenarios due to their flexibility and quicker development times. FPGAs are programmable hardware, allowing users to configure their functionality after manufacturing. In contrast, ASICs are hardwired for specific tasks. This flexibility makes FPGAs suitable for prototyping, testing, and adapting to changing requirements without the need for costly chip redesigns. Developing an ASIC is a complex and time-consuming process, often taking several months to years. FPGAs can be programmed and deployed much faster, making them advantageous when speedto-market is crucial. ASICs can be expensive to design and manufacture, especially for low-volume or rapidly changing applications. FPGAs have a lower initial cost, as they don�t require custom chip fabrication. This cost-effectiveness is notable for small production runs or research projects. Therefore, FPGAs are rapidly evolving towards becoming an alternative to custom ASICs to design DSAs because of their low power consumption and a higher degree of parallelism. Designing DSAs on an FPGA requires carefully calibrating the FPGA compute and memory resources to achieve optimal throughput from a given device. Hardware Descriptive Languages (HDL) like Verilog have been traditionally used to design FPGA hardware. HDLs are generic and not geared towards any domain. Also, the user has to put in much effort to describe the hardware at the register transfer level using the HDL. A recent trend is emerging wherein existing HDLs are used to create carefully handwritten templates suiting a specific domain. A compiler framework weaves together these templates to generate the DSA for accelerating the domain computations. This approach requires expensive design synthesis and FPGA re-flashing for accelerating different algorithms from the domain. In many edge and deeply embedded applications, this may not be feasible. Further, these days cloud companies are offering FPGA bases acceleration as a service. A large cluster of custom accelerators supports these services at the backend. In contrast to this fixed-function hardware approach, where the DSA gets tied with a specific function, an alternative design approach of overlay accelerators is gaining prominence. Overlays are DSAs resembling a processor, which is synthesized and flashed on the FPGA once but is flexible enough to process a broad class of computations soft reconfiguration. Over the last couple of years, a few design approaches have developed for overlay accelerators. Overlay designs exist that resemble a processor controlled through an instruction set; the Xilinx Vitis-AI Engine is a prime example of such a design. However, such a homogeneous approach often leads to inefficiencies arising out of the fetch-decodeexecute model of processor design. Instruction-based overlays spend significant energy on instruction overhead rather than actual computation. To solve this problem of homogeneous overlay accelerators, a heterogeneous design methodology is proposed. A heterogeneous overlay accelerator contains multiple small homogeneous units.

Abstract

In today�s sedentary lifestyle, a person spends a substantial amount of time in a sitting position. Having a poor sitting posture can put more stress on specific muscles and joints, forcing them to be overworked and causing them to fatigue, which results in back pains. �Health is Wealth� and when we are talking about a healthy body, posture is as important as eating in the right way and regular exercising. Hence, it is very important for us to sit in the correct posture. Incorrect sitting posture leads to widespread chronic back pain and other health-related issues, particularly in young adults. We developed a flexible pressure-sensor-array-based smart chair that analyses sitting posture using machine learning algorithms. This is an add-on feature that can be installed on any existing chair or can be integrated into chairs by manufacturers. The solution is completely flexible, requires very low power, and is low-cost. The smart chair works by mapping the body pressure at the seat and the backrest. Machine learning algorithms are used for the training and classification of different postures for posture recognition. To fabricate a flexible pressure sensor, we presented the synthesis of an organic polymer-polypyrrole used as a piezoresistive conductive material that was synthesized using in-situ chemical oxidative liquid polymerization. The sensor showed high sensitivity at low-pressure ranges and can measure pressure in the range of 160 Pa to 16 kPa. We then presented the design of a smaller sensor array mat (4 � 4) using a carbon-impregnated conductive polymer named velostat, and performed various tests to ensure that it could be effectively used for posture recognition applications. We presented the mechanical reliability of a velostat-based pressure sensor. We reported the bending response by examining its reliability when subjected to repeated mechanical stress for 150 bending cycles. We presented for the first time ever, the long-term reliability of velostat by testing it for 210 days. From the results, we observed that the particles of the velostat settle after a particular amount of time on repeated load applications. Once that happens, the change in the resistance of the velostat becomes practically invariant with time. We have observed that the decay ratio is closer to 1 after 210 days. This implies that we can expect reliable and repeatable results from the velostat sensor after the application of load 15 times with a load range from 1 to 12 kg. The relative error is also drastically reduced, and there is an overall error reduction by 53 percentage points in 210 days. A novel data acquisition circuit design with flexibility to read out both capacitive and resistive types of sensors has been presented. It requires less area and is more cost-effective compared to separate single-sensor read-out circuits. We finally present the design of a smart chair system. We tested different machine learning models for the classification of seven different postures and achieved the best accuracy of 95.89% using the Support Vector Machine (SVM) model and 95.61% using the Neural Network (NN). Though the training time for NN is longer compared to SVM, the prediction speed is almost double that of SVM.

Abstract

Speaker recognition (SR) involves automatic identification of individual speakers based on their voices, often representing acoustic traits as fixed-dimensional vectors through speaker embedding. A standard speaker recognition system (SRS) consists of three key phases: training, enrollment, and recognition. In each stage, acoustic features are extracted from raw speech signals using an acoustic feature extraction module, resulting in the acquisition of essential acoustic characteristics. Commonly used acoustic features include speech spectrogram, filter bank, and Mel-frequency cepstral coefficients. During the training stage, a background model is trained to establish a mapping from training voices to embeddings. The traditional background model employs a Gaussian Mixture Model (GMM) to generate identity-vector (ivector) embeddings. In contrast, more recent and promising background models leverage deep neural networks (DNNs) to generate deep embeddings, like xvector. In the enrollment stage, a voice spoken by an individual undergoing enrollment is mapped to an enrollment embedding using the previously trained background model. In the recognition stage, the process begins by retrieving the testing embedding of a given voice from the background model. Subsequently, the scoring module is engaged to measure the similarity between the enrollment and testing embeddings. The scoring module evaluates the similarity between the speaker and recorded embedding. Following the assessment, the scoring and decision module makes a decision based on the similarity score. A decision threshold is established, which serves as a criterion to determine whether the claimed identity of the speaker is accepted or rejected. The concept of voiceprint is rapidly gaining prominence as one of the emerging biometrics, primarily owing to its seamless integration with natural and human-centered Voice User Interface (VUI). The fast progress of Speaker Recognition Systems (SRSs) is intricately linked to the evolution of Neural Networks (NNs), with a particular emphasis on Deep Neural Networks (DNNs). With strides made in deep learning, Speaker Recognition (SR) has also benefitted and found extensive applications across hardware and software platforms. However, it has been shown that NNs are vulnerable to adversarial attacks, highlighting a challenge that needs to be addressed. Thus, even though users have the convenience of authentication with Speaker Recognition services, it has become evident that these solutions are vulnerable to adversarial attacks. This vulnerability highlights that Speaker Recognition (SR) is encountering security threats, raising significant concerns about user privacy. Adversarial attack was initially implemented with images, where an image classification model was successfully deceived using adversarial examples. Drawing inspiration from the progress made in adversarial attacks within the image domain, there is a growing interest in extending these techniques to the audio field. With emerging trends, convolutional neural networks have demonstrated instability to artificially crafted perturbations that remain undetectable to the human eye. Virtually every type of model, ranging from CNN to graphical neural network (GNN), has shown vulnerability to adversarial examples, particularly in the domain of image classification. Deep learning models typically get audio input by converting the audio into a spectrogram for further processing. A spectrogram serves as a condensed representation of an audio input. Given its image-like nature, the audio spectrogram is frequently used as input data for deep learning models, especially Convolutional Neural Networks (CNNs) adapted for audio tasks. CNN-based architectures were initially designed for image processing. This thesis contributes to the assessment of Convolutional Neural Networks (CNNs) for their resilience against adversarial attacks, a domain that is yet to be extensively investigated concerning endto-end trained CNNs for speaker recognition. This examination is essential for sustaining the integrity and security of speaker recognition systems. Our study fills this gap by exploring the variations of iterative Fast Gradient Sign Method (FGSM) to carry out adversarial attacks. We note that using a vanilla iterative FGSM technique can alter the identity of each speaker sample to any other speaker within the LibriSpeech dataset. Additionally, we introduce adversarial attacks specific to Mel spectrogram features by (a) constraining the number of manipulated pixels, (b) confining alterations to certain frequency bands, (c) limiting changes to particular time segments, and (d) employing a substitute model to generate the adversarial sample. Through comprehensive qualitative and quantitative analyses, we illustrate the vulnerability and counterintuitive behavior of existing CNN-based speaker recognition systems, wherein the predicted speak

Abstract

Imagine a robot navigating a complex and unfamiliar environment - underwater, subterranean, or even a dilapidated building. Current Visual Place Recognition (VPR) techniques often struggle with such diverse scenarios, requiring re-training for each new environment. This limitation hinders the development of truly autonomous robots. This work introduces AnyLoc, a novel VPR system taking a significant step towards universality. AnyLoc leverages feature representations learned by powerful foundation models, eliminating the need for VPR-specific training. Furthermore, by combining these features with unsupervised aggregation techniques, AnyLoc can uncover unique visual characteristics that define different environments. Our experiments demonstrate AnyLoc�s potential to function across multiple environments (outdoor, indoor, aerial, underwater, subterranean, and dilapidated) without retraining, laying the groundwork for VPR solutions that could be deployed anywhere, anytime, and across anyview.

Abstract

Localization and tracking play a critical role in various fields, ranging from wireless communications and robotics to surveillance and autonomous vehicles. In the context of signal processing, localization typically involves measuring signals received from multiple sensors to estimate the source�s position. Tracking, on the other hand, extends localization to a temporal context. In this thesis, we explore the data-driven approaches to enhance the direction of arrival (DOA) trajectory estimates. DOA estimation methods involve processing multi-channel sensor array data to determine the directions from which signals originate. Classical methods have two key limitations: first, they rely on analytical properties of observed signals which do not generalize well to non-ideal conditions; second they employ block-level processing, assuming static DOA within each block. In this thesis, the first challenge is resolved with a data-driven approachs, while the second is addressed by operating in DOA-trajectory space instead of DOA space. So, instead of estimating individual DOA, the approach focuses on estimating DOA trajectories. We proposed two data-driven approaches: one grid-based and the other grid-free. In the grid-based approach, we develop a fully convolutional neural network (FCNN) inspired by the computer vison techniques of image translation. The FCNN processes the 2D low resolution spectrum as input and outputs a refined 2D spectrum. The input spectrum typically has broad peaks, while the transformed spectrum has sharp peaks, which allows precise identification of DOA trajectory parameters. In another study, we develop a data-driven approach that is both grid-free and re-useable. This addresses two issues: first, the limitation of parameter estimation on predefined grids, which affects resolution; and second, the problem of estimating all sources at once, which can make traditional methods dependent on the number of sources. A deep complex network is proposed that directly processes the complex sensor array data, with outputs comprising of complex signal amplitude (per snapshot) and trajectory parameters for a single source. We obtain a residual by removing contribution of the identified source from the input data and this residual is again fed back into the network to identify the next source making it re-useble and independent of the number of sources to be estimated. These proposed methods are rigorously evaluated through extensive experiments and analysis, demonstrating their advantages over existing approaches. These findings have the potential to impact a variety of applications, with room for further improvement.

Abstract

This thesis explores the critical challenges and provides solutions associated with automatic spoken data validation in the complex multilingual and multicultural context of India, which is crucial for developing efficient human-computer interaction (HCI) systems such as automatic speech recognition (ASR) and Text-to-speech synthesis (TTS). The diversity in linguistic backgrounds and the prevalence of non-native language speakers create unique challenges in speech communication. These challenges are exacerbated by the frequent mismatches between recorded speech and its reference text, referred to as misspoken utterances. To tackle some of these challenges, this work introduces novel unsupervised techniques for detecting spoken content mismatches. The developed methods leverage state-of-the-art self-supervised speech representation models such as Wav2Vec-2.0 and HuBERT, integrating them with Dynamic Time Warping (DTW) as well as its variants such as Phone level cost maximised DTW approach (Ph-DTW), and Phone level cost maximised weighted DTW approach (Ph-WDTW) along with cross-attention mechanisms. This work develops and tests the techniques on specially curated datasets such as IIITH MM2 Speech-Text and Indic TIMIT, which include a wide variety of phonetic and linguistic features reflective of India�s language diversity. The methodologies proposed are rigorously evaluated for their effectiveness in improving the accuracy and efficiency of spoken data validation in an unsupervised manner. The results demonstrate significant advancements in the automatic detection of mismatches, thereby enhancing the reliability of speech data for training sophisticated HCI systems. By reducing the reliance on labour-intensive manual validation processes, these approaches significantly contribute to the scalability of speech data processing. Overall, this thesis not only addresses a significant gap in the technological handling of spoken data validation but also sets a foundation for future research and development in speech technology applications within diverse linguistic landscapes. The implications of this work are broad, offering potential improvements in various data-intensive speech applications such as ASR, TTS, and Computer-aided language learning systems (CALL) to name a few. This would be achieved by ensuring a readily accessible clean training, testing, and validation set for the development of target models for the aforementioned use-cases, thus addressing the reliable data scarcity to a great extent