Abstract

The social isolation required by the pandemic further amplified the internet’s utility as a medium of communication and information for people with digital access. Social media was one of the most popular forms of communication among young people with digital access during the COVID-19 pandemic. Consequently, crucial debates and discussions about the pandemic crisis have also developed on social media platforms, making them a great primary source to study the experiences of specific groups and communities during the pandemic. Furthermore, online news media outlets were an important source of information for people with digital access during the COVID–19 pandemic. This study consisted of research on social media and online news websites in order to understand the experiences of as well as attitudes towards LGBTQ+ people during the pandemic. We used LDA topic modelling and sentiment analysis on Reddit data to understand the themes and attitudes in circulation within five subreddits devoted to LGBTQ+ experiences and issues. In the process, we attempt to make sense of the role that social media platforms may have played in the lives of LGBTQ+ people who were online during the pandemic (10 March 2020–31 August 2021) and whether this was marked by any continuities or discontinuities from before the pandemic period (January 2019–December 2019). We also examined articles from online editions of two English–language Indian newspaper websites, which differed vastly in their circulation figures—The Times of India and The Indian Express. Focusing on the period spanning March 2020 to August 2021, we analysed articles from our chosen newspaper websites using distil–RoBERTa–base and ChatGPT–3.5 for sentiment analysis and BERTopic for topic modelling. Furthermore, we manually analysed a part of our dataset obtained from Reddit and online newspaper websites to arrive at a nuanced, granular understanding of our quantitative results. The results of our research on Reddit reveal that institutions such as the health care system, justice system, and legislative system perpetuated systemic inequalities against LGBTQ+ communities, thereby adding to the pre–existing stigma against them during a global crisis. Furthermore, the results from our topic model suggest a shift in the discussions of prominent topics in the subreddits from before the pandemic period, with a preponderance of discussions on political events. Similarly, the results from our sentiment analysis show an increase in positive comments and neutral comments and a decrease in negative comments, which does not necessarily indicate a loss of users’ interest in the subreddits. While analysing online newspaper websites, we found a contrast between the quality, quantity, and attitudes prevalent across The Times of India and The Indian Express during the COVID–19 pandemic in the context of LGBTQ+ communities. Our topic modelling results indicate more meaningful coverage by The Indian Express compared to The Times of India, which, however, does not mean sufficient coverage by The Indian Express to provide a comprehensive picture. Similarly, our sentiment analysis results indicate a difference in prevailing sentiment as depicted by the two models. Both newspaper websites exhibited a greater focus on transgender communities. Our research on Reddit has demonstrated what non–cis–heteronormative spaces—virtual and real— can look or feel like. By offering a more rigorous engagement with the data available on social media platforms, this research contributes to current debates in queer and social media studies about the empowering or disempowering nature of social media platforms for LGBTQ+ voices online. Furthermore, it captures the visibility and representation of the LGBTQ+ communities during the pandemic in online Indian news media outlets. By doing so, we contribute to the larger understanding of how Indian media narrated the lived realities of LGBTQ+ communities and what attitudes such representation reveal towards vulnerable communities in India during the pandemic. Overall, this thesis expands the understanding of the kind of discursive spaces and representation offered by online platforms for LGBTQ+ communities during a global crisis, demonstrating both possibilities and potential as well as challenges and problems.

Abstract

Spoken utterances have an inherent structure in the sense that some words seem to group naturally together and some words seem to have a notable break or disjuncture between them. This can be described in terms of prosodic phrasing, meaning that a spoken utterance has a prosodic phrase structure, similar to how a written utterance has a syntactic phrase structure. Phrase breaks in natural speech are important; they are physiologically essential, help emphasize content, and improve the intelligibility of speech. The process of inserting phrase breaks in an utterance is called phrasing. In the context of speech synthesis, phrasing is a crucial step. It breaks long utterances into meaningful units of information and improves the intelligibilty of the synthesized speech. Traditional methods of phrase break prediction have used discrete linguistic resources like part-ofspeech (POS) sequence information for modeling these breaks. These methods cannot be used for languages where the necessary linguistic resources are not readily available, resulting in unsupervised methods of inducing word representations which can be used as surrogates for POS tags in phrase break prediction. However, these methods are not suitable in the context of Indian languages, which are aggutinative in nature resulting in an increase in vocabulary size. This thesis presents the use of word terminal syllables (last syllable of the word) to model phrase breaks. We demonstrate the correlation between these terminal syllables and acoustic breaks found in the speech signal. We utilize these terminal syllables in building models for phrase break prediction from text in six Indian languages and demonstrate by means of objective and subjective measures that these models perform as well as traditional models using POS sequence information. The discrete linguistic representations (like POS tags, induced POS tags, word-terminal syllables) used by traditional methods of phrase break prediction require a hard classification of words into a set of predefined discrete classes, which raises issues when there is ambiguity in the linguistic representation of a word. Moreover, such a representation does not capture the co-occurrence statistics of the words, i. e., they do not take into account the distributional behaviour of words. Both these issues can be addressed by the use of continuous dimensional representations of words also known as word embeddings. This thesis presents a neural network architecture to induce task specific word embeddings for phrase break prediction. We show that our proposed architecture combines feature induction and phrase break prediction in a single framework and thus avoids the two stage process required while using features derived from Latent Semantic Analysis (LSA). We train our model on audiobook data and show that these task specific word features are better word representations as compared to those derived using LSA, for phrase break prediction. With the advent of deep learning, there have been attempts to apply deep neural networks (DNNs) to phrase break prediction. While deep neural networks are able to effectively capture dependencies across features, they lack the ability to capture long-term relations that are spread over time. On the other hand, recurrent neural networks (RNNs) are able to capture long-term temporal relations and thus are better suited for tasks where sequences have to be modeled. This thesis presents the use of RNNs for phrase break prediction, and shows by means of experimental results that they perform better than DNNs. Traditional techniques for learning word embddings learn static or context-free word embeddings, meaning that for each word in the vocabulary, one fixed embedding is learnt for all possible contexts in which that particular word appears in the text corpus. While static embeddings have proved to be effective in NLP applications, they do not take into account the fact that words have different meanings in different contexts. To address this issue, context-sensitive word representations, also called dynamic or contextual word embeddings were developed. The most popular technique for learning dynamic word embeddings is BERT (Bidirectional Encoder Representations from Transformers). This thesis presents work on phrase break prediction using bidirectional encoder representations learnt from fine-tuning a pretrained BERT model, with an additional token classification layer, on phrase break prediction. We show that representations using this model outperform task-specific static word embeddings learnt using a BLSTM token classification model trained from scratch. Finally, this thesis presents work which attempts to answer the following questions: (a) Is there any utility in incorporating an explicit external phrasing model in an End-to-End TTS system? and (b) How do you evaluate the effectiveness of a phrasing model in an End-to-End TTS system?

Abstract

A Completely Automated Public Turing Test to Tell Computers and Humans Apart (CAPTCHA) is one of the primary barriers between notorious bots and legitimate human users. However, advancements in Artificial Intelligence (AI) have enabled malicious bots to circumvent CAPTCHA challenges effectively. As a result, several types of CAPTCHA have been rendered ineffective. In this work, we introduce Swiss Cheese CAPTCHA, a novel sensor-based solution designed to be easily solvable by humans while presenting multiple obstructions for bots (similar to the Swiss Cheese Model) even when the sensor outputs can be predicted and interfered with. We leverage a range of human cognitive abilities and Generic Sensor API in modern devices to provide robust protection against automated attacks by making it more computationally expensive for bots to produce a valid answer within a stipulated time. We conducted two user studies to assess our proposal’s effectiveness: one involving 116 participants to assess the likability and improvise the design, and the other, with 107 participants, to investigate the impact of improvised design changes on cognitive abilities. Our results from these studies show an average completion time of 4.76 seconds and 6.12 seconds, with a success rate of 90.3% and 83.25%, respectively. By analyzing the 2141 resultant trajectories from both user studies, we assess the learnability, error recovery rate, efficiency, and satisfaction of users using the scheme. Finally, we devise an automated attack against our proposal to analyze its security in the real world; we find the probability of attack success is low. We also make our dataset available for further research.

Abstract

This thesis presents a comprehensive exploration of Multi-Object Multi-Part Scene Parsing in 2D images, showcasing significant advancements through two novel approaches, FLOAT and OLAF, each tailored to enhance scene parsing performance and scalability. The first paper introduces FLOAT, a factorized label space framework designed to independently predict object categories and part attributes, thereby simplifying the segmentation task and enhancing scalability. Notably, FLOAT incorporates a unique ’zoom’ refinement technique at inference time, significantly elevating segmentation accuracy, particularly for smaller objects and parts. Empirical results on the Pascal-Part datasets underscore FLOAT’s superior performance, achieving notable improvements in mean Intersection Over Union (mIOU) and segmentation quality IOU (sqIOU), especially on the most comprehensive PascalPart-201 dataset, reflecting its effectiveness in handling diverse and complex scenes. The second paper delves into OLAF, a plug-and-play methodology that augments traditional RGB inputs with object-based structural cues to better capture the complexities of scene structures. This approach leverages a weight adaptation technique, allowing pre-trained RGB models to seamlessly integrate augmented data, thus stabilizing the optimization process. Additionally, the introduction of the LDF encoder module aids in providing low-level dense feature guidance, enhancing the segmentation of smaller parts. OLAF demonstrates its versatility across various architectures and datasets, achieving significant mIOU gains on multiple Pascal-Part benchmarks, highlighting its broad applicability and robust performance enhancements in challenging segmentation scenarios.

Abstract

In Indian urban and rural driving scenarios, small objects are pervasive and often crucial for safe navigation. These objects can include pedestrians crossing roads, children playing near streets, cyclists, stray animals, as well as small vehicles like scooters and motorbikes. Additionally, traffic signs, signal lights, potholes, and road markings (such as lane dividers or zebra crossings) are often small in size but essential for driving decisions. In such contexts, missing or inaccurately segmenting these small objects can lead to critical errors in detection, causing accidents or delays in the vehicle’s decision-making process. Automated understanding of such objects need detection and segmentation to start with. Semantic Segmentation is a critical task in computer vision with a wide range of applications. The objective is to partition an image—a collection of pixels—into distinct labeled regions, each corresponding to specific objects or parts of the scene. This process is crucial for scene understanding and enables the localization of objects within the image. Over time, significant progress has been made in semantic segmentation, especially with the advent of deep learning. The advances in this area have revolutionized computer vision, pushing beyond traditional methods and achieving remarkable improvements in performance. When discussing semantic segmentation, we often focus on datasets, the objects within those datasets, and their corresponding segmentations. While many datasets exist for road scenarios, particularly those representing Western road conditions, there is relatively little research on road conditions specific to India. One notable exception is the Indian Driving Dataset (IDD), a dataset specifically designed for semantic segmentation of Indian road scenarios. Road and driving datasets typically contain objects of varying sizes within each class label. These objects can be broadly categorized into three types: small, medium, and large. The importance of segmentation is well understood across several domains such as medical imaging, autonomous vehicles, aerial imagery, robotics, surveillance, and industrial automation. However, one of the most challenging problems in segmentation is the segmentation of small objects. Small object segmentation is particularly difficult due to factors such as (i) the limited number of pixels representing small objects, (ii) class imbalance during training, and (iii) the inherent challenges posed by small object representations. These factors hinder the performance of deep learning architectures, making it harder for modern techniques to accurately handle small objects. This issue becomes evident when evaluating deep learning models, as performance metrics tend to be significantly worse for small objects. In this study, we delve into the challenges of segmenting small objects and explore potential strategies for improving algorithm design for this task. It is widely acknowledged that loss functions play a key role in segmentation performance, alongside the architecture of deep learning models. Indeed, the segmentation of small objects is highly influenced by the choice of loss function during model training. Various small objects commonly encountered in road scenarios are highlighted in this study, emphasizing their importance in Indian road environments. Objects such as pedestrians, traffic signs, and traffic lights—despite their small size—are crucial for tasks like autonomous navigation and driver assistance systems. The Indian Driving Dataset (IDD) offers a valuable resource, containing a wide range of small objects across different class labels, some of which dominate specific categories. These objects are captured and annotated in real-world, unstructured environments, making this dataset particularly relevant for segmentation tasks in Indian road scenarios. Therefore, our focus in this study is primarily on detection and segmentation of small objects within the IDD dataset and explore the impact of loss functions.

Abstract

Principal component analysis(PCA) is one of the most popular linear dimensionality reduction techniques in machine learning. In this thesis, we present a method for performing PCA on encrypted data using a homomorphic encryption scheme. We used the CKKS homomorphic encryption scheme for our purpose since it is most suitable for machine learning tasks allowing approximate computations on real numbers. In this thesis, we propose a new Homomorphic PCA(HPCA) algorithm that performs PCA. Unlike previously proposed algorithms, our HPCA algorithm is non-interactive in nature. This requires an approximation of the inverse sqrt function. This thesis presents two methods to approximate and securely perform the inverse sqrt function using CKKS homomorphic encryption scheme - Constrained Linear Regression and Pivot-Tangent method. Since the CKKS homomorphic scheme allows only the computation of polynomial functions, we propose a method to approximate the inverse sqrt function polynomially. Ultimately, we implement our approach for the inverse sqrt function. We provide an implementation of our method for the inverse sqrt function. We use the inverse sqrt approximations in our HPCA algorithm to make it non-interactive. In the end, we present the experimental results of our proposed HPCA algorithm on a few datasets computing a few principal components. We measure the R2 score on the reconstructed data and use it as an evaluation metric for our HPCA algorithm.

Abstract

This thesis explores the dual objectives of reviewing material characteristics of soft polymers for microfluidic and Lab-on-a-Chip (LOC) devices and developing a cost-effective, flexible microheater for LOC applications. Soft materials such as polydimethylsiloxane (PDMS), hydrogels, thermoplas- tic elastomers, and conducting polymers are analyzed for their mechanical, thermal, chemical, optical, and biocompatibility properties. These polymers play a crucial role in LOC systems due to their abil- ity to provide flexibility, durability, and compatibility with biological samples. The review delves into their fabrication techniques, solubility in common solvents, adhesion, and surface modification meth- ods, highlighting their relevance for applications such as diagnostics, cell culture, and chemical assays. Experimental studies further assess their optical properties and cytotoxicity, offering a comprehensive understanding of their potential for use in biomedical and chemical systems. By comparing the advan- tages and limitations of these materials, this work aids in optimizing material selection for enhanced performance and sustainability in LOC applications. The developed flexible microheater utilizes nichrome wire embedded in a PDMS substrate, chosen for its affordability and durability. The microheater operates within a temperature range of 40°C to 150°C, with precise control achieved through a pulse-width modulation (PWM)-based circuit and feed- back from a 100k Ω NTC thermistor. This setup provides closed-loop temperature regulation, ensuring stability and adaptability for diverse applications. The microheater’s efficacy is demonstrated in two sig- nificant applications: glucose detection and the synthesis of silver nanoparticles (AgNPs). In the glucose detection application, the microheater integrated into a microwell device reliably facilitates Benedict’s test, confirming its potential for point-of-care diagnostics. The microheater is also employed in the syn- thesis of AgNPs using green chemistry principles, utilizing ascorbic acid as a reducing agent and sodium citrate as a stabilizer. This process yields spherical nanoparticles with UV-Vis absorption peaks charac- teristic of AgNPs (between 405 nm and 411 nm). Both applications underline the microheater’s capacity to deliver consistent performance while maintaining low fabrication costs. Experimental results validate the heater’s capability to reach steady-state temperatures quickly and efficiently. The integration of the PWM-based control system ensures rapid response times and enhanced accuracy, addressing challenges in conventional heating systems. Furthermore, the cost of the microheater device is kept below INR 1000, making it an accessible solution for resource-constrained settings.

Abstract

In the field of artificial intelligence, the generation of human-like motion from natural language descriptions has garnered increasing attention across various research domains. Computer vision focuses on understanding and replicating visual cues for motion, while computer graphics aims to create and edit visually realistic animations. Similarly, multimedia research explores the intersection of data modalities, such as text, motion, and image, to enhance user experiences. Robotics and human-computer interaction are pivotal areas where language-driven motion systems improve the autonomy and responsiveness of machines, facilitating more efficient and meaningful human-robot interactions. Despite its significance, existing approaches still encounter significant difficulties, particularly when generating motions from unseen or novel text descriptions. These models often lack the ability to fully capture intricate, low-level motion nuances that go beyond basic action labels. This limitation arises from the reliance on brief and simplistic textual descriptions, which fail to convey the complex and fine-grained characteristics of human motion, resulting in less diverse and realistic outputs. As a result, the generated motions frequently lack the subtlety and depth required for more dynamic and context-specific applications. This thesis introduces two key contributions to overcome these limitations and advance text-conditioned human motion generation. First, we present Action-GPT, a novel framework aimed at significantly enhancing text-based action generation models by incorporating Large Language Models (LLMs). Traditional motion capture datasets tend to provide action descriptions that are brief and minimalistic, often failing to convey the full range of complexities involved in human movement. Such sparse descriptions limit the ability of models to generate diverse and nuanced motion sequences. Action-GPT leverages LLMs to create richer, more detailed descriptions of actions, capturing finer aspects of movement. By doing so, it improves the alignment between text and motion spaces, enabling models to generate more precise and contextually accurate motion sequences. This framework is designed to work with both stochastic models (e.g., VAE-based) and deterministic models offering flexibility across different types of motion generation architectures. Experimental results demonstrate that Action-GPT not only enhances the quality of synthesized motions—both in terms of realism and diversity—but also excels in zero-shot generation, effectively handling previously unseen text descriptions. Second, we introduce MoRAG, a sophisticated retrieval-augmented generation strategy tailored to enhance the performance of motion diffusion models. MoRAG adopts a multi-part fusion retrieval mechanism that allows for improved generalization of motion retrieval across a wide range of language inputs, addressing the limitations of current retrieval methods that struggle with unseen or atypical descriptions. By incorporating low-level, part-specific motion details into the retrieval process, MoRAG constructs more accurate and varied motion sequences. The retrieval process is refined by prompting LLMs to handle issues like spelling errors, rephrasing, and ambiguous language, ensuring that the retrieved motions are contextually relevant and diverse. These augmented motion samples are then used as additional knowledge within the motion generation pipeline, enhancing the system’s ability to generate complex and realistic motions from diverse textual inputs. This retrieval-augmented approach increases both the robustness and generalization capacity of the motion generation models, making them more adaptable to complex and unseen scenarios. Together, these contributions represent a substantial advancement in text-conditioned human motion generation. By refining both the action description process and the motion retrieval strategy, this work enhances the ability of models to generate diverse, realistic motions from natural language inputs, particularly in zero-shot settings and when handling detailed, complex descriptions.

Abstract

The performance of wireless communication systems is fundamentally constrained by the propagation environment, which is uncertain, volatile, and constantly changing due to factors such as multipath fading, doppler shift, path loss, etc. Legacy solutions such as multiple input multiple output, relays, etc., have circumvented these issues to a certain extent in order to establish reliable communication links. Nonetheless, as communication systems are heading towards a future with data rate requirements in the order of gigabits/second and latencies in the order of microseconds, efficient system design is the need of the hour. Especially, solutions that merely circumvent the ill-effects of wireless propagation environments are no longer sufficient. Towards this, a new technology named Reconfigurable Intelligent Surface (RIS) is emerging as a potential solution that has the capability to alter the propagation environment to a large extent, and hence is envisioned to be a part of next-generation cellular standards. There are many research directions for the implementation of RIS-aided communication systems, such as RIS hardware design, channel modeling, channel estimation, optimal beamforming, capacity and outage analysis, optimal RIS deployment, etc. These directions must be explored in-depth in order to realize the benefits of such RIS-aided communication systems fully. This thesis aims to explore RIS-aided communication systems design and their performance analysis in terms of capacity and outage, with a special emphasis on proposing low-complexity beamforming solutions and their performance analysis. The main contribution of the thesis focuses on developing statistically optimal, low-complexity beamforming and RIS-phase shift matrix solution for a RIS-aided downlink MISO system under different propagation environments: Rician-Rician, Rician-Rayleigh, and Rayleigh-Rayleigh. The study considers both correlated and i.i.d. fading scenarios and provides closed-form expressions for optimal transmit beamformers and RIS phase shifts. These solutions offer significant advantages over instantaneous CSI-based beamforming schemes by eliminating the need for reliable feedback channels. The work also derives the ergodic capacity and outage probability performances in closed-form, demonstrating how the number of RIS elements and the presence of LoS components influence system performance under various fading conditions. Next, the thesis investigates beamforming and outage probability analysis for RIS-aided MISO downlink systems under Rician fading along both the direct and the RIS-assisted indirect links. Toward this, the focus has been on maximizing the capacity for two transmitter architectures: fully digital and fully analog. This capacity maximization problem with optimally configured RIS is shown to be L1 norm-maximization with respect to the transmit beamformer. To obtain the optimal fully digital beamformer, a complex L1-PCA-based algorithm has been proposed that has low computational complexity. Another low-complexity optimal beamforming algorithm has been proposed to obtain the fully analog beamforming solution. Additionally, analytical upper bounds on the SNR achievable by the proposed algorithms are derived and used to determine lower bounds on outage probabilities. The derived bounds are numerically shown to be exact for a unit-rank channel matrix. The final part of the thesis explores the design of a RIS-NOMA-aided multi-user downlink system with a low-complexity transceiver architecture. It proposes a fully analog architecture for the base station (BS), using a single radio frequency (RF) chain, which limits it to single-user transmission. To enable multi-user communication, NOMA is integrated with the system. Moreover, the use of RIS enhances the receive SNR for each user and also makes the propagation environment conducive for NOMA, enhancing successive interference cancellation capability. Towards this, the sum rate and energy efficiency of the proposed system are investigated while ensuring minimum rates for all users. Furthermore, the non-convex optimization sum rate and energy efficiency problems are addressed using a quadratic transform-based approach. The proposed RIS-NOMA-aided fully analog system outperforms fully digital architecture in sum rate at lower SNR and energy efficiency across a wide SNR range, demonstrating that RIS can significantly reduce transceiver complexity while improving performance.

Abstract

Imagine a situation where an interior designer is working on a new project and plans to add a few objects in the home to enhance its look. He made all the purchases and now the objects are not fitting into the designed environment. Should he still use them? Instead he can check the suitability beforehand by placing them in the environment virtually and then buy if he is satisfied. Also, consider a scenario in a game where a player is surrounded by enemies approaching from all directions. The player must strategically place obstacles to hold off the enemies, but has no assistance in this task. Can the player survive under these conditions? Can he perform these actions just by voice control or text instruction? Motivated by these ideas, we devised a method to modify the scene based on text prompt in this thesis. Text driven diffusion models have shown remarkable capabilities in editing images. These edits include substituting the objects in scene, inserting new instances, deleting unwanted things and changing the textures. Execution of these edits is quick in images and does not need much guidance. However, when editing 3D scenes, existing works mostly rely on training a Neural Radiance Field (NeRF) for 3D editing. Recent NeRF editing methods leverage edit operations by deploying 2D diffusion models and project these edits into 3D space. They require strong positional priors alongside the text prompt to identify the edit location. In absence of this guidance, edits are decentralized from target location and are diverse in each view. When the changes are different in multiple views, 3D reconstruction becomes inaccurate, changing the overall aesthetics of the scene. These methods are operational on small 3D scenes and are more generalized to a particular scene. They require training for each specific edit and cannot be exploited in real-time edits as each training cycle requires ample amount of time. These limitations make NeRF approach a gridlock for real-time edits. To address these limitations, we propose a novel method, FreeEdit, to make edits in training free manner using mesh representations as a substitute for NeRF. Training-free methods are now a possibility because of the advances in foundation model’s space. We leverage these models to bring a training-free alternative and introduce solutions for insertion, replacement and deletion. We consider insertion, replacement and deletion as basic blocks for performing intricate edits, which are certain combinations of these operations. Given a text prompt and a 3D scene, our model is capable of identifying what object should be inserted/replaced or deleted and the location where the edit should be performed. We also introduce a novel algorithm as part of FreeEdit to find the optimal location on the grounding object for placement. We evaluate our model by comparing it with baseline models on a wide range of scenes using quantitative and qualitative metrics and showcase the merits of our method with respect to others. We hope this work motivates future research on text driven 3D scene edits using mesh representation. As this is an initial step towards training free approach for scene edits, adding few complex components can enhance the user experience that drives the research here after.