Abstract

In recent years, the Internet of Drones (IoD) consisting of Unmanned Aerial Vehicles (UAVs), also called drones, achieves a great momentum due to its high mobility to difficult-to-access places with minimum intervention. The drones are remotely piloted aircrafts, which are widely installed from military mission to civilian applications. For various IoD applications, the drones communicate over public (insecure) channels under the wireless sensor networks (WSN) where the drones are usually deployed in various crucial applications and terrains. Various security threats like replay, man-in-the-middle, impersonation, privileged-insider, physical drones capture attacks, etc. exist in an IoD environment. Therefore, it becomes a challenging job to design security mechanisms for providing the authenticity of transmitted information during communication over the public channels. An access control mechanism is a security mechanism that controls who or what can see, use or access the assets in the IoD system to ensure security/access control for the private data. The IoD applications produce a huge volume of data that is mainly confidential and it needs to be stored securely. Once the data is gathered by the UAVs, the data can be stored securely in the distributed servers (ledgers), such as blockchain, due to a single server failure issue in a traditional centralized storage platform. Once the data is stored in a blockchain, the data cannot be modified, deleted or altered by a malicious entity. Motivated by these issues, in this thesis, the blockchain-based access control schemes have been designed in order to provide strong security in IoD environment to store and access the information for the UAVs applications. The first study presents a new blockchain-based access control scheme in an Internet of Things (IoT)-enabled IoD deployment. In this scheme, several drones are deployed in different flying zones where the drones residing in each zone can securely communicate with each other in order to exchange crucial information. Next, the information is securely collected by their respective Ground Station Server (GSS) of the drones. Secure data gathered by the GSS form transactions, and the transactions are then made into the blocks. The blocks are finally added in the blockchain by the cloud servers which form a peer-to-peer (P2P) cloud servers network via the voting-based “Ripple Protocol Consensus Algorithm (RPCA)”. We provide all sorts of security analysis including formal security under the random oracle model, informal security and simulation-based formal security verification using the widely recognized “Automated Validation of Internet Security Protocols and Applications (AVISPA)” tool to assure that the proposed scheme can resist various potential attacks with high probability needed for an IoD environment. A meticulous comparative analysis among the proposed scheme and other closely related existing schemes shows that our scheme offers more functionality attributes and better security, and also low communication and computation costs as compared to other schemes. In addition, a real testbed experiment has been also demonstrated to show the feasibility study of the proposed scheme for the access control part. In the second study, we propose a novel access control scheme for unauthorized UAV detection and mitigation in an IoD environment, called ACSUD-IoD. With the help of the blockchain-based solution incorporated in ACSUD-IoD, the transactional data having both the normal secure data from a drone (UAV) to the GSS and the abnormal (suspected) data for detection of unauthorized UAVs by the GSS are stored in private blockchain which are considered as authentic and genuine. As a result, the Big data analytics can be performed on the authenticated transactional data stored into the blockchain. Through the detailed security analysis including formal security under the broadly-accepted Real-Or-Random (ROR) model, formal security verification using the widely-applied AVISPA tool and non-mathematical security analysis, we show the robustness of the proposed scheme against a number of potential attacks needed in an IoD environment. The testbed experiments for various cryptographic primitives using the broadly-accepted “Multi-precision Integer and Rational Arithmetic Cryptographic Library (MIRACL)” have been performed under both server and Raspberry PI 3 configurations. Furthermore, a detailed comparative analysis and blockchain-based simulation study have been conducted to show the effectiveness of the proposed scheme. Finally, in the last but not least study, we focus on designing a new blockchain-envisioned secure data delivery and collection scheme for the 5th generation mobile network (5G)-based IoT-enabled IoD environment which relies on the elliptic curve cryptography (ECC). This scheme tackles efficiently the security and privacy challenges during commun

Abstract

Language identification (LID) refers to the task of recognizing the language (e.g., Hindi,English, Telugu, etc.) from the spoken utterance. In a multilingual country like India, automatic LID systems play a key role in developing speech applications. Indian languages majorly belong to two language families i.e., Indo-Aryan and Dravidian. The languages from these families share a common phonetic space, which makes language identification a challenging task in the Indian scenarios. Extracting language discriminative information from speech plays a vital role in developing LID systems. Lack of large scale datasets covering various diversities such as speakers and dialects adds additional level of complexity of the task. Large scale datasets are a pre-requisite for effectively adapting the recent advancements in deep neural networks (DNNs) for developing the LID systems. Lack of this data poses an additional challenge for developing LID systems in Indian scenarios. The goal of this thesis is to improve the performance of Indian LID systems. As an initial step we have collected a speech database comprising data of 23 Indian languages. The database has 103.5 hours of speech data comprising of 1150 speakers covering multiple dialects. In this thesis, we have explored various feature extraction techniques and modelling approaches for developing LID systems. Modeling the long-temporal context is a key for developing LID systems. We have explored the use of stacked shifted delta cepstral ( Stacked-SDC ) features for developing LID systems. These features capture the long-term temporal context to improve the performance of LID systems. To effectively capture this information, a feature that can model the long-term temporal context is required. This study aims to capture the long-term temporal context by appending stacked shifted delta cepstral ( Stacked-SDC ) features. We also explore DNN with attention model for developing LID systems. These models have the sequence-summarizing capability, which helps to better model the long-temporal context. A multi-head attention mechanism is explored to develop LID systems, as having multiple heads improves the capacity of the models. The neural hidden representations derived from a joint acoustic model (JAM) of a multilingual end-to-end automatic speech recognition system are used as features for developing LID system. These features represent contextual information learned by the acoustic model. In this work, an LSTM-CTC ( Connectionist Temporal Classification ) framework is used for training the JAM models. Recently Self-supervised pre-training (wav2vec2.0) has been used to train a neural network which can generate representations with discriminative properties. We explore the use of features from the pre-trained network for developing LID system. The performance of LID systems is at its best when there is a match between training and operating environments. A large degradation is observed when there is a mismatch in the quality of the data in the presence of noise. To overcome the mismatch problem due to noisy operating environments, curriculum learning strategies are explored for training LID systems. These training methods have led to more stable LID systems that can be operated in noisy environments

Abstract

Digital (discretized and quantized) samples are used to represent speech signals in a computer. Beyond recording and playback, these representations as such have limited use. Most speech processing applications involve extraction of relevant information from the speech signals. For example, speech recognition requires extraction of parameters or features that describe the different sound units uniquely. In speaker recognition, it is necessary to extract the speaker-specific features to discriminate one speaker from another. In emotion recognition, the emotion-specific characteristics need to be identified and extracted. In speech compression, the features of speech production need to be captured, so that the speech can be effectively and efficiently used for transmission and storage. This thesis focuses on extraction and representation of speech production features, not only for understanding the link between the speech signal and the dynamic vocal tract system, but also to examine the possibility of compact representation for transmission and storage. The first attempt is to produce a spectrographic display that shows different types of segments with good temporal resolution, and also changes in the formant contours reflecting the rapid movement of the articulators of the dynamic vocal tract system. Methods are proposed to extract formants and their bandwidths. Glottal source processing methods are developed to obtain the characteristics of the glottal vibration within each cycle. While these source and system features are useful in understanding the production process, they are not adequate to reproduce all the important characteristics of speech, as evidenced by the poor quality of the formant vocoder developed using the extracted features. For a compact representation of the speech signal for reproduction, the highly successful linear prediction (LP) based source-system model is used. Even approximate separation of the source and system components helps in effectively compressing the speech information using artificial neural network (ANN) models for these components. The compressed parameters are also useful for other applications such as speech recognition, speaker recognition and speech enhancement

Abstract

Numerical simulations of condensed matter systems, for example biomolecular systems, are notorious for slow dynamics and consequent incomplete sampling lead to incorrect inferences on system thermodynamics and kinetics. In this study, we propose a protocol for efficient exploration of relevant phase space using machine learning techniques. The suggested protocol explicitly performs machine learning in two concurrent parallel stages: (i) map different metastable states into different classes by a classification model with assumption that large jumps that result in anomalous data (to the current class) are indicative of transitions between metastable states and (ii) construct a non-linear map within individual free energy minima (here, a class) by use of regression model that can learn local information. We apply the protocol on alanine tetrapeptide and show that a significant increase in sampling in all relevant phase space of the system; specifically, in comparison to the benchmark study using metadynamics using a well designed collective variable. We report sampling of all highly probable metastable states, and a significant increase in the sampling of relevant low probability metastable states, along with the concomitant increase in the sampling of the transitions between all such metastable states. The protocol proposed in the current study will be useful in construction of complete metastable states connectivity graph, thus enabling computation of thermodynamic and kinetic process of systems with very large degrees of freedom.

Abstract

Centrality measures on graphs have applications in many domains, including modeling the spread of an infection/disease, social network analysis, and transportation networks. As a result, parallel algorithms for computing various centrality metrics on graphs are gaining significant research attention in recent years. Also, since the networks of importance are extremely large, and the exact computation of the metric requires super-quadratic time, finding efficient algorithms to approximate the measure is essential. We study parallel algorithms for the percolation centrality (PC) measure, which extends the betweenness centrality measure by incorporating a time-dependent state variable with every node. We present parallel algorithms that compute the source-based and source-destination variants of the percolation centrality values of nodes in a network. Our algorithms extend the algorithm of Brandes and introduce optimizations aimed at exploiting the structural properties of graphs. Experimental studies of our algorithms on an Intel Xeon(R) Silver 4116 CPU indicate that our algorithmic techniques offer a significant speedup. We also present a new randomized approximation algorithm, called Dual-Brandes, for the sourcedestination variant of PC for undirected graphs. Our algorithm is inspired by the source-sampling technique developed by Brandes and Pich [5]. We show that the estimates produced by our algorithm are equal to PC in expectation. We also experimentally evaluate our algorithm against baselines based on existing ideas on a collection of 12 real-world graphs. Our algorithm outperforms baselines in accuracy and other metrics. On a fixed sample size of 10% nodes, our algorithm achieves a mean squared error 61% lower than the nearest baseline

Abstract

The weather observation and forecasting systems have become vital to every country for improving the efficiency of several systems that deal with life and property. Over the years, the weather information and forewarning systems are gradually becoming powerful, domain-specific, and location-specific. Weather-based decision support systems (DSSs) are being built to improve the efficiency of healthcare delivery, agricultural production systems, transportation sector, governance systems, and so on. In weather based DSS, given a weather situation, the domain experts prepare the appropriate suggestions to improve the efficiency of the stakeholders for carrying out day-to-day operations. Once the domain experts prepare a suggestion for a given weather situation, there is a scope to reuse the same suggestion for the similar weather situations in the future. As a result, the performance of the weather-based DSS could be improved. Notably, the notion of reuse is widely applied to improve the performance of DSSs in multiple domains. For example, the notion of reuse is widely applied to reduce the software development cost in the software domain. Notably, in the literature, the concept of reuse is not being explored to improve the performance of weather-based DSS. In this thesis, we introduced the problem context and proposed two frameworks to improve reuse in weather-based DSSs. As a part of the problem context, we defined two notions, viz. weather condition (WC) and coupled WC, to capture two types of weather situations. Moreover, we have defined the notions of cycle, period, temporal reuse, spatial reuse and coverage percentage metric. As the first approach, we proposed a framework to improve reuse by proposing the notion of Categorybased WC (CWC). The basic idea is that it is possible to improve the performance of weather-based DSS by exploiting the weather-based categories of the given domain. By considering the weather categories provided by India Meteorological Department (IMD), we have analyzed the extent of temporal reuse among the daily and five-day CWCs by conducting extensive experiments on 30 years of weather data collected at Rajendranagar, Hyderabad, Telangana, India. By varying the number of weather variables in CWC from one to five, we have computed the extent of temporal reuse among one-day and five-day CWCs for the following period types: year, season, and phenophases (i.e., growth stages) of the Rice crop. The results show that it is possible to improve reuse significantly with the proposed framework. The results also show that by preparing agro advisories for the first two years, there is a scope to achieve about 80 percent reuse in the third year for all period types. We have also conducted experiments to analyze both temporal reuse and spatial reuse among the CWCs by considering weather data of 12 locations (blocks) in the Telangana state. The results show that

Abstract

The number of people consuming social media and the amount of time they dedicate to it increases every day. With it, cases of mental illness and suicide risk are escalating on social media platforms. The lack of mental health detection mechanisms capable of proactively reaching out to people in need is becoming increasingly evident. The space of automatic mental health detection is ripe with potential, as previous work on mental health has predominantly been theoretical and on a small scale. Work done to detect mental illness on a more expansive scale is few and far between. However, it is riddled with many practical and ethical challenges. We must consider the degree or severity of a case of mental illness in order to estimate whether a person requires intervention. Identifying these intervention points can help us contact the user and provide them with guidance and resources before their condition escalates to suicidal ideation. However, research efforts are limited due to the social stigma and discrimination around accessing mental health care. This has often kept users from disclosing their personal problems not only to public social media forums but to mental health professionals as well. In addition to this, there are other barriers to accessing the appropriate help when necessary, such as financial concerns, apprehensions about privacy and confidentiality, and long waiting periods to see a mental health professional. Anonymous social media platforms such as mental health blogs or Reddit forums have therefore become increasingly popular as they can share their personal stories without judgment. They also generate a shared sense of community as they no longer have to suffer alone. People who face similar issues can share their experiences, give advice, motivate and persuade them to seek counsel from professionals. Therefore, social media has become a valuable source of linguistic cues to help us identify mental health problems from textual data. As researchers, it is imperative that we maintain user privacy and do not disclose personal identities or any other information associated with users. Therefore, anonymous subreddits and mental health forums make for excellent data sources. We additionally preprocess author usernames, as they could contain sensitive information related to the names or locations of the user. Our work takes steps towards automated identification, classification, and analysis of mental health on large-scale social media platforms. Research has indicated that approximately two-thirds of people who die by suicide were dealing with depression or other mental illnesses at the time of death. Hence, we first turn our efforts toward the identification of depression. Our novel architecture considers various issues that arise from dealing with large social media datasets. A majority of these challenges stem from the inherent size and composition of the internet. Social media is quite vast, and the amount of content on depression is nearly negligible when comparing the number and variety of topics being discussed on these platforms. This can often lead to skewed results and can be incredibly damaging to our purpose due to the sensitivity and severity of our problem. We attempt to simulate and consequently adapt our system to these conditions. We also strive to decrease the load on time and computational resources. The resulting system executes in a quicker, more efficient manner, making it suitable for deployment. We also experimented with different deep learning methods to analyze and categorize mental health issues from social media forums. While our user-level depression detection system is at its crux, a binary classification problem, we now focus on tackling the challenges that come when dealing with including multiple labels or categories of mental health issues. The dataset used contains information pertaining to 15 specific mental health support groups collected from related subreddits. We explore different approaches and class imbalance techniques to develop more effective learning models and counteract the bias in any given dataset. While our work as computational linguists usually revolves around finding linguistic intricacies from textual data, we also attempt to evaluate sentiment from pictorial data. We must not assume that social media users would limit themselves to sharing their mental health issues on predominantly textual platforms such as Twitter and Reddit. With the rise of meme culture and the popularity of sites such as Instagram, pictorial formats are being utilized to a greater extent. We take our knowledge of textual deep learning methods to experiment with incorporating this mode of data, which includes a short amount of text in their images, as seen in memes. This would allow us to detect positive or negative sentiments from these images and is a step towards detecting mental health sufferers on multi-modal social med

Abstract

Depression has always been an exciting topic of research due to its high prevalence and complex underlying mechanisms. Cognitive impairment, alterations in mood, emotion dysregulation, aversion to activity are just a few of the consequences of depression. Despite there being an ocean of literature present there, attempting to understand depression, we have just scratched the surface. There is still a lot to explore about the intricacies of this perplexing subject. This thesis is yet another attempt to understand and evaluate how depression changes how we understand and represent information (or concepts). In this work, our primary focus is on emotional concepts. In a nutshell, we are trying to look for answers to the question, ’Whether or not there exist any differences in the way people exhibiting moderate to severe depressive symptoms encode emotional concepts, compared to people with no or minimal depressive symptoms?’. To address this question, we first conducted a pilot study involving 48 participants. A corpus of 50 emotional words of varying hedonic and arousing content was given to the participants in this study. They were instructed to rate their affective experience of valence and arousal for each word on a 9-point self-assessment manikin (SAM) scale. This study aimed to record the Indian affective experience of emotional words and construct a comprehensive affective word database for the following main study. Fifteen words were selected from the corpus, which according to the participants’ responses, were capable of accurately capturing the required affect and arousal. These 15 words were clustered on the arousal-valence bi-dimensional model into five groups (3 belonging to each) and used as stimuli for the second study. An additional objective of the study was to compare and contrast the affective responses of the Indian population with the data collected by other studies on the western people to see if there existed any cultural differences between them. Following the pilot study, we conducted a detailed psychological study involving 107 participants and instructed them to generate properties for 15 emotional words (from the pilot study) presented to them as stimuli. Using state-of-the-art computational models, we evaluated the data collected to efficaciously shed some light on the linguistic, semantic, and sentiment differences in the properties generated by the two groups of samples (participants exhibiting no or mild symptoms and participants showing moderate to severe symptoms). These two groups were created after assessing the predisposition emotional state of the entire sample individually by using Beck Depression Inventory-II (BDI). We found an overall increased negative sentiment and decreased positive sentiment in the properties generated by the sample showing depressive symptoms. Emotion analysis of the data reflected that the sample with

Abstract

Augmented Reality and Virtual Reality(AR/VR) applications can become far more widespread if they can photo-realistically capture our surroundings and modify them in different ways. It could include editing the scene’s lighting, changing the objects’ material, or augmenting virtual objects onto the scene. There has been a significant amount of work done in this domain. However, most of these works capture the data in a controlled setting consisting of expensive setups such as light stages. These methods are impractical and cannot scale. Thus, we must design solutions that capture scenes casually from offthe-shelf devices commonly available to the public. Further, the user should be able to interact with the captured scenes and modify these scenes in exciting directions, such as editing the material or augmenting new objects into the scene. In this thesis, we study how we can produce novel views of a casually captured scene and modify them in interesting ways. First, we present a neural rendering framework for simultaneous novel view synthesis and appearance editing of a casually captured scene using off-the-shelf smartphone cameras under known illumination. Existing approaches cannot perform novel view synthesis and edit the materials of the scene objects. We propose a method to explicitly disentangle appearance from lighting while estimating radiance and learn an independent lighting estimation of the scene. This allows us to generalize arbitrary changes in the scene’s materials while performing novel view synthesis. We demonstrate our results on synthetic and real scenes. Next, we present PanoHDR-NeRF, a neural representation of an indoor scene’s high dynamic range (HDR) radiance field that can be captured casually without elaborate setups or complex capture protocols. First, a user captures a low dynamic range (LDR) omnidirectional video of the scene by freely waving an off-the-shelf camera around the scene. Then, an LDR2HDR network converts the captured LDR frames to HDR, which are subsequently used to train a modified NeRF++ model. The resulting PanoHDR-NeRF representation can synthesize full HDR panoramas from any location in the scene. We also show that the HDR images produced by PanoHDR-NeRF can synthesize correct lighting effects, enabling the augmentation of indoor scenes with synthetic objects that are lit correctly. Through these works, we demonstrate how we can casually capture scenes for AR/VR applications that the user can further edit.

Abstract

Human action recognition, with its irrefutable and varied use cases across fields of surveillance, robotics, human object interaction analysis and many more, has gained critical importance and attention in the field of compute vision. Traditionally entirely based on RGB sequences, action recognition domain has shifted focus towards using skeleton sequences due to the easy availability of skeleton data capturing apparatus and the release of large scale datasets, in recent years. Skeleton based human action recognition, having superiority in terms of privacy, robustness and computational efficiency over traditional RGB based action recognition, is the primary focus of this thesis. Ever since the release of large scale skeleton action datasets namely NTURGB+D and NTURGB+D 120, the community has solely focused on developing complex approaches, ranging from CNNs to complex GCNs and more recently transformers, to achieve the best classification accuracy for these datasets. However, in this rat race for state of the art performance, the community turned a blind eye to a major drawback at the data level which bottlenecks even the most sophisticated approaches. This drawback is where we start our explorations in this thesis. The pose tree provided in the NTURGB+D datasets contains only 25 joints, out of which only 6 joints (3 for each hand) are finger joints. This is a major drawback since only 3 finger level joints are not sufficient enough to distinguish between action categories such as ”Thumbs up” and ”Thumbs down” or ”Make ok sign” and ”Make victory sign”. To specifically address this bottleneck, we introduce two new pose based human action datasets - NTU60-X and NTU120-X. Our datasets extend the largest existing action recognition dataset, NTU-RGBD. In addition to the 25 body joints for each skeleton as in NTURGBD, NTU60-X and NTU120-X dataset include finger and facial joints, enabling a richer skeleton representation. We appropriately modify the state of the art approaches to enable training using the introduced datasets. Our results demonstrate the effectiveness of these NTU-X datasets in overcoming the aforementioned bottleneck and improving the state of the art performance, overall and on previously worst performing action categories. Pose-based action recognition is predominantly tackled by approaches that treat the input skeleton in a monolithic fashion, i.e. joints in the pose tree are processed as a whole. However, such approaches ignore the fact that action categories are often characterized by localized action dynamics involving only small subsets of part joint groups involving hands (e.g. ‘Thumbs up’) or legs (e.g. ‘Kicking’). Although part-grouping based approaches exist, each part group is not considered within the global pose frame, causing such methods to fall short. Further, conventional approaches employ independent modality streams (e.g. joint, bone, joint velocity, bone velocity) and train their network multiple times on these streams, which massively increases the number of training parameters. To address these issues, we introduce PSUMNet, a novel approach for scalable and efficient pose-based action recognition. At the representation level, we propose a global frame based part stream approach as opposed to conventional modality based streams. Within each part stream, the associated data from multiple modalities is unified and consumed by the processing pipeline. Experimentally, PSUMNet achieves the state of the art performance on the widely used NTURGB+D 60/120 dataset and dense joint skeleton dataset NTU 60-X/120-X. PSUMNet is highly efficient and outperforms competing methods which use 100%-400% more parameters. PSUMNet also generalizes to the SHREC hand gesture dataset with competitive performance. Overall, PSUMNet’s scalability, performance and efficiency make it an attractive choice for action recognition and for deployment on compute-restricted embedded and edge devices. Finally, we conclude this thesis by exploring new and more challenging frontiers under the umbrella of skeleton action recognition namely ”in the wild” skeleton action recognition and ”non-contextual” skeleton action recognition. We introduce Skeletics-152, a curated and 3D pose dataset derived from the RGB videos included in the larger Kinetics-700 dataset to explore in the wild skeleton action recognition. We further introduce, Skeleton-mimetics, a 3D pose dataset derived from recently introduced non-contextual action dataset-Mimetics. By benchmarking and analysing various approaches on these two new dataset we lay the ground for future exploration in these two challenging problems within skeleton action recognition. Overall in this thesis, we draw attention to prevailing drawbacks in the existing skeleton action datasets and introduce extensions of these datasets to counter their shortcomings. We also introduce a novel, efficient and hi