Abstract

The question “what is art” is a forever open inquiry. The study proposes a formal ontological approach to theorizing art. This leads us to question the fascinating density of the realms of imagination and freedom of creation, which can only be accessed through chasing formal structures of the mind and self. Therefore, this inquiry deals with the folding of various categories of existence, in Art and in the specific act of painting. The research explores the Indian and Greco-European traditions of metaphysical thought to arrive at a framework for describing the artistic phenomenon of painting. While we finally commit to the frameworks given in the Kashmiri Saiva darshana and Spanda-shastra therein, the inquiry journeys into a brief history of formal thought that does not limit itself to regional interpretative silos. Therefore, in describing the universal Art phenomenon, we enable conversation across cultures around the same theme. This is a departure from the art-historical mode of theorizing art, while not dismissing the value of the sense of past in guiding such an exploration. We endeavor into observing the workings of the private mind in the work of art, rather than only the existential fact of its material presence. Traditions of the practice of observing mental functions, and strains from diverse formal thought traditions, enable opening the mysteries of the phenomena of art, especially painting. We attempt to clarify the formal states involved in the phenomenon of Painting.

Abstract

Disfluencies in speech are the abrupt breaks or hesitations that disrupt the forward flow of speech. The presence of disfluencies can adversely affect the performance of many speech based applications like Automatic Speech Recognition (ASR) systems. So, in order to make the systems perform effectively, disfluencies need to be identified in the speech signal. This task is referred to as disfluency detection. The works in this thesis focus on detecting multiple disfluencies in spontaneous lecture-mode speech. The IIITH-Indian English Disfluency (IIITH-IED) dataset is used for this purpose. The main focus of this thesis is to incorporate feature-level improvements in the disfluency detection pipeline to effectively model and capture the characteristics of disfluencies directly from speech. In order to improve frame-level disfluency detection, Shifted Delta Cepstral (SDC) coefficients, which have the ability to capture temporal variations in the speech signal effectively, are explored. Further different configurations of SDC features are experimented with for each type of disfluency to check the effect of varying the SDC parameters. Since, disfluencies also depend on the speaking style and can vary from speaker to speaker, speaker characteristics are also included in the disfluency detection systems by giving the d-vectors (speaker embeddings) as input along with cepstral and SDC features. An improvement in the detection accuracy is obtained for all the disfluencies, showing the importance of considering temporal variations and speaker characteristics in detecting disfluencies. In the literature, most works use frame-level or utterance-level analysis for detecting disfluencies. To analyse speech disfluencies at a supra-segmental level, syllable-level disfluency is performed here. Pre-linguistic Automatic syllabification is used to segment the input speech into perceptually distinct syllable-like units. Then, statistical prosody features are used to detect the presence of speech disfluencies in each syllable-like region. In order to add complementary acoustic information into the disfluency detection systems developed, a BiLSTM based feature extractor is used to get an acoustic representation of the syllable-like units. This acoustic representation is concatenated with the prosody features and it is found that the combined features provide an accuracy of 91.24% while distinguishing disfluent and non-disfluent speech syllables.

Abstract

Ancient historical handwritten documents were one of the earliest forms of written media and contributed to the most valuable cultural and natural heritage of many countries globally. They hold early written knowledge about subjects like science, medicine, Buddhist doctrines and astrology. India has the most extensive collection of manuscripts and studies have been conducted on their digitization for passing on their wealth of wisdom to future generations. Targeted annotation systems for automatic multi-region instance segmentation of their document images exist but with relatively inferior quality layout prediction compared to their human-annotated counterparts. Precise boundary annotations of image regions in historical document images are crucial for downstream applications like OCR, which rely on region-class semantics. Some document collections contain densely laid out, highly irregular, and overlapping multi-class region instances with a large range in aspect ratio. Addressing this, a web-based layout annotation and analytics system is proposed in this thesis. The system, called HInDoLA, features an intuitive annotation GUI, a graphical analytics dashboard, and interfaces with machine-learning-based intelligent modules on the backend. HInDoLA has successfully helped us create the first-ever large-scale dataset for layout parsing of Indic palm-leaf manuscripts named Indiscapes. Keeping the non-technical nature of domain experts in mind, the tool offers an interactive and relatively fast annotation process with the help of two modes, namely Fully Automatic mode and Semi-Supervised Intelligent Mode. We then discuss the semi-supervised approach superiority over fully automatic approaches for Historical document annotation. Fully automatic boundary estimation approaches tend to be data-intensive, cannot handle variable-sized images, and produce sub-optimal results for images mentioned above. BoundaryNet, a novel resizing-free approach for high-precision semi-automatic layout annotation, is another main contribution from this thesis. An attention-guided skip network first processes the variablesized user-selected region of interest. The network optimization is guided via Fast Marching distance maps to obtain a good quality initial-boundary estimate and an associated feature representation. These outputs are processed by a Residual Graph Convolution Network optimized using Hausdorff loss to get the final region boundary. A challenging image manuscript dataset demonstrates that BoundaryNet outperforms solid baselines and produces high-quality semantic region boundaries. Qualitatively, our approach generalizes across multiple document image datasets containing different script systems and layouts, all without additional fine-tuning.

Abstract

Tunnels are the most important components of infrastructure, as they play an important role in public transport, water transport, and hydroelectric power generation. The tunnel industry has long believed that tunnels are safe from any seismic activity. Recent rigorous damage to tunnels has proven the, unlike scenario. Several case histories raised the concern for the safety of underground structures. Examples include Chi-Chi Earthquake in Taiwan (1999), Wenchuan Earthquake in Sichuan (2008), Nepal Earthquake (2011, 2015) and Sikkim earthquake (2011). At this point, it will be prudent to understand the seismic performance of rock support systems while designing tunnels. The need for more power plants has led to significant amount of hydroelectric tunnelling work in the Himalayan region. The challenges for tunnelling in Himalayan region are enormous owing to the heterogeneous geology and the high seismic activity. The analysis and understanding of the prevalent rock mass in such areas are a challenge and necessary for the successful execution of tunnelling project. A tunnel construction typically consists of the ground support system and the tunnel lining construction. The effect of earthquake activity is measured only after there is a static design available. The main aim of this thesis is to provide a simplistic design approach as an effective mechanism for engineers to design tunnel support. This study explores the effects of earthquakes on the tunnel lining. The analyses are carried out to understand the effect of rockmass quality ‘Q’, size of tunnels, effect of discontinuities on the seismic performance of tunnel lining. In this thesis an attempt is made to improve the current design approach for the support design of tunnel using Q system and expected PGA values at the tunnel site. For this, case study of three hydropower tunnels from India has been selected: 1) (4111 MW) Vishnugarh Pipalkoti hydropower project (Uttarakhand), 2) 450 MW Shongtong Karcham hydroelectric project (Himachal Pradesh), and 3) 37.5 MW Parnai hydroelectric project (Jammu and Kashmir). Seismic parameters have been evaluated by carrying out probabilistic seismic hazard assessment (PSHA) for these specific sites. This study uses pseudo-static to understand tunnel behavior in static and seismic cases. It is found that axial force increases with rock mass quality Q and tunnel size. Three sedimentary rock mass classes are modelled with different geo-mechanical parameters determined empirically. The study used Phase2 version 8.0 numerical programme, which is frequently used to understand tunnel seismic response. In this study, a tunnel model with diameters of 6, 12, 18, and 24 m was assessed at a depth of 100 m surrounded by rock masses with Q values ranging from 1 to 30 . As a result of such an analysis, researchers and field engineers can account for a continuum's seismic response. A parametric study was conducted to assess the seismic activity of tunnels with discontinuities in a fragmented geological setting with high overburden. Determining the influence of joint orientation, spacing, friction, and stiffness on the dynamic forces developed in a liner requires numerous pseudo-static finite element analyses. The role of joint stiffness and shear strength in the tunnel response was found to be critical. Reduced joint dip angles improved tunnel stability, as observed in the field. For competent rocks with joints, neither seismicity nor joint orientation had an effect on tunneling. However, the weak rock mass deforms and is affected regardless of joint position. The results agreed with the Norwegian Rock Index System (Q) and Bhasin et al. (2006) earlier tests. It is necessary to modify the current Q-system tunnel design chart to use Qseismic instead of Qstatic to achieve the new support system. Barton (1984) advised halving the Q value. However, here it is suggested that Qseismic = Qstatic, where ‘’ is a coefficient mostly reliant on the PGA and the rock mass quality Q, respectively. It is hypothesized that ‘’ depends on other qualities of the rock mass, the size of the tunnel, and the parameters of the shotcrete liner. There is a strong correlation between tunnel deterioration and the Peak ground Acceleration (PGA). Increases in PGA put more strain on the tunnel's liner. There is a correlation developed between ‘’ and PGA for the three case study locations with varying seismic stress. For the optimistic design of the tunnel support system, the proposed research observation is also used to design a key tool for evaluating the support system for Himalayan tunnel projects.

Abstract

Action recognition is indispensable not only to the umbrella field of computer vision but in multitudes of allied fields such as video surveillance, human computer interaction, robotics and human robot interaction. Typically action recognition is performed over RGB videos, however, in recent years skeleton action recognition has also gained a lot of traction. Much of it is owed to the development of frugal motion capture systems, which enabled the curation of large scale skeleton action datasets. In this thesis, we focus on skeleton-based human action recognition. We begin our explorations with skeleton-action recognition in the wild by introducing Skeletics-152, a curated and 3-D pose-annotated subset of RGB videos sourced from Kinetics-700, a large-scale action dataset. We extend our study to include out-of-context actions by introducing Skeleton-Mimetics, a dataset derived from the recently introduced Mimetics dataset. We also introduce Metaphorics, a dataset with caption-style annotated YouTube videos of the popular social game Dumb Charades and interpretative dance performances. We benchmark state-of-the-art models on the NTU-120 dataset and provide multi-layered assessment of the results. The results from benchmarking the top performers of NTU-120 on the newly introduced datasets reveal the challenges and domain gap induced by actions in the wild. Overall, our work characterizes the strengths and limitations of existing approaches and datasets. Via the introduced datasets, our work enables new frontiers for human action recognition. When moving ahead from the traditional supervised recognition to the more challenging zero-shot recognition, the language component of the action name becomes important. Focusing on this, we introduce SynSE, a novel syntactically guided generative approach for Zero-Shot Learning (ZSL). Our end-to-end approach learns progressively refined generative embedding spaces constrained within and across the involved modalities (visual, language). The inter-modal constraints are defined between action sequence embedding and embeddings of Parts of Speech (PoS) tagged words in the corresponding action description. We deploy SynSE for the task of skeleton-based action sequence recognition. Our design choices enable SynSE to generalize compositionally, i.e., recognize sequences whose action descriptions contain words not encountered during training. We also extend our approach to the more challenging Generalized Zero-Shot Learning (GZSL) problem via a confidence-based gating mechanism. We are the first to present zero-shot skeleton action recognition results on the large scale NTU-60 and NTU-120 skeleton action datasets with multiple splits. Our results demonstrate SynSE’s state of the art performance in both ZSL and GZSL settings compared to strong baselines on the NTU-60 and NTU-120 datasets. 3-D virtual avatars are extensively used in gaming, educational animation and physical exercise coaching applications. The development of these visually interactive systems relies heavily of how humans perceive the actions performed by the 3-D avatars. To this end, we perform a short user-study to gain insights into the recognizability of human actions performed virtually. Our results reveal that actions performed by 3-D avatars are significantly easier to recognize as compared to those performed 3-D skeletons. Concrete actions, i.e actions which can only be performed using a fixed set of movements are recognized more quickly and accurately as compared to abstract actions. Overall, in this thesis we study various new-frontiers in skeleton action recognition by means of novel datasets and tasks. We drift from unimodal approaches to a multimodal setup by incorporating language in a unique syntactically aware fashion with hopes of utilizing similar ideas in more challenging problems like skeleton action generation.

Abstract

With a tremendous increase in the user population of the world wide web across the globe, the diversity amongst its users has also increased unpredictably. It is now more than ever, essential to provide accessibility and enhance universal inclusivity. Web accessibility enables equal access to information on the world wide web regardless of disability. It is imperative that we address the problem of web accessibility by leveraging the current trends in the state-of-the-art. In this dissertation, we present a systematic mapping study on web accessibility with an emphasis on the web annotation intervention. The aim of this study is to provide a structure to the body of knowledge and identify key knowledge gaps. We performed a systematic investigation of the publications in the areas of web accessibility in the context of web annotation. As a part of this study, we framed research questions, defined inclusion exclusion criteria and formulated a classification scheme. We then mapped and analysed the obtained set of research publications. From a preliminary set of 2018 articles, we selected a set of 55 for our study. The insights gained from this mapping study can be used to develop tools and formulate strategies for the enhancement of accessibility and overcome existing barriers. In this dissertation we focus on the human users and aid them in better knowledge formation. We suggest that users face accessibility challenges in gathering information, especially when it is in a different representation system than they are used to. We identify this as a semantic gap. To overcome this we propose a client-side, browser based information modification tool called Renarration UI. We focus on both the design and implementation aspects of this tool and validate it by conducting an empirical study (n=10). Results are encouraging and suggest that renarration of web pages has potential to address these information accessibility issues. We also propose another client-side, browser-based information augmentation tool called SeRena. This semantic renarration tool enhances the accessibility of the enduser by facilitating the end-user with relevant additional information. To accomplish this we develop an ontology and populate a Knowledge Base(KB) with instances of the ontology. We then semantically annotate the webpage to provide the end-user with additional information from the KB.

Abstract

Planar region understanding from a single RGB image constitutes two essential components – depth map estimation and surface normal map estimation. In this work, we focus on the latter. We have proposed an approach to predict surface normals for each pixel. While surface normal prediction problem has been extensively studied, most approaches are geared towards indoor scenes and often rely on multiple modalities (depth, multiple views) for accurate estimation of normal maps. Outdoor scenes pose a greater challenge as they exhibit significant lighting variation, often contain occluders, and structures like building facades are often ridden with numerous windows and protrusions. Conventional supervised learning schemes excel in indoor scenes, but do not exhibit competitive performance when trained and deployed in outdoor environments. Furthermore, they involve complex network architectures and require many more trainable parameters. To tackle these challenges, we present an adversarial learning scheme that regularizes the output normal maps from a neural network to appear more realistic, by using a small number of precisely annotated examples. We target surface normal estimation in outdoor scenes – for building facades, using only RGB image input. Our method presents a lightweight, simpler, end-to-end trainable, single view architecture that utilizes skip connection, residual learning and adversarial regularization to generate fairly accurate normal maps of buildings in the city scenes while improving performance by at least 1.5x across most metrics. We provide qualitative and quantitative comparisons of our method against the state-of-the-art methods on normal map estimation on two city scene datasets - one real, and another synthetic, and observe significant performance enhancements. These evaluations demonstrate how even without using other modalities like depth, our single-view method is able to outperform state-of-the-art methods.

Abstract

Personal Assistants like Amazon’s Alexa, Google Home, and Apple Siri, have become a part and parcel of our everyday life. These assistants not only work towards answering user queries but also aim to emulate a human conversation by generating interactive responses for user queries. In this thesis, we look at a two-fold manner of the addition of interactive nature in such open-domain chatbots. We observe that topic diversion occurs frequently with engaging open-domain dialogue systems like virtual assistants. The balance between staying on topic and rectifying the topic drift is important for a good collaborative system. To resolve this we work towards segmenting the conversation in a flat hierarchical manner with novel annotation tags of a major topic, minor topic, and off-topic. Here, we present a model which uses a fine-tuned XLNet-base to classify the utterances pertaining to the major topic of conversation and those which are not, with a precision of 84%. We propose a preliminary study, classifying utterances into their respective annotation tags, which further extends into a system initiative for diversion rectification. This task of classifying utterances into those which belong to the major theme or not would also help us in the identification of relevant sentences for tasks like dialogue summarization and information extraction from conversations. In addition, a case study was conducted where a system initiative is emulated as a response to the user going off-topic, mimicking a common occurrence of the mixed-initiative present in the natural human-human conversation. An important feature of our topic-shift detection system is that it doesn’t o perate with any pre-defined topic sets, and detects the shift in a dynamic manner, as expected in interactions between two humans. In addition, we also look at the requirement of automatic question generation, which works towards increasing the interactive nature and conversational ability of a personal assistant. This thesis presents a system that automatically generates multiple, natural language questions using relative pronouns and relative adverbs from complex English sentences. Our system is syntax-based, runs on dependency parses information of a single-sentence input, and achieves high accuracy in terms of syntactic correctness, semantic adequacy, fluency, and uniqueness. One of the key advantages of our system, in comparison with other rule-based approaches, is that we nearly eliminate the chances of getting a wrong wh-word in the generated question, by fetching the requisite wh-word from the input sentence itself. Depending upon the input, we generate both factoid and descriptive type questions. Our contribution towards the exploitation of wh-pronouns and wh-adverbs to generate questions is novel in the Automatic Question Generation task. We explore in this thesis, the intersection between these two fields, which helps us in not only generating interactive responses but also helping the user by guiding them towards their goal.

Abstract

Leptoquarks (LQs) are hypothetical particles that connect the quark and the lepton sectors of the Standard Model (SM). As a result, the phenomenology of leptoquarks becomes very interesting and has been extensively studied in recent years. However, the majority of the recent attention is mainly due to their ability to explain the long-standing anomalies observed in B-meson decays and the muon anomalous magnetic moment measurements. Generally, the models explaining the anomalies contain LQs that couple with the third-generation quarks and leptons. This gives additional motivation to search for such LQs (commonly known as the third-generation LQs in the literature) at the Large Hadron Collider (LHC). In this thesis, we explore a novel search strategy to efficiently identify third-generation scalar leptoquarks at 14 TeV High Luminosity (HL)-LHC with an integrated luminosity of 3 ab−1 . LQs here decay to a top quark plus a tau lepton (tτ) and carry a charge of either ±1/3 (φ1) or ±5/3 (φ5). Our analysis utilizes a combined signal which consists of events from both single and pair production channels of LQ to improve the discovery prospects. We have divided our search into different benchmark scenarios based on LQ branching fractions and lepton-quark coupling parameters. We make use of several opensource HEP packages to simulate the experiments. A Monte Carlo event generator called Madgraph is used to generate the signal and background processes at the leading order (LO) with higher-order corrections being applied to some of the processes. Parton showering and hadronization is performed using the Pythia package while the detector effects are simulated using Delphes. Once all the processes are generated, we introduce a series of event selection cuts that are optimized to retain events from both pair and single production processes while rejecting a majority of background events. Finally, we try to estimate the discovery reach and exclusion mass limits of scalar LQs projected at the HC-LHC integrated luminosity of 3 ab−1 . Analysing the final result, we conclude that the overall HL-LHC reach goes beyond 1.09 (1.33) TeV for φ1 with BR = 0.5 (1.0) and 1.33 TeV for φ5 with BR = (1.0). In the first chapter, We begin our discussion with a brief introduction to the Standard Model (SM), we then talk about some of its limitations and mention a few beyond the SM (BSM) theories. The next chapter provides an overview of the experimental particle setup at the LHC along with a brief summary of jet reconstruction algorithms and jet grooming techniques. Chapter three mainly talks about the LQ phenomenology and model details while providing a sneak peek of a recent LQ search performed at the CMS. In the fourth chapter, we discuss different aspects of our analysis including the simulation setup and search strategy. Finally, we present the results and conclusions in the fifth chapter.

Abstract

The use of the Internet and online social networks has increased tremendously around the world over the past decade, providing an opportunity to exchange thoughts, ideas, opinions, and feelings with other people. This exponential growth of social media networks has allowed the production, distribution, and consumption of data at a phenomenal rate. However, it has led to various forms of online problems such as offensive content, hate speech, fake news, racism, trolling, etc. Additionally, such issues arise due to the ability to express any content on social media and the possibility of writing anonymous posts or comments. Although every social media platform has community standards and guidelines against online abuse, often these guidelines are indistinct and subjective. So, it is vital to identify and mitigate such social media issues to prevent psychological impact on affected communities and to avert hate crimes. And simultaneously, we need to increase encouraging, positive, and supportive content on social media. This thesis predominantly focuses on developing automated models to detect and classify the (abovementioned) online issues in two distinct modalities. The first one is Text; another one is the combination of Text + Vision (multimodality). In the text modality, we worked on two different types of text, one of which is the monolingual English text and the other is code-mixed text. Code-mixing is a prevalent phenomenon that occurs in text or speech on social media, so we work on social media problems with the monolingual and code-mixed text. Internet memes appear in multimodality (text + vision) on social media networks. Their primary purpose is to communicate ideas and opinions through the use of combinations of text and images, which create a specific state for the receptor, depending on the message the meme is to send. Most of the memes are downright humorous, while others, behind an amusing presentation, seek to convey subtle subtleties, including hatred, fake, sarcasm, propaganda regarding an idea, or a motivational purpose. Therefore, it is necessary to identify and remove such content on social media and make it a safer place for everyone. Hence, this thesis presents the various traditional machine learning models to the latest advanced transformer-based models to detect burgeoning social media problems in text and multimodalities. Coming to language use in social media, we find both monolingual data and code-mixed data. Initially, in monolingual text, we investigate the work of detecting fake news in COVID-19 related social media posts through the adaption of transformers-based contextual word representations. We propose an ensemble model by concatenating the BERT, ALBERT, and XLNet representations. This contextual word representations-based ensemble model outperformed all other individual transformer models for this problem. Group of people increasing conflict and hatred among consumers by exploiting freedom of speech and expression on social media platforms. Therefore, it is essential to take a positive reinforcement approach to research on positive, helpful, and supportive social media content. Wherefore, we build a transformer-based BERT with a threshold-based language detection system to detect Hope Speech in Youtube comments. The primary intent of this work is to reduce negativity and strengthen encouraging, supportive, and efficacious social media content.Next, we examine two principal issues in code-mixed data. Language identification is a primary preprocessing step in numerous code-mixed applications. With this purpose, we develop a word level language identification system by utilizing the English-Telugu code-mixed content. Subsequently, we explore the problem of sentiment analysis with English-Hindi bilingual code-mixed data. For this task, we propose a character level and sub-word level word representations with long-short term memory architecture. Later in multimodality, we explore a very salient new issue on social media: the emotion analysis of Internet memes. This work analyzes three different types of issues that revolve around Internet memes. The first of them is predicting the sentiment polarity of a meme, the second is a multi-label classification task that assesses whether a meme is offensive, humorous, satirical, or motivational, and the third is a multi-output ordinal classification task that predicts the degree of offense, humor, sarcasm of a meme. To handle these tasks, we introduce a multimodal architecture with late fusion technique that combines LSTM for textual features with VGG-16 for image feature representations. Propaganda is a communication tool that nfluences the opinions and actions of other people to achieve a predetermined goal. Initially, it was seen in newspapers, advertisements, and so on, but now it is widely used on social media. Therefore we develop a multimodal fusion system to detect propaganda in memes. We have used a robust fusion