Abstract

Cross-lingual dubbing of lecture videos requires the transcription of the original audio, correction and removal of disfluencies, domain term discovery, text-to-text translation into the target language, chunking of text using target language rhythm, text-to-speech synthesis followed by isochronous lipsyncing to the original video. This task becomes challenging when the source and target languages belong to different language families, resulting in differences in generated audio duration. This is further compounded by the original speaker’s rhythm, especially for extempore speech. This paper describes the challenges in regenerating English lecture videos in Indian languages semi-automatically. A prototype is developed for dubbing lectures into 9 Indian languages. A mean-opinion-score (MOS) is obtained for two languages, Hindi and Tamil, on two different courses. The output video is compared with the original video in terms of MOS (1-5) and lip synchronisation with scores of 4.09 and 3.74, respectively. The human effort also reduces by 75%.

Abstract

In computer assisted pronunciation training (CAPT), robust automatic models are critical for pronunciation assessment and mispronunciation detection and diagnosis (MDD). In the modelling, besides the audio data of second language (L2) learners, CAPT requires manually annotated ratings of over- all pronunciation quality, and the MDD uses manually anno- tated phonetic transcriptions. Though the pronunciation qual- ity and the mispronunciation are interdependent, to the best of our knowledge, none of the existing corpora contains both ratings and phonetic transcriptions. This could be due to the cost involved in obtaining phonetic transcriptions. However, a corpus with both kinds of information could benefit the re- searchers to obtain robust models by exploring the interde- pendencies. For addressing this, we develop voisTUTOR 2.0 corpus considering the existing voisTUTOR corpus referred to as voisTUTOR 1.0. We obtain phonetic transcriptions man- ually from a linguist for the entire Indian L2 learners’ En- glish audio data (26529 utterances approximately 14 hours) in voisTUTOR 1.0 for which overall quality ratings and bi- nary scores of factors influencing the pronunciation quality are available. A preliminary analysis of voisTUTOR 2.0 sug- gests that the phonetic errors correlated with the ratings and the binar

Abstract

Previous research has found that Acoustic Models (AM) of an Automatic Speech Recognition (ASR) system are susceptible to dialect variations within a language, thereby adversely affecting the ASR. To counter this, researchers have proposed to build a dialectspecific AM while keeping the Language Model (LM) constant for all the dialects. This study explores the effect of dialect mismatched LM by considering three different Telugu regional dialects: Telangana, Coastal Andhra, and Rayalaseema. We show that dialect variations that surface in the form of a different lexicon, grammar, and occasionally semantics can significantly degrade the performance of the LM under mismatched conditions. Therefore, this degradation has an adverse effect on the ASR even when dialect-specific AM is used. We show a degradation of up to 13.13 perplexity points when LM is used under mismatched conditions. Furthermore, we show a degradation of over 9% and over 15% in Character Error Rate (CER) and Word Error Rate (WER), respectively, in the ASR systems when using mismatched LMs over matched LMs

Abstract

Alzheimer’s Dementia is a progressive neurological disorder characterized by cognitive impairment. It affects mem- ory, thinking skills, language, and the ability to perform simple tasks. Detection of Alzheimer’s Dementia from the speech is considered a primitive task, as most speech cues are preserved in it. Studies in the literature focused mainly on the lexical features and few acoustic features for detecting Alzheimer’s disease. The present work explores the single frequency filtering cepstral coefficients (SFCC) for the automatic detection of Alzheimer’s disease. In contrast to STFTs, the proposed feature has better temporal and spectral resolution and captures the transient part more appropriately. This offers a very compact and efficient way to derive the formant structure in the speech signal. The experiments were conducted on the ADReSSo dataset, using the support vector machine classifier. The classification performance was compared with several baseline features like Mel-frequency cepstral coefficients (MFCC), perceptual linear prediction (PLP), linear prediction cepstral coefficient (LPCC), Mel frequency cepstral coefficients of LP-residual (MFCC-WR), ZFF signal (MFCC-ZF) and eGeMAPS (openSMILE). The experiments conducted on Alzheimer’s Dementia classification task show that the proposed feature performs better than conventional MFCCs. Among all the features, SFCC offers the best classification accuracy of 65.1% and 60.6% for dementia detection on cross- validation and test data, respectively. The combination of baseline features with SFCC features further improved the performance. Index Terms—Alzheimer’s disease, Cognitive impairment, Sin- gle frequency filtering cepstral coefficients.

Abstract

Parkinson’s disease (PD) is a progressive neurodegenerative dis- order of the central nervous system identified by motor and non- motor activities abnormalities. PD affects respiration, laryngeal, articulation, resonance, and prosodic aspects of speech production. Detection of PD from speech is a non-invasive approach useful for automatic screening. Perceptual attributes of speech due to PD are manifested as temporal variations in speech. In this regard, current work investigated the use of LSTM and BiLSTM networks with shifted delta cepstral (SDC) features to detect PD from speech. Further in BiLSTM networks, a multi-head attention mechanism is introduced, assuming that each head captures distinct information to detect PD. SDC features obtained from MFCCs, and SFFCCs are used for developing the PD detection system. The performance of the experiments is validated using the PC-GITA database. The experimental results revealed that BiLSTM networks give a rela- tive improvement of 4-5% over the LSTM networks. The use of a multi-head attention mechanism further improved the detection accuracy of the PD detection system, showing that it can capture various discriminative features.

Abstract

Modelling Indian languages is difficult as the number of word forms they present is high. In such cases, prior work has proposed us- ing subwords instead of words as the basic units to model the language. This provides the potential to cover more word forms. Additionally, previous work revealed that all Indian languages have several phonetic similarities. Consequently, multilingual acoustic models have been proposed to counter the data scarcity issue most Indian language datasets have. However, these models use monolin- gual language models (LM). They do not exploit the common basic tokens that certain Indian languages have. This motivated us to im- plement a bilingual subword-based Automatic Speech Recognition (ASR) system for Hindi and Marathi. Further, we try a combination of word-based monolingual LM with bilingual acoustic model to examine the reason for degradation in word-based multilingual ASRs. Our experiments show that multilingual subword-based ASR models outperform their word-based counterparts by upto 9.77% and 26.35% relative Character Error Rate (CER) in the case of Hindi and Marathi respectively.

Abstract

Epoch location is an important parameter for analysis of excitation source information from the speech signals. From several years lot of research have been done to find accurate locations of epochs. Epochs are instants at which vocal tract system are excited signifi- cantly. The prominent location of epochs can be found during the production of speech signals. However, due to the time varying nature of excitation source and the vocal tract system it is difficult to find accurate location of epochs. From various epoch extraction methods, Zero Frequency Filtering (ZFF) is the simplest and most widely used method to find accurate locations of epochs due to its highest identification rate and lowest false alarm rate among other algorithms. ZFF uses Infinite Impulse Response (IIR) filter followed by trend removal blocks however, the filter used in ZFF method is unstable which makes it unsuitable for practical implementation. In the literature many stable implementations of ZFF algorithms have been proposed. Compared to other stable algorithms of ZFF, Zero-phase Zero Frequency Resonator has simple design and gives the highest identification rate among other epoch extraction algo- rithms including ZFF. In this paper, the implementation of ZFF and ZP-ZFR has been proposed on FPGA board using Verilog which is Hardware Description Language (HDL).

Abstract

In this paper, we investigate the use of shifted delta cepstral (SDC) coefficients for detecting disfluencies in two types of speech - stuttered speech and spontaneous lecture-mode speech. SDC features capture temporal variations in the speech signal effectively across several frames. The UCLASS stuttered speech dataset and IIITH-IED dataset are used here to develop frame-level automatic disfluency detection systems for four types of disfluencies and the effect of SDC features on the detection of each disfluency type is examined using MFCC and SFFCC cepstral representations. Overall, it is found that using MFCC+SDC features gives an absolute improvement of 2.98% and 6.02% for stuttered and spontaneous speech disfluency detection respectively over the static MFCC features, while SFFCC+SDC features give an absolute improvement of 4.62% for stutter disfluencies and 7.28% for spontaneous speech disfluencies over the static SFFCC features, showing the importance of considering temporal variations for disfluency detection.

Abstract

Speech systems are sensitive to accent variations. This is especially challenging in the Indian context, with an abundance of languages but a dearth of linguistic studies characterising pronunciation variations. The growing number of L2 English speakers in India reinforces the need to study accents and L1-L2 interactions. We investigate the accents of Indian English (IE) speakers and report in detail our observations, both specific and common to all regions. In particular, we observe the phonemic variations and phonotactics occurring in the speakers' native languages and apply this to their English pronunciations. We demonstrate the influence of 18 Indian languages on IE by comparing the native language pronunciations with IE pronunciations obtained jointly from existing literature studies and phonetically annotated speech of 80 speakers. Consequently, we are able to validate the intuitions of Indian language influences on IE pronunciations by justifying pronunciation rules from the perspective of Indian language phonology. We obtain a comprehensive description in terms of universal and region-specific characteristics of IE, which facilitates accent conversion and adaptation of existing ASR and TTS systems to different Indian accents.

Abstract

Abstract. We consider Lie algebroids over algebraic spaces (in short we call it as a-spaces) by considering the sheaf of Lie-Rinehart algebras. We discuss about properties of universal enveloping algebroid U (OX, L) of a Lie algebroid L over an a-space (X, OX). This is done by sheafification of the presheaf of universal enveloping algebras for Lie-Rinehart algebras. We review the extent to which structure of the universal enveloping algebroid of Lie algebroids (over special a-spaces) resembles a sheaf of bialgebras. In the sequel we present a version of Poincar´e-Birkhoff-Witt theorem and Cartier-Milnor-Moore theorem for the Lie algebroid.