Abstract

Language identification (LID) is vital frontend for spoken dialogue systems operating in diverse linguistic settings to reduce recognition and understanding errors. Existing LID systems which use low-level signal information for classification do not scale well due to exponential growth of parameters as the classes increase. They also suffer performance degradation due to the inherent variabilities of speech signal. In the proposed approach, we model the language-specific phonotactic information in speech using recurrent neural network for developing an LID system. The input speech signal is tokenized to phone sequences by using a common language-independent phone recognizer with varying phonetic coverage. We establish a causal relationship between phonetic coverage and LID performance. The phonotactics in the observed phone sequences are modeled using statistical and recurrent neural network language models to predict language-specific symbol from a universal phonetic inventory. Proposed approach is robust, computationally light weight and highly scalable. Experiments show that the convex combination of statistical and recurrent neural network language model (RNNLM) based phonotactic models significantly outperform a strong baseline system of Deep Neural Network (DNN) which is shown to surpass the performance of i-vector based approach for LID. The proposed approach outperforms the baseline models in terms of mean F1 score over 176 languages. Further we provide significant information-theoretic evidence to analyze the mechanism of the proposed approach.

Abstract

Recent improvement in usage of digital media has led people to share their opinions about specific entity through audio. In this paper, an approach to detect the sentiment of an online spoken reviews based on relative prosody features is presented. Most of the existing systems for audio based sentiment analysis use conventional audio features, but they are not problem specific features to extract the sentiment. In this work, relative prosody features are extracted from normal and stressed regions of audio signal to detect the sentiment. Stressed regions are identified using the strength of excitation. Support Vector Machine (SVM) and Gaussian Mixture Model (GMM) classifiers are used to build the sentiment models. MOUD database is used for the proposed study. Experimental results show that, the rate of detecting the sentiment is improved with relative prosody features compared with the prosody and Mel Frequency Cepstral Coefficients (MFCC) because the relative prosody features has more sentiment specific discrimination compared to prosody features.

Abstract

A mismatch in training and operating environments causes a performance degradation in speech recognition systems (ASR). One major reason for this mismatch is due to the presence of expressive (emotive) speech in operational environments. Emotions in speech majorly inflict the changes in the prosody parameters of pitch, duration and energy. This work is aimed at improving the performance of speech recognition systems in the presence of emotive speech. This work focuses on improving the speech recognition performance without disturbing the existing ASR system. The prosody modification of pitch, duration and energy is achieved by tuning the modification factors values for the relative differences between the neutral and emotional data sets. The neutral version of emotive speech is generated using uniform and non-uniform prosody modification methods for speech recognition. During the study, IITKGP-SESC corpus is used for building the ASR system. The speech recognition system for the emotions (anger, happy and compassion) is evaluated. An improvement in the performance of ASR is observed when the prosody modified emotive utterance is used for speech recognition in place of original emotive utterance. An average improvement around 5% in accuracy is observed due to the use of non-uniform prosody modification methods.

Abstract

The main focus of this paper is towards the development of a large vocabulary Telugu speech database. Telugu is a low resource language where there exists no standardized database for building the speech recognition system (ASR). The database consists of neutral speech samples collected from 100 speakers for building the Telugu ASR system and it was named as IIIT-H Telugu speech corpus. The speech and text corpus design and the procedure followed for the collection of the database have been discussed in detail. The preliminary ASR system results for the models built in this database are reported. The architectural choices of deep neural networks (DNNs) play a crucial role in improving the performance of ASR systems. ASR trained with hybrid DNNs (DNNHMM) with more hidden layers have shown better performance over the conventional GMMs (GMM-HMM). Kaldi tool kit is used for building the acoustic models required for the ASR system.

Abstract

The objective of this work is to develop a rule-based emotion conversion method for a better emotional perception. In this work, performance of emotion conversion using the linear modification model is improved by using vowel-based non-uniform prosody modification. In the present approach, attempts were made to integrate features like position and identity for addressing the non-uniformity in prosody generated due to the emotional state of the speaker. We mainly concentrate on the parameters such as strength, duration and pitch contour of vowels at different parts of the sentence. The influence of emotions on the above parameters is exploited to convert the speech from neutral emotion to the target emotion. Non-uniform prosody modification factors for emotion conversion are based on the position of vowels in the word, and the position of the word in the sentence. This study is carried out by using Indian Institute of Technology-Simulated Emotion speech corpus. Evaluation of the proposed algorithm is carried out by a subjective listening test. From the listening tests, it is observed that the performance of the proposed approach is better than the existing approaches

Abstract

Shouted speech signals have been studied mostlyfor utterances or word segments. In the production features derived over these segments, the differences between normal and shouted speech may sometimes be masked by variations due to pauses and unvoiced regions. Also, our recent study of electroglottograph signals has highlighted the usefulness of examining changes in the glottal excitation source characteristics during production of shouted speech. In this paper we examine changes in the shout production features in the vowel regions,that are found automatically by using the knowledge of accurate vowel onset points and epochs. Vowel regions are produced by periodic impulse-like excitation and contain relatively high signal energy. Hence the source features the instantaneous fundamental frequency and strength of excitation are examined, along with the filter feature the dominant frequency, for five different steady vowel regions. Changes in these features, that distinguish between normal and shouted speech, are more prominent in the automatically found vowel regions than for the entire word segments. This insight can help improving the automatic shout detection and other paralinguistic applications where acoustic cues need to be found in the vowel regions, e.g., detection of emotion categories.

Abstract

Automatic detection of shout prosody in continuous speech signal involves examining changes in its production characteristics. Our recent study of electroglottograph signals highlighted that significant changes occur in the glottal excitation source characteristics during production of shouted speech, especially in the vowel contexts. But the differences between normal and shouted speech, in the production features derived over utterances or word segments, may be masked sometimes by pauses or unvoiced regions related variations. Also, for such a real-time system, these vowel regions need to be found automatically. In this paper, changes in the shout production features are examined in the automatically detected vowel regions. Production of a vowel involves periodic impulse-like excitation and relatively high signal energy. Hence, the knowledge of epochs using zero-frequency filtering, and accurate vowel onset points can be used for detecting these regions. Changes in two excitation source features, the instantaneous fundamental frequency and strength of excitation, and in a vocal tract filter feature the dominant frequency, are examined for five steady vowel regions. Larger changes in these distinguishing features are observed in the automatically found vowel regions, than in word segments. This approach can help improving the systems for automatic detection of shout regions in continuous speech, and in paralinguistic applications that involve detection of prosody or emotions

Abstract

The main focus of this study is to analyze the performance of Automatic Speech Recognition (ASR) in different emotional environments using prosody modification. The majority of ASR systems are trained using neutral speech and the performance of such systems degrade when tested with the emotional speech. In this paper, the various components of speech that contribute to the emotion characteristics are studied. The prosody features of the source emotional utterancesare modified according to the target neutral utterances using Flexible Analysis Synthesis Tool (FAST). In the FAST, Dynamic Time Warping (DTW) is used to align the source emotional and target neutral utterances. Components of the prosody such as intonation, duration and excitation source are manipulated to incorporate the desired features into the source utterance. The modified (source emotional) utterances are then used for testing the ASR system which is trained using neutral speech. In this study, three emotions (compassion, happiness and anger) are considered for the analysis. Experimental results indicate an average improvement in the speech recognition system performance by considering prosody modified speech.

Abstract

The present study focuses on the text-independent speaker recognition in emotional conditions. In this paper, both system and source features are considered to represent speaker specific information. At the model level, Gaussian Mixture Models (GMMs), Gaussian Mixture Model-Universal Background Model (GMM-UBM) and Deep Neural Networks (DNN) are explored. The experiments are performed using 3 emotional databases, i.e. German emotional speech database (EMO-DB), IITKGP-SESC: Hindi and IITKGP-SESC: Telugu databases. The emotions considered in the present study are neutral, anger, happy and sad. The results show that, the performance of a speaker recognition system trained with clean speech is degrading while testing with emotional data irrespective of feature used or model used to build the system. The best results are obtained for the score level fusion of system and source features based systems when speakers are modeled with DNNs.

Abstract

Source and system features are extensively used in building Speaker Recognition (SR) systems. In this paper, we investigate the influence of source and system features on the performance of the SR system in emotional conditions. The Linear Prediction Residual Cepstral Coefficients (LPRCC) which corresponds to source features, and Mel Frequency Cepstral Coefficients (MFCC) and Linear Prediction Cepstral Coefficients (LPCC) that correspond to system features are used for modeling SR system. A maximum-likelihood classifier based on Gaussian mixture density functions is used and experiments are carried out on 3 standard emotional speech databases (Indian Institute of Technology-Simulated Emotion speech corpus (IITKGP-SESC): Hindi, IITKGP-SESC: Telugu and German Emotional Speech Database (EMO-DB)). The speaker models are trained with neutral utterances and tested with 3 types (anger, happy and sad) of emotional speech utterances. Based on experimental results, the performance degradation was observed in mismatched (trained with one emotion and tested with other emotion) case and the average percentage of degradation in SR task using source features is approximately 16% more compared to system features. The performance degradation of SR system is nullified when trained and tested with the same emotion.