Abstract

The major impulse-like excitation in the speech signal is due to abrupt closure of the vocal folds, which takes place at the glottal closure instant (GCI) or epoch in each cycle. GCIs are used in many areas of speech science and technology, such as in prosody modification, voice source analysis, formant extraction and speech synthesis. It is difficult to observe these discontinuities (corresponding to GCIs) in the speech signal because of the superimposed time-varying response of the vocal tract system. This paper examines the phase part of different frequency components of the speech signal to extract epochs. Three analysis methods to decompose the speech signal into different frequency components are considered. These methods are the short-time Fourier transform (STFT), narrow bandpass filtering (NBPF), and single frequency filtering (SFF). The locations of the discontinuities in the speech signal are obtained from the instantaneous frequency (IF) (i.e., the time derivative of the phase) of each of the frequency components. A method for automatic detection of epochs using the amplitude weighted IF

Abstract

This paper proposes a method for displaying the phase information in speech signals through group delay spectrogram, without the need for phase unwrapping. The method involves scaling down the phase values without affecting the shape of the phase or group delay function, thus preserving the information of the phase spectrum. This is accomplished using single-frequency filtering (SFF) of speech signals to obtain the instantaneous complex SFF spectrum. The SFF involves filtering a frequency-shifting signal using a resonator at half the sampling frequency. The SFF spectrum displays characteristics similar to the standard short-time Fourier transform (STFT) spectrum, but without the effects of truncation due to windowing operation. The objective of the present study is to show that features of speech production can also be observed in the phase information, displayed through the group delay spectrogram. The time-frequency resolution in the group delay spectrogram depends on the choice of the bandwidth of the resonator used in the SFF analysis. The speech production features displayed in the group delay spectrogram are examined for different types of speech signals at different time-frequency resolutions.

Abstract

This paper examines the phase derivatives of speech signals. The instantaneous complex spectra obtained in the single frequency filtering (SFF) analysis of signals is used to derive the phase function. The problem of phase wrapping is avoided by using the proposed modification to SFF analysis to derive a scaled down version of the phase function. We consider the derivatives of the exponent (i.e., logarithm) of the complex SFF spectra, with respect to frequency, time, and both frequency and time. The imaginary part of the exponent is the phase function, and the real part is the log magnitude function. The negative derivative of phase with respect to frequency is the group delay (GD) function, and the derivative of the phase with respect to time is the instantaneous frequency (IF) function. The features of speech production displayed through the GD function are compared with the features displayed through the derivative with respect to frequency of the corresponding log magnitude function. Likewise, the features of production displayed through the IF function are compared with the features displayed through the derivative with respect to time of the corresponding log magnitude function. The speech production characteristics reflected in these representations of phase derivatives are examined for different types of utterances.

Abstract

The time delay between signals received at two spatially separated microphones is estimated using the location of the peak in the cross-correlation sequence between the two signals. The estimated delay is usually expressed in integer multiple of the sampling interval. The robustness of the delay estimation is affected by the degradation of the signals due to waveform distortion, noise, reflections, and reverberation. The broadband nature of speech signals is exploited to derive multiple evidence for the time delay from each frequency component of the signal. This helps to reduce the effects of waveform distortion, as the cross-correlation is computed on individual components. The multiple evidence provides robustness in the estimation of the time delay. The accuracy of the estimated time delay depends on the error between the integer delay and the true fractional delay. Methods are proposed to estimate the fractional part of the sampling interval of the delay. Robustness of the proposed method of estimation of the fractional part of the delay is examined for different types and levels of additive noise, as well for reverberation. Experimental results are given for data collected in an actual live room

Abstract

This paper describes a new method for speech activity detection (SAD) based on the recently proposed single frequency filtering (SFF) analysis of speech signals and a neural network model. The SFF analysis gives instantaneous spectrum of the speech signal at each sampling instant. The frequency resolution of the spectrum is decided by the number of frequencies used in the SFF analysis, which in turn depends on the frequency spacing. Using a frequency spacing of 10 Hz and a sampling frequency of 8 kHz, a 401 dimensional spectrum, covering 0-4 kHz, is obtained at each sampling instant. This is used as a feature vector to train an artificial neural network (ANN) model to discriminate (noisy) speech and nonspeech (mostly noise). The output of the trained ANN model for a given test utterance gives speech/nonspeech decision at every sampling instant. Post processing of the decision is used for SAD. The system generated SAD is evaluated on the Apollo corpus for SAD task in terms of detection cost function (DCF). The DCF values of the proposed system on the development and evaluation datasets are 3.1% and 4.6%, respectively, whereas the DCF values of the reported baseline system are 8.6% and 11.7%, respectively.

Abstract

Nasalization of vowels is a phenomenon where the oral and nasal tracts participate simultaneously in the production of speech. Acoustic coupling of the oral and nasal tracts, together with the glottal activity, results in a complex dynamic production system. The extent of coupling determines the degree of nasalization in vowels. Changes in the instantaneous response of the vocal tract system are used to study the effects of co-articulatory load of nasals on vowels. Dominant resonance frequency (DRF) contour is extracted from the instantaneous response of the system to determine the duration of the vowel nasalization. The extent of coupling determines the relative dominance of the oral and nasal resonances during the open and closed phases of a glottal cycle. Larger coupling leads to more decay of the energy of the oral resonances, and hence the vowel spectra exhibit predominantly the nasal resonances characteristics. Comparison with the existing measures show the effectiveness of the proposed measures for determining vowel nasalization. Segments of vowels in the context of nasal consonants from utterances of both male and female speakers of English are used to illustrate different aspects of this study on vowel nasalization.

Abstract

Speech production can be regarded as a process where a time-varying vocal tract system (filter) is excited by a time-varying excitation. In addition to its linguistic message, the speech signal also carries information about, for example, the gender and age of the speaker. Moreover, the speech signal includes acoustical cues about several speaker traits, such as the emotional state and the state of health of the speaker. In order to understand the production of these acoustical cues by the human speech production mechanism and utilize this information in speech technology, it is necessary to extract features describing both the excitation and the filter of the human speech production mechanism. While the methods to estimate and parameterize the vocal tract system are well established, the excitation appears less studied. This article provides a review of signal processing approaches used for the extraction of excitation information from speech. This article highlights the importance of excitation information in the analysis and classification of phonation type and vocal emotions, in the analysis of nonverbal laughter sounds, and in studying pathological voices. Furthermore, recent developments of deep learning techniques in the context of extraction and utilization of the excitation information are discussed.

Abstract

Advances in deep learning have been instrumental in enhancing the performance of face verification systems. Despite their ability to attain high accuracy, most of these systems fail to provide interpretations of their decisions. With the increased demands in making deep learning models more interpretable, numerous post-hoc methods have been proposed to probe the workings of these systems. Yet, the quest for face verification systems that inherently provide interpretations still remains largely unexplored. Additionally, most of the existing face recognition models are highly susceptible to adversarial attacks. In this work, we propose a face verification system which addresses the issue of interpretability by employing modular neural networks. In this, representations for each individual facial parts such as nose, mouth, eyes etc. are learned separately. We also show that our method is significantly more resistant to adversarial attacks, thereby addressing another crucial weakness concerning deep learning models.

Abstract

Estimation of time delay from the received broadband signals like speech, collected at two or more spatially distributed microphones, has many applications. Methods like the cross-correlation of the signals directly and generalized crosscorrelation based methods (GCC and GCC-PHAT) have been used for several years to estimate the time delay. Performance of these methods degrades due to noise, multi-path reflections, and reverberation in a practical environment, like a live room. The estimated time delay is usually robust due to the averaging effect of the delay obtained over several frames in an utterance of a few seconds. The robustness is affected if the varying time delay of a moving speaker is desired. A smaller duration for averaging results in errors in the estimation of the time delay, and a longer duration for averaging results in loss of accuracy. Since the single frequency filtering (SFF) based analysis provides an estimation of the instantaneous time delay, it is possible to study the trade off between accuracy and robustness. This paper examines this trade-off in determining the number of stationary speakers from mixed signals and in tracking a speaker moving along a straight line path and along a circular path. The results are illustrated for actual data collected in a live room.

Abstract

Formants are resonances of the time varying vocal tract system, and their characteristics are reflected in the response of the system for a sequence of impulse-like excitation sequence originated at the glottis. This paper presents a method to enhance the formants information in the display of spectrogram of the speech signal, especially for high pitched voices. It is well known that in the narrowband spectrogram, the presence of pitch harmonics masks the formant information, whereas in the wideband spectrogram, the formant regions are smeared. Using single frequency filtering (SFF) analysis, we show that the wideband equivalent SFF spectrogram can be modified to enhance the formant information in the display by improving the frequency resolution. For this, we obtain two SFF spectrograms by using single frequency filtering of the speech signal at two closely spaced roots on the real axis in the z-plane. The ratio or difference of the two SFF spectrograms is processed to enhance the formant information in the spectrographic display. This will help in tracking rapidly changing formants and in resolving closely spaced formants. The effect is more pronounced in the case of high-pitched voices, like female and children speech. Keywords: formants, spectrograms, single frequency filtering