Effective Spectral and Excitation Modeling Techniques for LSTM-RNN-Based Speech Synthesis Systems

International Journal
2017-11-01 22:07
Authors : Eunwoo Song, Frank K. Soong, Hong-Goo Kang

Year : 2017

Publisher / Conference : IEEE/ACM Transactions on Audio, Speech, and Language Processing

Volume : 25, issue 11

Page : 2152-2161

In this paper, we report research results on modeling the parameters of an improved time-frequency trajectory excitation (ITFTE) and spectral envelopes of an LPC vocoder with a long short-term memory (LSTM)-based recurrent neural network (RNN) for high-quality text-to-speech (TTS) systems. The ITFTE vocoder has been shown to significantly improve the perceptual quality of statistical parameter-based TTS systems in our prior works. However, a simple feed-forward deep neural network (DNN) with a finite window length is inadequate to capture the time evolution of the ITFTE parameters. We propose to use the LSTM to exploit the time-varying nature of both trajectories of the excitation and filter parameters, where the LSTM is implemented to use the linguistic text input and to predict both ITFTE and LPC parameters holistically. In the case of LPC parameters, we further enhance the generated spectrum by applying LP bandwidth expansion and line spectral frequency-sharpening filters. These filters are not only beneficial for reducing unstable synthesis filter conditions but also advantageous toward minimizing the muffling problem in the generated spectrum. Experimental results have shown that the proposed LSTM-RNN system with the ITFTE vocoder significantly outperforms both similarly configured band aperiodicity-based systems and our best prior DNN-trainecounterpart, both objectively and subjectively.
전체 326
266 International Journal Eunwoo Song, Frank K. Soong, Hong-Goo Kang "Effective Spectral and Excitation Modeling Techniques for LSTM-RNN-Based Speech Synthesis Systems" in IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol.25, issue 11, pp.2152-2161, 2017
265 International Conference Seung-chul Shin, Junhyung Moon, Saewon Kye, Kyoungwoo Lee, Yong Seung Lee, Hong-Goo Kang "Continuous bladder volume monitoring system for wearable applications" in EMBC, 2017
264 Domestic Conference 김정규, 박영철, 강홍구 "저사양 TV 사운드 설계환경을 위한 IIR 필터 기반 주파수 등화기" in 대한전자공학회 학술대회, 2017
263 International Conference Jinkyu Lee, Keulbit Kim, Turaj Shabestary, Hong-Goo Kang "Deep bi-directional long short-term memory based speech enhancement for wind noise reduction" in HSCMA, 2017
262 International Conference JeeSok Lee, Soo-Whan Chung, Min-Seok Choi, Hong-Goo Kang "A study on search grid points for data-driven 3-D beamsteering" in HSCMA, 2017
261 Domestic Journal Ji-ho Seo, Dae Hee Youn, Young-Cheol Park "A Method of Designing Low-power Feedback Active Noise Control Filter for Headphones/Earphones" in 한국통신학회논문지, vol.10, 제 1호, pp.57-65, 2017
260 Domestic Journal Hyeongi Moon, Young-cheol Park, Yong Ju Lee, Young-soo Whang "MPEG-H 3D Audio Decoder Structure and Complexity Analysis" in 한국통신학회논문지, vol.42, 제 2호, pp.432-443, 2017
259 International Conference Young-Sun Joo, Won-Suk Jun, Hong-Goo Kang "Efficient deep neural networks for speech synthesis using bottleneck features" in APSIPA, 2016
258 Domestic Journal 문현기, 박영철, 황영수 "위상 일치와 가변 지수 감쇠 가중치 부여 방법이 적용된 가상 저음 시스템" in 방송공학회논문지, vol.21, 제 6호, pp.889-898, 2016
257 International Conference Ji-ho Seo, Young-cheol Park, Dae Hee Youn "Design of feedback active noise control system based on a constrained optimization for headphone/earphone applications" in IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia), 2016