creators_name: Kanevsky, Dimitri
creators_name: Povey , Daniel 
creators_name: Ramabhadran, Bhuvana 
creators_name: Sainath, Tara
creators_id: kanevsky@us.ibm.com
creators_id: dpovey@us.ibm.com
creators_id: bhuvana@us.ibm.com
creators_id: tsainath@MIT.EDU
type: preprint
datestamp: 2008-01-15 23:56:42
lastmod: 2011-03-11 08:57:02
metadata_visibility: show
title: Adapted Extended Baum-Welch transformations
subjects: comp-sci-speech
full_text_status: public
keywords: MMIE training, EBW transformations, gradient steepness
abstract: The discrimination technique for estimating parameters of Gaussian mixtures that is based on the Extended Baum-Welch transformations (EBW) has had significant impact on the speech recognition community.  
In this paper we introduce a general definition of a family of EBW transformations that can be associated with a weighted sum of updated and initial models. We compute a gradient steepness measurement for a family of EBW transformations that are applied to functions of  Gaussian mixtures and demonstrate the growth property of these transformations. We consider EBW transformations  of discriminative functions in which EBW controlled parameters are adapted to a gradient steepness measurement or to the likelihood of the data given the model. We present experimental results that show that adapted EBW transformations can significantly speed up estimating parameters of Gaussian mixtures and give better decoding results.
date: 2007-10-01
date_type: completed
refereed: FALSE
referencetext: S. Axelrod, V. Goel, R. Gopinath, P. Olsen, and K. Visweswariah, "Discriminative Training of Subspace Constrained GMMs for Speech Recognition," to be submitted to IEEE Transactions on Speech and Audio Processing. 

L.E.Baum and J.A. Eagon, "An inequality with applications to statistical prediction for functions of Markov processes and to a model of ecology," {\em Bull. Amer. Math. Soc.}, vol. 73, pp.360-363, 1967.

A. Gunawardana and W. Byrne, ``Discriminative Speaker Adaptation with Conditional Maximum Likelihood Linear Regression,'' ICASSP, 2002.

P.S. Gopalakrishnan, D. Kanevsky, D. Nahamoo and A. Nadas, "An inequality for rational functions with applications to some statistical estimation problems",  IEEE Trans. Information Theory, Vol. 37, No.1 January 1991 .

D. Kanevsky, "Growth Transformations for General Functions", RC22919 (W0309-163), September 25, 2003.

D. Kanevsky, "Extended Baum transformations for general functions", in Proc. ICASSP, 2004.

D. Kanevsky, ``Extended Baum Transformations for General Functions, II", tech. Rep. RC23645(W0506-120), Human Language technologies, IBM , 2005, http://cogprints.org/5058/01/rc23645.pdf .

D. Kanevsky, "Constrained corrective training for continuous parameter system", US patent 6,044,344, March 28, 2000. 

Cong Liu   Peng Liu   Hui Jiang   Soong, F.   Ren-Hua Wang, "A Constrained Line Search Optimization for Discriminative Training in Speech Recognition",
in Proc. ICASSP, 2007.

Y. Normandin, "An improved MMIE Training Algorithm for Speaker Independent, Small Vocabulary, Continuous Speech Recognition", Proc. ICASSP'91, pp. 537-540, 1991.
Daniel Povey, "Discriminative Training for Large Vocabulary Speech Recognition",  PhD Thesis March 1, 2003.

Daniel Povey, Dimitri Kanevsky, Brian Kingsbury, Bhuvana Ramabhadran, George Saon and Karthik Visweswariah "Boosted MMI for model and feature-space discriminative training", submitted for ICASSP'08.

Tara N. Sainath, Dimitri Kanevsky, Giridharan Iyengar,``Unsupervised Audio Segmentation Using Extended Baum-Welch Transformations,  in Proc. ICASSP, 2007.
Tara N. Sainath, Victor Zue, Dimitri Kanevsky, ``Audio-Classification using Extended Baum-Welch Transformations" , in Proc. Interspeech 2007.

Tara N. Sainath, Dimitri Kanevsky, Bhuvana Ramabhadran ``Broad Phoentic Recognition in a Hidden Markov Model Framework Using Extended Baum-Welch Transformations " , to appear in Proc. ASRU 2007.

Tara N. Sainath, Dimitri Kanevsky, Bhuvana Ramabhadran ``Gradient Steepness Metrics Using Extended Baum-Welch Transformations for Universal Pattern Recognition Tasks" ,  submitted for ICASSP 2008.

R. Schluter, W. Macherey, B. Muler and H. Ney, "Comparison of discriminative training criteria and optimization methods for speech recognition", Speech Communication, Vol. 34, pp.287-310, 2001.

V. Valtchev , P.C. Woodland and S. J. Young, "Lattice-based Discriminative Training for Large Vocabulary Speech Recognition Systems", Speech Communication, Vol. 22, pp. 303-314, 1996.


citation:   Kanevsky, Dr Dimitri and Povey , Dr Daniel  and Ramabhadran, Dr Bhuvana  and Sainath, Dr Tara  (2007) Adapted Extended Baum-Welch transformations.  [Preprint]     
document_url: http://cogprints.org/5902/1/rc24458.pdf