In electrical engineering

Electrical engineering

Electrical engineering is a field of engineering that generally deals with the study and application of electricity, electronics and electromagnetism. The field first became an identifiable occupation in the late nineteenth century after commercialization of the electric telegraph and electrical...

, computer science

Computer science

Computer science or computing science is the study of the theoretical foundations of information and computation and of practical techniques for their implementation and application in computer systems...

, statistical computing and bioinformatics

Bioinformatics

Bioinformatics is the application of computer science and information technology to the field of biology and medicine. Bioinformatics deals with algorithms, databases and information systems, web technologies, artificial intelligence and soft computing, information and computation theory, software...

, the Baum–Welch algorithm is used to find the unknown parameters of a hidden Markov model

Hidden Markov model

A hidden Markov model is a statistical Markov model in which the system being modeled is assumed to be a Markov process with unobserved states. An HMM can be considered as the simplest dynamic Bayesian network. The mathematics behind the HMM was developed by L. E...

 (HMM). It makes use of the forward-backward algorithm and is named for Leonard E. Baum and Lloyd R. Welch

Lloyd R. Welch

Lloyd Richard Welch is a noted American information theorist, and co-inventor of the Baum–Welch algorithm.Welch received his B.S. in mathematics from the University of Illinois, 1951, and Ph.D. in mathematics from the California Institute of Technology, 1958, under advisor Frederic Bohnenblust...

.

Explanation

The Baum–Welch algorithm is a particular case of a generalized expectation-maximization

Expectation-maximization algorithm

In statistics, an expectation–maximization algorithm is an iterative method for finding maximum likelihood or maximum a posteriori estimates of parameters in statistical models, where the model depends on unobserved latent variables...

 (GEM) algorithm. It can compute maximum likelihood

Maximum likelihood

In statistics, maximum-likelihood estimation is a method of estimating the parameters of a statistical model. When applied to a data set and given a statistical model, maximum-likelihood estimation provides estimates for the model's parameters....

 estimates and posterior mode estimates for the parameters (transition and emission probabilities) of an HMM, when given only emissions as training data.

For a given cell in the transition matrix, all paths to that cell are summed. There is a link (transition from that cell to a cell ). The joint probability of , the link, and can be calculated and normalized by the probability of the entire string. Call this .

Now, calculate the probability of all paths with all links emanating from . Normalize this by the probability of the entire string. Call this .

Now divide by . This is dividing the expected transition from to by the expected transitions from . As the corpus grows, and particular transitions are reinforced, they will increase in value, reaching a local maximum. No way to ascertain a global maximum is known.

External links

• An Interactive Spreadsheet for Teaching the Forward-Backward Algorithm (spreadsheet and article with step-by-step walkthrough)
• Formal derivation of the Baum-Welch algorithm
• Implementation of the Baum-Welch algorithm