G.J. McLachlan and T. Krishnan, The EM Algorithm and Extensions, New York: Wiley, 1997, pp. 274.

The Expectation-Maximization (EM) algorithm iteratively computes maximum likelihood estimates (MLE) in a frequentist framework or the mode of a posterior distribution in a Bayesian framework, when some observations can be viewed as missing. Since Dempster, Laird and Rubin (1977) have formalized the procedure and exhibited its versatility, the EM algorithm is used to solve a broad range of statistical problems, such as finite mixture analysis, factor analysis or neural network learning. It has been the subject of thousands of publications (see the review of Meng and Pedlow 1992). Some excellent books have already appeared, which are partly concerned with the EM algorithm (e.g., Little and Rubin 1987), but no book has been fully dedicated to the subject. The book by G.J. McLachlan and T. Krishnan fills this gap.

It is organized in six chapters. Many examples from very different statistical areas are provided, and used for illustrating details of implementation, extensions of the algorithms or theory. They help the reader throughout the book.

Chapter 1 is a very general introduction. It is a kind of overview of all the subjects tackled in the rest of the book. The algorithm is introduced by means of well detailed examples. Competitive algorithms, such as the Newton-Raphson method, are briefly presented. Advantages and problems related to the use of the EM algorithm are summarized. The chapter provides a comprehensive account of the algorithm and can be read independently for its own interest.

Chapter 2 presents a variety of different areas of application. Some examples concern parameter estimation in the context of missing observations (e.g., bivariate normal data with missing values). Other examples relate to problems where the likelihood equations are easier to manipulate when considering some “hidden variables” (e.g., finite mixture). Although the examples are well presented, others, such as the one about multinomial distribution, are sometimes redundant with the introductory examples and do not bring additional insight into the algorithm.

Chapter 3 given an account of the basic EM algorithm theory. The monotonicity of the EM and Generalized EM, convergence, convergence rates, missing information principle are explained. Once again the reader enjoys the examples which illustrate some theoretical points. This makes the chapter pleasant to read in spite of its technical content.

Chapter 4 deals with extensions of the algorithm that address the speeding of the rate of convergence and the estimation of the covariance matrix of the MLE.

Chapter 5 mainly focuses on the Expectation-Conditional Maximum (ECM) algorithm and some other related variants. These extensions were initially designed for solving an intractable or rather complicated M-step. They replace the unique M-step by a sequence of computationally simpler maximizations. The end of the chapter deals with different extensions, which have really different motivations form the ECM algorithm. However, all this material is very clearly presented and well illustrated.

Chapter 6 presents several interesting topics, which are not investigated deeply enough. For example, it is a pity that such an important and topical subject like iterative simulation algorithms is not more developed. These iterative techniques repalce the E or M steps by draws from conditional distributions. They are an attractive alternative when the E or M step proves intractable and also useful for tackling the problem of initial condition dependence. The presentation of neural network applications is so succinct that it is hard to follow. Moreover, the authors treat mush too briefly the interpretation of the EM algorithm as maximizing a joint function of the parameter and of the distribution over the missing variables (Hathaway 1986, Neal and Hinton 1993, Yuille, Stolorz and Utans 1994). Whereas it would have seemed important to give a more detailed account of this interpretation, because it makes it possible to justify a wide range of variants of the EM algorithm.

In conclusion, this is a good book, which is very pleasant to read. It contains a great variety of examples. Although some recent extensions and areas of application could have been more discussed in greater detail, the review is quite complete and comprehensive. In any case, as this subject is in full development, it will call for another book in a few years.

École des Mines de Paris

C. Ambroise