BOOK REVIEW Editor: Niels Keiding

1. the em algorithm and extensions. Geoffrey J. McLachlan and Thriyambakam Krishnan. Wiley, New York, 1997. No. of pages: xvii + 274. Price: $59.95. ISBN 0-471-12358-7

Two decades have passed since Dempster, Laird and Rubin published their seminal article^{1} introducing the EM algorithm. The paper was remarkable in at once revealing the simple unifying idea that underlay a scattering of seemingly unrelated maximum likelihood algorithms: showing how the idea could generate many more such algorithms; establishing general properties of these algorithms; and demonstrating the power of posing inference questions as incomplete-data problems. The paper gave rise to a flood of applications and refinements; a search of the CIS database reveals that the term‘EM algorithm’ appears in the title or keyword list of 576 documents up to 1997. The monograph by McLachlan and Krishnan is the first book-length treatment of EM.

The book contains six chapters. Chapter 1 describes Newton-type algorithms, introduces EM with the familiar example of the genetic linkage data, recounts the history of the algorithm and fixes notation. Chapter 2 gives detailed expositions of nine dey examples, and Chapter 3 reviews the theory of EM. Chapter 4 illustrates methods for accelerating convergence and computing standard errors. Chapter 5 discusses extensions of the algorithm, and Chapter 6 describes some related procedures, including Gibbs sampling, data augmentation and multiple imputation.

The authors illustrate EM in a range of examples, including regression with missing outcomes, analysis of medical imaging data, and estimation of multivariate t and finite mixture distributions. All examples include formulae for the E and M steps; some also include data and numerical results. The book contains no exercises. The writing leans heavily on displayed equations but is generally clear, and there is a moderate number of errors.

The authors provide excellent syntheses of EM’s theoretical underpinnings, the theory of its convergence, and the various ways in which it has been extended and refined. Chapter 6 clearly illustrates how the basic EM idea – of computation with the distribution of unobserved quantities conditional on observed quantities and provisional parameter estimates – has inspired much recent work on iterative simulation of Bayesian posterior distributions.

My only objection to the book is that it pays scant attention to practical issues of algorithm development and performance. When the E and M steps take simple closed forms, and the fraction of information missing is modest, EM is easy to program and converges without delay. When the E or M step is complicated, or one also needs standard errors, then EM is no easier to implement than other methods – and if moreover the fraction of information missing is large, it may be much slower than other methods. Thus, the author’s contention (p. 228) that ‘It is generally acknowledged that the EM algorithm and its extensions are the most suitable for handling the kind of incompletedata problems discussed in this book’ overstates both the value of EM relative to other methods and the degree of consensus about it.

Nevertheless, EM has become a core topic in the literature of incomplete data and statistical computing, and McLachlan and Krishnan have given us a thorough and readable account of its workings and history. Students and researchers who wish to learn about EM should look here first.

Daniel F. Heitjan Division of Biostatistics

Columbia University School of Public Health

600 W. 168th Street

New York, NY 10032, U.S.A.

References

[1] Dempster, A.P., Laird, N.M. and Rubin, D.B. ‘Maximum likelihood from incomplete data via the EM algorithm (with discussion)’, Journal of the Royal Statistical Society, Series B, 39, 1–38 (1976).