This paper by Franz Josef Och addresses the issue of training statistical machine translation models to optimize translation quality, particularly in terms of automatic evaluation metrics such as BLEU and multi-reference word error rate (mWER). The author critiques the common approach of using maximum likelihood or related criteria, which often do not align well with the final translation quality on unseen text. Instead, Och proposes training criteria that directly optimize these metrics, including a new algorithm for efficient training of unsmoothed error counts. The paper reviews various automatic evaluation criteria and presents two training criteria that aim to minimize errors. A new optimization algorithm for unsmoothed error counts is introduced, which is more efficient and stable than traditional grid-based line optimization methods. The results from experiments on the 2002 TIDES Chinese-English translation task show that optimizing error rates as part of the training criterion significantly improves translation quality, both in terms of error rates and evaluation metrics. The paper concludes by discussing future research directions, including the reliability of estimating parameters using unsmoothed error rate criteria and the choice of error rate to optimize during training.This paper by Franz Josef Och addresses the issue of training statistical machine translation models to optimize translation quality, particularly in terms of automatic evaluation metrics such as BLEU and multi-reference word error rate (mWER). The author critiques the common approach of using maximum likelihood or related criteria, which often do not align well with the final translation quality on unseen text. Instead, Och proposes training criteria that directly optimize these metrics, including a new algorithm for efficient training of unsmoothed error counts. The paper reviews various automatic evaluation criteria and presents two training criteria that aim to minimize errors. A new optimization algorithm for unsmoothed error counts is introduced, which is more efficient and stable than traditional grid-based line optimization methods. The results from experiments on the 2002 TIDES Chinese-English translation task show that optimizing error rates as part of the training criterion significantly improves translation quality, both in terms of error rates and evaluation metrics. The paper concludes by discussing future research directions, including the reliability of estimating parameters using unsmoothed error rate criteria and the choice of error rate to optimize during training.