February 1998 | Robert J. McEliece, Fellow, IEEE, David J. C. MacKay, and Jung-Fu Cheng
This paper describes the connection between the iterative turbo decoding algorithm and Pearl's belief propagation algorithm. Turbo codes, introduced in 1993, are a significant development in coding theory. While the performance of turbo decoding is well-documented, a theoretical explanation for its effectiveness remains elusive. Pearl's belief propagation algorithm, known in artificial intelligence but less so in information theory, is shown to be closely related to turbo decoding. However, Pearl's algorithm is only guaranteed to work on belief networks without loops, and turbo decoding networks contain loops, making the theoretical explanation incomplete. Despite this, Pearl's algorithm can be used to derive iterative decoding algorithms for various error-control systems, including Gallager's low-density parity-check codes, serially concatenated codes, and product codes. The paper demonstrates that belief propagation provides a general methodology for designing low-complexity iterative decoding algorithms for hybrid coded systems. The turbo decoding algorithm is shown to be an instance of Pearl's belief propagation algorithm applied to the belief network of a parallel concatenated code. The paper also discusses other decoding algorithms derived from belief propagation, including those for low-density parity-check codes, low-density generator matrix codes, serially concatenated codes, and product codes. The results show that belief propagation is a powerful tool for designing efficient decoding algorithms for various coding schemes.This paper describes the connection between the iterative turbo decoding algorithm and Pearl's belief propagation algorithm. Turbo codes, introduced in 1993, are a significant development in coding theory. While the performance of turbo decoding is well-documented, a theoretical explanation for its effectiveness remains elusive. Pearl's belief propagation algorithm, known in artificial intelligence but less so in information theory, is shown to be closely related to turbo decoding. However, Pearl's algorithm is only guaranteed to work on belief networks without loops, and turbo decoding networks contain loops, making the theoretical explanation incomplete. Despite this, Pearl's algorithm can be used to derive iterative decoding algorithms for various error-control systems, including Gallager's low-density parity-check codes, serially concatenated codes, and product codes. The paper demonstrates that belief propagation provides a general methodology for designing low-complexity iterative decoding algorithms for hybrid coded systems. The turbo decoding algorithm is shown to be an instance of Pearl's belief propagation algorithm applied to the belief network of a parallel concatenated code. The paper also discusses other decoding algorithms derived from belief propagation, including those for low-density parity-check codes, low-density generator matrix codes, serially concatenated codes, and product codes. The results show that belief propagation is a powerful tool for designing efficient decoding algorithms for various coding schemes.