### Abstract

In this paper, we discuss the difference in code lengths between the code based on the minimum description length (MDL) principle (the MDL code) and the Bayes code under the condition that the same prior distribution is assumed for both codes. It is proved that the code length of the Bayes code is smaller than that of the MDL code by o(1) or O(1) for the discrete model class and by O(1) for the parametric model class. Because we can assume the same prior for the Bayes code as for the code based on the MDL principle, it is possible to construct the Bayes code with equal or smaller code length than the code based on the MDL principle. From the viewpoint of mean code length per symbol unit (compression rate), the Bayes code is asymptotically indistinguishable from the MDL two-stage codes.

Original language | English |
---|---|

Pages (from-to) | 927-944 |

Number of pages | 18 |

Journal | IEEE Transactions on Information Theory |

Volume | 47 |

Issue number | 3 |

DOIs | |

Publication status | Published - 2001 Mar |

### Fingerprint

### Keywords

- Asymptotic normality
- Bayes code
- Minimum description length (MDL) principle
- Universal coding

### ASJC Scopus subject areas

- Information Systems
- Electrical and Electronic Engineering

### Cite this

**An analysis of the difference of code lengths between two-step codes based on MDL principle and Bayes codes.** / Goto, Masayuki; Matsushima, Toshiyasu; Hirasawa, S.

Research output: Contribution to journal › Article

*IEEE Transactions on Information Theory*, vol. 47, no. 3, pp. 927-944. https://doi.org/10.1109/18.915647

}

TY - JOUR

T1 - An analysis of the difference of code lengths between two-step codes based on MDL principle and Bayes codes

AU - Goto, Masayuki

AU - Matsushima, Toshiyasu

AU - Hirasawa, S.

PY - 2001/3

Y1 - 2001/3

N2 - In this paper, we discuss the difference in code lengths between the code based on the minimum description length (MDL) principle (the MDL code) and the Bayes code under the condition that the same prior distribution is assumed for both codes. It is proved that the code length of the Bayes code is smaller than that of the MDL code by o(1) or O(1) for the discrete model class and by O(1) for the parametric model class. Because we can assume the same prior for the Bayes code as for the code based on the MDL principle, it is possible to construct the Bayes code with equal or smaller code length than the code based on the MDL principle. From the viewpoint of mean code length per symbol unit (compression rate), the Bayes code is asymptotically indistinguishable from the MDL two-stage codes.

AB - In this paper, we discuss the difference in code lengths between the code based on the minimum description length (MDL) principle (the MDL code) and the Bayes code under the condition that the same prior distribution is assumed for both codes. It is proved that the code length of the Bayes code is smaller than that of the MDL code by o(1) or O(1) for the discrete model class and by O(1) for the parametric model class. Because we can assume the same prior for the Bayes code as for the code based on the MDL principle, it is possible to construct the Bayes code with equal or smaller code length than the code based on the MDL principle. From the viewpoint of mean code length per symbol unit (compression rate), the Bayes code is asymptotically indistinguishable from the MDL two-stage codes.

KW - Asymptotic normality

KW - Bayes code

KW - Minimum description length (MDL) principle

KW - Universal coding

UR - http://www.scopus.com/inward/record.url?scp=0035269617&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0035269617&partnerID=8YFLogxK

U2 - 10.1109/18.915647

DO - 10.1109/18.915647

M3 - Article

AN - SCOPUS:0035269617

VL - 47

SP - 927

EP - 944

JO - IEEE Transactions on Information Theory

JF - IEEE Transactions on Information Theory

SN - 0018-9448

IS - 3

ER -