An analysis of the difference of code lengths between two-step codes based on MDL principle and Bayes codes

Masayuki Goto, Toshiyasu Matsushima, Shigeichi Hirasawa

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

In this paper, we discuss the difference in code lengths between the code based on the minimum description length (MDL) principle (the MDL code) and the Bayes code under the condition that the same prior distribution is assumed for both codes. It is proved that the code length of the Bayes code is smaller than that of the MDL code by o(1) or O(1) for the discrete model class and by O(1) for the parametric model class. Because we can assume the same prior for the Bayes code as for the code based on the MDL principle, it is possible to construct the Bayes code with equal or smaller code length than the code based on the MDL principle. From the viewpoint of mean code length per symbol unit (compression rate), the Bayes code is asymptotically indistinguishable from the MDL two-stage codes.

Original languageEnglish
Pages (from-to)927-944
Number of pages18
JournalIEEE Transactions on Information Theory
Volume47
Issue number3
DOIs
Publication statusPublished - 2001 Mar

    Fingerprint

Keywords

  • Asymptotic normality
  • Bayes code
  • Minimum description length (MDL) principle
  • Universal coding

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications
  • Library and Information Sciences

Cite this