A study on difference of codelengths between codes based on MDL principle and bayes codes for given prior distributions

Masayuki Goto, Toshiyasu Matsushima, Shigeichi Hirasawa

    Research output: Contribution to journalArticle

    Abstract

    The principle of the Minimum Description Length (MDL) proposed by J. Rissanen provides a type of structure for the model estimation based on probabilistic model selection allowing minimization of the codelength. On the other hand, the use of Bayes codes makes it possible to find a coding function from a mix of probabilistic models without specifying any concrete model. It has been pointed out that codes based on the MDL principle (MDL codes) are closely related to Bayes theory because in the definition of the description length of the probabilistic model, an unknown prior distribution is assumed. In this paper, we apply asymptotic analysis to the codelength difference between the MDL codes and Bayes codes, including cases of different prior distributions. The results of the analysis clearly show that in the case of discrete model families, codes having a high prior distribution in true models (that is, the models for which an advantageous prior distribution is assumed) are favorable, but in the case of parametric model families, Bayes codes have shorter codelength than the MDL codes even in the cases of advantageous prior distribution assumed for the MDL codes.

    Original languageEnglish
    Pages (from-to)30-40
    Number of pages11
    JournalElectronics and Communications in Japan, Part III: Fundamental Electronic Science (English translation of Denshi Tsushin Gakkai Ronbunshi)
    Volume84
    Issue number4
    Publication statusPublished - 2001

      Fingerprint

    Keywords

    • Bayes code
    • Information source coding
    • MDL principle
    • Prior distribution

    ASJC Scopus subject areas

    • Electrical and Electronic Engineering
    • Computer Networks and Communications

    Cite this