An analysis of the difference of code lengths between two-step codes based on MDL principle and Bayes codes

    Research output: Contribution to journalArticle

    4 Citations (Scopus)

    Abstract

    In this paper, we discuss the difference in code lengths between the code based on the minimum description length (MDL) principle (the MDL code) and the Bayes code under the condition that the same prior distribution is assumed for both codes. It is proved that the code length of the Bayes code is smaller than that of the MDL code by o(1) or O(1) for the discrete model class and by O(1) for the parametric model class. Because we can assume the same prior for the Bayes code as for the code based on the MDL principle, it is possible to construct the Bayes code with equal or smaller code length than the code based on the MDL principle. From the viewpoint of mean code length per symbol unit (compression rate), the Bayes code is asymptotically indistinguishable from the MDL two-stage codes.

    Original languageEnglish
    Pages (from-to)927-944
    Number of pages18
    JournalIEEE Transactions on Information Theory
    Volume47
    Issue number3
    DOIs
    Publication statusPublished - 2001 Mar

    Fingerprint

    symbol

    Keywords

    • Asymptotic normality
    • Bayes code
    • Minimum description length (MDL) principle
    • Universal coding

    ASJC Scopus subject areas

    • Information Systems
    • Electrical and Electronic Engineering

    Cite this

    @article{df29055d11d44e30b194a7171391633d,
    title = "An analysis of the difference of code lengths between two-step codes based on MDL principle and Bayes codes",
    abstract = "In this paper, we discuss the difference in code lengths between the code based on the minimum description length (MDL) principle (the MDL code) and the Bayes code under the condition that the same prior distribution is assumed for both codes. It is proved that the code length of the Bayes code is smaller than that of the MDL code by o(1) or O(1) for the discrete model class and by O(1) for the parametric model class. Because we can assume the same prior for the Bayes code as for the code based on the MDL principle, it is possible to construct the Bayes code with equal or smaller code length than the code based on the MDL principle. From the viewpoint of mean code length per symbol unit (compression rate), the Bayes code is asymptotically indistinguishable from the MDL two-stage codes.",
    keywords = "Asymptotic normality, Bayes code, Minimum description length (MDL) principle, Universal coding",
    author = "Masayuki Goto and Toshiyasu Matsushima and S. Hirasawa",
    year = "2001",
    month = "3",
    doi = "10.1109/18.915647",
    language = "English",
    volume = "47",
    pages = "927--944",
    journal = "IEEE Transactions on Information Theory",
    issn = "0018-9448",
    publisher = "Institute of Electrical and Electronics Engineers Inc.",
    number = "3",

    }

    TY - JOUR

    T1 - An analysis of the difference of code lengths between two-step codes based on MDL principle and Bayes codes

    AU - Goto, Masayuki

    AU - Matsushima, Toshiyasu

    AU - Hirasawa, S.

    PY - 2001/3

    Y1 - 2001/3

    N2 - In this paper, we discuss the difference in code lengths between the code based on the minimum description length (MDL) principle (the MDL code) and the Bayes code under the condition that the same prior distribution is assumed for both codes. It is proved that the code length of the Bayes code is smaller than that of the MDL code by o(1) or O(1) for the discrete model class and by O(1) for the parametric model class. Because we can assume the same prior for the Bayes code as for the code based on the MDL principle, it is possible to construct the Bayes code with equal or smaller code length than the code based on the MDL principle. From the viewpoint of mean code length per symbol unit (compression rate), the Bayes code is asymptotically indistinguishable from the MDL two-stage codes.

    AB - In this paper, we discuss the difference in code lengths between the code based on the minimum description length (MDL) principle (the MDL code) and the Bayes code under the condition that the same prior distribution is assumed for both codes. It is proved that the code length of the Bayes code is smaller than that of the MDL code by o(1) or O(1) for the discrete model class and by O(1) for the parametric model class. Because we can assume the same prior for the Bayes code as for the code based on the MDL principle, it is possible to construct the Bayes code with equal or smaller code length than the code based on the MDL principle. From the viewpoint of mean code length per symbol unit (compression rate), the Bayes code is asymptotically indistinguishable from the MDL two-stage codes.

    KW - Asymptotic normality

    KW - Bayes code

    KW - Minimum description length (MDL) principle

    KW - Universal coding

    UR - http://www.scopus.com/inward/record.url?scp=0035269617&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=0035269617&partnerID=8YFLogxK

    U2 - 10.1109/18.915647

    DO - 10.1109/18.915647

    M3 - Article

    AN - SCOPUS:0035269617

    VL - 47

    SP - 927

    EP - 944

    JO - IEEE Transactions on Information Theory

    JF - IEEE Transactions on Information Theory

    SN - 0018-9448

    IS - 3

    ER -