Properties of a word-valued source with a non-prefix-free word set

Takashi Ishida, Masayuki Goto, Toshiyasu Matsushima, Shigeichi Hirasawa

    Research output: Contribution to journalArticle

    3 Citations (Scopus)

    Abstract

    Recently, a word-valued source has been proposed as a new class of information source models. A word-valued source is regarded as a source with a probability distribution over a word set. Although a word-valued source is a nonstationary source in general, it has been proved that an entropy rate of the source exists and the Asymptotic Equipartition Property (AEP) holds when the word set of the source is prefix-free. However, when the word set is not prefix-free (non-prefix-free), only an upper bound on the entropy density rate for an i.i.d. word-valued source has been derived so far. In this paper, we newly derive a lower bound on the entropy density rate for an i.i.d. word-valued source with a finite non-prefix-free word set. Then some numerical examples are given in order to investigate the behavior of the bounds.

    Original languageEnglish
    Pages (from-to)3710-3723
    Number of pages14
    JournalIEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences
    VolumeE89-A
    Issue number12
    DOIs
    Publication statusPublished - 2006 Dec

    Fingerprint

    Entropy
    Probability distributions
    Prefix-free
    Equipartition
    Probability Distribution
    Lower bound
    Upper bound
    Numerical Examples

    Keywords

    • Asymptotic Equipartition Property (AEP)
    • Entropy density rate
    • Entropy rate
    • Non-prefix-free
    • Word set
    • Word-valued source

    ASJC Scopus subject areas

    • Electrical and Electronic Engineering
    • Hardware and Architecture
    • Information Systems

    Cite this

    Properties of a word-valued source with a non-prefix-free word set. / Ishida, Takashi; Goto, Masayuki; Matsushima, Toshiyasu; Hirasawa, Shigeichi.

    In: IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, Vol. E89-A, No. 12, 12.2006, p. 3710-3723.

    Research output: Contribution to journalArticle

    @article{35bb8e22e83147788f62bbb6e29f1e82,
    title = "Properties of a word-valued source with a non-prefix-free word set",
    abstract = "Recently, a word-valued source has been proposed as a new class of information source models. A word-valued source is regarded as a source with a probability distribution over a word set. Although a word-valued source is a nonstationary source in general, it has been proved that an entropy rate of the source exists and the Asymptotic Equipartition Property (AEP) holds when the word set of the source is prefix-free. However, when the word set is not prefix-free (non-prefix-free), only an upper bound on the entropy density rate for an i.i.d. word-valued source has been derived so far. In this paper, we newly derive a lower bound on the entropy density rate for an i.i.d. word-valued source with a finite non-prefix-free word set. Then some numerical examples are given in order to investigate the behavior of the bounds.",
    keywords = "Asymptotic Equipartition Property (AEP), Entropy density rate, Entropy rate, Non-prefix-free, Word set, Word-valued source",
    author = "Takashi Ishida and Masayuki Goto and Toshiyasu Matsushima and Shigeichi Hirasawa",
    year = "2006",
    month = "12",
    doi = "10.1093/ietfec/e89-a.12.3710",
    language = "English",
    volume = "E89-A",
    pages = "3710--3723",
    journal = "IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences",
    issn = "0916-8508",
    publisher = "Maruzen Co., Ltd/Maruzen Kabushikikaisha",
    number = "12",

    }

    TY - JOUR

    T1 - Properties of a word-valued source with a non-prefix-free word set

    AU - Ishida, Takashi

    AU - Goto, Masayuki

    AU - Matsushima, Toshiyasu

    AU - Hirasawa, Shigeichi

    PY - 2006/12

    Y1 - 2006/12

    N2 - Recently, a word-valued source has been proposed as a new class of information source models. A word-valued source is regarded as a source with a probability distribution over a word set. Although a word-valued source is a nonstationary source in general, it has been proved that an entropy rate of the source exists and the Asymptotic Equipartition Property (AEP) holds when the word set of the source is prefix-free. However, when the word set is not prefix-free (non-prefix-free), only an upper bound on the entropy density rate for an i.i.d. word-valued source has been derived so far. In this paper, we newly derive a lower bound on the entropy density rate for an i.i.d. word-valued source with a finite non-prefix-free word set. Then some numerical examples are given in order to investigate the behavior of the bounds.

    AB - Recently, a word-valued source has been proposed as a new class of information source models. A word-valued source is regarded as a source with a probability distribution over a word set. Although a word-valued source is a nonstationary source in general, it has been proved that an entropy rate of the source exists and the Asymptotic Equipartition Property (AEP) holds when the word set of the source is prefix-free. However, when the word set is not prefix-free (non-prefix-free), only an upper bound on the entropy density rate for an i.i.d. word-valued source has been derived so far. In this paper, we newly derive a lower bound on the entropy density rate for an i.i.d. word-valued source with a finite non-prefix-free word set. Then some numerical examples are given in order to investigate the behavior of the bounds.

    KW - Asymptotic Equipartition Property (AEP)

    KW - Entropy density rate

    KW - Entropy rate

    KW - Non-prefix-free

    KW - Word set

    KW - Word-valued source

    UR - http://www.scopus.com/inward/record.url?scp=33845565335&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=33845565335&partnerID=8YFLogxK

    U2 - 10.1093/ietfec/e89-a.12.3710

    DO - 10.1093/ietfec/e89-a.12.3710

    M3 - Article

    AN - SCOPUS:33845565335

    VL - E89-A

    SP - 3710

    EP - 3723

    JO - IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences

    JF - IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences

    SN - 0916-8508

    IS - 12

    ER -