On the complexity of hypothesis space and the sample complexity for machine learning

Makoto Nakazawa, Toshiyuki Kohnosu, Toshiyasu Matsushima, Shigeichi Hirasawa

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    The problem of learning a concept from examples in the model introduced by Valiant is discussed. According to the traditional ways of thinking, it is assumed that the learnability is independent of the occurrence probability of instance. By utilizing this probability, we propose the metric as a new measure to determine the complexity of hypothesis space. The metric measures the hardness of discrimination between hypotheses. Furthermore, we obtain the average metric dependent on prior information. This metric is the measure of complexity for hypothesis space in the average. Similarly in the worst case, we obtain the minimum metric. We make clear the relationship between these measures and the Vapnik - Chervonenkis (VC) dimension. Finally, we show the upper bound on sample complexity utilizing the metric. This results can be applied in the discussion on the learnability of the class with an infinite VC dimension.

    Original languageEnglish
    Title of host publicationProceedings of the IEEE International Conference on Systems, Man and Cybernetics
    PublisherIEEE
    Pages132-137
    Number of pages6
    Volume1
    Publication statusPublished - 1994
    EventProceedings of the 1994 IEEE International Conference on Systems, Man and Cybernetics. Part 1 (of 3) - San Antonio, TX, USA
    Duration: 1994 Oct 21994 Oct 5

    Other

    OtherProceedings of the 1994 IEEE International Conference on Systems, Man and Cybernetics. Part 1 (of 3)
    CitySan Antonio, TX, USA
    Period94/10/294/10/5

    Fingerprint

    Learning systems
    Hardness

    ASJC Scopus subject areas

    • Hardware and Architecture
    • Control and Systems Engineering

    Cite this

    Nakazawa, M., Kohnosu, T., Matsushima, T., & Hirasawa, S. (1994). On the complexity of hypothesis space and the sample complexity for machine learning. In Proceedings of the IEEE International Conference on Systems, Man and Cybernetics (Vol. 1, pp. 132-137). IEEE.

    On the complexity of hypothesis space and the sample complexity for machine learning. / Nakazawa, Makoto; Kohnosu, Toshiyuki; Matsushima, Toshiyasu; Hirasawa, Shigeichi.

    Proceedings of the IEEE International Conference on Systems, Man and Cybernetics. Vol. 1 IEEE, 1994. p. 132-137.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Nakazawa, M, Kohnosu, T, Matsushima, T & Hirasawa, S 1994, On the complexity of hypothesis space and the sample complexity for machine learning. in Proceedings of the IEEE International Conference on Systems, Man and Cybernetics. vol. 1, IEEE, pp. 132-137, Proceedings of the 1994 IEEE International Conference on Systems, Man and Cybernetics. Part 1 (of 3), San Antonio, TX, USA, 94/10/2.
    Nakazawa M, Kohnosu T, Matsushima T, Hirasawa S. On the complexity of hypothesis space and the sample complexity for machine learning. In Proceedings of the IEEE International Conference on Systems, Man and Cybernetics. Vol. 1. IEEE. 1994. p. 132-137
    Nakazawa, Makoto ; Kohnosu, Toshiyuki ; Matsushima, Toshiyasu ; Hirasawa, Shigeichi. / On the complexity of hypothesis space and the sample complexity for machine learning. Proceedings of the IEEE International Conference on Systems, Man and Cybernetics. Vol. 1 IEEE, 1994. pp. 132-137
    @inproceedings{687b9ff6d82f436cb1a292cf88b9b031,
    title = "On the complexity of hypothesis space and the sample complexity for machine learning",
    abstract = "The problem of learning a concept from examples in the model introduced by Valiant is discussed. According to the traditional ways of thinking, it is assumed that the learnability is independent of the occurrence probability of instance. By utilizing this probability, we propose the metric as a new measure to determine the complexity of hypothesis space. The metric measures the hardness of discrimination between hypotheses. Furthermore, we obtain the average metric dependent on prior information. This metric is the measure of complexity for hypothesis space in the average. Similarly in the worst case, we obtain the minimum metric. We make clear the relationship between these measures and the Vapnik - Chervonenkis (VC) dimension. Finally, we show the upper bound on sample complexity utilizing the metric. This results can be applied in the discussion on the learnability of the class with an infinite VC dimension.",
    author = "Makoto Nakazawa and Toshiyuki Kohnosu and Toshiyasu Matsushima and Shigeichi Hirasawa",
    year = "1994",
    language = "English",
    volume = "1",
    pages = "132--137",
    booktitle = "Proceedings of the IEEE International Conference on Systems, Man and Cybernetics",
    publisher = "IEEE",

    }

    TY - GEN

    T1 - On the complexity of hypothesis space and the sample complexity for machine learning

    AU - Nakazawa, Makoto

    AU - Kohnosu, Toshiyuki

    AU - Matsushima, Toshiyasu

    AU - Hirasawa, Shigeichi

    PY - 1994

    Y1 - 1994

    N2 - The problem of learning a concept from examples in the model introduced by Valiant is discussed. According to the traditional ways of thinking, it is assumed that the learnability is independent of the occurrence probability of instance. By utilizing this probability, we propose the metric as a new measure to determine the complexity of hypothesis space. The metric measures the hardness of discrimination between hypotheses. Furthermore, we obtain the average metric dependent on prior information. This metric is the measure of complexity for hypothesis space in the average. Similarly in the worst case, we obtain the minimum metric. We make clear the relationship between these measures and the Vapnik - Chervonenkis (VC) dimension. Finally, we show the upper bound on sample complexity utilizing the metric. This results can be applied in the discussion on the learnability of the class with an infinite VC dimension.

    AB - The problem of learning a concept from examples in the model introduced by Valiant is discussed. According to the traditional ways of thinking, it is assumed that the learnability is independent of the occurrence probability of instance. By utilizing this probability, we propose the metric as a new measure to determine the complexity of hypothesis space. The metric measures the hardness of discrimination between hypotheses. Furthermore, we obtain the average metric dependent on prior information. This metric is the measure of complexity for hypothesis space in the average. Similarly in the worst case, we obtain the minimum metric. We make clear the relationship between these measures and the Vapnik - Chervonenkis (VC) dimension. Finally, we show the upper bound on sample complexity utilizing the metric. This results can be applied in the discussion on the learnability of the class with an infinite VC dimension.

    UR - http://www.scopus.com/inward/record.url?scp=0028726019&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=0028726019&partnerID=8YFLogxK

    M3 - Conference contribution

    VL - 1

    SP - 132

    EP - 137

    BT - Proceedings of the IEEE International Conference on Systems, Man and Cybernetics

    PB - IEEE

    ER -