Back-channel feedback generation using linguistic and nonlinguistic information and its application to spoken dialogue system

Shinya Fujie, Kenta Fukushima, Tetsunori Kobayashi

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    14 Citations (Scopus)

    Abstract

    A conversational system which can generate back-channel feedback of proper content in proper timing by utilizing FST based early detectable decoder and prosody analysis is proposed. In human conversation, we do not take turns in order, but we give the back-channel feedbacks during the partner's speech. By receiving these feedbacks, speakers can know the partner's state and feel comfortable to speak. Therefore, spoken dialogue systems should be able to generate back-channel feedbacks in synchronization with user's utterances. The appropriateness of these feedbacks depends on the contents and the timings. The contents strongly depend on the contents of the dialogue partner's utterance, and the timings strongly depend on the prosody of the partner's utterance. In order to determine the content of the feedback earlier than the end of the utterance, we use finite state transducer based speech recognizer. We used prosody information, especially F0 and power of the utterance, to extract the proper timing of the feedback. We implemented these modules and applied them to the spoken dialogue system on the humanoid robot ROBISUKE. Experimental results show the effectiveness of our methods.

    Original languageEnglish
    Title of host publication9th European Conference on Speech Communication and Technology
    Pages889-892
    Number of pages4
    Publication statusPublished - 2005
    Event9th European Conference on Speech Communication and Technology - Lisbon
    Duration: 2005 Sep 42005 Sep 8

    Other

    Other9th European Conference on Speech Communication and Technology
    CityLisbon
    Period05/9/405/9/8

    Fingerprint

    Linguistics
    Feedback
    Transducers
    Synchronization
    Robots

    ASJC Scopus subject areas

    • Engineering(all)

    Cite this

    Fujie, S., Fukushima, K., & Kobayashi, T. (2005). Back-channel feedback generation using linguistic and nonlinguistic information and its application to spoken dialogue system. In 9th European Conference on Speech Communication and Technology (pp. 889-892)

    Back-channel feedback generation using linguistic and nonlinguistic information and its application to spoken dialogue system. / Fujie, Shinya; Fukushima, Kenta; Kobayashi, Tetsunori.

    9th European Conference on Speech Communication and Technology. 2005. p. 889-892.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Fujie, S, Fukushima, K & Kobayashi, T 2005, Back-channel feedback generation using linguistic and nonlinguistic information and its application to spoken dialogue system. in 9th European Conference on Speech Communication and Technology. pp. 889-892, 9th European Conference on Speech Communication and Technology, Lisbon, 05/9/4.
    Fujie S, Fukushima K, Kobayashi T. Back-channel feedback generation using linguistic and nonlinguistic information and its application to spoken dialogue system. In 9th European Conference on Speech Communication and Technology. 2005. p. 889-892
    Fujie, Shinya ; Fukushima, Kenta ; Kobayashi, Tetsunori. / Back-channel feedback generation using linguistic and nonlinguistic information and its application to spoken dialogue system. 9th European Conference on Speech Communication and Technology. 2005. pp. 889-892
    @inproceedings{f532d8a4df2f4704b09444a2be8ac052,
    title = "Back-channel feedback generation using linguistic and nonlinguistic information and its application to spoken dialogue system",
    abstract = "A conversational system which can generate back-channel feedback of proper content in proper timing by utilizing FST based early detectable decoder and prosody analysis is proposed. In human conversation, we do not take turns in order, but we give the back-channel feedbacks during the partner's speech. By receiving these feedbacks, speakers can know the partner's state and feel comfortable to speak. Therefore, spoken dialogue systems should be able to generate back-channel feedbacks in synchronization with user's utterances. The appropriateness of these feedbacks depends on the contents and the timings. The contents strongly depend on the contents of the dialogue partner's utterance, and the timings strongly depend on the prosody of the partner's utterance. In order to determine the content of the feedback earlier than the end of the utterance, we use finite state transducer based speech recognizer. We used prosody information, especially F0 and power of the utterance, to extract the proper timing of the feedback. We implemented these modules and applied them to the spoken dialogue system on the humanoid robot ROBISUKE. Experimental results show the effectiveness of our methods.",
    author = "Shinya Fujie and Kenta Fukushima and Tetsunori Kobayashi",
    year = "2005",
    language = "English",
    pages = "889--892",
    booktitle = "9th European Conference on Speech Communication and Technology",

    }

    TY - GEN

    T1 - Back-channel feedback generation using linguistic and nonlinguistic information and its application to spoken dialogue system

    AU - Fujie, Shinya

    AU - Fukushima, Kenta

    AU - Kobayashi, Tetsunori

    PY - 2005

    Y1 - 2005

    N2 - A conversational system which can generate back-channel feedback of proper content in proper timing by utilizing FST based early detectable decoder and prosody analysis is proposed. In human conversation, we do not take turns in order, but we give the back-channel feedbacks during the partner's speech. By receiving these feedbacks, speakers can know the partner's state and feel comfortable to speak. Therefore, spoken dialogue systems should be able to generate back-channel feedbacks in synchronization with user's utterances. The appropriateness of these feedbacks depends on the contents and the timings. The contents strongly depend on the contents of the dialogue partner's utterance, and the timings strongly depend on the prosody of the partner's utterance. In order to determine the content of the feedback earlier than the end of the utterance, we use finite state transducer based speech recognizer. We used prosody information, especially F0 and power of the utterance, to extract the proper timing of the feedback. We implemented these modules and applied them to the spoken dialogue system on the humanoid robot ROBISUKE. Experimental results show the effectiveness of our methods.

    AB - A conversational system which can generate back-channel feedback of proper content in proper timing by utilizing FST based early detectable decoder and prosody analysis is proposed. In human conversation, we do not take turns in order, but we give the back-channel feedbacks during the partner's speech. By receiving these feedbacks, speakers can know the partner's state and feel comfortable to speak. Therefore, spoken dialogue systems should be able to generate back-channel feedbacks in synchronization with user's utterances. The appropriateness of these feedbacks depends on the contents and the timings. The contents strongly depend on the contents of the dialogue partner's utterance, and the timings strongly depend on the prosody of the partner's utterance. In order to determine the content of the feedback earlier than the end of the utterance, we use finite state transducer based speech recognizer. We used prosody information, especially F0 and power of the utterance, to extract the proper timing of the feedback. We implemented these modules and applied them to the spoken dialogue system on the humanoid robot ROBISUKE. Experimental results show the effectiveness of our methods.

    UR - http://www.scopus.com/inward/record.url?scp=33745216252&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=33745216252&partnerID=8YFLogxK

    M3 - Conference contribution

    AN - SCOPUS:33745216252

    SP - 889

    EP - 892

    BT - 9th European Conference on Speech Communication and Technology

    ER -