Fair assessment of group work by mutual evaluation based on trust network

Yumeno Shiba, Toshiharu Sugawara

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    3 Citations (Scopus)

    Abstract

    We propose a method for fair and accurate assessment of group work based on trust networks generated by mutual evaluations. Group work is often used for educational activities in universities since it is an effective way to acquire useful knowledge in a number of practical subjects. One drawback is the difficulty of deciding on final marks. Some students may work quite hard whereas others may rarely participate in the group work, but it is almost impossible for professors/instructors to identify contributions of individual students in detail. In contrast, students in the same group are obvious choices for appropriate evaluators of other members since they have first-hand knowledge of the collaborative work. However, some students may be irresponsible for their ratings and submit disputable evaluations, resulting in inaccurate marks. We introduce a simple mutual evaluation method and generate trust networks expressing the distances between evaluations in this paper. After that, disputable evaluations are excluded and students are marked again. We also examine a grouping strategy to detect irresponsible students more accurately. We demonstrate the effectiveness and limitations of our method using multi-agent simulation. Results show that our method can help with the marking of individual students in a group work.

    Original languageEnglish
    Title of host publicationProceedings - Frontiers in Education Conference, FIE
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    Volume2015-February
    EditionFebruary
    DOIs
    Publication statusPublished - 2015 Feb 17
    Event44th Annual Frontiers in Education Conference, FIE 2014 - Madrid, Spain
    Duration: 2014 Oct 222014 Oct 25

    Other

    Other44th Annual Frontiers in Education Conference, FIE 2014
    CountrySpain
    CityMadrid
    Period14/10/2214/10/25

    Fingerprint

    group work
    Students
    evaluation
    student
    educational activities
    grouping
    instructor
    university teacher
    rating
    simulation
    university
    Group

    Keywords

    • Cooperative learning
    • Fair assessment
    • Mutual evaluation
    • Trust network

    ASJC Scopus subject areas

    • Computer Science Applications
    • Software
    • Education

    Cite this

    Shiba, Y., & Sugawara, T. (2015). Fair assessment of group work by mutual evaluation based on trust network. In Proceedings - Frontiers in Education Conference, FIE (February ed., Vol. 2015-February). [7044121] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/FIE.2014.7044121

    Fair assessment of group work by mutual evaluation based on trust network. / Shiba, Yumeno; Sugawara, Toshiharu.

    Proceedings - Frontiers in Education Conference, FIE. Vol. 2015-February February. ed. Institute of Electrical and Electronics Engineers Inc., 2015. 7044121.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Shiba, Y & Sugawara, T 2015, Fair assessment of group work by mutual evaluation based on trust network. in Proceedings - Frontiers in Education Conference, FIE. February edn, vol. 2015-February, 7044121, Institute of Electrical and Electronics Engineers Inc., 44th Annual Frontiers in Education Conference, FIE 2014, Madrid, Spain, 14/10/22. https://doi.org/10.1109/FIE.2014.7044121
    Shiba Y, Sugawara T. Fair assessment of group work by mutual evaluation based on trust network. In Proceedings - Frontiers in Education Conference, FIE. February ed. Vol. 2015-February. Institute of Electrical and Electronics Engineers Inc. 2015. 7044121 https://doi.org/10.1109/FIE.2014.7044121
    Shiba, Yumeno ; Sugawara, Toshiharu. / Fair assessment of group work by mutual evaluation based on trust network. Proceedings - Frontiers in Education Conference, FIE. Vol. 2015-February February. ed. Institute of Electrical and Electronics Engineers Inc., 2015.
    @inproceedings{64f655e5e9c74d4c97522459b09db51e,
    title = "Fair assessment of group work by mutual evaluation based on trust network",
    abstract = "We propose a method for fair and accurate assessment of group work based on trust networks generated by mutual evaluations. Group work is often used for educational activities in universities since it is an effective way to acquire useful knowledge in a number of practical subjects. One drawback is the difficulty of deciding on final marks. Some students may work quite hard whereas others may rarely participate in the group work, but it is almost impossible for professors/instructors to identify contributions of individual students in detail. In contrast, students in the same group are obvious choices for appropriate evaluators of other members since they have first-hand knowledge of the collaborative work. However, some students may be irresponsible for their ratings and submit disputable evaluations, resulting in inaccurate marks. We introduce a simple mutual evaluation method and generate trust networks expressing the distances between evaluations in this paper. After that, disputable evaluations are excluded and students are marked again. We also examine a grouping strategy to detect irresponsible students more accurately. We demonstrate the effectiveness and limitations of our method using multi-agent simulation. Results show that our method can help with the marking of individual students in a group work.",
    keywords = "Cooperative learning, Fair assessment, Mutual evaluation, Trust network",
    author = "Yumeno Shiba and Toshiharu Sugawara",
    year = "2015",
    month = "2",
    day = "17",
    doi = "10.1109/FIE.2014.7044121",
    language = "English",
    volume = "2015-February",
    booktitle = "Proceedings - Frontiers in Education Conference, FIE",
    publisher = "Institute of Electrical and Electronics Engineers Inc.",
    edition = "February",

    }

    TY - GEN

    T1 - Fair assessment of group work by mutual evaluation based on trust network

    AU - Shiba, Yumeno

    AU - Sugawara, Toshiharu

    PY - 2015/2/17

    Y1 - 2015/2/17

    N2 - We propose a method for fair and accurate assessment of group work based on trust networks generated by mutual evaluations. Group work is often used for educational activities in universities since it is an effective way to acquire useful knowledge in a number of practical subjects. One drawback is the difficulty of deciding on final marks. Some students may work quite hard whereas others may rarely participate in the group work, but it is almost impossible for professors/instructors to identify contributions of individual students in detail. In contrast, students in the same group are obvious choices for appropriate evaluators of other members since they have first-hand knowledge of the collaborative work. However, some students may be irresponsible for their ratings and submit disputable evaluations, resulting in inaccurate marks. We introduce a simple mutual evaluation method and generate trust networks expressing the distances between evaluations in this paper. After that, disputable evaluations are excluded and students are marked again. We also examine a grouping strategy to detect irresponsible students more accurately. We demonstrate the effectiveness and limitations of our method using multi-agent simulation. Results show that our method can help with the marking of individual students in a group work.

    AB - We propose a method for fair and accurate assessment of group work based on trust networks generated by mutual evaluations. Group work is often used for educational activities in universities since it is an effective way to acquire useful knowledge in a number of practical subjects. One drawback is the difficulty of deciding on final marks. Some students may work quite hard whereas others may rarely participate in the group work, but it is almost impossible for professors/instructors to identify contributions of individual students in detail. In contrast, students in the same group are obvious choices for appropriate evaluators of other members since they have first-hand knowledge of the collaborative work. However, some students may be irresponsible for their ratings and submit disputable evaluations, resulting in inaccurate marks. We introduce a simple mutual evaluation method and generate trust networks expressing the distances between evaluations in this paper. After that, disputable evaluations are excluded and students are marked again. We also examine a grouping strategy to detect irresponsible students more accurately. We demonstrate the effectiveness and limitations of our method using multi-agent simulation. Results show that our method can help with the marking of individual students in a group work.

    KW - Cooperative learning

    KW - Fair assessment

    KW - Mutual evaluation

    KW - Trust network

    UR - http://www.scopus.com/inward/record.url?scp=84938152606&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=84938152606&partnerID=8YFLogxK

    U2 - 10.1109/FIE.2014.7044121

    DO - 10.1109/FIE.2014.7044121

    M3 - Conference contribution

    AN - SCOPUS:84938152606

    VL - 2015-February

    BT - Proceedings - Frontiers in Education Conference, FIE

    PB - Institute of Electrical and Electronics Engineers Inc.

    ER -