Fair assessment of group work by mutual evaluation with irresponsible and collusive students using trust networks

Yumeno Shiba, Haruna Umegaki, Toshiharu Sugawara

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    1 Citation (Scopus)

    Abstract

    We propose a fair peer assessment method for group work using a multi-agent trust network. Although group work is an effective educational method, accurately assessing individual students is not easy. Mutual evaluation is often used for such assessment, but often presents some potential problems such as irresponsible evaluations and collusion. Our method identifies and excludes such cheating and unfair ratings on the basis of trust networks that are often used to evaluate sellers in e-market places by using customers’ ratings. We assume a group-work course in a semester in which students mutually evaluate other group members a few (three to five) times. We introduce the iterative method for alternately generating trust networks using cluster-trust values, which represent similarity of evaluations in a cluster network. We experimentally show that our method can find the irresponsible students and collusive groups and considerably improve accuracy of final marks with only a few chances for mutual evaluations, and thereby, can provide useful information for assessments to instructors.

    Original languageEnglish
    Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    PublisherSpringer Verlag
    Pages528-537
    Number of pages10
    Volume9387
    ISBN (Print)9783319255231, 9783319255231
    DOIs
    Publication statusPublished - 2015
    Event18th International Conference on Principles and Practice of Multi-Agent Systems, PRIMA 2015 - Bertinoro, Italy
    Duration: 2015 Oct 262015 Oct 30

    Publication series

    NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    Volume9387
    ISSN (Print)03029743
    ISSN (Electronic)16113349

    Other

    Other18th International Conference on Principles and Practice of Multi-Agent Systems, PRIMA 2015
    CountryItaly
    CityBertinoro
    Period15/10/2615/10/30

    Fingerprint

    Students
    Evaluation
    Iterative methods
    Unfair
    Collusion
    Potential Problems
    Evaluate
    Customers
    Iteration

    ASJC Scopus subject areas

    • Computer Science(all)
    • Theoretical Computer Science

    Cite this

    Shiba, Y., Umegaki, H., & Sugawara, T. (2015). Fair assessment of group work by mutual evaluation with irresponsible and collusive students using trust networks. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 9387, pp. 528-537). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9387). Springer Verlag. https://doi.org/10.1007/978-3-319-25524-8_35

    Fair assessment of group work by mutual evaluation with irresponsible and collusive students using trust networks. / Shiba, Yumeno; Umegaki, Haruna; Sugawara, Toshiharu.

    Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 9387 Springer Verlag, 2015. p. 528-537 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9387).

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Shiba, Y, Umegaki, H & Sugawara, T 2015, Fair assessment of group work by mutual evaluation with irresponsible and collusive students using trust networks. in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). vol. 9387, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9387, Springer Verlag, pp. 528-537, 18th International Conference on Principles and Practice of Multi-Agent Systems, PRIMA 2015, Bertinoro, Italy, 15/10/26. https://doi.org/10.1007/978-3-319-25524-8_35
    Shiba Y, Umegaki H, Sugawara T. Fair assessment of group work by mutual evaluation with irresponsible and collusive students using trust networks. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 9387. Springer Verlag. 2015. p. 528-537. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-25524-8_35
    Shiba, Yumeno ; Umegaki, Haruna ; Sugawara, Toshiharu. / Fair assessment of group work by mutual evaluation with irresponsible and collusive students using trust networks. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 9387 Springer Verlag, 2015. pp. 528-537 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
    @inproceedings{3f5c4da01d3b4417914b3358fdc7bcfb,
    title = "Fair assessment of group work by mutual evaluation with irresponsible and collusive students using trust networks",
    abstract = "We propose a fair peer assessment method for group work using a multi-agent trust network. Although group work is an effective educational method, accurately assessing individual students is not easy. Mutual evaluation is often used for such assessment, but often presents some potential problems such as irresponsible evaluations and collusion. Our method identifies and excludes such cheating and unfair ratings on the basis of trust networks that are often used to evaluate sellers in e-market places by using customers’ ratings. We assume a group-work course in a semester in which students mutually evaluate other group members a few (three to five) times. We introduce the iterative method for alternately generating trust networks using cluster-trust values, which represent similarity of evaluations in a cluster network. We experimentally show that our method can find the irresponsible students and collusive groups and considerably improve accuracy of final marks with only a few chances for mutual evaluations, and thereby, can provide useful information for assessments to instructors.",
    author = "Yumeno Shiba and Haruna Umegaki and Toshiharu Sugawara",
    year = "2015",
    doi = "10.1007/978-3-319-25524-8_35",
    language = "English",
    isbn = "9783319255231",
    volume = "9387",
    series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
    publisher = "Springer Verlag",
    pages = "528--537",
    booktitle = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

    }

    TY - GEN

    T1 - Fair assessment of group work by mutual evaluation with irresponsible and collusive students using trust networks

    AU - Shiba, Yumeno

    AU - Umegaki, Haruna

    AU - Sugawara, Toshiharu

    PY - 2015

    Y1 - 2015

    N2 - We propose a fair peer assessment method for group work using a multi-agent trust network. Although group work is an effective educational method, accurately assessing individual students is not easy. Mutual evaluation is often used for such assessment, but often presents some potential problems such as irresponsible evaluations and collusion. Our method identifies and excludes such cheating and unfair ratings on the basis of trust networks that are often used to evaluate sellers in e-market places by using customers’ ratings. We assume a group-work course in a semester in which students mutually evaluate other group members a few (three to five) times. We introduce the iterative method for alternately generating trust networks using cluster-trust values, which represent similarity of evaluations in a cluster network. We experimentally show that our method can find the irresponsible students and collusive groups and considerably improve accuracy of final marks with only a few chances for mutual evaluations, and thereby, can provide useful information for assessments to instructors.

    AB - We propose a fair peer assessment method for group work using a multi-agent trust network. Although group work is an effective educational method, accurately assessing individual students is not easy. Mutual evaluation is often used for such assessment, but often presents some potential problems such as irresponsible evaluations and collusion. Our method identifies and excludes such cheating and unfair ratings on the basis of trust networks that are often used to evaluate sellers in e-market places by using customers’ ratings. We assume a group-work course in a semester in which students mutually evaluate other group members a few (three to five) times. We introduce the iterative method for alternately generating trust networks using cluster-trust values, which represent similarity of evaluations in a cluster network. We experimentally show that our method can find the irresponsible students and collusive groups and considerably improve accuracy of final marks with only a few chances for mutual evaluations, and thereby, can provide useful information for assessments to instructors.

    UR - http://www.scopus.com/inward/record.url?scp=84950341951&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=84950341951&partnerID=8YFLogxK

    U2 - 10.1007/978-3-319-25524-8_35

    DO - 10.1007/978-3-319-25524-8_35

    M3 - Conference contribution

    AN - SCOPUS:84950341951

    SN - 9783319255231

    SN - 9783319255231

    VL - 9387

    T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

    SP - 528

    EP - 537

    BT - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

    PB - Springer Verlag

    ER -