Student placement and skill ranking predictors for programming classes using class attitude, psychological scales, and code metrics

Ryosuke Ishizue, Kazunori Sakamoto, Hironori Washizaki, Yoshiaki Fukazawa

    Research output: Contribution to journalArticle

    Abstract

    In some situations, it is necessary to measure personal programming skills. For example, often students must be divided according to skill level and motivation to learn or companies recruiting employees must rank candidates by evaluating programming skills through programming tests, programming contests, etc. This process is burdensome because teachers and recruiters must prepare, implement, and evaluate a placement examination. This paper tries to predict the placement and ranking results of programming contests via machine learning without such an examination. Explanatory variables used for machine learning are classified into three categories: Psychological Scales, Programming Tasks, and Student-answered Questionnaires. The participants are university students enrolled in a Java programming class. One target variable is the placement result based on an examination by a teacher of a class and the ranking results of the programming contest. Our best classification model with a decision tree has an F-measure of 0.912, while our best ranking model with an SVM-rank has an nDCG of 0.962. In both prediction models, the best explanatory variable is from the Programming Task followed in order by Psychological Sale and Student-answered Questionnaire. Our classification model uses 9 explanatory variables, while our ranking model uses 20 explanatory variables. These include all three types of explanatory variables. The source code complexity, which is a source code metrics from Programming Task, shows best performance when the prediction uses only one explanatory variable. Contribution (1), this method can automate some of the teacher’s workload, which may improve educational quality and increase the number of acceptable students in the course. Contribution (2), this paper shows the potential of using difficult-to-formulate information for an evaluation such as a Psychological Scale is demonstrated. These are the contributions and implications of this paper.

    Original languageEnglish
    Article number7
    JournalResearch and Practice in Technology Enhanced Learning
    Volume13
    Issue number1
    DOIs
    Publication statusPublished - 2018 Dec 1

    Fingerprint

    ranking
    programming
    Students
    Psychology
    student
    Learning systems
    Decision Trees
    Decision trees
    Computer programming
    Workload
    examination
    Motivation
    Sales
    teacher
    Placement
    Predictors
    Ranking
    Programming
    Psychological
    Personnel

    Keywords

    • Machine learning
    • Placement
    • Programming class
    • Psychological Scale

    ASJC Scopus subject areas

    • Education
    • Social Psychology
    • Media Technology
    • Management of Technology and Innovation

    Cite this

    @article{e77466237ae84c2cb8980340cbc46baf,
    title = "Student placement and skill ranking predictors for programming classes using class attitude, psychological scales, and code metrics",
    abstract = "In some situations, it is necessary to measure personal programming skills. For example, often students must be divided according to skill level and motivation to learn or companies recruiting employees must rank candidates by evaluating programming skills through programming tests, programming contests, etc. This process is burdensome because teachers and recruiters must prepare, implement, and evaluate a placement examination. This paper tries to predict the placement and ranking results of programming contests via machine learning without such an examination. Explanatory variables used for machine learning are classified into three categories: Psychological Scales, Programming Tasks, and Student-answered Questionnaires. The participants are university students enrolled in a Java programming class. One target variable is the placement result based on an examination by a teacher of a class and the ranking results of the programming contest. Our best classification model with a decision tree has an F-measure of 0.912, while our best ranking model with an SVM-rank has an nDCG of 0.962. In both prediction models, the best explanatory variable is from the Programming Task followed in order by Psychological Sale and Student-answered Questionnaire. Our classification model uses 9 explanatory variables, while our ranking model uses 20 explanatory variables. These include all three types of explanatory variables. The source code complexity, which is a source code metrics from Programming Task, shows best performance when the prediction uses only one explanatory variable. Contribution (1), this method can automate some of the teacher’s workload, which may improve educational quality and increase the number of acceptable students in the course. Contribution (2), this paper shows the potential of using difficult-to-formulate information for an evaluation such as a Psychological Scale is demonstrated. These are the contributions and implications of this paper.",
    keywords = "Machine learning, Placement, Programming class, Psychological Scale",
    author = "Ryosuke Ishizue and Kazunori Sakamoto and Hironori Washizaki and Yoshiaki Fukazawa",
    year = "2018",
    month = "12",
    day = "1",
    doi = "10.1186/s41039-018-0075-y",
    language = "English",
    volume = "13",
    journal = "Research and Practice in Technology Enhanced Learning",
    issn = "1793-7078",
    publisher = "Springer Open",
    number = "1",

    }

    TY - JOUR

    T1 - Student placement and skill ranking predictors for programming classes using class attitude, psychological scales, and code metrics

    AU - Ishizue, Ryosuke

    AU - Sakamoto, Kazunori

    AU - Washizaki, Hironori

    AU - Fukazawa, Yoshiaki

    PY - 2018/12/1

    Y1 - 2018/12/1

    N2 - In some situations, it is necessary to measure personal programming skills. For example, often students must be divided according to skill level and motivation to learn or companies recruiting employees must rank candidates by evaluating programming skills through programming tests, programming contests, etc. This process is burdensome because teachers and recruiters must prepare, implement, and evaluate a placement examination. This paper tries to predict the placement and ranking results of programming contests via machine learning without such an examination. Explanatory variables used for machine learning are classified into three categories: Psychological Scales, Programming Tasks, and Student-answered Questionnaires. The participants are university students enrolled in a Java programming class. One target variable is the placement result based on an examination by a teacher of a class and the ranking results of the programming contest. Our best classification model with a decision tree has an F-measure of 0.912, while our best ranking model with an SVM-rank has an nDCG of 0.962. In both prediction models, the best explanatory variable is from the Programming Task followed in order by Psychological Sale and Student-answered Questionnaire. Our classification model uses 9 explanatory variables, while our ranking model uses 20 explanatory variables. These include all three types of explanatory variables. The source code complexity, which is a source code metrics from Programming Task, shows best performance when the prediction uses only one explanatory variable. Contribution (1), this method can automate some of the teacher’s workload, which may improve educational quality and increase the number of acceptable students in the course. Contribution (2), this paper shows the potential of using difficult-to-formulate information for an evaluation such as a Psychological Scale is demonstrated. These are the contributions and implications of this paper.

    AB - In some situations, it is necessary to measure personal programming skills. For example, often students must be divided according to skill level and motivation to learn or companies recruiting employees must rank candidates by evaluating programming skills through programming tests, programming contests, etc. This process is burdensome because teachers and recruiters must prepare, implement, and evaluate a placement examination. This paper tries to predict the placement and ranking results of programming contests via machine learning without such an examination. Explanatory variables used for machine learning are classified into three categories: Psychological Scales, Programming Tasks, and Student-answered Questionnaires. The participants are university students enrolled in a Java programming class. One target variable is the placement result based on an examination by a teacher of a class and the ranking results of the programming contest. Our best classification model with a decision tree has an F-measure of 0.912, while our best ranking model with an SVM-rank has an nDCG of 0.962. In both prediction models, the best explanatory variable is from the Programming Task followed in order by Psychological Sale and Student-answered Questionnaire. Our classification model uses 9 explanatory variables, while our ranking model uses 20 explanatory variables. These include all three types of explanatory variables. The source code complexity, which is a source code metrics from Programming Task, shows best performance when the prediction uses only one explanatory variable. Contribution (1), this method can automate some of the teacher’s workload, which may improve educational quality and increase the number of acceptable students in the course. Contribution (2), this paper shows the potential of using difficult-to-formulate information for an evaluation such as a Psychological Scale is demonstrated. These are the contributions and implications of this paper.

    KW - Machine learning

    KW - Placement

    KW - Programming class

    KW - Psychological Scale

    UR - http://www.scopus.com/inward/record.url?scp=85062674635&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85062674635&partnerID=8YFLogxK

    U2 - 10.1186/s41039-018-0075-y

    DO - 10.1186/s41039-018-0075-y

    M3 - Article

    VL - 13

    JO - Research and Practice in Technology Enhanced Learning

    JF - Research and Practice in Technology Enhanced Learning

    SN - 1793-7078

    IS - 1

    M1 - 7

    ER -