Development of an analytic rating scale for Japanese L1 writing

The present study developed an analytic rating scale for Japanese university students' first language (L1) expository writing. Research proceeded in four stages. In Stage 1, we devised a questionnaire to investigate Japanese L1 teachers' criteria for evaluating expository writing. A total of 102 teachers responded to the questionnaire, and rated the 35 related descriptions according to their importance. In Stage 2, based on the questionnaire results, we either eliminated or reorganized these 35 descriptions under six analytic criteria: 1) Clarity of the theme; 2) Appeal to the readers; 3) Expression; 4) Organization; 5) Knowledge of language forms; and 6) Social awareness. Then, in Stage 3, we attempted to investigate possible weighting of these six criteria. We asked 106 teachers to rate two to four compositions with varied profiles using both a holistic scale and the obtained analytic scale. The results showed that the explanatory power of each criterion can vary from composition to composition, and thus we concluded that the six criteria should have equal weighting. Finally, in Stage 4, we pilot-tested the obtained scale using a new set of 69 compositions. The results indicate that the present scale is both valid and reliable, and that it is superior to a traditional analytic scale in capturing such composition qualities as appeal to the readers and social awareness.