Conducting laboratory experiments properly with statistical tools: An easy hands-on tutorial

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    3 Citations (Scopus)

    Abstract

    This hands-on half-day tutorial consists of two 90-minute sessions. Part I covers the following topics: paired and two-sample t -tests, confidence intervals (with Excel and R); familywise error rate, multiple comparison procedures; ANOVA (with Excel and R); Tukey's HSD test, simultaneous confidence intervals (with R). Part II covers the following topics: randomised Tukey HSD test (with Discpower); what's wrong with statistical significance tests?; effect sizes, statistical power; topic set size design (with Excel); power analysis (with R); summary: how to report your results. Participants should have some prior knowledge about the very basics of statistical significance testing and are strongly encouraged to bring a laptop with R already installed. The tutorial participants will be able to design and conduct statistical significance tests for comparing the mean effectiveness scores of two or more systems appropriately, and to report on the test results in an informative manner.

    Original languageEnglish
    Title of host publication41st International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2018
    PublisherAssociation for Computing Machinery, Inc
    Pages1369-1370
    Number of pages2
    ISBN (Electronic)9781450356572
    DOIs
    Publication statusPublished - 2018 Jun 27
    Event41st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2018 - Ann Arbor, United States
    Duration: 2018 Jul 82018 Jul 12

    Other

    Other41st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2018
    CountryUnited States
    CityAnn Arbor
    Period18/7/818/7/12

    Fingerprint

    Statistical tests
    Analysis of variance (ANOVA)
    Experiments
    Testing

    Keywords

    • Confidence intervals
    • Effect sizes
    • Multiple comparison procedures
    • Randomisation test
    • Sample sizes
    • Statistical power
    • Statistical significance
    • T-test
    • Tukey's honestly significant difference test

    ASJC Scopus subject areas

    • Software
    • Computer Graphics and Computer-Aided Design
    • Information Systems

    Cite this

    Sakai, T. (2018). Conducting laboratory experiments properly with statistical tools: An easy hands-on tutorial. In 41st International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2018 (pp. 1369-1370). Association for Computing Machinery, Inc. https://doi.org/10.1145/3209978.3210182

    Conducting laboratory experiments properly with statistical tools : An easy hands-on tutorial. / Sakai, Tetsuya.

    41st International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2018. Association for Computing Machinery, Inc, 2018. p. 1369-1370.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Sakai, T 2018, Conducting laboratory experiments properly with statistical tools: An easy hands-on tutorial. in 41st International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2018. Association for Computing Machinery, Inc, pp. 1369-1370, 41st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2018, Ann Arbor, United States, 18/7/8. https://doi.org/10.1145/3209978.3210182
    Sakai T. Conducting laboratory experiments properly with statistical tools: An easy hands-on tutorial. In 41st International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2018. Association for Computing Machinery, Inc. 2018. p. 1369-1370 https://doi.org/10.1145/3209978.3210182
    Sakai, Tetsuya. / Conducting laboratory experiments properly with statistical tools : An easy hands-on tutorial. 41st International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2018. Association for Computing Machinery, Inc, 2018. pp. 1369-1370
    @inproceedings{0cbeafe1fadc44e89e167419d633994a,
    title = "Conducting laboratory experiments properly with statistical tools: An easy hands-on tutorial",
    abstract = "This hands-on half-day tutorial consists of two 90-minute sessions. Part I covers the following topics: paired and two-sample t -tests, confidence intervals (with Excel and R); familywise error rate, multiple comparison procedures; ANOVA (with Excel and R); Tukey's HSD test, simultaneous confidence intervals (with R). Part II covers the following topics: randomised Tukey HSD test (with Discpower); what's wrong with statistical significance tests?; effect sizes, statistical power; topic set size design (with Excel); power analysis (with R); summary: how to report your results. Participants should have some prior knowledge about the very basics of statistical significance testing and are strongly encouraged to bring a laptop with R already installed. The tutorial participants will be able to design and conduct statistical significance tests for comparing the mean effectiveness scores of two or more systems appropriately, and to report on the test results in an informative manner.",
    keywords = "Confidence intervals, Effect sizes, Multiple comparison procedures, Randomisation test, Sample sizes, Statistical power, Statistical significance, T-test, Tukey's honestly significant difference test",
    author = "Tetsuya Sakai",
    year = "2018",
    month = "6",
    day = "27",
    doi = "10.1145/3209978.3210182",
    language = "English",
    pages = "1369--1370",
    booktitle = "41st International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2018",
    publisher = "Association for Computing Machinery, Inc",

    }

    TY - GEN

    T1 - Conducting laboratory experiments properly with statistical tools

    T2 - An easy hands-on tutorial

    AU - Sakai, Tetsuya

    PY - 2018/6/27

    Y1 - 2018/6/27

    N2 - This hands-on half-day tutorial consists of two 90-minute sessions. Part I covers the following topics: paired and two-sample t -tests, confidence intervals (with Excel and R); familywise error rate, multiple comparison procedures; ANOVA (with Excel and R); Tukey's HSD test, simultaneous confidence intervals (with R). Part II covers the following topics: randomised Tukey HSD test (with Discpower); what's wrong with statistical significance tests?; effect sizes, statistical power; topic set size design (with Excel); power analysis (with R); summary: how to report your results. Participants should have some prior knowledge about the very basics of statistical significance testing and are strongly encouraged to bring a laptop with R already installed. The tutorial participants will be able to design and conduct statistical significance tests for comparing the mean effectiveness scores of two or more systems appropriately, and to report on the test results in an informative manner.

    AB - This hands-on half-day tutorial consists of two 90-minute sessions. Part I covers the following topics: paired and two-sample t -tests, confidence intervals (with Excel and R); familywise error rate, multiple comparison procedures; ANOVA (with Excel and R); Tukey's HSD test, simultaneous confidence intervals (with R). Part II covers the following topics: randomised Tukey HSD test (with Discpower); what's wrong with statistical significance tests?; effect sizes, statistical power; topic set size design (with Excel); power analysis (with R); summary: how to report your results. Participants should have some prior knowledge about the very basics of statistical significance testing and are strongly encouraged to bring a laptop with R already installed. The tutorial participants will be able to design and conduct statistical significance tests for comparing the mean effectiveness scores of two or more systems appropriately, and to report on the test results in an informative manner.

    KW - Confidence intervals

    KW - Effect sizes

    KW - Multiple comparison procedures

    KW - Randomisation test

    KW - Sample sizes

    KW - Statistical power

    KW - Statistical significance

    KW - T-test

    KW - Tukey's honestly significant difference test

    UR - http://www.scopus.com/inward/record.url?scp=85051473247&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85051473247&partnerID=8YFLogxK

    U2 - 10.1145/3209978.3210182

    DO - 10.1145/3209978.3210182

    M3 - Conference contribution

    AN - SCOPUS:85051473247

    SP - 1369

    EP - 1370

    BT - 41st International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2018

    PB - Association for Computing Machinery, Inc

    ER -