Conducting laboratory experiments properly with statistical tools

An easy hands-on tutorial

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    This hands-on half-day tutorial consists of two sessions. Part I covers the following topics: Preliminaries; Paired and two-sample t-tests, confidence intervals; One-way ANOVA and two-way ANOVA without replication; Familiwise error rate. Part II covers the following topics: Tukey's HSD test, simultaneous confidence intervals; Randomisation test and randomised Tukey HSD test; What's wrong with statistical significance tests?; Effect sizes, statistical power; Topic set size design and power analysis; Summary: how to report your results. Participants should have some prior knowledge about the very basics of statistical significance testing and are strongly encouraged to bring a laptop with R already installed. They will learn how to design and conduct statistical significance tests for comparing the mean effectiveness scores of two or more systems appropriately, and to report on the test results in an informative manner.

    Original languageEnglish
    Title of host publicationWSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining
    PublisherAssociation for Computing Machinery, Inc
    Pages830-831
    Number of pages2
    ISBN (Electronic)9781450359405
    DOIs
    Publication statusPublished - 2019 Jan 30
    Event12th ACM International Conference on Web Search and Data Mining, WSDM 2019 - Melbourne, Australia
    Duration: 2019 Feb 112019 Feb 15

    Publication series

    NameWSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining

    Conference

    Conference12th ACM International Conference on Web Search and Data Mining, WSDM 2019
    CountryAustralia
    CityMelbourne
    Period19/2/1119/2/15

    Fingerprint

    Statistical tests
    Analysis of variance (ANOVA)
    Experiments
    Testing

    Keywords

    • Confidence intervals
    • Effect sizes
    • Multiple comparison procedures
    • Randomisation test
    • Sample sizes
    • Statistical power
    • Statistical significance
    • T-test
    • Tukey's honestly significant difference test

    ASJC Scopus subject areas

    • Computer Networks and Communications
    • Software
    • Computer Science Applications

    Cite this

    Sakai, T. (2019). Conducting laboratory experiments properly with statistical tools: An easy hands-on tutorial. In WSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining (pp. 830-831). (WSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining). Association for Computing Machinery, Inc. https://doi.org/10.1145/3289600.3291378

    Conducting laboratory experiments properly with statistical tools : An easy hands-on tutorial. / Sakai, Tetsuya.

    WSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining. Association for Computing Machinery, Inc, 2019. p. 830-831 (WSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining).

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Sakai, T 2019, Conducting laboratory experiments properly with statistical tools: An easy hands-on tutorial. in WSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining. WSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining, Association for Computing Machinery, Inc, pp. 830-831, 12th ACM International Conference on Web Search and Data Mining, WSDM 2019, Melbourne, Australia, 19/2/11. https://doi.org/10.1145/3289600.3291378
    Sakai T. Conducting laboratory experiments properly with statistical tools: An easy hands-on tutorial. In WSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining. Association for Computing Machinery, Inc. 2019. p. 830-831. (WSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining). https://doi.org/10.1145/3289600.3291378
    Sakai, Tetsuya. / Conducting laboratory experiments properly with statistical tools : An easy hands-on tutorial. WSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining. Association for Computing Machinery, Inc, 2019. pp. 830-831 (WSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining).
    @inproceedings{5d874a575ea84356b72adf1d295aa52e,
    title = "Conducting laboratory experiments properly with statistical tools: An easy hands-on tutorial",
    abstract = "This hands-on half-day tutorial consists of two sessions. Part I covers the following topics: Preliminaries; Paired and two-sample t-tests, confidence intervals; One-way ANOVA and two-way ANOVA without replication; Familiwise error rate. Part II covers the following topics: Tukey's HSD test, simultaneous confidence intervals; Randomisation test and randomised Tukey HSD test; What's wrong with statistical significance tests?; Effect sizes, statistical power; Topic set size design and power analysis; Summary: how to report your results. Participants should have some prior knowledge about the very basics of statistical significance testing and are strongly encouraged to bring a laptop with R already installed. They will learn how to design and conduct statistical significance tests for comparing the mean effectiveness scores of two or more systems appropriately, and to report on the test results in an informative manner.",
    keywords = "Confidence intervals, Effect sizes, Multiple comparison procedures, Randomisation test, Sample sizes, Statistical power, Statistical significance, T-test, Tukey's honestly significant difference test",
    author = "Tetsuya Sakai",
    year = "2019",
    month = "1",
    day = "30",
    doi = "10.1145/3289600.3291378",
    language = "English",
    series = "WSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining",
    publisher = "Association for Computing Machinery, Inc",
    pages = "830--831",
    booktitle = "WSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining",

    }

    TY - GEN

    T1 - Conducting laboratory experiments properly with statistical tools

    T2 - An easy hands-on tutorial

    AU - Sakai, Tetsuya

    PY - 2019/1/30

    Y1 - 2019/1/30

    N2 - This hands-on half-day tutorial consists of two sessions. Part I covers the following topics: Preliminaries; Paired and two-sample t-tests, confidence intervals; One-way ANOVA and two-way ANOVA without replication; Familiwise error rate. Part II covers the following topics: Tukey's HSD test, simultaneous confidence intervals; Randomisation test and randomised Tukey HSD test; What's wrong with statistical significance tests?; Effect sizes, statistical power; Topic set size design and power analysis; Summary: how to report your results. Participants should have some prior knowledge about the very basics of statistical significance testing and are strongly encouraged to bring a laptop with R already installed. They will learn how to design and conduct statistical significance tests for comparing the mean effectiveness scores of two or more systems appropriately, and to report on the test results in an informative manner.

    AB - This hands-on half-day tutorial consists of two sessions. Part I covers the following topics: Preliminaries; Paired and two-sample t-tests, confidence intervals; One-way ANOVA and two-way ANOVA without replication; Familiwise error rate. Part II covers the following topics: Tukey's HSD test, simultaneous confidence intervals; Randomisation test and randomised Tukey HSD test; What's wrong with statistical significance tests?; Effect sizes, statistical power; Topic set size design and power analysis; Summary: how to report your results. Participants should have some prior knowledge about the very basics of statistical significance testing and are strongly encouraged to bring a laptop with R already installed. They will learn how to design and conduct statistical significance tests for comparing the mean effectiveness scores of two or more systems appropriately, and to report on the test results in an informative manner.

    KW - Confidence intervals

    KW - Effect sizes

    KW - Multiple comparison procedures

    KW - Randomisation test

    KW - Sample sizes

    KW - Statistical power

    KW - Statistical significance

    KW - T-test

    KW - Tukey's honestly significant difference test

    UR - http://www.scopus.com/inward/record.url?scp=85061708391&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85061708391&partnerID=8YFLogxK

    U2 - 10.1145/3289600.3291378

    DO - 10.1145/3289600.3291378

    M3 - Conference contribution

    T3 - WSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining

    SP - 830

    EP - 831

    BT - WSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining

    PB - Association for Computing Machinery, Inc

    ER -