Master - Lab - Status Quo. How is effect size reported in CHI publications?

Supervisor: Anna-Marie (

In quantitative analysis, researchers investigate the relationship between one or more independent, or predictor variables, and one or more dependent or outcome variable(s) [1]. Effect sizes measure the strength of the relationship of independent variables with dependent variables [1] or, when comparing two or more groups, the size of the difference between the groups [2]. However, effect sizes are often not or not sufficiently reported in scientific publications [3,4]. In this lab, you should determine the status quo of effect size reporting in CHI publications.

Your tasks

  • Choose a sample of quantitative CHI-Papers, i.e. papers with hypothesis tests in them. Sampling possibilities could be a random sample, or only papers with an award.
  • For each hypothesis test in the paper, determine and note:
    • Was an effect size reported (Which type? Standardized or unstandardized?)
    • Was a judgment of the size of the effect size reported (e.g. small, medium, large…)
    • Was the effect size interpreted further?
  • Use qualitative analysis methods to identify strategies for effect size reporting and interpretation.

This lab works well for a group. Each group member would analyze their own papers.

Literature to start with:

Papers to analyze:


It is helpful to have some basic knowledge of inferential statistical testing / null hypothesis statistical testing. English slides and German videos for an introduction, as well as an introduction to qualitative analysis can be provided.