Master - Lab - Usability evaluation of a tutorial for power analysis

Supervisor: Anna-Marie (

You can do this lab as a group.

In Usable Security and Privacy, a priori power analysis is only rarely used to calculate necessary sample sizes prior to conducting a study [1]. Reasons for this could be that the procedure of power analysis and necessary values (effectsizes) vary depending on the planned statistical test [2,3] and the terminology which is used may also differ between different tools and publications [2,3]. In our work in [1], we created tutorials for power analysis and published them on a companion website ( - first as a PDF, but they will be integrated into the website in the future.

Your task:

  • Familiarize yourself with the basic concept of power analysis
  • Usability evalution (e.g. heuristic evaluation, cognitive walkthrough) of the tutorials [4]
  • Development of improvement for the identified usability problems

Literature to start with:

  • [1] Ortloff et al. (2023). SoK: I Have the (Developer) Power! Sample Size Estimation for Fisher’s Exact, Chi-Squared, McNemar’s, Wilcoxon Rank-Sum, Wilcoxon Signed-Rank and t-tests in Developer-Centered Usable Security, available online her: , companion website:

  • [2] Faul et al (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences, tool available online here:

  • [3] Ellis (2010). The Essential Guide to Effect Sizes. Statistical Power, Meta-Analysis, and the Interpretation of Research Results. (Book can be lent from the work group if interested)

  • [4] Usability Methoden: Kapitel 14 aus Baxter et al. (2015). Understanding your users: a practical guide to user research methods. (PDF can be provided if interested)