Volume 2 — Analysis of Quantitative Ratings
Volume 2 covers the intraclass correlation coefficient (ICC) and related methods for quantitative ratings — one-factor, random-factorial, and mixed-factorial designs, Finn's coefficient, measures of association and concordance, and multivariate inter-rater reliability — along with the statistical inference and sample-size calculations needed to use them correctly.
This edition splits the previous single-volume Handbook into two focused books: Volume 1 for categorical ratings, Volume 2 for quantitative ratings and intraclass correlation. New to this volume is a full chapter on structuring a ratings database before analysis, an expanded treatment of sample-size and power calculations for every ICC design, a new chapter on Finn's coefficient, and a new multivariate approach to inter-rater reliability based on principal component analysis.
Nine chapters, free
Full PDF excerpts covering the book's foundational and applied chapters, exactly as they appear in the book.
Introduction
What inter-rater reliability means for quantitative ratings, how to scope and design a reliability experiment, and how to choose the right ICC-based method for your data.
Download PDF ↓Setting Up a Database of Ratings for Analysis
New to the 5th edition. Practical guidance on organizing quantitative rating data — wide and long formats — before any coefficient is computed.
Download PDF ↓Intraclass Correlation: A Measure of Rater Agreement
The statistical framework behind the ICC — the ANOVA models it's built on, the Bland-Altman plot, sample size, and the multivariate ICC.
Download PDF ↓Intraclass Correlations in One-Factor Studies
Computing the ICC and its precision under Models 1A and 1B, with confidence intervals, p-values, and sample-size calculations for single-factor designs.
Download PDF ↓Intraclass Correlations under the Random Factorial Design
Inter- and intra-rater reliability coefficients when both rater and subject factors are random, with statistical inference and sample-size methods.
Download PDF ↓Intraclass Correlations under the Mixed Factorial Design
ICC methods for a fixed group of raters and a random sample of subjects, including confidence intervals, p-values, and optimal sample size.
Download PDF ↓Finn's Coefficient of Reliability
New to the 5th edition. An alternative to the ICC for small or homogeneous subject populations, where the traditional ICC approach can fail.
Download PDF ↓Measures of Association and Concordance
Pearson and Spearman correlation, Kendall's Tau and coefficient of concordance, and Lin's concordance correlation coefficient — with their advantages and limits.
Download PDF ↓Intraclass Correlation & Multivariate Analysis
Benchmarking the ICC, influence analysis to spot problem raters, and a new multivariate approach to inter-rater reliability based on principal component analysis.
Download PDF ↓Full table of contents
- 1 Introduction1
- 2 Setting Up a Database of Ratings for Analysis3
- 3 Intraclass Correlation: A Measure of Rater Agreement47
- 4 Intraclass Correlations in One-Factor Studies67
- 5 Intraclass Correlations under the Random Factorial Design115
- 6 Intraclass Correlations under the Mixed Factorial Design174
- 7 Finn's Coefficient of Reliability225
- 8 Measures of Association and Concordance239
- 9 Intraclass Correlation & Multivariate Analysis264
- A Data Tables295
Looking for categorical ratings instead?
Volume 1 covers chance-corrected agreement coefficients — Cohen's Kappa, Gwet's AC1/AC2, Krippendorff's alpha, and related measures — the companion to this book for researchers working with categories rather than continuous measurements.
See Volume 1