AgreeStat Analytics
Home / Research Papers
Free to download

Research papers

Published and working papers by Kilem L. Gwet, Ph.D. on chance-corrected agreement coefficients, intraclass correlation, and inter-rater reliability study design — freely available to researchers and reviewers.

2021
PDF ↓
2018
PDF ↓
2016

Testing the Difference of Correlated Agreement Coefficients for Statistical Significance

By Kilem L. Gwet · Educational and Psychological Measurement (2016)
The methods discussed in this paper are implemented in AgreeTest, a cloud-based app you can access through your browser.
PDF ↓
2014

Quantification of Myocardial Blood Flow in Absolute Terms Using 82Rb PET Imaging: Results of the RUBY-10 Study

By Sergey V. Nesterov, Emmanuel Deshayes, et al., including Kilem L. Gwet, and Robert A. deKemp · Journal of the American College of Cardiology: Cardiovascular Imaging (2014)
PDF ↓
2013

A Comparison of Cohen's Kappa and Gwet's AC1 When Calculating Inter-Rater Reliability Coefficients: A Study Conducted With Personality Disorder Samples

By Nahathai Wongpakaran, Tinakon Wongpakaran, Danny Wedding, and Kilem L. Gwet · BMC Medical Research Methodology (2013)
PDF ↓
2011

On Krippendorff's Alpha Coefficient

By Kilem L. Gwet, Ph.D. · August 2011
PDF ↓
2008

Computing Inter-Rater Reliability and Its Variance in the Presence of High Agreement

By Kilem L. Gwet, Ph.D. · British Journal of Mathematical and Statistical Psychology (2008), 61, 29–48
PDF ↓
2008

Variance Estimation of Nominal-Scale Inter-Rater Reliability With Random Selection of Raters

By Kilem Li Gwet, Ph.D. · Psychometrika, Vol. 73, No. 3, 407–430 (September 2008)
PDF ↓
2008

Intrarater Reliability

By Kilem L. Gwet · Wiley Encyclopedia of Clinical Trials (2008)
PDF ↓
2002

Kappa Statistic Is Not Satisfactory for Assessing the Extent of Agreement Between Raters

By Kilem L. Gwet · Statistical Methods for Inter-Rater Reliability Assessment, No. 1, April 2002
PDF ↓
2002

Computing Inter-Rater Reliability With the SAS System

By Kilem L. Gwet · Statistical Methods for Inter-Rater Reliability Assessment, No. 3, October 2002
PDF ↓
2002

Inter-Rater Reliability: Dependency on Trait Prevalence and Marginal Homogeneity

By Kilem L. Gwet · Statistical Methods for Inter-Rater Reliability Assessment, No. 2, May 2002
PDF ↓
Companion materials

Additional resources & datasets

Excel walkthrough

Appendix B, worked in Excel

A step-by-step Excel walkthrough of the paired t-test method from "Testing the Difference of Correlated Agreement Coefficients," matching Appendix B of that paper cell for cell.

Download the .xlsx file
Cloud app

Test it interactively with AgreeTest

Prefer not to build the test yourself? AgreeTest runs the same correlated- and uncorrelated-coefficient significance test from your browser — no spreadsheet required.

Try AgreeTest
Commentary

Commentary: Do Not Force Agreement

A short commentary by Guangchao Charles Feng and Xinshu Zhao in response to Klaus Krippendorff's criticism of their paper in 2016.

Download the PDF
Applied study

Assessing agreement on nominal judgement variables

A working paper by Gareth McCray on assessing inter-rater agreement for nominal judgment variables.

Download the PDF