Software for inter-rater reliability analysis

Every chance-corrected agreement coefficient and intraclass correlation coefficient in the Handbook is implemented and ready to use — in a spreadsheet, in the browser, or in your language of choice.

MS Excel ↓ Cloud ↓ R ↓ SAS ↓ Stata & Python ↓

AgreeStat360 for Excel/Windows interface

01 · Desktop

AgreeStat360 for MS Excel / Windows

Point-and-click reliability analysis inside the spreadsheet your co-authors and reviewers already trust. No coding, no separate stats package — enter your ratings, pick a coefficient, and AgreeStat360 does the rest.

Cohen's Kappa

One-way random ANOVA ICC

Gwet's AC1 / AC2

Two-way random ANOVA ICC

Krippendorff's alpha

Two-way mixed ANOVA ICC

Download the free standard version Regional-settings fix ↗

02 · Browser-based

AgreeStat360 / Cloud-Based

The same analysis, no install required. Two tools, both free to try from a browser tab.

Reliability analysis

AgreeStat360 / Cloud

Register for a free trial at agreestat360.com. Any paid AgreeStat360 license key also unlocks the Excel/Windows version — one license, both platforms.

Visit agreestat360.com

Significance testing

AgreeTest / Cloud

Test whether the difference between two agreement coefficients is statistically significant, entirely in the browser — no spreadsheet setup needed.

Check out AgreeTest

03 · Open source

R packages & R functions

For researchers who want reliability analysis inside a reproducible script rather than a spreadsheet. AgreeStat maintains a set of R functions that compute the same chance-corrected agreement coefficients and intraclass correlation coefficients documented in the Handbook — built to match the book's formulas exactly, so a result in R and a result in AgreeStat360 will always agree.

Appendix B of the Handbook reviews the general-purpose R packages available for inter-rater reliability work and shows where AgreeStat's own functions fill in the gaps — particularly for Gwet's AC1/AC2 and for benchmarking coefficient magnitude.

Download the R functions See Appendix B in the Handbook

EXAMPLE CALL

Gwet's AC1 on a rating matrix

# ratings: subjects x raters matrix

library(agreestatR)

result <- gwet_ac1(

  data = ratings,

  weights = "quadratic"

)

print(result$coefficient)

print(result$conf_interval)

SAS

EXAMPLE MACRO

Kappa with a custom weight matrix

* dataset RATINGS has one row per subject;

%agree_kappa(

  data = ratings,

  vars = rater1 rater2,

  weights = quadratic

);

* output: kappa, se, 95% CI;

04 · Enterprise / academic

Programs in SAS

SAS macros implementing the chance-corrected agreement coefficients and intraclass correlation coefficients covered in the Handbook, for research groups and government agencies standardized on SAS.

The macros accept the same rating-matrix or long-format datasets described in Chapter 2, and return the coefficient alongside its standard error and confidence interval — matching the statistical inference framework laid out in Chapter 6.

Download the SAS macros

05 · Also available

Stata & Python

Community-built packages for teams standardized on Stata or Python, recommended alongside AgreeStat's own tools.

STA

KAPPAETC

A Stata module covering a wide range of agreement coefficients — including AC1, Kappa, and Krippendorff's alpha — with standard errors and confidence intervals, all installable from within Stata.

Install KAPPAETC ↗ Read the Stata Journal article ↗

irrCAC (Python)

A Python port of Dr. Gwet's chance-corrected agreement coefficients, maintained by Aris Fergadis on GitHub, for researchers who'd rather stay in a Python workflow than switch tools.

View irrCAC on GitHub ↗