advanced analytics

Handbook of Inter-Rater Reliability
(4th Edition), by Kilem L. Gwet. Ph.D.


Handbook's Main Page Error Page  

Errors found in the fourth edition of the book

  • Page 37 (Example 2.1) - A big thank to D. Klein.
    The calculation of chance agreement is wrong. The last paragraph of Example 2.1 should read as follows:

percent chance agreement

The last sentence of paragraph following example 2.1 is revised as follows:

kappa coefficient


  • Page 54 (Equation 2.4.6) - A big thank to Nilden Tutalar.
    Equation 2.4.6 is erroneous (the denominator in the expression of pe is (q-1), and not q*(q-1) as specified in the book). The correct equation iac1 statistics the following:



  • Page 54 (Equations 2.4.8 and 2.4.9)
    Equation 2.4.8 is inaccurate. It must be replaced by the following equation:
    pa of krippendorff's alpha
  • and equation 2.4.9 would be replaced with the following:
    Krippendorff's alpha coefficient


  • Page 55 (Equations 2.4.10)
    Equation 2.4.10 contains a typo, and must read as follows:
    pi_k for krippendorff's alpha

  • Page 56 (Last bullet point)
    The last bullet point on page 56 must be replaced with the following paragraph:
    example 2.4 for krippendorff's alpha coefficient

  • Page 64 (Tables 2.24 and 2.25) - A big thank to Bruce Burkemper (South Carolina)
    Below are the corrected tables, where the revisions are circled in red.

    Table 2.24

    Table 2.25

  • Page 81 (first paragraph)
    Replace "... these quadratic weights from equation 3.5 ..." with "... these quadratic weights from equation 3.2.5 ..."

  • Page 117 (Second paragraph)
    The first sentence of the second paragraph should read as follows:
    One way to verify how well equation (4.4.6) works is to simulate a population of subjects similar to what is described in Table 4.3, to define the "true" inter-rater reliability according to equation 4.4.1, and to study the statistical properties of the coecient given by equation 4.2.1.

  • Page 118 (Second paragraph)
    (the hat indicating that it is an approximation of the fixed and unknown abstract of equation 4.4.1). .....

  • Page 118 (Second paragraph)
    . . . a good approximation of the population parameter of equation 4.4.1 as shown by Gwet (2008a).

  • Page 124 (Table 4.8)
    Gwet's Gwet's AC1 should be replaced by Gwet's Gwet's AC2
  • Page 137(first paragraph)
    . . . the population-based coefficient takes (e.g. equation 5.2.3) ...

  • Page 148 (last sentence of the first paragraph)
    "... is defined by equation 4.18 of chapter 4 ..." must be replaced by "... is defined by equation 4.6.1 of chapter 4 ..."

  • Page 149 (first sentence associated with "Krippendorff's Alpha Coefficient" bullet point
    The reference in parenthese "(see section 3.5.2)" must be replaced by "(see section 3.4)"

  • Page 149 (last two bullet points)
    The last two bullet points must be replaced by the following two:
    conger's coefficient variance

  • Page 173 (Second sentence)
    The sentence "However qualifying an agreement coefficient of 0.3 as fair would be a highly
    questionable characterization of the extent of agreement among 2 raters if the number
    of raters is limited to 5." should read:
    "However qualifying an agreement coefficient of 0.3 as fair would be a highly
    questionable characterization of the extent of agreement among 2 raters if the number
    of subjects is limited to 5."


  • Page 253 (Figure 9.4.7)
    Figure 9.4.7 in the book is incorrect.. The correct figure is the following:

sample size for intraclass correaltion


  • Page 282 (last bullet point)
    "... the summation by M/n - r." must be replaced with "... the summation by M - rn."