Free Krippendorff's alpha calculator: it measures agreement among two or more raters even with missing data, using distance functions for nominal, ordinal, interval or ratio data. It returns alpha with its interpretation, the coincidence matrix and a ready-to-paste APA 7 sentence.
Worked example: 3 coders classified 20 open-ended answers into 4 categories, with 2 uncoded cells; that leaves 58 pairable values and nominal alpha = 1 − 57 × 4 / 1259 = .819, above the .80 threshold Krippendorff regards as acceptable.
What value is acceptable? Krippendorff recommends relying on the data only when alpha ≥ .80; between .667 and .80 only tentative conclusions are warranted. Alpha or kappa? Cohen's kappa fits exactly 2 raters with complete nominal data; with more raters, missing cells or another scale use alpha, and for continuous measures, the ICC.
Where kappa falls short: Krippendorff's alpha with more than two raters, missing data and nominal, ordinal, interval or ratio scales. With APA 7 sentence.