Compute Cohen's kappa from a 2×2 table, with 95% CI, the Landis & Koch interpretation and the APA 7 sentence to report it, ready to copy into your paper.
Cohen’s (1960) kappa measures agreement between two raters who classify the same cases into categories, discounting the agreement expected by chance: κ = (pₒ − pₑ) / (1 − pₑ), where pₒ is the observed proportion of agreement and pₑ the agreement expected by chance from the marginal totals. κ = 1 is perfect agreement; κ = 0 is no better than chance; a negative value is worse than chance. It is the standard index of inter-rater reliability for categorical ratings in psychology, medicine and nursing research.
Two psychologists review 100 clinical records and decide whether each patient meets criteria for depression. They agree on "yes" in 40 cases and on "no" in 45; the first says yes and the second no in 10, and the other way round in 5. Observed agreement is pₒ = 85/100 = .85. They said "yes" in 50 and 45 cases respectively, so chance agreement is pₑ = (50 × 45 + 50 × 55) / 100² = .50. Kappa = (.85 − .50) / (1 − .50) = .70, 95% CI [.56, .84]: substantial agreement by the Landis and Koch (1977) benchmarks.
| Kappa | Strength of agreement |
|---|---|
| < 0 | Poor |
| .01 – .20 | Slight |
| .21 – .40 | Fair |
| .41 – .60 | Moderate |
| .61 – .80 | Substantial |
| .81 – 1 | Almost perfect |
Landis and Koch (1977) offered these bands as rough guides: for an instrument used to make decisions about patients, a kappa of .61 may not be enough.
With a very rare category you can have 95% agreement and a low kappa, because chance agreement is already very high. That is not a calculator error: it is a property of the index. Report it alongside the observed agreement and the 2×2 table, as explained in how to report Cohen’s kappa in APA 7.
One sentence is enough: "Inter-rater agreement was substantial, κ = .70, 95% CI [.56, .84]." Give the number of cases rated, the categories and how disagreements were resolved. The confidence interval uses the asymptotic variance of Fleiss, Cohen and Everitt (1969) and is truncated to [−1, 1].
If you use it in your thesis or in a paper, cite it so that readers of your Methods section know how the calculation was done.
Ciudad-Fernández, V. (2026). Cohen's kappa calculator [Web application]. Ciudad Science. Retrieved October 18, 2026, from https://analisisdedatospsicologia.com/en/tools/cohen-kappa-calculator
For your Methods section: Inter-rater agreement was estimated with Cohen's kappa (Cohen, 1960), with its large-sample standard error (Fleiss et al., 1969), and interpreted following Landis and Koch (1977); computations were done with the Ciudad Science Cohen's kappa calculator (Ciudad-Fernández, 2026).
@misc{ciudadfernandez2026kappa,
author = {Ciudad-Fern{\'a}ndez, V{\'i}ctor},
title = {{Cohen's kappa calculator}},
year = {2026},
publisher = {Ciudad Science},
howpublished = {Web application},
url = {https://analisisdedatospsicologia.com/en/tools/cohen-kappa-calculator},
urldate = {2026-10-18},
note = {Accessed 2026-10-18},
}