Assessment Reliability Calculator

Classical test theory treats an observed score as a true score plus random error, and reliability is defined as the proportion of observed score variance that is true score variance. Coefficient alpha estimates that proportion from a single administration by comparing the sum of the individual item variances to the variance of the total score. If items covary strongly, the total variance greatly exceeds the summed item variances and alpha approaches 1; if items are unrelated, the two quantities converge and alpha approaches 0. The k over k minus 1 term corrects for the fact that a finite set of items underestimates the covariance structure. For dichotomously scored items where each item variance is p times 1 minus p, the same formula is known as Kuder-Richardson 20. Common interpretive bands are 0.90 and above for high-stakes individual decisions such as licensure or admissions, 0.80 to 0.89 for course examinations, and 0.70 to 0.79 as a floor for group-level research. The standard error of measurement, the standard deviation times the square root of 1 minus reliability, translates reliability into score units: with a standard deviation of 10 and reliability of 0.84, the SEM is 4 points, so a 95 percent confidence band around an observed score spans roughly 15.7 points. Two limitations deserve emphasis. Alpha rises with test length regardless of item quality, so a long test of mediocre items can post a high coefficient, and alpha assumes essentially tau-equivalent, unidimensional items, which is why omega coefficients are preferred for multidimensional scales. Alpha is also a property of scores in a particular sample, not a fixed property of the instrument, and it shrinks when the group tested is homogeneous.

The mathematical groundwork was laid by Charles Spearman, whose 1904 papers introduced correction for attenuation and the idea that observed correlations are depressed by measurement error. In 1910 Spearman and William Brown independently published the prophecy formula that predicts how reliability changes when a test is lengthened or shortened, which is why it carries both names. Split-half estimation followed from that work, though it had the awkward property that different ways of splitting a test produced different reliability values. G. Frederic Kuder and Marion Richardson resolved much of that ambiguity in a 1937 Psychometrika paper that derived a family of internal consistency formulas, of which KR-20 became the standard for right or wrong scoring. Louis Guttman generalized the approach in 1945 with a set of lower bounds, one of which, lambda-3, is algebraically identical to the coefficient later popularized by Lee Cronbach in his 1951 Psychometrika article on coefficient alpha and the internal structure of tests. Cronbach extended the logic to items scored on any scale and framed alpha as a general lower bound on reliability, and the paper became one of the most cited in the social sciences. Later work, including generalizability theory in the 1970s and renewed advocacy for omega coefficients from the 1990s onward, addressed alpha's known limitations. Professional expectations for reporting reliability evidence are codified in the Standards for Educational and Psychological Testing, published jointly by AERA, APA, and NCME and most recently revised in 2014.