T-Test Calculator (One & Two Sample)

An independent two-sample t-test asks whether the observed gap between two group means is larger than the noise in the estimate of that gap. The numerator is the difference of the means; the denominator is the standard error of that difference, formed by adding the two squared standard errors and taking the square root. Welch's version keeps each group's own variance rather than pooling them, which is why it stays accurate when the groups differ in spread or in size. The cost is that the sampling distribution is no longer exactly t, so the Welch-Satterthwaite formula supplies an effective degrees of freedom that is usually fractional and lies between the smaller group's n-1 and the pooled n1+n2-2. With means of 25 and 22, standard deviations of 5 and 4 and 30 per group, t is about 2.57 on roughly 55.3 degrees of freedom. The two-tailed p-value is the area beyond plus and minus t, computed here from the exact t distribution through the regularised incomplete beta function rather than a normal approximation. Report Cohen's d alongside it, since d divides the same mean difference by the pooled standard deviation and is unaffected by sample size; the usual reference points are 0.2, 0.5 and 0.8. Assumptions are independence and an approximately normal sampling distribution of each mean, which the central limit theorem generally supplies by around 30 per group. Strong skew, heavy tails or small samples argue for the Mann-Whitney test or a bootstrap, before-and-after measurements on the same subjects require a paired test, and multiple comparisons need an adjustment.

The test originates in industrial quality control. William Sealy Gosset, an Oxford-trained chemist hired by Arthur Guinness Son and Company in Dublin in 1899, worked on barley varieties and yeast cell counts where experiments produced only a handful of observations and the large-sample normal theory of the day did not apply. His 1908 Biometrika paper The Probable Error of a Mean derived the distribution of a mean divided by its estimated standard error. Guinness forbade employees from publishing under their own names after an earlier disclosure of trade information, so the paper appeared under the pseudonym Student, and the statistic has carried that name ever since. Ronald Fisher, who corresponded with Gosset for years, recast the result in terms of degrees of freedom and put the modern t ratio into general use through Statistical Methods for Research Workers in 1925. The unequal-variance case, known as the Behrens-Fisher problem, resisted an exact solution. B. L. Welch published approximate treatments in Biometrika in 1938 and 1947, and F. E. Satterthwaite gave the matching degrees of freedom approximation in 1946. Simulation studies much later, including work by Graeme Ruxton in Behavioral Ecology in 2006 and by Delacre, Lakens and Leys in 2017, showed that Welch's test loses almost nothing when variances are equal and protects the error rate when they are not, which is why it is now the recommended default.