Regression Wizard Auto Model Calculator

Least squares fits a curve by choosing parameters that minimize the sum of squared vertical distances between observed and predicted values. For the straight line y = a + bx the solution is closed form: the slope equals the covariance of x and y divided by the variance of x, and the fitted line always passes through the point of means. The quadratic fit here solves the three-by-three normal equations by Cramer rule, while the exponential and power fits are linearized by taking logarithms, which is why both require strictly positive data. That linearization is not innocent: minimizing squared error in log space effectively weights small values more heavily than a direct nonlinear fit would, so the coefficients differ slightly from a true nonlinear least squares result. R-squared reports the fraction of variance explained, but it never decreases when parameters are added, so comparing a quadratic against a line by raw R-squared always favours the quadratic. Adjusted R-squared applies a penalty factor of (n-1)/(n-p-1) to the unexplained share and is the appropriate ranking statistic when candidate models differ in parameter count. Expectations vary sharply by field: above 0.95 is routine in calibrated physical measurement, 0.3 to 0.5 can be a strong result in social science, and 0.1 can be economically meaningful in finance. The prediction interval widens with distance from the mean of x, which is the formal reason extrapolation is risky. Least squares also assumes independent observations with constant error variance, assumptions that time series and count data routinely violate.

The method of least squares first appeared in print in 1805, in an appendix by Adrien-Marie Legendre on determining the orbits of comets. Carl Friedrich Gauss claimed in 1809 to have used the method since about 1795 and supplied the probabilistic justification, deriving it from an assumption of normally distributed observation errors, and Pierre-Simon Laplace soon connected it to the central limit theorem. The immediate applications were astronomical and geodetic: reconciling redundant and slightly inconsistent measurements of orbits and of the shape of the Earth. The word regression comes from Francis Galton, who in an 1886 paper on hereditary stature described how the children of unusually tall parents tended back toward the population mean, calling it regression toward mediocrity. Karl Pearson formalized the correlation coefficient in the 1890s and introduced the chi-squared test in 1900 as part of the same statistical programme. Ronald A. Fisher supplied much of the inferential machinery during the 1920s, including analysis of variance and the distribution theory that makes standard errors and confidence intervals usable, gathered in Statistical Methods for Research Workers in 1925. Adjusted R-squared, the complexity-penalized version used here for ranking models, was proposed by Henri Theil in 1961, and information criteria such as the one Hirotugu Akaike introduced in 1974 generalized the same trade-off between goodness of fit and number of parameters.