Linear Regression

1 Basic Explanation

The term “regression” (回归 in Chinese) comes from statistics and is a translation of the word Regression; linear regression is LinearRegression. In earlier chapters we mentioned the concept of fitting: the closer the curve fitted in a Cartesian coordinate system is to the points, the better. To optimize the fitted curve, there are two approaches: minimizing the sum of the absolute values of the deviations, and minimizing the sum of the squares of the deviations. Minimizing the sum of absolute deviations is hard to compute, whereas minimizing the sum of squared deviations can be explained by orthogonal projection in an inner product space of vectors. For a combination of technical and mathematical reasons, the optimization method based on minimizing the sum of squared deviations was ultimately adopted for fitting the curve relationship of a set of data, and this is the least squares method.

2 Fitting and Regression

Fitting and regression are very similar, but there are also some differences. First, for known data, a fitting method may find its key curve, but we do not know whether it also fits unknown data well. When the amount of data is small, there is actually no obvious difference, but when the amount of data becomes very large, the results obtained by fitting become inaccurate.