Regression is a set of statistical methods that are widely used to analyze the relationship of one variable to one or more others.
For a long time, the regression of a random variable y on the random variable vector x denoted the conditional average of y knowing x. Today, the regression term denotes any element of the conditional distribution of y knowing x, considered as a function of x. For example, we can look at the conditional mean, the conditional median, the conditional mode, the conditional variance…
The term « regression » was introduced by Francis Galton following a study on the size of the descendants of large individuals, which decreases from generations to an average size (thus their size is decreasing).
In machine learning, we distinguish regression problems from classification problems. Thus, it is considered that the problems of prediction of a quantitative variable are regression problems whereas problems of prediction of a qualitative variable are problems of classification. Some methods, such as logistic regression, are both regression methods in the sense of predicting the probability of belonging to each class.