Regression

Use

Regression is a technique for predicting a mapping (mathematical function) between input attributes and output values. Basically, you're given points; find the line or other function that passes through them as well as possible. It can also be used to find trends. It is a supervised learning approach and the continous-valued analogue to classification.

Example

Predict the relationship between working experience and salary or the trajectory a stock price is following.

Popular Techniques

Linear Regression, Polynomial Regression, Exponential Regression, Moving Averages, k-Nearest Neighbor, Neural Networks, SVMs.

Measuring

Mean-Squared Error (MSE) is the most common performance measure - the smaller it is, the better. Likelihood is also popular; minimizing MSE is equivalent to maximizing likelihood. The R² coefficient (also known as the coefficient of determination) is used to determine the appropriateness of the model itself; specifically the degree of variation covered by the model.

Caveats

• Same problems as classification.
• It generally helps to have an idea of the underlying model before regressing.