Linear Regression is one of the most basic and important techniques for predicting a value of an attribute (y). e.g. price of a house, number of people employed based on some factors. It is used to fit values in a forecasting or predictive model. The attributes are usually fitted using the least squares approach. The cost function which it involves for minimizing the error can be minimized using many mathematical tricks and algorithms (Gradient Descent, Derivative test, Newton's Method). Using the derivative method on the least squares approach and with the help of properties of matrices, it reduces the problem of linear regression to a consolidated equation.
whilst linear regression is used to predict or forecast a value, logistic regression, which can also be interpreted as a perceptron, is used to classify between two distinct classes. Logistic Regression is one of the most popular and most widely used learning algorithms. The value predicted by logistic regression is a discrete value. Here are some examples of classification problems:
- Email: Spam or Not Spam
- Online Transactions: Fraudulent or Legitimate
- Tumor: Malignant or Benign
In all of these examples, the value that we are trying to predict is a variable Y that we can think of as taking on two values, either zero or one (Zero = negative class, One = Positive class).