Logistic Regression is another very important algorithm in Machine Learning. It is also widely used in Machine Learning Algorithms to classify incoming data based on historical data. It also plays a very important role in data preparation steps by allowing data sets to be put into specifically predefined buckets during the extract, transform, load (ETL) process in order to stage the information for analysis.
It can predict the dependent data variables by analyzing the relationship between one or more existing independent variables. For example, logistic regression could be used to predict whether a high school student will be admitted or not to a particular college. It is a statistical analysis method to predict a binary outcome, such as yes or no, based on prior observations of a data set.
Why did Logistic Regression Come into the Picture?
Some Problems like Is the mail spam or not?
Will it rain or not?
Will the property will have a good neighbour or not? are the type of classification problems and we can not answer with linear regression but we can easily do this by logistic regression.
Logistic Regression is used for the statistical classification model which deals with the categorical dependent variables which can be binary or dichotomous.
We can call a Logistic Regression a Linear Regression model but the Logistic Regression uses a more complex cost function, this cost function can be defined as the ‘Sigmoid function’ or also known as the ‘logistic function’ instead of a linear function.
It is a classification algorithm used to assign observations to a discrete set of classes. Some of the examples of classification problems are Email spam or not spam, Online transactions Fraud or not Fraud, Tumor Malignant or Benign. Logistic regression transforms its output using the logistic sigmoid function to return a probability value.
It aims to model the behaviour of an objective variable as a combination of effects from a number of predictor variables. Among these tried-and-true techniques is Logistic Regression, which was originally developed by statistician David Cox in 1958.
Advantages
- It performs well when the dataset is linearly separable.
- It is less prone to over-fitting but it can overfit in high dimensional datasets.
- It not only gives a measure of how relevant a predictor (coefficient size) is, but also its direction of association (positive or negative).
- It is easier to implement, interpret and very efficient to train.
Disadvantages
- It has the assumption of linearity between the dependent variable and the independent variables.
- It can only be used to predict discrete functions. The dependent variable of Logistic Regression is restricted to the discrete number set. This restriction itself is problematic, as it is prohibitive to the prediction of continuous data.
- The number of observations is lesser than the number of features, Logistic Regression should not be used, otherwise, it may lead to overfitting.
- Log in to post comments