Feature Scaling

Let’s understand feature scaling.

Here is the housing dataset

In this dataset, the total number of rooms ranges from about 6 to 39320

while the median incomes range from 0 to 15.

Machine Learning algorithms don’t perform well when the input numerical features have very different scales. So how should we handle the case when the input data contains such a varied scale? There are two ways to make all attributes on the same scale.

min-max scaling and standardization.

So what is min-max scaling? Min-max scaling is also known as normalization.

In min-max scaling, all the values are shifted and rescaled so that they end up ranging between 0 to 1.

The minimum value in the original data

becomes 0 in the normalized data

and the maximum value in the original data

becomes 1 in the normalized data.

The remaining values in the original data take values between 0 and 1 in the normalized data.

To calculate the normalized value we first subtract the original value with the minimum value of the list. And then we divide it with the range of the list i.e difference of maximum value and the minimum value in the list.

Let’s find the normalized value of 50

Original data consist of minus 100, minus 50, 0, 50 and 100.

In the original data, the minimum value is minus 100

And the maximum value is 100

So normalized value of 50 will be 50 minus negative 100 divided by 100 minus negative 100

which is 0.75. In short in min-max scaling values are shifted and rescaled so that the new values are between 0 and 1.

The second approach of scaling is standardization. It is quite different from min-max scaling. As you can see in the above chart, min-max scaling scaled the input data in the range of 0 and 1

But standardization does not bound values to a specific range.

In the standardization, we scale the values by calculating how many standard deviations away the value is from the mean. In standardization, features are rescaled so that output has the properties of standard normal distribution with ...

Zero mean and Unit variance.

So which approach should we use for feature scaling - min-max scaling or standardization?

Min-max scaling is good for neural network algorithms as neural network algorithms often expect input values in the range of 0 and 1.

Unlike min-max scaling, standardization does not bound values to a specific range.

In other words, the min-max scaling always results in values between 0 and 1 while the standardization may result in larger range.

Compared to min-max scaling, standardization is less affected by outliers. If we are using machine learning algorithms like support vector machines and logistic regression, we use standardization for scaling.

One important point to note is scaling the target values or label is generally not required.

Previous Index Next

End-to-End ML Project- Beginner friendly

Feature Scaling

XP

Loading comments...