Scaling and transforming numbers

Why scale?

Features come in very different units: age (18–80), monthly income (₹10,000–₹10,00,000), number of past loans (0–10). k-NN, k-means, SVMs, PCA and anything trained by gradient descent are affected by these sizes. Income would dominate distances and slow down gradient descent. Scaling removes the unfair advantage of big numbers.

Min-max scaling

x_scaled = (x − min) ÷ (max − min). The smallest training value becomes 0 and the largest becomes 1. It keeps the shape of the data, but one extreme outlier squashes everything else into a tiny range. New values outside the training range will fall below 0 or above 1, which is fine.

Standardisation (z-scores)

x_scaled = (x − mean) ÷ standard deviation. The result has mean 0 and standard deviation 1, like the z-scores from module 2. It is the most common choice and is less squeezed by single outliers than min-max. RobustScaler goes further, using the median and IQR, which are barely moved by outliers.

values = [10, 20, 30, 40]
mn, mx = min(values), max(values)
print([(v - mn) / (mx - mn) for v in values])

import statistics
m, sd = statistics.mean(values), statistics.pstdev(values)
print([round((v - m) / sd, 4) for v in values])

Fit on train, apply to test

The min, max, mean and standard deviation are learned values, just like model weights. Learn them from the training data only, then use the same numbers to transform validation, test and future data. If you compute them from all the data, information from the test set leaks into training. In scikit-learn: scaler.fit(X_train), then scaler.transform(X_test); a Pipeline does this automatically.

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler().fit(X_train)
X_train_s = scaler.transform(X_train)
X_test_s = scaler.transform(X_test)     # same mean and std as training

Log and other transforms

Incomes, house prices and city populations are right-skewed: most values are small and a few are huge. Taking the logarithm, log(1 + x) to handle zeros, pulls in the long tail and often makes relationships more linear. If you transform the target, remember to transform predictions back (for example with exp) before reporting them.

Scaling inside cross-validation

The same rule applies inside cross-validation: in every fold, the scaler must be fitted on the training part of that fold only. Doing this by hand is easy to get wrong, which is why the scaler is normally placed inside a scikit-learn Pipeline together with the model (topic 5). The pipeline is then passed to cross_val_score or GridSearchCV, and it refits the scaler correctly in each fold without any extra code from you.

Which models need scaling?

Common mistakes

• Fitting the scaler on all data before splitting (leakage).
• Scaling the target for a regression model and forgetting to unscale the predictions.
• Dividing by a zero standard deviation for a constant column. Fix: drop constant columns or guard the division.
• Scaling one-hot columns unnecessarily; it is usually harmless but adds confusion.