Choosing and explaining a model

There is no free lunch

The "no free lunch" idea says no algorithm wins on every problem. A model that is perfect for one dataset can be poor on another. So data scientists try a few sensible candidates, evaluate them fairly with cross-validation, and choose using both the score and practical needs.

Two more classic models

• Naive Bayes: applies Bayes' theorem (module 3) and assumes features are independent given the class. The assumption is "naive", but it is fast and works well for text, such as spam filtering.
• Support vector machines (SVM): find the boundary between classes with the widest margin (gap) to the nearest points. With kernels, they can draw curved boundaries. Strong on small and medium datasets; they need scaled features and get slow on very large ones.

A practical comparison

How to choose

1. Start with a baseline and a simple model (logistic or linear regression).
2. Try a tree ensemble (random forest or gradient boosting).
3. Compare with cross-validation on the metric that matters.
4. If scores are close, prefer the simpler, faster, more explainable model.
5. Check practical limits: prediction speed, memory, how often it must be retrained, and whether regulators or users need explanations.

Explaining predictions

Permutation importance measures how much the model's score drops when one feature's values are shuffled, which breaks its link with the target. A big drop means the model relies on that feature. It works for any model and is computed on validation data. SHAP values go further and split each individual prediction into contributions from each feature, so you can say "this loan was declined mainly because of two missed EMIs and high existing debt".

from sklearn.inspection import permutation_importance

result = permutation_importance(model, X_val, y_val, n_repeats=10, random_state=0)
for i in result.importances_mean.argsort()[::-1][:5]:
    print(X_val.columns[i], round(result.importances_mean[i], 4))

Common mistakes

• Picking the fanciest model by default. Fix: let fair comparison decide.
• Ignoring a 0.2-point gain that costs 10× more compute and no explainability.
• Computing importance on training data, where overfit features look important. Fix: use validation data.