Other clustering methods and dimensionality reduction

Beyond k-means

k-means is fast and simple, but it needs k in advance and expects round clusters. Real data may have groups of odd shapes, outliers that belong to no group, or a natural hierarchy (city → area → street). Different methods suit these cases.

Hierarchical (agglomerative) clustering

Start with every point as its own cluster. Repeatedly merge the two closest clusters until only one remains. The record of merges is a tree called a dendrogram. Cutting it at a chosen height gives any number of clusters. It is useful when you want to see structure at several levels, but it is slow on very large datasets.

DBSCAN: density-based clustering

DBSCAN groups points that have many neighbours within a radius (eps). A core point has at least min_samples neighbours. Clusters grow from core points, and points that are not near any dense region are labelled noise (−1). It finds clusters of any shape, does not need k, and naturally flags outliers, which makes it handy for things like grouping GPS pings into delivery hotspots. It struggles when clusters have very different densities.

from sklearn.cluster import DBSCAN, AgglomerativeClustering

db = DBSCAN(eps=0.5, min_samples=5).fit(X_scaled)
print(set(db.labels_))          # e.g. {0, 1, 2, -1}; -1 means noise

agg = AgglomerativeClustering(n_clusters=4, linkage="ward").fit(X_scaled)

Dimensionality reduction

Datasets can have hundreds of columns. Many are related: height in cm and height in inches carry the same information. Dimensionality reduction creates a smaller set of new columns that keep most of the useful information. Benefits: faster models, less overfitting, and the ability to draw the data in 2-D.

PCA in plain words

Principal component analysis (PCA) finds the direction in which the data varies most; that is the first principal component. The second component is the direction of most remaining variation, at right angles to the first, and so on. Projecting the data onto the first few components gives new features. The explained variance ratio says what share of the total variation each component keeps, such as "the first 3 components keep 85%". PCA needs scaled features, and its components are mixtures of the original columns, so they are harder to explain.

from sklearn.decomposition import PCA

pca = PCA(n_components=0.90)       # keep enough components for 90% of the variance
X_small = pca.fit_transform(X_scaled)
print(X_small.shape, pca.explained_variance_ratio_.round(2))

t-SNE and UMAP

t-SNE and UMAP squeeze high-dimensional data into 2-D pictures that keep nearby points nearby. They make beautiful plots for exploring, for example of embeddings of support tickets. But distances and cluster sizes in the picture can be misleading, and the result changes with settings. Use them to look, not as features for models or as proof of groups.

Common mistakes

• Running PCA without scaling, so the column with the largest units dominates the first component.
• Fitting PCA on all data before the train/test split (leakage). Fit on training data only.
• Reading clusters in a t-SNE plot as real, measured groups.
• Using DBSCAN with a default eps on unscaled data and getting everything as noise.