Hierarchical Clustering tutorial

Overview

What is Hierarchical Clustering?

Hierarchical Clustering is a unsupervised learning technique. Builds a hierarchy of clusters using distance metrics. Its central idea is summarized by d(A,B) = Link({d(a,b) : a ∈ A, b ∈ B}).

This guide connects theory to practice. You will trace the input, intermediate process, and output; decode the notation; run a dependency-light implementation; then repeat the workflow with scikit-learn and evaluate the result.

Preparation

Prerequisites and tools

You do not need an advanced software stack. Start with a recent Python environment and the fundamentals below, then install only the packages used by the practical example.

• Comfort with Python functions, NumPy arrays, and basic descriptive statistics.
• A clear distinction between training data, validation data, and untouched test data.
• Familiarity with features, targets, preprocessing, and task-appropriate evaluation metrics.

Process

How Hierarchical Clustering works

1. 1

   Input

   Unlabelled numeric observations.

2. 2

   Prepare and configure

   Check shapes, value ranges, ordering assumptions, missing values, and the parameters that control the algorithm’s behavior.

3. 3

   Algorithm process

   Merge closest clusters iteratively (agglomerative).

4. 4

   Output

   A dendrogram showing cluster hierarchy.

5. 5

   Validate

   Compare linkage choices and inspect whether cluster assignments remain stable.

Core concept

Formula and intuition

The notation captures the main operation or complexity statement behind Hierarchical Clustering. Read it together with the symbol key above and the step-by-step process in this guide.

Applications

When to use Hierarchical Clustering

Taxonomy, document clustering, and multi-scale analysis.

Implementation

Hierarchical Clustering from scratch in Python

This dependency-light example emphasizes the algorithm’s mechanics so each important step remains visible.

import numpy as np

rng = np.random.default_rng(0)
X = rng.uniform(0, 10, (12, 2))
linkage = "average"  # how distance between two clusters is measured

def cluster_distance(a, b):
    dists = [np.linalg.norm(X[i] - X[j]) for i in a for j in b]
    if linkage == "single":
        return min(dists)     # distance between the closest pair of points
    if linkage == "complete":
        return max(dists)     # distance between the farthest pair of points
    return np.mean(dists)     # average distance across all pairs

# agglomerative clustering: start with every point as its own cluster,
# then repeatedly merge the two closest clusters until only one remains
clusters = [[i] for i in range(len(X))]
merges = []
while len(clusters) > 1:
    best = (np.inf, 0, 1)
    for i in range(len(clusters)):
        for j in range(i + 1, len(clusters)):
            d = cluster_distance(clusters[i], clusters[j])
            if d < best[0]:
                best = (d, i, j)
    _, i, j = best
    merges.append((clusters[i], clusters[j], best[0]))  # record this merge (builds the dendrogram)
    clusters[i] = clusters[i] + clusters.pop(j)

Practical tutorial

Build Hierarchical Clustering with scikit-learn

AgglomerativeClustering supports common linkage rules, distance metrics, and either a cluster count or distance threshold.

1. Prepare the data.Unlabelled numeric observations. Validate its shape, type, range, and ordering before training or execution.
2. Configure the algorithm.Begin with explicit, conservative parameters and a fixed random seed whenever the library supports one.
3. Fit or execute.Merge closest clusters iteratively (agglomerative).
4. Inspect the result.A dendrogram showing cluster hierarchy. Then apply the evaluation checks in the next section.

from sklearn.cluster import AgglomerativeClustering
from sklearn.datasets import make_blobs
from sklearn.metrics import silhouette_score
from sklearn.preprocessing import StandardScaler

X, _ = make_blobs(
    n_samples=500, centers=4, cluster_std=0.9, random_state=7
)
X = StandardScaler().fit_transform(X)
model = AgglomerativeClustering(n_clusters=4, linkage="ward")
labels = model.fit_predict(X)
print("cluster sizes:", [(labels == i).sum() for i in set(labels)])
print("silhouette:", silhouette_score(X, labels))

API details and version-specific options: official scikit-learn reference →

Evaluation

How to evaluate the result

A successful run is not enough. Evaluate the output against the intended use, compare it with a simple baseline, and preserve a genuinely unseen test case whenever the task involves learned parameters.

• Compare linkage choices and inspect whether cluster assignments remain stable.
• Use a dendrogram or distance threshold when the hierarchy matters more than one fixed partition.

Common mistakes

Pitfalls and how to avoid them

These failure modes are common in tutorials and production systems. Treat them as review questions, not just after-the-fact debugging advice.

• Ward linkage requires Euclidean distance.
• Agglomerative clustering can be costly for very large datasets.
• Early merges cannot be undone later in the hierarchy.

Project checklist

Before using Hierarchical Clustering in a project

• ✓Define the prediction or discovery objective before selecting the algorithm.
• ✓Split data before fitting preprocessing and tune only inside cross-validation.
• ✓Record data versions, random seeds, features, hyperparameters, and evaluation metrics.
• ✓Compare against a simple baseline and inspect errors by meaningful subgroups.
• ✓Monitor input drift and real-world performance after deployment.

Further reading

Official documentation and next steps

This guide is an educational introduction, not a substitute for domain validation. For consequential applications, review the source documentation, test against representative data, and involve a subject-matter expert.

Continue learning

Related machine learning guides

• Linear RegressionFits a straight line that minimizes squared prediction errors.
• Logistic RegressionMaps a weighted feature score to a class probability.
• Decision TreeSplits the feature space into rule-based regions.