K-Means Clustering tutorial

Overview

What is K-Means Clustering?

K-Means Clustering is a unsupervised learning technique. Partitions observations around movable cluster centroids. Its central idea is summarized by J = ∑ᵢ₌₁ⁿ ‖xᵢ − μ(cᵢ)‖₂².

This guide connects theory to practice. You will trace the input, intermediate process, and output; decode the notation; run a dependency-light implementation; then repeat the workflow with scikit-learn and evaluate the result.

Preparation

Prerequisites and tools

You do not need an advanced software stack. Start with a recent Python environment and the fundamentals below, then install only the packages used by the practical example.

• Comfort with Python functions, NumPy arrays, and basic descriptive statistics.
• A clear distinction between training data, validation data, and untouched test data.
• Familiarity with features, targets, preprocessing, and task-appropriate evaluation metrics.

Process

How K-Means Clustering works

1. 1

   Input

   Unlabelled numeric observations.

2. 2

   Prepare and configure

   Check shapes, value ranges, ordering assumptions, missing values, and the parameters that control the algorithm’s behavior.

3. 3

   Algorithm process

   Assign points to centroids, then recompute centroid positions.

4. 4

   Output

   A cluster label for each observation.

5. 5

   Validate

   Compare silhouette score and inertia across plausible values of k.

Core concept

Formula and intuition

The notation captures the main operation or complexity statement behind K-Means Clustering. Read it together with the symbol key above and the step-by-step process in this guide.

Applications

When to use K-Means Clustering

Segmentation, compression, and pattern discovery.

Implementation

K-Means Clustering from scratch in Python

This dependency-light example emphasizes the algorithm’s mechanics so each important step remains visible.

import numpy as np

# three loosely separated blobs of points
rng = np.random.default_rng(0)
X = np.vstack([rng.normal(c, 0.6, (50, 2)) for c in [(-2, -2), (2, 2), (2, -2)]])

# classic k-means loop: alternate between assigning points to the nearest
# centroid and moving each centroid to the mean of its assigned points
k = 3
centers = X[rng.choice(len(X), k, replace=False)]  # start from k random data points
for _ in range(10):
    distances = np.linalg.norm(X[:, None] - centers, axis=2)
    labels = distances.argmin(axis=1)                                    # assignment step
    centers = np.array([X[labels == c].mean(axis=0) for c in range(k)])  # update step

Practical tutorial

Build K-Means Clustering with scikit-learn

KMeans provides robust centroid initialization, repeated starts, convergence checks, and inertia reporting.

1. Prepare the data.Unlabelled numeric observations. Validate its shape, type, range, and ordering before training or execution.
2. Configure the algorithm.Begin with explicit, conservative parameters and a fixed random seed whenever the library supports one.
3. Fit or execute.Assign points to centroids, then recompute centroid positions.
4. Inspect the result.A cluster label for each observation. Then apply the evaluation checks in the next section.

from sklearn.cluster import KMeans
from sklearn.datasets import make_blobs
from sklearn.metrics import silhouette_score
from sklearn.preprocessing import StandardScaler

X, _ = make_blobs(
    n_samples=600, centers=4, cluster_std=1.1, random_state=7
)
X = StandardScaler().fit_transform(X)
model = KMeans(n_clusters=4, n_init="auto", random_state=42)
labels = model.fit_predict(X)
print("inertia:", model.inertia_)
print("silhouette:", silhouette_score(X, labels))
print("centroids:", model.cluster_centers_)

API details and version-specific options: official scikit-learn reference →

Evaluation

How to evaluate the result

A successful run is not enough. Evaluate the output against the intended use, compare it with a simple baseline, and preserve a genuinely unseen test case whenever the task involves learned parameters.

• Compare silhouette score and inertia across plausible values of k.
• Run several random seeds and verify that the solution is stable.

Common mistakes

Pitfalls and how to avoid them

These failure modes are common in tutorials and production systems. Treat them as review questions, not just after-the-fact debugging advice.

• K-means assumes roughly spherical clusters under Euclidean distance.
• Scaling changes the geometry and therefore the cluster assignments.
• A low inertia always improves with larger k, so it is not sufficient alone.

Project checklist

Before using K-Means Clustering in a project

• ✓Define the prediction or discovery objective before selecting the algorithm.
• ✓Split data before fitting preprocessing and tune only inside cross-validation.
• ✓Record data versions, random seeds, features, hyperparameters, and evaluation metrics.
• ✓Compare against a simple baseline and inspect errors by meaningful subgroups.
• ✓Monitor input drift and real-world performance after deployment.

Further reading

Official documentation and next steps

This guide is an educational introduction, not a substitute for domain validation. For consequential applications, review the source documentation, test against representative data, and involve a subject-matter expert.

Continue learning

Related machine learning guides

• Linear RegressionFits a straight line that minimizes squared prediction errors.
• Logistic RegressionMaps a weighted feature score to a class probability.
• Decision TreeSplits the feature space into rule-based regions.