Clustering searches for structure without class labels. K-means partitions points into a user-selected
number k of clusters by alternating between nearest-center assignments and mean updates.
The two steps are simple; choosing k, initializing the centers, and matching the model to
the shape of the data are not.
The two Lloyd steps
- Assignment: assign each point to its nearest centroid under squared Euclidean distance.
- Update: replace each centroid with the arithmetic mean of the points assigned to it.
The shaded Voronoi cells contain all locations closest to each centroid. After an update, the cells
move with the centroids. Points that now lie inside another cell will change assignment on the next step.
Objective and convergence
Lloyd's algorithm minimizes the within-cluster sum of squared distances, also called inertia or SSE.
With fixed assignments, the mean minimizes squared distance for each cluster. With fixed centroids,
nearest-center assignment minimizes each point's contribution. Each completed step therefore does not
increase the objective. Because a finite dataset has finitely many assignments, the procedure reaches a
fixed point in finitely many assignment changes, subject to a consistent rule for ties and empty clusters.
Convergence means the local update rule has stopped changing the solution. It does not mean the selected
clustering is globally best, stable across starts, or meaningful for the application.
Initialization changes the destination
- Random data points: simple, but several centers may begin in one visible group.
- Farthest point: repeatedly chooses the point farthest from its nearest existing center;
it spreads coverage but can favor outliers.
- K-means++: chooses later centers randomly with probability proportional to squared
distance from the nearest selected center. It usually improves starting coverage while retaining randomness.
- Manual placement: exposes how a deliberately poor start can redirect later assignments.
Repeating the algorithm from several starts and keeping the lowest-inertia result reduces, but does not remove,
sensitivity to local optima. K-means++ is available in this applet so its variability can be compared directly
with uniform random initialization.
Shape assumptions and choosing k
Squared Euclidean distance favors compact, roughly convex groups of comparable scale. Curved, nested, highly
unequal, or density-defined clusters can be split in ways that look unnatural. The smiley data makes that mismatch visible.
There is no universally correct automatic value of k. Inertia always weakly decreases as more
centers are added, so it cannot select k by itself. Elbow plots, silhouette scores, stability checks,
external validation, and domain constraints provide different evidence, and they can disagree because
clustering objectives do not define one uniquely true grouping.
What this model leaves out
This applet uses Euclidean distance, equal-weight points, and Lloyd updates. Alternatives include k-medoids
for greater outlier robustness, Gaussian mixture models for probabilistic soft assignments, fuzzy c-means,
spectral clustering, and DBSCAN or HDBSCAN for density-defined shapes and noise points. Production libraries
may accelerate the same objective with triangle-inequality bounds, vectorized computation, and multiple starts.
The initialization comparison and failure-mode datasets are inspired by Naftali Harris's
Visualizing K-Means Clustering. The explanation and classroom sequence are adapted for this suite.
For teachers
Curriculum: Unsupervised learning, centroids, assignment and update steps,
inertia, convergence, initialization, silhouette score, and model-data mismatch.
Pre-exploration prompts:
- What information is missing when training examples have no class labels?
- What does the number
k require the algorithm to produce?
- Why might two reasonable clusterings exist for the same points?
Post-exploration prompts:
- Compare random and k-means++ starts on packed circles over several runs.
- Place all centers in one corner. Which later step can recover, and where can the path remain trapped?
- Run the smiley data with several values of
k. Why does changing k not repair the shape assumption?
- Compare inertia and silhouette while increasing
k. Why do the measures behave differently?
Misconceptions to test: k-means discovers the correct number of clusters;
convergence establishes global optimality; lower inertia always identifies a better model;
k-means++ guarantees the final optimum.
聚类在没有类别标签的情况下寻找结构。K 均值把点划分为用户指定的 k 个簇,并在最近中心分配与均值更新之间交替。两个步骤很简单,选择 k、初始化中心以及判断模型是否适合数据形状则并不简单。
Lloyd 算法的两个步骤
- 分配:按照平方欧几里得距离,把每个点分配给最近质心。
- 更新:把每个质心替换为当前分配到该簇的点的算术平均位置。
阴影 Voronoi 区域包含所有最接近相应质心的位置。质心更新后,区域也会随之移动。现在落入另一颜色区域的点,会在下一次分配步骤中改变所属簇。
目标函数与收敛
Lloyd 算法最小化簇内平方距离总和,也称惯性或 SSE。在分配固定时,均值是使每个簇平方距离最小的位置;在质心固定时,最近中心分配使每个点的贡献最小。因此每个完整步骤都不会增大目标函数。有限数据集只有有限种分配,在平局与空簇处理规则一致的条件下,算法会在有限次分配变化后达到固定点。
收敛只表示局部更新规则不再改变当前解,并不表示该聚类是全局最佳、不同起点下稳定,或对实际应用具有意义。
初始化会改变终点
- 随机数据点:方法简单,但多个中心可能同时落在一个可见群组中。
- 最远点:反复选择距离最近已有中心最远的点,能够分散覆盖,却可能偏向离群点。
- K-means++:后续中心按到最近已选中心的平方距离成比例随机选择,通常改善初始覆盖,同时保留随机性。
- 手动放置:可以直接观察刻意较差的起点如何改变之后的分配路径。
从多个起点重复运行并保留惯性最低的结果,可以降低但不能消除对局部最优的敏感性。本工具已包含 K-means++,可以直接把它的波动与均匀随机初始化比较。
形状假设与选择 k
平方欧几里得距离偏好紧密、近似凸且尺度相近的群组。弯曲、嵌套、大小差异很大或由密度定义的簇,可能被切分成视觉上不自然的结果。笑脸数据让这种不匹配直接可见。
不存在适用于所有问题的自动正确 k。增加中心后,惯性总是不增,因此不能单独用惯性选择 k。肘部图、轮廓系数、稳定性检验、外部验证和领域约束提供不同证据,而且它们可能不一致,因为聚类目标并不会定义唯一真实的分组。
这个模型省略了什么
本工具使用欧几里得距离、等权重点和 Lloyd 更新。其他方法包括对离群点更稳健的 K 中心点、提供概率软分配的高斯混合模型、模糊 C 均值、谱聚类,以及处理密度形状和噪声点的 DBSCAN 或 HDBSCAN。生产库还可使用三角不等式界、向量化计算和多次初始化加速相同目标。
初始化比较和失败情境数据受到 Naftali Harris 的 Visualizing K-Means Clustering 启发,解释与课堂顺序则为本套件重新编写。
教师指导
课程目标:无监督学习、质心、分配与更新步骤、惯性、收敛、初始化、轮廓系数和模型与数据不匹配。
探索前问题:
- 训练样本没有类别标签时缺少了什么信息?
- 数值
k 要求算法产生什么?
- 为什么同一组点可能存在两个合理聚类?
探索后问题:
- 在紧密圆群上多次比较随机初始化与 K-means++。
- 把全部中心放在一个角落。之后哪个步骤能够恢复,路径又可能在哪里被困?
- 用多个
k 运行笑脸数据。改变 k 为什么不能修复形状假设?
- 提高
k 时比较惯性和轮廓系数。两个指标为什么表现不同?
需要检验的误解:K 均值会发现正确聚类数;收敛证明全局最优;惯性越低模型一定越好;K-means++ 保证最终最优。