Hierarchical Clustering in Python

The purpose here is to write a script in Python that uses the aggregative clustering method in order to partition in k meaningful clusters the dataset (shown in the 3D graph below) containing mesures (area, perimeter and asymmetry coefficient) of three different varieties of wheat kernels : Kama (red), Rosa (green) and Canadian (blue).

3D graph showing the dataset of mesures of AREA, PERIMETER and ASYMMETRY of three different varieties of wheat kernels : Kama (red), Rosa (green) and Canadian (blue)

First, we import the basic libraries and we load our dataset :

#importing libraries
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
import cmath as math
import sys

#loading the dataset
dataset = pd.read_csv('dataset-2-FINAL.csv')
X = dataset.iloc[:,[0,1,2]].values
y_set = dataset.iloc[:,[3]].values


The 30 first rows (over 210) of the dataset in Python are as follows :

Dataset in Python containing mesures (area, perimeter and asymmetry coefficient) of three different varieties of wheat kernels : Kama (red), Rosa (green) and Canadian (blue)

It contains the mesures (AREA, PERIMETER and ASYMMETRY COEFFICIENT) of three different varieties of wheat kernels : Kama (red), Rosa (green) and Canadian (blue) and a fourth column (SPECIES.

One easy way to do clustering in Python consists in using a dendrogram in order to partition the dataset into an optimal number of clusters.

The following code creates the dendrogram and browse the dendrogram tree-like structure in order to retrieve the membership assignments between the data points and the clusters.

import scipy.cluster.hierarchy as sch
Z = sch.linkage(X, method = 'median')
den = sch.dendrogram(Z)
plt.title('Dendrogram for the clustering of the dataset on three different varieties of wheat kernels (Kama, Rosa and Canadian)')
plt.xlabel('Wheat kernels')
plt.ylabel('Euclidean distance in the space with dimensions AREA, PERIMETER and ASYMMETRY');
plt.show()

y_pred = getClusterAssignments(X, den)

This displays the following dendrogram :

Dendrogram for the hierarchical clustering of the dataset of three kernel wheat varieties.

We can visualize the distribution of the data inside the clusters in 3D with the following script :

#visualising the clusters
t1 = getTrace(X[y_pred == 1, 0], X[y_pred == 1, 1], X[y_pred == 1, 2], s= 4, c='red', label = '1') #
t2 = getTrace(X[y_pred == 2, 0], X[y_pred == 2, 1], X[y_pred == 2, 2], s= 4, c='green', label = '2') #-> blue, 3
t3 = getTrace(X[y_pred == 3, 0], X[y_pred == 3, 1], X[y_pred == 3, 2], s= 4, c='blue', label = '3') #-> green, 2

x=X[:,0]
y=X[:,1]
z=X[:,2]
showGraph("Clustering results of the wheat kernels dataset represented in the 3D space with dimensions for AREA, PERIMETER and ASYMMETRY", "AREA", [min(x),max(x)], "PERIMETER", [min(y),max(y)], "ASYMMETRY", [min(z)-1,max(z)], [t1,t2,t3])

that displays the following 3D graph in which each measures of area, perimeter and asymmetry coefficient corresponds to an axis and data points are drawn in a color corresponding to the cluster they belong to.

Results of the dendrogram clustering of the dataset of mesures of AREA, PERIMETER and ASYMMETRY of three different varieties of wheat kernels : Kama (red), Rosa (green) and Canadian (blue)

Fortunately, the clusters assignments found in the dendrogram match the actual classifiction in the dataset (in column SPECIES). Therefore, we can evaluate the success ratio of the clustering with the following code :

from sklearn.metrics import confusion_matrix
cm = confusion_matrix(y_set, y_pred)
print("success ratio : ",success_ratio(cm=cm), "%")

which prints a score of 85.2380952381 % of success.

A second way to partition a dataset into clusters in Python, consists in using the AgglomerativeClustering class from the sklearn.cluster package, with the following code :

from sklearn.cluster import AgglomerativeClustering
hc = AgglomerativeClustering(n_clusters = 3, affinity = 'euclidean', linkage = 'ward')
y_hc = hc.fit_predict(X)

In this case, we need to provide the number of clusters as input (here n_clusters = 3).

We can see the results of the clustering in a 3D graph with the following code :

#visualising the clusters
t1 = getTrace(X[y_hc == 0, 0], X[y_hc == 0, 1], X[y_hc == 0, 2], s= 4, c='red', label = '1') #
t2 = getTrace(X[y_hc == 1, 0], X[y_hc == 1, 1], X[y_hc == 1, 2], s= 4, c='green', label = '2') #
t3 = getTrace(X[y_hc == 2, 0], X[y_hc == 2, 1], X[y_hc == 2, 2], s= 4, c='blue', label = '3') #

x=X[:,0]
y=X[:,1]
z=X[:,2]
showGraph("Clustering results of the wheat kernels dataset represented in the 3D space with dimensions for AREA, PERIMETER and ASYMMETRY", "AREA", [min(x),max(x)], "PERIMETER", [min(y),max(y)], "ASYMMETRY", [min(z)-1,max(z)], [t1,t2,t3])

Which displays the following 3D graph :

Results of the clustering with the AgglomerativeClustering class of the dataset of mesures of AREA, PERIMETER and ASYMMETRY of three different varieties of wheat kernels : Kama (red), Rosa (green) and Canadian (blue)

If we reassign the correct cluster labels so that they correspond to the initial classes given in the dataset (column SPECIES) without changing the content of those clusters, we can evaluate the success ratio of the clustering by comparing it to the original classification. This is done with the following code :

y_pred = np.array(y_hc)
y_pred[y_hc == 0] = 1
y_pred[y_hc == 1] = 2
y_pred[y_hc == 2] = 3

print("success ratio : ",success_ratio(cm=cm), "%")

the printed success ratio shows that 83.333333% of the data points were assigned to the correct classes. As the previous one, it is not a bad result.

The scripts written in this page use the functions defined below :

def getClusterAssignments(X, den):
cluster_idxs = dict()
cluster_rows = dict()
colors_nums = dict()
col_num=0
for c in np.unique(den['color_list']):
cluster_idxs[c] = []
colors_nums[c] = col_num
cluster_rows[col_num] = []
col_num = col_num + 1