Before start our lesson please download the datasets.
Problem Statement
Face detection problem consists of 564 images of 20 people. Each covering a range of poses from profile to frontal views. Subjects cover a range of race/sex/appearance. Each subject exists in their own directory labelled 1a, 1b, 1t and images are numbered consequetively as they were taken. The files are all in PGM format, approximately 220 x 220 pixels in 256 shades of grey. We need to train a model using the features extracted from these images. Our model should recognize the faces of these people in any new image.
Data Exploration
All the images are stored in a single .mat file. We read the .mat file using scipy package
import warnings
warnings.filterwarnings('ignore') # to hide warnings
import scipy.io
faces = scipy.io.loadmat('D:intern bangpython case studyFace Detection - UMist Facesumist_cropped.mat')
type(faces)
faces variable has all images data. It is a dict variable. Lets’ see the keys
faces.keys()
It has five keys. We will check each key.
header
type(faces['__header__'])
faces['__header__']
‘header‘ key has information about the .mat file.
version
type(faces['__version__'])
faces['__version__']
‘version‘ key indicates version of the file
globals
type(faces['__globals__'])
faces['__globals__']
‘globals‘ key has no value
facedat
type(faces['facedat'])
faces['facedat'].shape
faces['facedat'][0][0].shape
length=[]
sum=0
for i in range(0,20):
length.append(faces['facedat'][0][i].shape)
sum=sum+faces['facedat'][0][i].shape[2]
print(length)
print(sum)
‘facedat‘ has pixel values of all the images. It is a multidimensional array. It has images of 20 persons. Number of images of each person is different. Size of each image is 112×92. Total number of images are 575.
dirnames
type(faces['dirnames'])
len(faces['dirnames'])
faces['dirnames']
‘dirnames‘ consists of person id’s. As there are twenty people faces, there are twenty id’s from 1a,1b to 1t.
Image viewing
Now we will see some images present in the data set
%matplotlib inline
from matplotlib import pyplot as plt
image1=faces['facedat'][0][0][:,:,0]
plt.imshow(image1,cmap='Greys_r')
plt.show()
from matplotlib import pyplot as plt
image1=faces['facedat'][0][1][:,:,0]
plt.imshow(image1,cmap='Greys_r')
plt.show()
Model Building
Images pixel values are in multidimensional array. Every model trains on a 2-D data frame or array. So we should convert our image pixel values array into a 2-D array.
import numpy as np
a=np.concatenate((faces['facedat'][0][0:]),axis=2) # concatenating all 20 images arrays into a single array(4D to 3D)
a.shape
b=a.reshape((112*92,575)) #chnaging 3D array 'a' into 2D array
b.shape
facedat_2d=b.swapaxes(1,0) #swaping axes to get a image pixel values into a row
facedat_2d.shape
facedat_2d is our required 2D array. Next we should create a label array. There are twenty persons, each have different number of images. I am assigning labels from 0 to 19 for 1a to 1t respectively.
labels=np.zeros((575,)) #defining labels array as zero matrix
ip=0
out=faces['facedat'][0][0].shape[2]
for i in range(0,20):
labels[ip:out]=i
if (i<19):
out=out+faces['facedat'][0][i+1].shape[2]
ip=ip+faces['facedat'][0][i].shape[2]
labels.shape
labels array has labels of the images which are present in rows of feature_2d array. Next we convert the feature_2d,labels arrays into data frames for easy handling of the data.
import pandas as pd
x_train=pd.DataFrame(facedat_2d)
y_train=pd.DataFrame(labels)
Now our data is good for training any model. We will build different models and we choose the model which has high accuracy.
Neural Network
For building neural networks we will use neurolab package. We will directly use image pixel values for training.
import neurolab as nl
import pylab as pl
#function to find min,max of each column
def minMax(x):
return pd.Series(index=['min','max'],data=[x.min(),x.max()])
#storing min,max values into a list
listvalues = x_train.apply(minMax).T.values.tolist()
error = []
# Creating network with 1 hidden layer and random initialized
net = nl.net.newff(listvalues,[20,1],transf=[nl.trans.LogSig()] * 2)
net.trainf = nl.train.train_rprop
# Training network
import time #to take note of time take to train the network
start_time = time.time()
error.append(net.train(x_train, y_train, show=0, epochs = 250,goal=0.02))
print("--- %s seconds ---" % (time.time() - start_time))
#predicting using trained network
predicted_values = net.sim(x_train)
Even with very few nodes in hidden layer, neural network is taking huge time(around 1 hour) to train. So we will build another model.
SVM
We use image pixel values for training the model.
from sklearn import svm
clf = svm.SVC()
model =clf.fit(x_train,y_train) #training the model
clf
Predicted=clf.predict(x_train) #predictions of training data
#confusion matrix
from sklearn.metrics import confusion_matrix as cm
ConfusionMatrix = cm(y_train,Predicted)
print(ConfusionMatrix)
#accuracy
accuracy=np.trace(ConfusionMatrix)/sum(sum(ConfusionMatrix))
print(accuracy)
Accuracy is one. It means that the model may be overfitting the data. Lets check for overfitting by doing cross validation on the model.
Cross Validation
We will do shuffleshift cross validation. Our model has 575 samples consisting of 20 persons faces. Each person has around 20 to 30 face images. So we should keep the ratio of testing to training samples in cross validation very low. If it is high cross validation method may assign most of the samples of some classes into testing set, which will result in low score. To avoid this we will keep the ratio very small.
from sklearn import cross_validation
cv = cross_validation.ShuffleSplit(575, n_iter=10,test_size=0.05, random_state=None)
scores = cross_validation.cross_val_score(clf,x_train,y_train,cv = cv)
scores_mean = scores.mean()
print('score=',scores_mean) #mean of all the scores
print('# support vectors of the model are')
len(clf.support_) # gives Number of support vectors in the model
Score is very low.Which indicates overfitting of the model. Number of support vectors in the model is close total number of images. It is taking every image as a support vector. Our samples are way less than features(pixels). This is leading to overfitting.So we extract features of the images for face detection.
PCA
PCA method has been popular technique in face recognition problem. We will do PCA on our images.
from sklearn.decomposition import RandomizedPCA
#Number of eigen faces to be used
n_components = 50
#pca implementation
pca = RandomizedPCA(n_components=n_components, whiten=True).fit(x_train)
#Projecting the input data on the eigenfaces orthonormal basis
x_train_pca = pca.transform(x_train)
x_train_pca.shape
x_train_pca has weights of images. We use this data for training our svm model.
from sklearn import svm
clf = svm.SVC()
model =clf.fit(x_train_pca,y_train)
print("Number of support vectors of the model are")
len(clf.support_) # gives Number of support vectors in the model
Predicted=clf.predict(x_train_pca)
from sklearn.metrics import confusion_matrix as cm
ConfusionMatrix = cm(y_train,Predicted)
print(ConfusionMatrix)
#accuracy
accuracy=np.trace(ConfusionMatrix)/sum(sum(ConfusionMatrix))
print(accuracy)
Cross Validation
from sklearn import cross_validation
cv = cross_validation.ShuffleSplit(575, n_iter=10,test_size=0.05, random_state=None)
scores = cross_validation.cross_val_score(clf,x_train_pca,y_train,cv = cv)
scores_mean = scores.mean()
print('score=',scores_mean) #mean of all the scores
Score is very good. PCA weights are good features for face recognition.
Other Features
We used pca to get features of the images. Now we use mean,variance,skewness,kurtosis of the images for training the model
means=x_train.mean(axis=1)
variances=x_train.var(axis=1)
skewness=x_train.skew(axis=1)
kurtosis=x_train.kurtosis(axis=1)
features = pd.concat([ means,variances, skewness, kurtosis], axis =1)
features.shape
features.head(5)
features variable has mean,variance,skewness,kurtosis of every image in rows. We use this for building the model.
clf = svm.SVC()
model =clf.fit(features,y_train)
Predicted=clf.predict(features)
print("Number of support vectors of the model are")
len(clf.support_) # gives Number of support vectors in the model
from sklearn.metrics import confusion_matrix as cm
ConfusionMatrix = cm(y_train,Predicted)
print(ConfusionMatrix)
#accuracy
accuracy=np.trace(ConfusionMatrix)/sum(sum(ConfusionMatrix))
print(accuracy)
Cross Validation
from sklearn import cross_validation
cv = cross_validation.ShuffleSplit(575, n_iter=10,test_size=0.05, random_state=None)
scores = cross_validation.cross_val_score(clf,x_train_pca,y_train,cv = cv)
scores_mean = scores.mean()
print('score=',scores_mean) #mean of all the scores
Cross validation score is good for both the models using pca and features of mean,variance,kurtosis,skewnees, but the later model is using most of the samples as support vectors which is not good. So we dont use that model. We take the model which is using pca features.
Random Forest
We will build Random Forest model using PCA weights.
from sklearn.ensemble import RandomForestClassifier
forest=RandomForestClassifier(n_estimators=10, criterion='gini', max_depth=None, min_samples_split=2,
min_samples_leaf=1, min_weight_fraction_leaf=0.0, max_features='auto',
max_leaf_nodes=None, bootstrap=True, oob_score=False, n_jobs=1, random_state=None,
verbose=0, warm_start=False, class_weight=None)
forest.fit(x_train_pca,y_train)
Predicted=forest.predict(x_train_pca)
from sklearn.metrics import confusion_matrix as cm
ConfusionMatrix = cm(y_train,Predicted)
print(ConfusionMatrix)
#accuracy
accuracy=np.trace(ConfusionMatrix)/sum(sum(ConfusionMatrix))
print(accuracy)
Cross Validation
from sklearn import cross_validation
cv = cross_validation.ShuffleSplit(575, n_iter=10,test_size=0.05, random_state=None)
scores = cross_validation.cross_val_score(forest,x_train_pca,y_train,cv = cv)
scores_mean = scores.mean()
print('score=',scores_mean) #mean of all the scores
Cross validation Score is good. Random Forest model is also a good model.
Bagging
Now we will do bagging on the x_train_pca data set.
from sklearn.ensemble import BaggingClassifier
bagging=BaggingClassifier(base_estimator=svm.SVC(), n_estimators=5, max_samples=1.0,
max_features=1.0, bootstrap=True, bootstrap_features=False, oob_score=False,
warm_start=False, n_jobs=1, random_state=None, verbose=0)
bagging.fit(x_train_pca,y_train)
Predicted=bagging.predict(x_train_pca)
from sklearn.metrics import confusion_matrix as cm
ConfusionMatrix = cm(y_train,Predicted)
print(ConfusionMatrix)
#accuracy
accuracy=np.trace(ConfusionMatrix)/sum(sum(ConfusionMatrix))
print(accuracy)
Cross Validation
cv = cross_validation.ShuffleSplit(575, n_iter=10,test_size=0.05, random_state=None)
scores = cross_validation.cross_val_score(forest,x_train_pca,y_train,cv = cv)
scores_mean = scores.mean()
print('score=',scores_mean) #mean of all the scores
Conclusion
Cross validation scores of all the models are good. SVM is giving highest score. So we take that as our final model.
