In the current article, I am presenting the results of my experiments with Fashion-MNIST using Deep Learning (Convolutional Neural Network – CNN) which I have implemented using TensorFlow Keras APIs (version 2.1.6-tf).<\/p>\n\n\n\n\n\n\n\n
The complete code for this project you can find here :<\/p>\n\n\n\n
https:\/\/github.com\/cloudxlab\/ml\/tree\/master\/projects\/Fashion-MNIST<\/a><\/p>\n\n\n\n Fashion-MNIST is a dataset of Zalando’s fashion article images \u2014consisting of a training set of 60,000<\/strong> examples and a test set of 10,000<\/strong> examples. Each instance is a 28×28 grayscale image, associated with a label.<\/p>\n\n\n\n This work is part of my experiments with Fashion-MNIST dataset using Convolutional Neural Network (CNN) which I have implemented using TensorFlow Keras APIs(version 2.1.6-tf). The objective is to identify (predict) different fashion products from the given images using a CNN model. For regularization, I have used \u2018dropout\u2019 technique for this problem.<\/p>\n\n\n\n I have used Fashion-MNIST dataset for this experiment with Deep Learning. Fashion-MNIST dataset is a collection of fashion articles images provided by Zalando<\/a> . Thanks to Zalando Research for hosting the dataset<\/a>. <\/p>\n\n\n\n Fashion MNIST Training dataset consists of 60,000 images and each image has 784 features (i.e. 28×28 pixels). Each pixel is a value from 0 to 255, describing the pixel intensity. 0 for white and 255 for black.<\/p>\n\n\n\n The class labels for Fashion MNIST are: Let us have a look at one instance (an article image), say at index 220, of the training dataset.<\/p>\n\n\n\n So, we see that image at index (instance no.) 220 is a bag, and the corresponding label value also indicates the same (8 – Bag).<\/p>\n\n\n\n The \u2018target\u2019 dataset has 10 class labels, as we can see from above (0 – T-shirt\/top, 1 \u2013 Trouser,,\u2026.9 \u2013 Ankle Boot). <\/p>\n\n\n\n Given the images of the articles, we need to classify them into one of these classes, hence, it is essentially a \u2018Multi-class Classification\u2019<\/strong> problem. We will be using CNN to come up with a model for this problem and will use \u201cAccuracy\u201d as the performance measure.<\/p>\n\n\n\n We already have the splitted dataset (training and test) available in ratio 85:15 (60,000:10,000) from Zalando Research, we will use the same for this experiment.<\/p>\n\n\n\n As part of data preparation, following techniques were applied on the dataset:<\/p>\n\n\n We shuffle the training dataset to get uniform samples for cross-validation. This also ensures that we don’t miss out any digit in a cross-validation fold. Each image (instance) in the dataset has 784 pixels (features) and value of each feature(pixel) ranges from 0 to 255, this range is too wide, hence we have performed Normalization on the training and test dataset, by dividing the pixels by 255, so that values of all features (pixels) are in a small range (0 to 1). Below is the architecture of my final CNN model. It has 3 convolutional layers, 2 max. Pool layers, 2 dropout layers, 1 fully connected (dense) layer and 1 output (softmax) layer. Apart from these, it also has a flatten layer whose purpose is just to \u2018flatten\u2019 the output, i.e. convert a 2-D output to a 1-D output (which is then fed to the dense layer).<\/p>\n\n\n\n During the training phase itself the cross validation (validation dataset = 10% of training dataset) and hyperparameter tuning were performed on the training dataset. First, I tried with a CNN model without any regularization technique applied to it, below table shows the results of training and cross validation (validation) for the same. <\/p>\n\n\n\n Deep learning models have a high tendency to overfit (perform well on training dataset than on the validation\/test dataset), the same can be seen in the below results.<\/p>\n\n\n\n Results of CNN model without applying Regularization<\/strong><\/p>\n\n\n\n A training accuracy of 99% and test accuracy of 92% confirms that model is overfitting. To solve the model overfitting issue, I applied regularization technique called \u2018Dropout\u2019 and also introduced a few more max. pool layers. The architecture diagram for this CNN model is shown above (under section – CNN Model Architecture). Below are the results for the same.<\/p>\n\n\n\n Results of the CNN model with regularization (dropout) applied<\/strong><\/p>\n\n\n\n A training accuracy value of 94% and test accuracy of 93% confirms that model is performing fine and there is no overfitting<\/strong>. Thus, this is our final CNN model. Please note that \u2018Dropout\u2019 should only be applied during the training phase and not during the test phase. For the test phase, TensorFlow Keras API (evaluate() method) automatically takes care of this internally, we needn\u2019t specify any parameter for this (to not apply Dropout). Below are the plots for ‘Accuracy’ and ‘Loss’ for training and validation(test) phases for this final CNN model. Let us make a prediction for one of the instances\/images (say instance no. 88) using our final CNN model (new_model).<\/p>\n\n\n\n Thus, we see that, our CNN model has predicted it right, actual instance (image) at instance no. 88 is a T-shirt and our predicted value also confirms the same (test labels[88] = 0; test_images[88] – T-shirt).<\/p>\n\n\n\n With our final CNN model, we could achieve a training accuracy of 94% and test accuracy of 93%<\/strong> confirming that model is fine with no overfitting<\/strong>. If you remember, with Machine Learning<\/strong> model (XGBoost) I had achieved a test accuracy of 84.72 %<\/strong>, and with Deep Learning <\/strong>model (CNN) here I could achieve a test accuracy of 93 %<\/strong>. Thus, we got around 8% improvement in accuracy<\/em> by using Deep Learning.<\/p>\n\n\n\n Though, in this case, we got a good improvement in accuracy score (8%), still there may be a chance to improve performance further, by say, increasing the number of convolutional layers (and neurons\/filters) or trying out different combinations of different layers.<\/p>\n\n\n\n The complete code for this project you can find here : https:\/\/github.com\/cloudxlab\/ml\/tree\/master\/projects\/Fashion-MNIST<\/a> For the complete course on Machine Learning, please visit Specialization Course on Machine Learning & Deep Learning<\/a>![\"Fashion-MNIST](\"https:\/\/blog.cloudxlab.com\/wp-content\/uploads\/2019\/06\/Screenshot-2019-03-13-at-10.46.28.png\")
Objective<\/strong>
<\/h2>\n\n\n\nAcknowledgements <\/strong>
<\/h2>\n\n\n\nUnderstanding and Analysing the dataset<\/strong>
<\/h2>\n\n\n\n
<\/p>\n\n\n\nLabel <\/strong><\/td> Description <\/strong><\/td><\/tr> 0<\/td> T-shirt\/top<\/td><\/tr> 1<\/td> Trouser<\/td><\/tr> 2<\/td> Pullover<\/td><\/tr> 3<\/td> Dress<\/td><\/tr> 4<\/td> Coat<\/td><\/tr> 5<\/td> Sandal<\/td><\/tr> 6<\/td> Shirt<\/td><\/tr> 7<\/td> Sneaker<\/td><\/tr> 8<\/td> Bag<\/td><\/tr> 9<\/td> Ankle boot <\/td><\/tr><\/tbody><\/table>\n\n\n\n plt.figure() \nplt.imshow(np.squeeze(train_images[220]))<\/code><\/pre>\n\n\n\n![\"One](\"https:\/\/blog.cloudxlab.com\/wp-content\/uploads\/2019\/06\/Screenshot-2019-06-17-at-17.48.37.png\")
train_labels[220]\n8<\/code><\/pre>\n\n\n\nProblem Definition<\/strong>
<\/h2>\n\n\n\n
<\/p>\n\n\n\nPreparing the Data<\/strong>
<\/h2>\n\n\n\nfashion_mnist = keras.datasets.fashion_mnist\n\n(train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data()\n\ntrain_images.shape\n\n(60000, 28, 28)\n\ntrain_labels.shape\n\n(60000,)\n\ntest_images.shape\n\n(10000, 28, 28)\n\ntest_labels.shape\n\n(10000,)<\/code><\/pre>\n\n\n\n\n
\n
<\/p>\n\n\n\n
<\/p>\n\n\ntrain_images_norm = train_images \/ 255.0\ntest_images_norm = test_images \/ 255.0\n<\/pre>\n\n\n
CNN Model Architecture<\/strong>
<\/h2>\n\n\n\n![\"CNN](\"https:\/\/blog.cloudxlab.com\/wp-content\/uploads\/2019\/06\/Screenshot-2019-06-17-at-15.01.43.png\")
new_model = models.Sequential()\n\nnew_model.add(conv1)\n\n# No Pooling Layer and Dropout layer for first Convolutional layer 'conv1'\n\nnew_model.add(conv2)\nnew_model.add(max_pool_2)\nnew_model.add(drop_2)\nnew_model.add(conv3)\nnew_model.add(max_pool_3)\nnew_model.add(drop_3)\nnew_model.add(flat_layer)\nnew_model.add(fc)\nnew_model.add(output)<\/code><\/pre>\n\n\n\n![\"CNN](\"https:\/\/blog.cloudxlab.com\/wp-content\/uploads\/2019\/06\/Screenshot-2019-06-17-at-16.43.25.png\")
Training, Validation and Test Results<\/strong><\/h2>\n\n\n\n
<\/p>\n\n\n\n![\"Results](\"https:\/\/blog.cloudxlab.com\/wp-content\/uploads\/2019\/06\/Screenshot-2019-06-17-at-18.12.21.png\")
<\/p>\n\n\n\n![\"Results](\"https:\/\/blog.cloudxlab.com\/wp-content\/uploads\/2019\/06\/Screenshot-2019-06-17-at-18.15.06.png\")
<\/p>\n\n\n\n
<\/p>\n\n\n\n
<\/p>\n\n\n\n![\"'Accuracy'](\"https:\/\/blog.cloudxlab.com\/wp-content\/uploads\/2019\/06\/Screenshot-2019-06-17-at-16.16.26.png\")
![\"'Loss'](\"https:\/\/blog.cloudxlab.com\/wp-content\/uploads\/2019\/06\/Screenshot-2019-06-17-at-16.16.48.png\")
# Making predictions on Test dataset\n\npredicted_test_labels = new_model.predict(test_images_norm)\n\npredicted_test_labels.shape\n\n(10000, 10)\n\npredicted_test_labels[88]\n\narray([9.9878651e-01, 1.5129776e-09, 1.0235499e-04, 1.5311174e-05, 6.6746189e-08, 5.6005112e-10, 1.0947105e-03, 1.0985579e-12,\n9.4132378e-07, 1.5505348e-10], dtype=float32)\n\npredicted_test_labels_index = np.argmax(predicted_test_labels[88])\n\npredicted_test_labels_index\n\n0\n\ntest_labels[88]\n\n0\n\nplt.figure()\n\nplt.imshow(np.squeeze(test_images[88]))\n<\/code><\/pre>\n\n\n\n![\"\"](\"https:\/\/blog.cloudxlab.com\/wp-content\/uploads\/2019\/06\/Screenshot-2019-06-17-at-18.24.58.png\")
<\/figure>\n\n\n\nConclusion<\/strong>
<\/h2>\n\n\n\n
<\/p>\n\n\n\n
<\/p>\n\n\n\n
<\/p>\n\n\n\n
<\/p>\n","protected":false},"excerpt":{"rendered":"