{"id":2161,"date":"2019-04-29T07:27:15","date_gmt":"2019-04-29T07:27:15","guid":{"rendered":"https:\/\/cloudxlab.com\/blog\/?p=2161"},"modified":"2019-04-29T09:39:28","modified_gmt":"2019-04-29T09:39:28","slug":"regression-using-tensorflow-keras-api","status":"publish","type":"post","link":"https:\/\/cloudxlab.com\/blog\/regression-using-tensorflow-keras-api\/","title":{"rendered":"Regression with Neural Networks using TensorFlow Keras API"},"content":{"rendered":"\n<p>As part of this blog post, I am going to walk you through how an Artificial Neural Network figures out a complex relationship in data by itself without much of our hand-holding. You should modify the data generation function and observe if it is able to predict the result correctly. I am going to use the Keras API of TensorFlow. Keras API makes it really easy to create Deep Learning models. <\/p>\n\n\n\n<p>Machine learning is about computer figuring out relationships in data by itself as opposed to programmers figuring out and writing code\/rules. Machine learning generally is categorized into two types: Supervised and Unsupervised. In supervised, we have the supervision available. And supervised learning is further classified into Regression and Classification. In classification, we have training data with features and labels and the machine should learn from this training data on how to label a record. In regression, the computer\/machine should be able to predict a value &#8211; mostly numeric. An example of Regression is predicting the salary of a person based on various attributes: age, years of experience, the domain of expertise, gender.<\/p>\n\n\n\n<p>The notebook having all the code is available <a href=\"https:\/\/github.com\/cloudxlab\/ml\/blob\/master\/deep_learning\/tensorflow_keras_regression.ipynb\">here<\/a> on GitHub as part of <a href=\"https:\/\/github.com\/cloudxlab\/ml\">cloudxlab repository<\/a> at the location <em>deep_learning\/tensorflow_keras_regression.ipynb . <\/em>I am going to walk you through the code from this notebook here.<\/p>\n\n\n\n<p><strong>Generate Data:<\/strong> Here we are going to generate some data using our own function. This function is a non-linear function and a usual line fitting may not work for such a function<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>def myfunc(x):\n    if x &lt; 30:\n        mult = 10\n    elif x &lt; 60:\n        mult = 20\n    else:\n        mult = 50\n    return x*mult\n<\/code><\/pre>\n\n\n\n<!--more-->\n\n\n\n<p>Let us check what does this function return.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>print(myfunc(10))\nprint(myfunc(30))\nprint(myfunc(60))<\/code><\/pre>\n\n\n\n<p>It should print something like:<br><span id=\"gc-number-1\" class=\"gc-cs-link\" title=\"Call with Google Voice\">100<\/span><br><span id=\"gc-number-1\" class=\"gc-cs-link\" title=\"Call with Google Voice\">600<\/span><br><span id=\"gc-number-1\" class=\"gc-cs-link\" title=\"Call with Google Voice\">3000<\/span><\/p>\n\n\n\n<p>Now, let us generate data. here <strong>x<\/strong> is a <em>numpy<\/em> array of input values. Let us import <em>numpy<\/em> library as <em>np<\/em>. Then using <em>arange<\/em> function we are generating values between <em>0<\/em> and <em>100<\/em> with a gap of <em>0.01.<\/em> It generates a <em>numpy<\/em> array.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>import numpy as np\nx = np.arange(0, 100, .01)<\/code><\/pre>\n\n\n\n<p>To call a function repeatedly on a <em>numpy<\/em> array we first need to convert the function using vectorize. Afterwards, we are converting 1-D array to 2-D array having only one value in the second dimension &#8211; you can think of it as a table of data with only one column.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>myfuncv = np.vectorize(myfunc)\ny = myfuncv(x)\nX = x.reshape(-1, 1)<\/code><\/pre>\n\n\n\n<p>Now, we have X representing the input data with single feature and y representing the output. We will now split this data into two parts: training set (<em>X_train, y_train)<\/em> and test set (<em>X_test  y_test<\/em>). We are going make neural network learn from training data, and once it has learnt &#8211; how to produce y from X &#8211; we are going to test the model on the test set.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>import sklearn.model_selection as sk\n\nX_train, X_test, y_train, y_test = sk.train_test_split(X,y,test_size=0.33, random_state = 42)<\/code><\/pre>\n\n\n\n<p>Let us visualize how does our data looks like. Here, we are plotting only X_train vs y_train. You can try plotting X vs y, as well as, X_test vs y_test.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>import matplotlib.pyplot as plt\nplt.scatter(X_train,y_train)<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image\"><img width=\"384\" height=\"252\" src=\"https:\/\/blog.cloudxlab.com\/wp-content\/uploads\/2019\/04\/x_train.png\" alt=\"\" class=\"wp-image-2164\" srcset=\"https:\/\/cloudxlab.com\/blog\/wp-content\/uploads\/2019\/04\/x_train.png 384w, https:\/\/cloudxlab.com\/blog\/wp-content\/uploads\/2019\/04\/x_train-300x197.png 300w\" sizes=\"(max-width: 384px) 85vw, 384px\" \/><\/figure>\n\n\n\n<p>Let us now create a neural network and see if it can figure out the relationship. There are three steps involved: Create Neural Network, Train it and Test it.<\/p>\n\n\n\n<p>Let us import TensorFlow libraries and check the version.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>import tensorflow as tf\nimport numpy as np\nprint(tf.__version__)<\/code><\/pre>\n\n\n\n<p>It should print something like this:<br>&#8216;1.10.0&#8217;<\/p>\n\n\n\n<p>Now, let us create a neural network using Keras API of TensorFlow.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># Import the kera modules\nfrom keras.layers import Input, Dense\nfrom keras.models import Model\n\n# This returns a tensor. Since the input only has one column\ninputs = Input(shape=(1,))\n\n# a layer instance is callable on a tensor, and returns a tensor\n# To the first layer we are feeding inputs\nx = Dense(32, activation='relu')(inputs)\n# To the next layer we are feeding the result of previous call here it is h\nx = Dense(64, activation='relu')(x)\nx = Dense(64, activation='relu')(x)\n\n# Predictions are the result of the neural network. Notice that the predictions are also having one column.\npredictions = Dense(1)(x)\n\n# This creates a model that includes\n# the Input layer and three Dense layers\nmodel = Model(inputs=inputs, outputs=predictions)\n# Here the loss function is mse - Mean Squared Error because it is a regression problem.\nmodel.compile(optimizer='rmsprop',\n              loss='mse',\n              metrics=['mse'])<\/code><\/pre>\n\n\n\n<p>Let us now train the model. First, it would initialize the weights of each neuron with random values and the using backpropagation it is going to tweak the weights in order to get the appropriate result.  Here we are running the iteration 500 times and we are feeding 100 records of X at a time.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>model.fit(X_train, y_train,  epochs=500, batch_size=100)  # starts training<\/code><\/pre>\n\n\n\n<p>It might show something like this on screen:<\/p>\n\n\n\n<p><em>Epoch 1\/500 6700\/6700 [==============================] &#8211; 0s 36us\/step &#8211; loss: 6084593.4888 &#8211; mean_squared_error: 6084593.4888 <\/em><\/p>\n\n\n\n<p><em>Epoch 2\/500 6700\/6700 [==============================] &#8211; 0s 13us\/step &#8211; loss: 2762668.9375 &#8211; mean_squared_error: 2762668.9375<br>&#8230;..<\/em><\/p>\n\n\n\n<p>Once we have trained the model. We can do predictions using the <em>predict<\/em> method of the model. In our case, this model should predict y using X. Let us test it over our test set. Plot the results.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># x_test = np.arange(0, 100, 0.02)\n# X_test = x_test.reshape(-1, 1)\ny_test = model.predict(X_test)\nplt.scatter(X_test, y_test)<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image\"><img width=\"384\" height=\"252\" src=\"https:\/\/blog.cloudxlab.com\/wp-content\/uploads\/2019\/04\/test_predictions.png\" alt=\"\" class=\"wp-image-2168\" srcset=\"https:\/\/cloudxlab.com\/blog\/wp-content\/uploads\/2019\/04\/test_predictions.png 384w, https:\/\/cloudxlab.com\/blog\/wp-content\/uploads\/2019\/04\/test_predictions-300x197.png 300w\" sizes=\"(max-width: 384px) 85vw, 384px\" \/><figcaption>Predictions of the neural network<\/figcaption><\/figure>\n\n\n\n<p>So, the predictions are very similar to the actual values.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>As part of this blog post, I am going to walk you through how an Artificial Neural Network figures out a complex relationship in data by itself without much of our hand-holding. You should modify the data generation function and observe if it is able to predict the result correctly. I am going to use &hellip; <a href=\"https:\/\/cloudxlab.com\/blog\/regression-using-tensorflow-keras-api\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Regression with Neural Networks using TensorFlow Keras API&#8221;<\/span><\/a><\/p>\n","protected":false},"author":14,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[29,30,1],"tags":[17],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v16.2 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Regression with Neural Networks using TensorFlow Keras API | CloudxLab Blog<\/title>\n<meta name=\"description\" content=\"Regression with Artificial Neural Networks using Keras API of Tensorflow\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/cloudxlab.com\/blog\/regression-using-tensorflow-keras-api\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Regression with Neural Networks using TensorFlow Keras API | CloudxLab Blog\" \/>\n<meta property=\"og:description\" content=\"Regression with Artificial Neural Networks using Keras API of Tensorflow\" \/>\n<meta property=\"og:url\" content=\"https:\/\/cloudxlab.com\/blog\/regression-using-tensorflow-keras-api\/\" \/>\n<meta property=\"og:site_name\" content=\"CloudxLab Blog\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/cloudxlab\" \/>\n<meta property=\"article:published_time\" content=\"2019-04-29T07:27:15+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2019-04-29T09:39:28+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/blog.cloudxlab.com\/wp-content\/uploads\/2019\/04\/x_train.png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@CloudxLab\" \/>\n<meta name=\"twitter:site\" content=\"@CloudxLab\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\">\n\t<meta name=\"twitter:data1\" content=\"5 minutes\">\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebSite\",\"@id\":\"https:\/\/cloudxlab.com\/blog\/#website\",\"url\":\"https:\/\/cloudxlab.com\/blog\/\",\"name\":\"CloudxLab Blog\",\"description\":\"Learn AI, Machine Learning, Deep Learning, Devops &amp; Big Data\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":\"https:\/\/cloudxlab.com\/blog\/?s={search_term_string}\",\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/cloudxlab.com\/blog\/regression-using-tensorflow-keras-api\/#primaryimage\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/blog.cloudxlab.com\/wp-content\/uploads\/2019\/04\/x_train.png\",\"contentUrl\":\"https:\/\/blog.cloudxlab.com\/wp-content\/uploads\/2019\/04\/x_train.png\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/cloudxlab.com\/blog\/regression-using-tensorflow-keras-api\/#webpage\",\"url\":\"https:\/\/cloudxlab.com\/blog\/regression-using-tensorflow-keras-api\/\",\"name\":\"Regression with Neural Networks using TensorFlow Keras API | CloudxLab Blog\",\"isPartOf\":{\"@id\":\"https:\/\/cloudxlab.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/cloudxlab.com\/blog\/regression-using-tensorflow-keras-api\/#primaryimage\"},\"datePublished\":\"2019-04-29T07:27:15+00:00\",\"dateModified\":\"2019-04-29T09:39:28+00:00\",\"author\":{\"@id\":\"https:\/\/cloudxlab.com\/blog\/#\/schema\/person\/4835f1b3d5000626cb15e9311d748e09\"},\"description\":\"Regression with Artificial Neural Networks using Keras API of Tensorflow\",\"breadcrumb\":{\"@id\":\"https:\/\/cloudxlab.com\/blog\/regression-using-tensorflow-keras-api\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/cloudxlab.com\/blog\/regression-using-tensorflow-keras-api\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/cloudxlab.com\/blog\/regression-using-tensorflow-keras-api\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"item\":{\"@type\":\"WebPage\",\"@id\":\"https:\/\/cloudxlab.com\/blog\/\",\"url\":\"https:\/\/cloudxlab.com\/blog\/\",\"name\":\"Home\"}},{\"@type\":\"ListItem\",\"position\":2,\"item\":{\"@id\":\"https:\/\/cloudxlab.com\/blog\/regression-using-tensorflow-keras-api\/#webpage\"}}]},{\"@type\":\"Person\",\"@id\":\"https:\/\/cloudxlab.com\/blog\/#\/schema\/person\/4835f1b3d5000626cb15e9311d748e09\",\"name\":\"Sandeep Giri\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/cloudxlab.com\/blog\/#personlogo\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/1393214840cf7455bb4cba055cb30468?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/1393214840cf7455bb4cba055cb30468?s=96&d=mm&r=g\",\"caption\":\"Sandeep Giri\"},\"sameAs\":[\"https:\/\/cloudxlab.com\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","_links":{"self":[{"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/posts\/2161"}],"collection":[{"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/users\/14"}],"replies":[{"embeddable":true,"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/comments?post=2161"}],"version-history":[{"count":21,"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/posts\/2161\/revisions"}],"predecessor-version":[{"id":2188,"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/posts\/2161\/revisions\/2188"}],"wp:attachment":[{"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/media?parent=2161"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/categories?post=2161"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/tags?post=2161"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}