{"id":2969,"date":"2020-04-21T01:57:12","date_gmt":"2020-04-21T01:57:12","guid":{"rendered":"https:\/\/cloudxlab.com\/blog\/?p=2969"},"modified":"2020-04-22T16:35:34","modified_gmt":"2020-04-22T16:35:34","slug":"how-to-build-a-number-plate-reader-part-2","status":"publish","type":"post","link":"https:\/\/cloudxlab.com\/blog\/how-to-build-a-number-plate-reader-part-2\/","title":{"rendered":"How to build a Number Plate Reader &#8211; Part 2"},"content":{"rendered":"\n<p>In the <a href=\"https:\/\/cloudxlab.com\/blog\/number-plate-reader\/\">previous blog<\/a> of this series, we trained a model to identify a numberplate in a picture. Here we will learn how to use OpenCV and PyTesseract to get the final number from the plate. <\/p>\n\n\n\n<p>We will start from where we ended in the last session. We had trained an ssd_inception model and used it Tensorflow Object Detection API to detect number plates. <\/p>\n\n\n\n<!--more-->\n\n\n\n<figure class=\"wp-block-image size-large\"><img width=\"1280\" height=\"960\" src=\"https:\/\/blog.cloudxlab.com\/wp-content\/uploads\/2020\/04\/licence_plate_detected-1.png\" alt=\"number plate reader - number plate detected\" class=\"wp-image-2973\" \/><figcaption>Number plate detected<\/figcaption><\/figure>\n\n\n\n<h2>Number Plate Reader<\/h2>\n\n\n\n<p>We will first give the code for the number plate reader. The code is commented, but we will go through to explain each line.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code class=\"language-python line-numbers\">import numpy as np\nimport tensorflow as tf\nimport cv2 as cv\nimport pytesseract\nimport re\nimport os\nfrom os import listdir\nfrom os.path import isfile, join\nimport imutils\n\n\n#Change PATH names to match your file paths for the model and INPUT_FILE'\nMODEL_NAME='ssd_inception_output_inference_graph_v1.pb'\nPATH_TO_FROZEN_GRAPH = MODEL_NAME + '\/frozen_inference_graph.pb'\nINPUT_FILE='test_images\/img6.jpeg'\n\n\n#Read the model from the file\nwith tf.gfile.FastGFile(PATH_TO_FROZEN_GRAPH, 'rb') as f:\n    graph_def = tf.GraphDef()\n    graph_def.ParseFromString(f.read())\n\n\ndef cleanup_image(img):\n    #Resize the image with interpolation\n    cv.imwrite(\"number_plate_original.png\", img)\n    img = cv.resize(img, None, fx=2.5, fy=2.5, interpolation=cv.INTER_CUBIC)\n    cv.imwrite(\"number_plate_resized.png\", img)\n    #Convert to Grey scale\n    img = cv.cvtColor(img, cv.COLOR_BGR2GRAY)\n    cv.imwrite(\"number_plate_greyed.png\", img)\n\n    #Define the kernel for morphological operations\n    kernel = np.ones((3, 3), np.uint8)\n    # Dilation expands whitespace breaking up characters and removing noise\n    img = cv.dilate(img, kernel, iterations=1)\n    cv.imwrite(\"number_plate_dilated.png\", img)\n\n    # Erosion makes the dark characters bigger\n\n    img = cv.erode(img, kernel, iterations=1)\n    cv.imwrite(\"number_plate_eroded.png\", img)\n\n    string=(pytesseract.image_to_string(img))\n    string=re.sub(r'\\W+', '', string)\n    return (string)\n\n\n\n\n\n\n\n#Create the tensorflow session\nwith tf.Session() as sess:\n\n    sess.graph.as_default()\n    tf.import_graph_def(graph_def, name='')\n    # Read the input file\n    img = cv.imread(INPUT_FILE)\n\n\n    rows = img.shape[0]\n    cols = img.shape[1]\n    inp = cv.resize(img, (300, 300))\n    inp = inp[:, :, [2, 1, 0]]  # BGR2RGB\n\n    # Run the model\n    out = sess.run([sess.graph.get_tensor_by_name('num_detections:0'),\n                    sess.graph.get_tensor_by_name('detection_scores:0'),\n                    sess.graph.get_tensor_by_name('detection_boxes:0'),\n                    sess.graph.get_tensor_by_name('detection_classes:0')],\n                   feed_dict={'image_tensor:0': inp.reshape(1, inp.shape[0], inp.shape[1], 3)})\n\n    # Visualize detected bounding boxes.\n    num_detections = int(out[0][0])\n\n    for i in range(num_detections):\n        classId = int(out[3][0][i])\n\n        score = float(out[1][0][i])\n        bbox = [float(v) for v in out[2][0][i]]\n\n        if score &gt; 0.9:\n            # Creating a box around the detected number plate\n            x = int(bbox[1] * cols)\n            y = int(bbox[0] * rows)\n            right = int(bbox[3] * cols)\n            bottom = int(bbox[2] * rows)\n            #Extract the detected number plate\n            tmp=img[ y: bottom, x: right ]\n            text=cleanup_image(tmp)\n            text_height=0.5\n            cv.rectangle(img, (x, y), (right, bottom), (125, 255, 51), thickness=2)\n            cv.putText(img, text, (x, y - 5),\n                cv.FONT_HERSHEY_SIMPLEX, text_height, (125, 255, 51), 2)\n            cv.imwrite('licence_plate_read.png', img)\n<\/code><\/pre>\n\n\n\n<p>The first step is to create an image of only the license plate. This is done by the following code<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code class=\"\">        #Extract the detected number plate\n        tmp=img[ y: bottom, x: right ]<\/code><\/pre>\n\n\n\n<p>TMP is an np array that represents the detected number plate. We then send this array to a function cleanup_image.<\/p>\n\n\n\n<p>The following steps are done in the function cleanup image.<\/p>\n\n\n\n<h4>upscale<\/h4>\n\n\n\n<p>We will upscale the image by 2.5 times and use basic interpolation so that detail is not lost. This makes is easier for us to manipulate the image<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img width=\"660\" height=\"168\" src=\"https:\/\/blog.cloudxlab.com\/wp-content\/uploads\/2020\/04\/number_plate_resized.png\" alt=\"number plate reader\" class=\"wp-image-2974\" \/><figcaption>Upscaled image<\/figcaption><\/figure>\n\n\n\n<h4>Convert To GREYSCAle<\/h4>\n\n\n\n<p>We convert the image to greyscale as it easier to delineate and read text in black and white.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img width=\"660\" height=\"168\" src=\"https:\/\/blog.cloudxlab.com\/wp-content\/uploads\/2020\/04\/number_plate_greyed.png\" alt=\"number plate reader\" class=\"wp-image-2975\" \/><figcaption>Greyed Image<\/figcaption><\/figure>\n\n\n\n<h4>Dilation and erosion<\/h4>\n\n\n\n<p>Dilation and Erosion are morphological operations in OpenCV. We first create a 3&#215;3 kernel or matrix and then run a convolution over the entire image. Dilation increases the whitespace, reducing noise or small dark spots in the image. After small dark pixels are eliminated, the bigger numbers are now clearly delineated, but may be smaller. Erosion now will make the dark letters a little bigger and easier to read.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img width=\"660\" height=\"168\" src=\"https:\/\/blog.cloudxlab.com\/wp-content\/uploads\/2020\/04\/number_plate_dilated.png\" alt=\"number plate reader\" class=\"wp-image-2976\" \/><figcaption>Dilated Image<\/figcaption><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large\"><img width=\"660\" height=\"168\" src=\"https:\/\/blog.cloudxlab.com\/wp-content\/uploads\/2020\/04\/number_plate_eroded.png\" alt=\"number plate reader\" class=\"wp-image-2977\" \/><figcaption>Eroded Image<\/figcaption><\/figure>\n\n\n\n<h4>PyTesseract OCR<\/h4>\n\n\n\n<p>We will now use Pytesseract OCR to read the text from the image. The text in image is now quite clear for Pytesseract to read. After running Pytesseract we return the string back to the main function. The main function now makes a square around with number plate with the read vehicle number.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img width=\"1280\" height=\"960\" src=\"https:\/\/blog.cloudxlab.com\/wp-content\/uploads\/2020\/04\/licence_plate_read.png\" alt=\"number plate reader working\" class=\"wp-image-2979\" \/><figcaption>Number Plate Read successfully<\/figcaption><\/figure>\n\n\n\n<h2>Further Improvement<\/h2>\n\n\n\n<p>The proposed algorithm has worked well for this number plate. However the same algorithm may not work well universally. There will be issues where number plates are blurred, the orientations are not correct, sunlight and shade causes disturbance, etc. I will propose a few improvements which can be worked on later.<\/p>\n\n\n\n<ol><li>We can train the detector to identify number plates on a variety of real pictures. More pictures the better the detection.<\/li><li>We can try the <a href=\"https:\/\/github.com\/idealo\/image-super-resolution\">super-resolution module<\/a>, for improving low resolution pictures.<\/li><li>The above algorithm works well when the number plate is black characters on white background. For other combinations the module may need tweaking.<\/li><li>For some difficult cases, we will have to <a href=\"https:\/\/gurus.pyimagesearch.com\/lesson-sample-segmenting-characters-from-license-plates\/\">extract contours<\/a> and run OCR on those extracted contours. <\/li><li>We can multiple algorithms and compare the outputs to a certain rule. The rule can be as simple as number of characters or sizes of characters. Rules can also be based on regex.<\/li><\/ol>\n","protected":false},"excerpt":{"rendered":"<p>In the previous blog of this series, we trained a model to identify a numberplate in a picture. Here we will learn how to use OpenCV and PyTesseract to get the final number from the plate. We will start from where we ended in the last session. We had trained an ssd_inception model and used &hellip; <a href=\"https:\/\/cloudxlab.com\/blog\/how-to-build-a-number-plate-reader-part-2\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;How to build a Number Plate Reader &#8211; Part 2&#8221;<\/span><\/a><\/p>\n","protected":false},"author":26,"featured_media":2997,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[67],"tags":[61],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v16.2 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>How to build a Number Plate Reader - Part 2 | CloudxLab Blog<\/title>\n<meta name=\"description\" content=\"Here we will learn how to use OpenCV and PyTesseract to get the final number from the plate. Previously we detected a number plate.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/cloudxlab.com\/blog\/how-to-build-a-number-plate-reader-part-2\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How to build a Number Plate Reader - Part 2 | CloudxLab Blog\" \/>\n<meta property=\"og:description\" content=\"Here we will learn how to use OpenCV and PyTesseract to get the final number from the plate. Previously we detected a number plate.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/cloudxlab.com\/blog\/how-to-build-a-number-plate-reader-part-2\/\" \/>\n<meta property=\"og:site_name\" content=\"CloudxLab Blog\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/cloudxlab\" \/>\n<meta property=\"article:published_time\" content=\"2020-04-21T01:57:12+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2020-04-22T16:35:34+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/blog.cloudxlab.com\/wp-content\/uploads\/2020\/04\/licence_plate_read-1.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1280\" \/>\n\t<meta property=\"og:image:height\" content=\"960\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@CloudxLab\" \/>\n<meta name=\"twitter:site\" content=\"@CloudxLab\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\">\n\t<meta name=\"twitter:data1\" content=\"5 minutes\">\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebSite\",\"@id\":\"https:\/\/cloudxlab.com\/blog\/#website\",\"url\":\"https:\/\/cloudxlab.com\/blog\/\",\"name\":\"CloudxLab Blog\",\"description\":\"Learn AI, Machine Learning, Deep Learning, Devops &amp; Big Data\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":\"https:\/\/cloudxlab.com\/blog\/?s={search_term_string}\",\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/cloudxlab.com\/blog\/how-to-build-a-number-plate-reader-part-2\/#primaryimage\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/cloudxlab.com\/blog\/wp-content\/uploads\/2020\/04\/licence_plate_read-1.png\",\"contentUrl\":\"https:\/\/cloudxlab.com\/blog\/wp-content\/uploads\/2020\/04\/licence_plate_read-1.png\",\"width\":1280,\"height\":960,\"caption\":\"number plate reader\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/cloudxlab.com\/blog\/how-to-build-a-number-plate-reader-part-2\/#webpage\",\"url\":\"https:\/\/cloudxlab.com\/blog\/how-to-build-a-number-plate-reader-part-2\/\",\"name\":\"How to build a Number Plate Reader - Part 2 | CloudxLab Blog\",\"isPartOf\":{\"@id\":\"https:\/\/cloudxlab.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/cloudxlab.com\/blog\/how-to-build-a-number-plate-reader-part-2\/#primaryimage\"},\"datePublished\":\"2020-04-21T01:57:12+00:00\",\"dateModified\":\"2020-04-22T16:35:34+00:00\",\"author\":{\"@id\":\"https:\/\/cloudxlab.com\/blog\/#\/schema\/person\/e2c5cc7b933ebd4b15f9b463dc7cf1b4\"},\"description\":\"Here we will learn how to use OpenCV and PyTesseract to get the final number from the plate. Previously we detected a number plate.\",\"breadcrumb\":{\"@id\":\"https:\/\/cloudxlab.com\/blog\/how-to-build-a-number-plate-reader-part-2\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/cloudxlab.com\/blog\/how-to-build-a-number-plate-reader-part-2\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/cloudxlab.com\/blog\/how-to-build-a-number-plate-reader-part-2\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"item\":{\"@type\":\"WebPage\",\"@id\":\"https:\/\/cloudxlab.com\/blog\/\",\"url\":\"https:\/\/cloudxlab.com\/blog\/\",\"name\":\"Home\"}},{\"@type\":\"ListItem\",\"position\":2,\"item\":{\"@id\":\"https:\/\/cloudxlab.com\/blog\/how-to-build-a-number-plate-reader-part-2\/#webpage\"}}]},{\"@type\":\"Person\",\"@id\":\"https:\/\/cloudxlab.com\/blog\/#\/schema\/person\/e2c5cc7b933ebd4b15f9b463dc7cf1b4\",\"name\":\"Praveen Pavithran\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/cloudxlab.com\/blog\/#personlogo\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/03c8d253347dcf9e04ec550cd6144973?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/03c8d253347dcf9e04ec550cd6144973?s=96&d=mm&r=g\",\"caption\":\"Praveen Pavithran\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","_links":{"self":[{"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/posts\/2969"}],"collection":[{"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/users\/26"}],"replies":[{"embeddable":true,"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/comments?post=2969"}],"version-history":[{"count":4,"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/posts\/2969\/revisions"}],"predecessor-version":[{"id":3010,"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/posts\/2969\/revisions\/3010"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/media\/2997"}],"wp:attachment":[{"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/media?parent=2969"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/categories?post=2969"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/tags?post=2969"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}