{"id":3908,"date":"2022-09-20T12:45:26","date_gmt":"2022-09-20T12:45:26","guid":{"rendered":"https:\/\/cloudxlab.com\/blog\/?p=3908"},"modified":"2022-09-21T10:18:34","modified_gmt":"2022-09-21T10:18:34","slug":"how-to-interact-with-apache-zookeeper-using-python","status":"publish","type":"post","link":"https:\/\/cloudxlab.com\/blog\/how-to-interact-with-apache-zookeeper-using-python\/","title":{"rendered":"How to Interact with Apache Zookeeper using Python?"},"content":{"rendered":"\n<p>In the <a href=\"https:\/\/cloudxlab.com\/blog\/big-data-solution-apache-hadoop-and-spark\/\" target=\"_blank\" rel=\"noreferrer noopener\">Hadoop ecosystem<\/a>, <a href=\"https:\/\/zookeeper.apache.org\/doc\/r3.4.14\/\" target=\"_blank\" rel=\"noreferrer noopener\">Apache Zookeeper<\/a> plays an important role in coordination amongst distributed resources. Apart from being an important component of Hadoop, it is also a very good concept to learn for a system design interview.<\/p>\n\n\n\n<h2>What is Apache Zookeeper?<\/h2>\n\n\n\n<p><strong>Apache ZooKeeper<\/strong> is a coordination tool to let people build distributed systems easier. In very simple words, it is a central data store of key-value pairs, using which distributed systems can coordinate. Since it needs to be able to handle the load, Zookeeper itself runs on many machines.<\/p>\n\n\n\n<p>Zookeeper provides a simple set of primitives and it is very easy to program. <\/p>\n\n\n\n<p>It is used for:<\/p>\n\n\n\n<ul><li>synchronization<\/li><li>locking<\/li><li>maintaining configuration<\/li><li>failover management.<\/li><\/ul>\n\n\n\n<p>It does not suffer from <a href=\"https:\/\/cloudxlab.com\/blog\/race-condition-and-deadlock\/\" target=\"_blank\" rel=\"noreferrer noopener\">Race Conditions and Dead Locks<\/a>.<\/p>\n\n\n\n<!--more-->\n\n\n\n<div class=\"wp-block-group\"><div class=\"wp-block-group__inner-container\">\n<pre class=\"wp-block-verse\"><strong>Note: <\/strong>This blog assumes that you are already familiar with basics of Apache Zookeeper. If you aren't, then please go through the following blog first:<\/pre>\n\n\n\n<p><blockquote class=\"wp-embedded-content\"><a href=\"https:\/\/cloudxlab.com\/blog\/introduction-to-apache-zookeeper\/\">Introduction to Apache Zookeeper<\/a><\/blockquote><script type='text\/javascript'><!--\/\/--><![CDATA[\/\/><!--\t\t\/*! This file is auto-generated *\/\t\t!function(c,d){\"use strict\";var e=!1,n=!1;if(d.querySelector)if(c.addEventListener)e=!0;if(c.wp=c.wp||{},!c.wp.receiveEmbedMessage)if(c.wp.receiveEmbedMessage=function(e){var t=e.data;if(t)if(t.secret||t.message||t.value)if(!\/[^a-zA-Z0-9]\/.test(t.secret)){for(var r,a,i,s=d.querySelectorAll('iframe[data-secret=\"'+t.secret+'\"]'),n=d.querySelectorAll('blockquote[data-secret=\"'+t.secret+'\"]'),o=0;o<n.length;o++)n[o].style.display=\"none\";for(o=0;o<s.length;o++)if(r=s[o],e.source===r.contentWindow){if(r.removeAttribute(\"style\"),\"height\"===t.message){if(1e3<(i=parseInt(t.value,10)))i=1e3;else if(~~i<200)i=200;r.height=i}if(\"link\"===t.message)if(a=d.createElement(\"a\"),i=d.createElement(\"a\"),a.href=r.getAttribute(\"src\"),i.href=t.value,i.host===a.host)if(d.activeElement===r)c.top.location.href=t.value}}},e)c.addEventListener(\"message\",c.wp.receiveEmbedMessage,!1),d.addEventListener(\"DOMContentLoaded\",t,!1),c.addEventListener(\"load\",t,!1);function t(){if(!n){n=!0;for(var e,t,r=-1!==navigator.appVersion.indexOf(\"MSIE 10\"),a=!!navigator.userAgent.match(\/Trident.*rv:11.\/),i=d.querySelectorAll(\"iframe.wp-embedded-content\"),s=0;s<i.length;s++){if(!(e=i[s]).getAttribute(\"data-secret\"))t=Math.random().toString(36).substr(2,10),e.src+=\"#?secret=\"+t,e.setAttribute(\"data-secret\",t);if(r||a)(t=e.cloneNode(!0)).removeAttribute(\"security\"),e.parentNode.replaceChild(t,e)}}}}(window,document);\/\/--><!]]><\/script><iframe title=\"&#8220;Introduction to Apache Zookeeper&#8221; &#8212; CloudxLab Blog\" sandbox=\"allow-scripts\" security=\"restricted\" src=\"https:\/\/cloudxlab.com\/blog\/introduction-to-apache-zookeeper\/embed\/\" width=\"600\" height=\"338\" frameborder=\"0\" marginwidth=\"0\" marginheight=\"0\" scrolling=\"no\" class=\"wp-embedded-content\"><\/iframe><\/p>\n<\/div><\/div>\n\n\n\n<p>Now, let&#8217;s come back to our topic, that is:<\/p>\n\n\n\n<h2>How to interact with Apache Zookeeper?<\/h2>\n\n\n\n<p>We interact with Apache Zookeeper through clients. A client is any process that connects to the ZooKeeper ensemble using the ZooKeeper client API. It establishes a session with the ZooKeeper service by creating a handle to the service using a language binding. Apache ZooKeeper ships with API bindings for&nbsp;<a href=\"https:\/\/zookeeper.apache.org\/doc\/r3.4.9\/zookeeperProgrammers.html#Java+Binding\">Java<\/a>&nbsp;and&nbsp;<a href=\"https:\/\/zookeeper.apache.org\/doc\/r3.4.9\/zookeeperProgrammers.html#C+Binding\">C<\/a> by default.&nbsp; But, we can also interact with Zookeeper using <strong>Python<\/strong>. It has been made possible by the python library <a href=\"https:\/\/kazoo.readthedocs.io\/en\/latest\/\">Kazoo<\/a>. Kazoo implements a higher level API to&nbsp;<a href=\"http:\/\/zookeeper.apache.org\/\">Apache Zookeeper<\/a>&nbsp;for Python clients.<\/p>\n\n\n\n<h2>Why Kazoo?<\/h2>\n\n\n\n<p>Using&nbsp;<a href=\"https:\/\/kazoo.readthedocs.io\/en\/latest\/glossary.html#term-zookeeper\"><em>Zookeeper<\/em><\/a>&nbsp;in a safe manner can be difficult due to the variety of edge-cases in&nbsp;<a href=\"https:\/\/kazoo.readthedocs.io\/en\/latest\/glossary.html#term-zookeeper\"><em>Zookeeper<\/em><\/a>&nbsp;and other bugs that have been present in the Python C binding. Due to how the C library utilizes a separate C thread for&nbsp;<a href=\"https:\/\/kazoo.readthedocs.io\/en\/latest\/glossary.html#term-zookeeper\"><em>Zookeeper<\/em><\/a>&nbsp;communication some libraries like&nbsp;<a href=\"http:\/\/gevent.org\/\">gevent<\/a>&nbsp;(or&nbsp;<a href=\"http:\/\/eventlet.net\/\">eventlet<\/a>) also don\u2019t work properly by default.<\/p>\n\n\n\n<p>There were several issues By utilizing a pure Python implementation, Kazoo handles all of these cases and provides a new asynchronous API which is consistent when using threads or&nbsp;<a href=\"http:\/\/gevent.org\/\">gevent<\/a>&nbsp;(or&nbsp;<a href=\"http:\/\/eventlet.net\/\">eventlet<\/a>) greenlets.<\/p>\n\n\n\n<h2>How to use Kazoo?<\/h2>\n\n\n\n<p>Kazoo makes the connection process much simple and hassle-free. Let&#8217;s learn how to use it.<\/p>\n\n\n\n<p>It is recommended to create a virtual environment to avoid conflicts between the packages. It lets you have a stable, reproducible, and portable environment. So, we will start by creating a virtual environment and installing <a href=\"https:\/\/kazoo.readthedocs.io\/en\/latest\/\">Kazoo<\/a> in it.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"># Create virtual environment\nvirtualenv zk\n\n#activate it\nsource zk\/bin\/activate\n\n# Install python client\npip3 install kazoo\n\n# Launch python prompt\npython3<\/pre>\n\n\n\n<p>To begin using Kazoo, a&nbsp;<a href=\"https:\/\/kazoo.readthedocs.io\/en\/latest\/api\/client.html#kazoo.client.KazooClient\">KazooClient<\/a>&nbsp;object must be created and a connection established:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">from kazoo.client import KazooClient\n\nzk = KazooClient(hosts='127.0.0.1:2181')\nzk.start()<\/pre>\n\n\n\n<p>By default, the client will connect to a local Zookeeper server on the default port (2181). You should make sure Zookeeper is actually running there first, or the&nbsp;start&nbsp;command will be waiting until its default timeout.<\/p>\n\n\n\n<p>Once connected, the client will attempt to stay connected regardless of intermittent connection loss or Zookeeper session expiration.&nbsp;<\/p>\n\n\n\n<p>Now, let&#8217;s check whether it&#8217;s working or not. We will write a function that can be triggered either when the node has changed or when the children of the node change.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">def my_func(event):\n    # check to see what the children are now\n        print(\"event: \", event)\n        zk.get_children(\"\/cloudxlab\", watch=my_func)\n\n\n# Call my_func when the children change\nchildren = zk.get_children(\"\/cloudxlab\/\", watch=my_func)<\/pre>\n\n\n\n<p>Now in another terminal let&#8217;s try creating znodes inside \/cloudxlab using <code>zookeeper-client<\/code>.  So, when I created a znode inside \/cloudxlab, the following output was displayed automatically in the python terminal:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">&gt;&gt; ('event: ', WatchedEvent(type='CHILD', state='CONNECTED', path=u'\/cloudxlab'))<\/pre>\n\n\n\n<p>Hence, it&#8217;s working fine. You are also advised to try it out. <\/p>\n\n\n\n<p>So, today we have learned how to use Kazoo to Interact with Apache Zookeeper. That&#8217;s all for this blog. Happy Learning!!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In the Hadoop ecosystem, Apache Zookeeper plays an important role in coordination amongst distributed resources. Apart from being an important component of Hadoop, it is also a very good concept to learn for a system design interview. What is Apache Zookeeper? Apache ZooKeeper is a coordination tool to let people build distributed systems easier. In &hellip; <a href=\"https:\/\/cloudxlab.com\/blog\/how-to-interact-with-apache-zookeeper-using-python\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;How to Interact with Apache Zookeeper using Python?&#8221;<\/span><\/a><\/p>\n","protected":false},"author":36,"featured_media":3910,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[24,1],"tags":[178,113],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v16.2 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>How to Interact with Apache Zookeeper using Python? | CloudxLab Blog<\/title>\n<meta name=\"description\" content=\"Apache ZooKeeper ships with API bindings for Java and C by default. But, we can also interact with Zookeeper using Python with the help of Kazoo.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/cloudxlab.com\/blog\/how-to-interact-with-apache-zookeeper-using-python\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How to Interact with Apache Zookeeper using Python? | CloudxLab Blog\" \/>\n<meta property=\"og:description\" content=\"Apache ZooKeeper ships with API bindings for Java and C by default. But, we can also interact with Zookeeper using Python with the help of Kazoo.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/cloudxlab.com\/blog\/how-to-interact-with-apache-zookeeper-using-python\/\" \/>\n<meta property=\"og:site_name\" content=\"CloudxLab Blog\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/cloudxlab\" \/>\n<meta property=\"article:published_time\" content=\"2022-09-20T12:45:26+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2022-09-21T10:18:34+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/blog.cloudxlab.com\/wp-content\/uploads\/2022\/09\/zkservice.jpeg\" \/>\n\t<meta property=\"og:image:width\" content=\"600\" \/>\n\t<meta property=\"og:image:height\" content=\"185\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@CloudxLab\" \/>\n<meta name=\"twitter:site\" content=\"@CloudxLab\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\">\n\t<meta name=\"twitter:data1\" content=\"3 minutes\">\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebSite\",\"@id\":\"https:\/\/cloudxlab.com\/blog\/#website\",\"url\":\"https:\/\/cloudxlab.com\/blog\/\",\"name\":\"CloudxLab Blog\",\"description\":\"Learn AI, Machine Learning, Deep Learning, Devops &amp; Big Data\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":\"https:\/\/cloudxlab.com\/blog\/?s={search_term_string}\",\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/cloudxlab.com\/blog\/how-to-interact-with-apache-zookeeper-using-python\/#primaryimage\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/cloudxlab.com\/blog\/wp-content\/uploads\/2022\/09\/zkservice.jpeg\",\"contentUrl\":\"https:\/\/cloudxlab.com\/blog\/wp-content\/uploads\/2022\/09\/zkservice.jpeg\",\"width\":600,\"height\":185},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/cloudxlab.com\/blog\/how-to-interact-with-apache-zookeeper-using-python\/#webpage\",\"url\":\"https:\/\/cloudxlab.com\/blog\/how-to-interact-with-apache-zookeeper-using-python\/\",\"name\":\"How to Interact with Apache Zookeeper using Python? | CloudxLab Blog\",\"isPartOf\":{\"@id\":\"https:\/\/cloudxlab.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/cloudxlab.com\/blog\/how-to-interact-with-apache-zookeeper-using-python\/#primaryimage\"},\"datePublished\":\"2022-09-20T12:45:26+00:00\",\"dateModified\":\"2022-09-21T10:18:34+00:00\",\"author\":{\"@id\":\"https:\/\/cloudxlab.com\/blog\/#\/schema\/person\/4438d405318314ec50940bde93ef548a\"},\"description\":\"Apache ZooKeeper ships with API bindings for Java and C by default. But, we can also interact with Zookeeper using Python with the help of Kazoo.\",\"breadcrumb\":{\"@id\":\"https:\/\/cloudxlab.com\/blog\/how-to-interact-with-apache-zookeeper-using-python\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/cloudxlab.com\/blog\/how-to-interact-with-apache-zookeeper-using-python\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/cloudxlab.com\/blog\/how-to-interact-with-apache-zookeeper-using-python\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"item\":{\"@type\":\"WebPage\",\"@id\":\"https:\/\/cloudxlab.com\/blog\/\",\"url\":\"https:\/\/cloudxlab.com\/blog\/\",\"name\":\"Home\"}},{\"@type\":\"ListItem\",\"position\":2,\"item\":{\"@id\":\"https:\/\/cloudxlab.com\/blog\/how-to-interact-with-apache-zookeeper-using-python\/#webpage\"}}]},{\"@type\":\"Person\",\"@id\":\"https:\/\/cloudxlab.com\/blog\/#\/schema\/person\/4438d405318314ec50940bde93ef548a\",\"name\":\"Shubh Tripathi\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/cloudxlab.com\/blog\/#personlogo\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/76bb13891affbf9da48fa9701d774ff0?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/76bb13891affbf9da48fa9701d774ff0?s=96&d=mm&r=g\",\"caption\":\"Shubh Tripathi\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","_links":{"self":[{"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/posts\/3908"}],"collection":[{"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/users\/36"}],"replies":[{"embeddable":true,"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/comments?post=3908"}],"version-history":[{"count":10,"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/posts\/3908\/revisions"}],"predecessor-version":[{"id":3922,"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/posts\/3908\/revisions\/3922"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/media\/3910"}],"wp:attachment":[{"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/media?parent=3908"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/categories?post=3908"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cloudxlab.com\/blog\/wp-json\/wp\/v2\/tags?post=3908"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}