{"id":3524,"date":"2021-05-02T06:30:23","date_gmt":"2021-05-02T06:30:23","guid":{"rendered":"https:\/\/cloudxlab.com\/blog\/?p=3524"},"modified":"2021-06-24T12:57:15","modified_gmt":"2021-06-24T12:57:15","slug":"how-to-design-a-large-scale-system-to-process-emails-using-multiple-machines-zookeeper-use-case-study","status":"publish","type":"post","link":"https:\/\/cloudxlab.com\/blog\/how-to-design-a-large-scale-system-to-process-emails-using-multiple-machines-zookeeper-use-case-study\/","title":{"rendered":"How to design a large-scale system to process emails using multiple machines [Zookeeper Use Case Study]?"},"content":{"rendered":"\n

Introduction<\/h2>\n\n\n\n

As part of this blog we are going to discuss various ways of large scale system design and the pros-cons of each.<\/p>\n\n\n\n

To get a fair understanding of this post, you should know what is distributed computing<\/a>, what is deadlock and race conditions<\/a>, locking in distributed systems<\/a> and Zookeeper<\/a> etc. Let’s get started.<\/p>\n\n\n\n

Scenario<\/h2>\n\n\n\n

Consider a situation where we have an email inbox that consists of emails, and emails are to be processed. For example, processing those emails and classifying each of the emails as spam or non-spam. The other example of the processing could be we are indexing the email so that the search could be performed.<\/p>\n\n\n\n

We have an email-processor program, running on various machines distributed physically from each other. <\/p>\n\n\n\n

\"\"
Email processor program running on distributed systems<\/figcaption><\/figure><\/div>\n\n\n\n

Now these machines need to somehow coordinate such that:<\/p>\n\n\n\n