How to access databases using Jupyter Notebook

SQL is a very important skill. You not only can access the relational databases but also big data using Hive, Spark-SQL etcetera. Learning SQL could help you excel in various roles such as Business Analytics, Web Developer, Mobile Developer, Data Engineer, Data Scientist, and Data Analyst. Therefore having access to SQL client is very important via browser. In this blog, we are going to walk through the examples of interacting with SQLite and MySQL using Jupyter notebook.

A Jupyter notebook is a great tool for analytics and interactive computing. You can interact with various tools such as Python, Linux, File System, Scala, Lua, Spark, R, and SQL from the comfort of the browser. For almost every interactive tool, there is a kernel in Jupyter. Let us walk through how would you use SQL to interact with various databases from the comfort of your browser.

Using Jupyter to access databases such SQLite and MySQL.
Continue reading “How to access databases using Jupyter Notebook”

Getting Started with Apache Airflow

Apache Airflow

When you are building a production system whether it’s a machine learning model deployment or simple data cleaning, you would need to run multiple steps with multiple different tools and you would want to trigger some processes periodically. This is not possible to do it manually more than once. Therefore, you need a workflow manager and a scheduler. In workflow manager, you would define which processes to run and their interdependencies and in scheduler, you would want to execute them at a certain schedule.

When I started using Apache Hadoop in 2012, we used to get the HDFS data cleaned using our multiple streaming jobs written in Python, and then there were shell scripts and so on. It was cumbersome to run these manually. So, we started using Azkaban for the same, and later on Oozie came. Honestly, Oozie was less than impressive but it stayed due to the lack of alternatives.

As of today, Apache Airflow seems to be the best solution for creating your workflow. Unlike Oozie, Airflow is not really specific to Hadoop. It is an independent tool – more like a combination of Apache Ant and Unix Cron jobs. It has many more integrations. Check out Apache Airflow’s website.

Continue reading “Getting Started with Apache Airflow”

Distributed Computing with Locks

Introduction

Having known of the prevalence of BigData in real-world scenarios, it’s time for us to understand how they work. This is a very important topic in understanding the principles behind system design and coordination among machines in big data. So let’s dive in.

Scenario:

Consider a scenario where there is a resource of data, and there is a worker machine that has to accomplish some task using that resource. For example, this worker is to process the data by accessing that resource. Remember that the data source is having huge data; that is, the data to be processed for the task is very huge.

Continue reading “Distributed Computing with Locks”

Online Courses Free of Cost during #NoPayJan

Malcolm X once said, “Education is our passport to the future”. This has become more relevant than ever in the last year. The COVID-19 pandemic gave a big jolt to the economy and the existing strata of professions across the world. Many succumbed to the pandemic by losing their jobs and facing extensive pay cuts. 

People who were up-to-date with technology made it through the darkest times, making online education become the next big thing across the globe. According to a recent LinkedIn survey, around more than 60% of professionals have increased the amount of time spent on online learning for upskilling during the lockdown period. But the challenge here was that online education was becoming more and more expensive with a consistent fall in the quality of content. Online education slowly started becoming a very far-fetched dream for the common man. 

At CloudxLab, we strive to ensure that education does not feel like a luxury but a basic need that everybody is entitled to. Keeping this in mind, we bring forth the “#NoPayJan” where you can access some of the most sought after and industry-relevant courses completely free of cost. During #NoPayJan anybody who is signing up at CloudxLab will be able to access the contents of all the self-paced courses. This offer will be running from January 1 till January 31, 2021. CloudxLab provides an online learning platform where you can learn and practice Data Science, Deep Learning, Machine Learning, Big Data, Python, etc.

When the highly competitive and commercialized education providers have cluttered the online learning platform, CloudxLab tries to break through with a disruptive change by making upskilling affordable and accessible and thus, achievable.

Happy New Year & Happy Learning!

Improving the Performance of Deep-Learning based Flask App with ZMQ

Introduction

It is a well-known fact that deep learning models are heavy; with a lot of weights for the deep layers. And it is obviously an overhead to load the model every time we need to get the predictions from the model. Thus this is costly in terms of the time of execution.

In this project, we will mainly focus on addressing this issue, by uniquely integrating the networking functionalities provided by ZMQ library. We will build a server-client based architecture to make the model load exactly once(that is during the starting of the app). The predictions from the model will be served by the model server, as long as it listens to its Flask client which requests it for the predictions for an input image.

Continue reading “Improving the Performance of Deep-Learning based Flask App with ZMQ”

REVA University partners with CloudxLab for setting up Center of Excellence in AI and Deep Technologies

REVA University signs an MoU with CloudxLab to set up a Center of Excellence in AI and Deep Technologies: In the picture, Dr. K.M Babu, Vice Chancellor, Dr. Dhanamjaya, Pro-Vice Chancellor, Sandeep Giri, Founder, CloudX Labs.

REVA University’s REVA Academy for Corporate Excellence (RACE) has inked a memorandum of understanding with CloudxLab for setting up a Center of Excellence in AI and Deep Technologies and providing a platform for promoting research and innovation

REVA University and Cloudxlab research collaboration intends to work on technologies involving, deep learning, reinforced learning, curiosity-based machines, distributed computing, and launching specialized courses in these advanced technologies.

This collaboration will be aimed at providing and launching some highly sought-after courses in deep technologies involving experts from Academia as well as the industry. These courses will be delivered in hybrid mode – a combination of physical classroom, online instructor-led, self-paced, and project-based modes.

Dr. P. Shyama Raju, honorable Chancellor, REVA University  said “This one-of-a-kind collaboration is aimed at being a launchpad for those who are planning to step into the world of AI, Deep Learning and other advanced technologies. It affirms REVA University’s commitment to make high-end technical education available to everyone in the world.”

“With Artificial Intelligence, machine learning, and other high-end technologies influencing every aspect of our lives, we are optimistic that this collaboration will help professionals in shaping their career”, says Sandeep Giri, CEO, and Founder at CloudxLab.

About REVA University 

REVA University is one of the top-ranked private Universities in Bangalore, India, offering a wide range of UG, PG and PhD programs. REVA Academy for Corporate Excellence (RACE) is one of the initiatives of REVA University focused on corporate training to develop visionary enterprise leaders through progressive and integrated learning capabilities. RACE offers best-in-class, specialized, techno-functional, and interdisciplinary programs that are designed to suit the needs of working professionals. 

How to label custom images for YOLO – YOLO 3

In this blog we will show how to label custom images for making your own YOLO detector. We have other blogs that cover how to setup Yolo with Darknet, running object detection on images, videos and live CCTV streams. If you want to detect items not covered by the general model, you need custom training.

In our case we will build a truck type detector. There are 4 types of trucks we will try to identify

Continue reading “How to label custom images for YOLO – YOLO 3”

Setup Yolo with Darknet- Yolo 1

We will explore YOLO for image recognition in a series of blogs. This is the first one. In this blog, we will see how to setup YOLO with darknet and run it. We will also demonstrate the various choices you have with YOLO in terms of accuracy, speed and cost, enabling you to make a more informed choice of how you would want to run your models.

Setup Yolo with Darknet

The content in the blog is not unique. However if you are starting with YOLO, this is the first thing you need to do.

Continue reading “Setup Yolo with Darknet- Yolo 1”