Enrollments closing soon for Post Graduate Certificate Program in Applied Data Science & AI By IIT Roorkee | 3 Seats Left

  Apply Now

Project- How to build low-latency deep-learning-based flask app

17 / 17

Modifying the app.py

Now that we witnessed a huge performance advantage with the usage of ZMQ, we shall proceed to make the necessary changes to app.py such that:

(1) resnet_model_server.py serves the model predictions. The Server listens to the port 5576 for any incoming requests.

(2)the client in app.py sends a request(along with the encoded image) to the Server for predictions.

(3) the Server responds to the client in app.py with the class predictions of the image.

INSTRUCTIONS
  • Let us view the files in ~/Flask-ZMQ-App-Folder/Flask-Server-Folder:

    ls ~/Flask-ZMQ-App-Folder/Flask-Server-Folder
    

    We could see app.py and app_helper.py.

  • Previously, it is in app_helper.py that the model was imported and the predictions are made. Now we no longer need this file, since all this functionality is defined in the RequestHandler of the resnet_model_server.py. So we can remove this file.

    • Remove the file app_helper.py and rm app_helper.pyc:

      rm app_helper.py
      rm app_helper.pyc
      
  • Now we shall remove the app.py so that we could freshly create the file and write the modified code so that we could establish a connection between app.py and the server.

    rm app.py
    
  • Freshly create the app.py using vi command:

    vi app.py
    
  • Press i key and copy-paste the following code into app.py:

    from flask import Flask, request, render_template
    from werkzeug.utils import secure_filename
    import os
    
    import base64
    import uuid
    import zmq
    
    
    # Define a flask app
    app = Flask(__name__)
    
    @app.route("/")
    def index():
        return render_template("index.html")
    
    @app.route('/uploader', methods = ['POST'])
    def upload_file():
        predictions=""
    
        if request.method == 'POST':
            f = request.files['file']
    
            # Save the file to ./uploads
            basepath = os.path.dirname(__file__)
            file_path = os.path.join(basepath, 'static','uploads', secure_filename(f.filename))
            f.save(file_path)
    
            global img_str
            with open(file_path, "rb") as image_file:
                img_str = base64.b64encode(image_file.read())
    
            context = zmq.Context()
            socket = context.socket(zmq.DEALER)
            _rid = "{}".format(str(uuid.uuid4()))
            socket.setsockopt(zmq.IDENTITY, _rid)
            socket.connect('tcp://localhost:5576')
            poll = zmq.Poller()
            poll.register(socket, zmq.POLLIN)
            obj = socket.send_json({"payload": img_str, "_rid": _rid})
    
            received_reply = False
            while not received_reply:
                sockets = dict(poll.poll(1000))
                if socket in sockets:
                    if sockets[socket] == zmq.POLLIN:
                        result_dict = socket.recv_json()                    
                        predictions = result_dict['preds']
                        received_reply = True                    
                        return render_template("upload.html", predictions=predictions, display_image=f.filename) 
    
            socket.close()
            context.term()
    
    
    if __name__ == "__main__":
        app.run(host="0.0.0.0",debug=True,port="4114")
    
  • Press theesc key, then :wq and hit Enter.

  • Run the program:

    python app.py
    
  • Open another web console, and run the resnet_model_server.py after activating its virtual environment:

    cd ~/Flask-ZMQ-App-Folder/Model-Server-Folder
    deactivate
    source model-env/bin/activate
    python resnet_model_server.py
    
  • Open a new tab in the browser, and go to http://f.cloudxlab.com:4114/.

  • We could see that the results are shown up on the browser with very less latency, almost as soon as we submit the image.

Let us have a brief look at the app.py code:

  • We imported the necessary packages:

    from flask import Flask, request, render_template
    from werkzeug.utils import secure_filename
    import os
    
    import base64
    import uuid
    import zmq
    
  • We define the port number in main method of our program:

    if __name__ == "__main__":
        app.run(host="0.0.0.0",debug=True,port="4114")
    
  • We defined the Flask app for our program.

    # Define a flask app
    app = Flask(__name__)
    
  • We define the default route and function for our web app:

    @app.route("/")
    def index():
        return render_template("index.html")
    
  • We define the upload_file function which is invoked upon submitting the image via UI for predictions:

    @app.route('/uploader', methods = ['POST'])
    def upload_file():
        predictions=""
        if request.method == 'POST':
            f = request.files['file']
    
            # Save the file to ./uploads
            basepath = os.path.dirname(__file__)
            file_path = os.path.join(basepath, 'static','uploads', secure_filename(f.filename))
            f.save(file_path)
    
            global img_str
            with open(file_path, "rb") as image_file:
                img_str = base64.b64encode(image_file.read())
    
            context = zmq.Context()
            socket = context.socket(zmq.DEALER)
            _rid = "{}".format(str(uuid.uuid4()))
            socket.setsockopt(zmq.IDENTITY, _rid)
            socket.connect('tcp://localhost:5576')
            poll = zmq.Poller()
            poll.register(socket, zmq.POLLIN)
            obj = socket.send_json({"payload": img_str, "_rid": _rid})
    
            received_reply = False
            while not received_reply:
                sockets = dict(poll.poll(1000))
                if socket in sockets:
                    if sockets[socket] == zmq.POLLIN:
                        result_dict = socket.recv_json()                    
                        predictions = result_dict['preds']
                        received_reply = True                    
                        return render_template("upload.html", predictions=predictions, display_image=f.filename) 
    
            socket.close()
            context.term()
    

    (1) If there is any POST request, then we are accessing the file in the request:

    if request.method == 'POST':
        f = request.files['file']
    
            # Save the file to ./uploads
            basepath = os.path.dirname(__file__)
            file_path = os.path.join(basepath, 'static','uploads', secure_filename(f.filename))
            f.save(file_path)
    

    Then we are saving the image in the uploads folder of static as already discussed in the previous project.

    (2) Nextly, we are reading the image in binary format and then encoding it into base64 form as already discussed while creating the test_client.py:

            global img_str
            with open(file_path, "rb") as image_file:
                img_str = base64.b64encode(image_file.read())
    

    (3) Then, we create socket, assign the unique id value to it, and connect it to the server which listens to port 5576 :

            context = zmq.Context()
            socket = context.socket(zmq.DEALER)
            _rid = "{}".format(str(uuid.uuid4()))
            socket.setsockopt(zmq.IDENTITY, _rid)
            socket.connect('tcp://localhost:5576')
    

    (4) We create a roller poll and register the socket with it:

            poll = zmq.Poller()
            poll.register(socket, zmq.POLLIN)
    

    (5) We make a dictionary with the 'payload' being the encoded image and the '_rid' being the unique id. We send this in the JSON format to the server using obj = socket.send_json({"payload": img_str, "_rid": _rid}).

    (6) We set the received_reply=False and keep polling to check if we have messages to receive from the server. Basically, we are waiting for a response from the server.

            received_reply = False
            while not received_reply:
                sockets = dict(poll.poll(1000))
    

    (7) We poll the sockets to check if we have messages to receive and work on it.

            if sockets[socket] == zmq.POLLIN:
                result_dict = socket.recv_json()                    
                predictions = result_dict['preds']
                received_reply = True
    

    If there are, then we receive those JSON messages, and store the predictions in variable predictions and turn the received_reply to True. Once this is True, the loop breaks, and the socket and context close.


No hints are availble for this assesment

Answer is not availble for this assesment


Note - Having trouble with the assessment engine? Follow the steps listed here

Loading comments...