Capture face in webcam and display real time to flask

Question

Basically I have three file which is app.py camera.py and gallery.html. I attach my code for your reference.

app.py

from flask import Flask, Response, json, render_template
from werkzeug.utils import secure_filename
from flask import request
from os import path, getcwd
import time
import os

app = Flask(__name__)
import cv2
from camera import VideoCamera


app.config['file_allowed'] = ['image/png', 'image/jpeg']
app.config['train_img'] = path.join(getcwd(), 'train_img')


def gen(camera):
    while True:
        frame = camera.get_frame()
        yield (b'--frame\r\n'
               b'Content-Type: image/jpeg\r\n\r\n' + frame + b'\r\n\r\n')

@app.route('/video_feed')
def video_feed():
    return Response(gen(VideoCamera()),
                    mimetype='multipart/x-mixed-replace; boundary=frame')

@app.route('/')
def index():
    return render_template('index.html')

@app.route('/gallery')
def get_gallery():
   images = os.listdir(os.path.join(app.static_folder, "capture_image"))
   return render_template('gallery.html', images=images)
app.run()

camera.py

import cv2
import face_recognition
from PIL import Image
import os
import time



dir_path = "C:/tutorial/face_recognition/venv/src4/capture_image"

class VideoCamera(object):
    def __init__(self):
        self.video = cv2.VideoCapture(0)

    def get_frame(self):
        success, frame = self.video.read()
        small_frame = cv2.resize(frame, (0, 0), fx=0.25, fy=0.25)
        rgb_small_frame = small_frame[:, :, ::-1]
        face_locations = face_recognition.face_locations(rgb_small_frame,number_of_times_to_upsample=2)

        for face_location in face_locations:
            top, right, bottom, left = face_location

            face_image = rgb_small_frame[top:bottom, left:right]
            pil_image = Image.fromarray(face_image)
            File_Formatted = ("%s" % (top)) + ".jpg"
            file_path = os.path.join( dir_path, File_Formatted) 
            pil_image.save(file_path)



        ret, jpeg = cv2.imencode('.jpg', frame)
        return jpeg.tobytes()

gallery.html

<section class="row">
  {% for image in images %}
    <section class="col-md-4 col-sm-6" style="background-color: green;">
      <img src="{{ url_for('static', filename='capture_image/' + image) }}">
    </section>
  {% endfor %}
</section>

This what i have done so far, the webcam will capture the face in webcam and save in folder. Then send the image to gallery.html. Currently, i am want to display the image real time in html templete without refresh when the face is captured it will automatically display in html gallery.html dynamically or real time.For your information i am using flask,python and openCV

My question is how i can display the face capture real time without refresh. When new face captured it will automatically display in gallery.html?

Hope someone can regarding on this matter.Thank you

Answer 1

OKAY. The first thing is to download this module : webcamJS . This is the javascript module that will allow you to capture photos from the client side. Test it to familiarize yourself with it (there are many alternatives, but in my opinion it is one of the simplest solutions).

But I'm nice, I still put a minimal code to show you how to use it:

You configure your HTML page and add the following div (do not blame me for the structure of the code, I know it's not pretty, all this hodgepodge between html and javascript, but it works):

<div id="my_photo_booth">
    <div id="my_camera"></div>

        <script src="{{url_for('static',filename='js/webcam.min.js')}}"></script>
        <script src="{{url_for('static',filename='audio/shutter.mp3')}}"></script>
        <script src="{{url_for('static',filename='audio/shutter.ogg')}}"></script>

        <!-- Configure a few settings and attach camera -->
        <script language="JavaScript">
            Webcam.set({
                // live preview size
                width: 320,
                height: 240,

                // device capture size
                dest_width: 640,
                dest_height: 480,

                // final cropped size
                crop_width: 480,
                crop_height: 480,

                // format and quality
                image_format: 'jpeg',
                jpeg_quality: 90,

                // flip horizontal (mirror mode)
                flip_horiz: true
            });
            Webcam.attach( '#my_camera' );
        </script>

        <br>
        <div id="results" style="display:none">
            <!-- Your captured image will appear here... -->
        </div>

        <!-- A button for taking snaps -->
        <form>
            <div id="pre_take_buttons">
                <!-- This button is shown before the user takes a snapshot -->
                <input type=button class="btn btn-success btn-squared" value="CAPTURE" onClick="preview_snapshot()">
            </div>

            <div id="post_take_buttons" style="display:none">
                <!-- These buttons are shown after a snapshot is taken -->
                <input type=button class="btn btn-danger btn-squared responsive-width" value="&lt; AGAIN" onClick="cancel_preview()">
                <input type=button class="btn btn-success btn-squared responsive-width" value="SAVE &gt;" onClick="save_photo()" style="font-weight:bold;">
            </div>
        </form>

</div>

A bit of javascript to manipulate the photo capture and send the photo to the server:

<script language="JavaScript">
    // preload shutter audio clip
    var shutter = new Audio();
    shutter.autoplay = false;
    shutter.src = navigator.userAgent.match(/Firefox/) ? '/static/audio/shutter.ogg' : '/static/audio/shutter.mp3';

    function preview_snapshot() {
        // play sound effect
        try { shutter.currentTime = 0; } catch(e) {;} // fails in IE
        shutter.play();

        // freeze camera so user can preview current frame
        Webcam.freeze();

        // swap button sets
        document.getElementById('pre_take_buttons').style.display = 'none';
        document.getElementById('post_take_buttons').style.display = '';
    }

    function cancel_preview() {
        // cancel preview freeze and return to live camera view
        Webcam.unfreeze();

        // swap buttons back to first set
        document.getElementById('pre_take_buttons').style.display = '';
        document.getElementById('post_take_buttons').style.display = 'none';
    }

    function save_photo() {
        // actually snap photo (from preview freeze).
        Webcam.snap( function(data_uri) {
            // display results in page
            console.log(data_uri);

            // shut down camera, stop capturing
            Webcam.reset();

            $.getJSON($SCRIPT_ROOT + '/_photo_cap', {
                photo_cap: data_uri,
            },function(data){
                var response = data.response;
            });

        } );
    }
</script>

Obviously this code you add it to the bottom of your html code.

I hope you'll manage with all this. But the interesting part here is the save_photo() function. In this function, I get the data uri from the photo and send it to flask via ajax (Check this link to see how to use jquery / ajax to send data to flask).

On the flask side:

import base64

@bp.route('/photo')
def photo():
    return render_template('photo.html')


@bp.route('/_photo_cap')
def photo_cap():
    photo_base64 = request.args.get('photo_cap')
    header, encoded = photo_base64.split(",", 1)
    binary_data = base64.b64decode(encoded)
    image_name = "photo.jpeg"

    with open(os.path.join("app/static/images/captures",image_name), "wb") as f:
        f.write(binary_data)
    //facial recognition operations
    response = 'your response'

    return jsonify(response=response)

Here there are two routes, one to render the photo capture page, and another to receive the data uri sent via ajax.

Basically what happens in the second route is that I get the data uri, I convert it to base64 and I store it on my disk. Then that's where you intervene. You do your facial recognition operations, then you return a response to your page.