Распознаем эмоции с Tensorflow

05.06.2017
By

Tensorflow позволяет сделать распознавание эмоций.

Nimish Ronge сделал неплохой пример распознавания с использованием OpenCV и Tensorflow - https://github.com/nimish1512/Emotion-recognition-and-prediction

Несмотря  на то, что в требованиях есть использование Python 3.x, программа нормально работает с Python 2.7

Кроме того, nimish1512 также отдает обученную нейросеть, которую можно скачать по следующим ссылкам:

1) model_1_nimish.tflearn.data-00000-of-00001 https://drive.google.com/open?id=0B8_K9DW3E9PlV0phWlFfRGFfcEk
2) model_1_nimish.tflearn.index https://drive.google.com/open?id=0B8_K9DW3E9PlSmJySGM2Z0lwdlU
3) model_1_nimish.tflearn.meta https://drive.google.com/open?id=0B8_K9DW3E9Plb0ZVeHg0cEJuNlE

Итак, работает под Ubuntu 14.04, Tensorflow 1.1., OpenCV 3.0.0:

Cначала ставим OpenCV по следующей инструкции:

Step 1:

Open up a terminal and update the apt-get  package manager followed by upgrading any pre-installed packages:

Install OpenCV 3.0 and Python 2.7+ on Ubuntu

Shell
1
2
$ sudo apt-get update
$ sudo apt-get upgrade

Step 2:

Now we need to install our developer tools:

Install OpenCV 3.0 and Python 2.7+ on Ubuntu

Shell
1
$ sudo apt-get install build-essential cmake git pkg-config

The pkg-config  is likely already installed, but be sure to include it just in case. We’ll be using git  to pull down the OpenCV repositories from GitHub. The  cmake  package is used to configure our build.

Step 3:

OpenCV needs to be able to load various image file formats from disk, including JPEG, PNG, TIFF, etc. In order to load these image formats from disk, we’ll need our image I/O packages:

Install OpenCV 3.0 and Python 2.7+ on Ubuntu

Shell
1
$ sudo apt-get install libjpeg8-dev libtiff4-dev libjasper-dev libpng12-dev

Step 4:

At this point, we have the ability to load a given image off of disk. But how do we display the actual image to our screen? The answer is the GTK development library, which the highgui  module of OpenCV depends on to guild Graphical User Interfaces (GUIs):

Install OpenCV 3.0 and Python 2.7+ on Ubuntu

Shell
1
$ sudo apt-get install libgtk2.0-dev

Step 5:

We can load images using OpenCV, but what about processing video streams and accessing individual frames? We’ve got that covered here:

Install OpenCV 3.0 and Python 2.7+ on Ubuntu

Shell
1
$ sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev

Step 6:

Install libraries that are used to optimize various routines inside of OpenCV:

Install OpenCV 3.0 and Python 2.7+ on Ubuntu

Shell
1
$ sudo apt-get install libatlas-base-dev gfortran

Step 7:

Install pip , a Python package manager:

Install OpenCV 3.0 and Python 2.7+ on Ubuntu

Shell
1
2
$ wget https://bootstrap.pypa.io/get-pip.py
$ sudo python get-pip.py

Step 8:

Install virtualenv and virtualenvwrapper. These two packages allow us to create separate Python environments for each project we are working on. While installing virtualenv  and virtualenvwrapper  is not a requirement to get OpenCV 3.0 and Python 2.7+ up and running on your Ubuntu system, I highly recommend it and the rest of this tutorial will assume you have them installed!

Install OpenCV 3.0 and Python 2.7+ on Ubuntu

Shell
1
2
$ sudo pip install virtualenv virtualenvwrapper
$ sudo rm -rf ~/.cache/pip

Now that we have virtualenv  and virtualenvwrapper  installed, we need to update our ~/.bashrc  file:

Install OpenCV 3.0 and Python 2.7+ on Ubuntu

Shell
1
2
3
# virtualenv and virtualenvwrapper
export WORKON_HOME=$HOME/.virtualenvs
source /usr/local/bin/virtualenvwrapper.sh

This quick update will ensure that both virtualenv  and virtualenvwrapper  are loaded each time you login.

To make the changes to our ~/.bashrc  file take effect, you can either (1) logout and log back in, (2) close your current terminal window and open a new one, or preferably, (3) reload the contents of your ~/.bashrc  file:

Install OpenCV 3.0 and Python 2.7+ on Ubuntu

Shell
1
$ source ~/.bashrc

Lastly, we can create our cv  virtual environment where we’ll be doing our computer vision development and OpenCV 3.0 + Python 2.7+ installation:

Install OpenCV 3.0 and Python 2.7+ on Ubuntu

Shell
1
$ mkvirtualenv cv

Step 9:

As I mentioned above, this tutorial covers how to install OpenCV 3.0 and Python 2.7+ (I’ll have a OpenCV 3.0 + Python 3 tutorial available later this month), so we’ll need to install our Python 2.7 development tools:

Install OpenCV 3.0 and Python 2.7+ on Ubuntu

Shell
1
$ sudo apt-get install python2.7-dev

Since OpenCV represents images as multi-dimensional NumPy arrays, we better install NumPy into our cv  virtual environment:

Install OpenCV 3.0 and Python 2.7+ on Ubuntu

Shell
1
$ pip install numpy

Step 10:

Our environment is now all setup — we can proceed to change to our home directory, pull down OpenCV from GitHub, and checkout the 3.0.0  version:

Install OpenCV 3.0 and Python 2.7+ on Ubuntu

Shell
1
2
3
4
$ cd ~
$ git clone https://github.com/Itseez/opencv.git
$ cd opencv
$ git checkout 3.0.0

Update (3 January 2016): You can replace the 3.0.0  version with whatever the current release is (as of right now, it’s 3.1.0 ). Be sure to check OpenCV.org for information on the latest release.

As I mentioned last week, we also need the opencv_contrib repo as well. Without this repository, we won’t have access to standard keypoint detectors and local invariant descriptors (such as SIFT, SURF, etc.) that were available in the OpenCV 2.4.X version. We’ll also be missing out on some of the newer OpenCV 3.0 features like text detection in natural images:

Install OpenCV 3.0 and Python 2.7+ on Ubuntu

Shell
1
2
3
4
$ cd ~
$ git clone https://github.com/Itseez/opencv_contrib.git
$ cd opencv_contrib
$ git checkout 3.0.0

Again, make sure that you checkout the same version for opencv_contrib  that you did for opencv  above, otherwise you could run into compilation errors.

Time to setup the build:

Install OpenCV 3.0 and Python 2.7+ on Ubuntu

Shell
1
2
3
4
5
6
7
8
9
$ cd ~/opencv
$ mkdir build
$ cd build
$ cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D INSTALL_C_EXAMPLES=ON \
-D INSTALL_PYTHON_EXAMPLES=ON \
-D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib/modules \
-D BUILD_EXAMPLES=ON ..

Update (3 January 2016): In order to build OpenCV 3.1.0 , you need to set -D INSTALL_C_EXAMPLES=OFF  (rather than ON ) in the cmake  command. There is a bug in the OpenCV v3.1.0 CMake build script that can cause errors if you leave this switch on. Once you set this switch to off, CMake should run without a problem.

Notice how compared to last week our CMake command is substantially less verbose and requires less manual tweaking — this is because CMake is able to better automatically tune our install parameters (at least compared to OSX).

Now we can finally compile OpenCV:

Install OpenCV 3.0 and Python 2.7+ on Ubuntu

Shell
1
$ make -j4

Where you can replace the 4 with the number of available cores on your processor to speedup the compilation.

Here’s an example of OpenCV 3.0 compiling on my system:

Figure 1: OpenCV 3.0 with Python 2.7+ support compiling on my Ubuntu 14.04 system.Figure 1: OpenCV 3.0 with Python 2.7+ support compiling on my Ubuntu 14.04 system.

Assuming that OpenCV compiled without error, you can now install it on your Ubuntu system:

Install OpenCV 3.0 and Python 2.7+ on Ubuntu

Shell
1
2
$ sudo make install
$ sudo ldconfig

Step 11:

If you’ve reached this step without an error, OpenCV should now be installed in  /usr/local/lib/python2.7/site-packages

However, our cv  virtual environment is located in our home directory — thus to use OpenCV within our cv  environment, we first need to sym-link OpenCV into the site-packages  directory of the cv  virtual environment:

Install OpenCV 3.0 and Python 2.7+ on Ubuntu

Shell
1
2
$ cd ~/.virtualenvs/cv/lib/python2.7/site-packages/
$ ln -s /usr/local/lib/python2.7/site-packages/cv2.so cv2.so

Step 12:

Congratulations! You have successfully installed OpenCV 3.0 with Python 2.7+ bindings on your Ubuntu system!

To confirm your installation, simply ensure that you are in the cv  virtual environment, followed by importing cv2 :

Install OpenCV 3.0 and Python 2.7+ on Ubuntu

Shell
1
2
3
4
5
$ workon cv
$ python
>>> import cv2
>>> cv2.__version__
’3.0.0′

Here’s an example of demonstrating the OpenCV 3.0 and Python 2.7+ install on my own Ubuntu machine:

Figure 2: OpenCV 3.0 with Python 2.7+ bindings has been successfully installed on Ubuntu!Figure 2: OpenCV 3.0 with Python 2.7+ bindings has been successfully installed on Ubuntu!

Step 13:

Now that OpenCV has been configured and installed, let’s build a quick Python script to detect the red game cartridge in the image named games.jpg  below:

Figure 2: Our goal is to detect the red game cartridge (on the right) in this image.Figure 3: Our goal is to detect the red game cartridge (on the right) in this image.

Open up your favorite editor, create a new file, name it find_game.py , and insert the following code:

Install OpenCV 3.0 and Python 2.7+ on Ubuntu

Python
# import the necessary packages
import numpy as np
import cv2
 
# load the games image
image = cv2.imread("games.jpg")
 
# find the red color game in the image
upper = np.array([65, 65, 255])
lower = np.array([0, 0, 200])
mask = cv2.inRange(image, lower, upper)
 
# find contours in the masked image and keep the largest one
(_, cnts, _) = cv2.findContours(mask.copy(), cv2.RETR_EXTERNAL,
    cv2.CHAIN_APPROX_SIMPLE)
c = max(cnts, key=cv2.contourArea)
 
# approximate the contour
peri = cv2.arcLength(c, True)
approx = cv2.approxPolyDP(c, 0.05 * peri, True)
 
# draw a green bounding box surrounding the red game
cv2.drawContours(image, [approx], -1, (0, 255, 0), 4)
cv2.imshow("Image", image)
cv2.waitKey(0)

You’ll also need to download the games.jpg image and place it in the same directory as your find_game.py  file. Once the games.jpg  file has been downloaded, you can execute the script via:

Install OpenCV 3.0 and Python 2.7+ on Ubuntu

Shell
1
$ python find_game.py

Assuming that you have downloaded the games.jpg  image and placed it in the same directory as our find_game.py  script, you should see the following output:

Figure 4: We have successfully detected the red game cartridge in the image!Figure 4: We have successfully detected the red game cartridge in the image!

Notice how our script was able to successfully detect the red game cartridge in the right portion of the image, followed by drawing a green bounding box surrounding it.

Obviously this isn’t the most exciting example in the world — but it has demonstrated that we have OpenCV 3.0 with Python 2.7+ bindings up and running on our Ubuntu system!

 

Затем устанавливаем Tensorflow также в виртуальное окружение cv.

Устанавливаем tflean – $pip install tflearn

Клонируем творение nimish1512

git clone https://github.com/nimish1512/Emotion-recognition-and-prediction.git

Для простоты переименовываем директорию, куда клонировали программу в emotion.

В эту же директорию сохраняем обученную нейросеть, которую скачали в самом начале. Получится так:

Запускаем виртуальное окружение. В моем случае $ source ~/.virtualenvs/cv/bin/activate

Запускаем $python em_model.py

Результат налицо :)

немного теории:

Как обучить собственную нейросеть - https://github.com/isseu/emotion-recognition-neural-networks

Откуда взять картинки эмоций – https://www.kaggle.com/c/challenges-in-representation-learning-facial-expression-recognition-challenge/data

Как сконвертировать картинки из формата csv в обычные png – https://www.kaggle.com/c/challenges-in-representation-learning-facial-expression-recognition-challenge/discussion/29428

Программа на Python (автор NoBugs)

#! /usr/bin/env
# -*-coding: utf-8-*-

__author__ = 'fer_2013'

import numpy as np
import cv2
import mxnet as mx
import pandas as pd
import random
import os

curdir = os.path.abspath(os.path.dirname(__file__))

def gen_record(csvfile,channel):
    data = pd.read_csv(csvfile,delimiter=',',dtype='a')
    labels = np.array(data['emotion'],np.float)
    # print(labels,'\n',data['emotion'])

    imagebuffer = np.array(data['pixels'])
    images = np.array([np.fromstring(image,np.uint8,sep=' ') for image in imagebuffer])
    del imagebuffer
    num_shape = int(np.sqrt(images.shape[-1]))
    images.shape = (images.shape[0],num_shape,num_shape)
    # img=images[0];cv2.imshow('test',img);cv2.waitKey(0);cv2.destroyAllWindow();exit()
    dirs = set(data['Usage'])
    subdirs = set(labels)
    class_dir = {}
    for dr in dirs:
        dest = os.path.join(curdir,dr)
        class_dir[dr] = dest
        if not os.path.exists(dest):
            os.mkdir(dest)

    data = zip(labels,images,data['Usage'])

    for d in data:
        destdir = os.path.join(class_dir[d[-1]],str(int(d[0])))
        if not os.path.exists(destdir):
            os.mkdir(destdir)
        img = d[1]
        filepath = unique_name(destdir,d[-1])
        print('[^_^] Write image to %s' % filepath)
        if not filepath:
            continue
        sig = cv2.imwrite(filepath,img)
        if not sig:
            print('Error')
            exit(-1)

def unique_name(pardir,prefix,suffix='jpg'):
    filename = '{0}_{1}.{2}'.format(prefix,random.randint(1,10**8),suffix)
    filepath = os.path.join(pardir,filename)
    if not os.path.exists(filepath):
        return filepath
    unique_name(pardir,prefix,suffix)

if __name__ == '__main__':
    filename = 'fer2013.csv'
    filename = os.path.join(curdir,filename)
    gen_record(filename,1)

    # ##################### test
    # tmp = unique_name('./Training','Training')
    # print(tmp)

Тренинг собственнойц нейросети может занять очень много времени ( недели) -= зависит от вашего компа.

 

 

Добавить комментарий