[Part-1] Home Appliance Automation with Raspberry Pi

At Azilen, we believe in creating smart solutions for an interconnected and intelligent world. With emergence of IoT we try to achieve advanced connectivity and ensure efficiency through connected devices and network sensors.

Our recent work on the automation of home appliances through gesture recognition is one such example where we aim to control and automate home appliances such as air conditioners and televisions. We have achieved gesture recognition for implementing functionality like Turning On-Off, Increasing and decreasing the temperature for Air-conditioner and Turning On-Off for Television.

Through an extensive research, our team explored various feasible options for selecting the right platform and suitable libraries to make real time computer vision possible for gesture recognition. From Omega ,Chip, Aurduino ,Raspberry Pi amongst the platform and from BoofCV, OpenIMAJ, MATLAB, OpenCV and JavaCV among the libraries, the team selected Raspberry Pi and leading open source library ”Open CV” and its java port “JavaCV” as the right platform to make computer vision and image processing possible.

The library has more than 2500 optimized algorithms, which includes a comprehensive set of both classic and state-of-the-art computer vision and machine learning algorithms. These algorithms can be used to detect and recognize faces, identify objects, classify human actions in videos, track camera movements, track moving objects, etc.

This blog talks about the complete step by step guide to setup OpenCV and JavaCV on Raspberry Pi.

Note:

We assume that you have pre-installed latest Raspbian operating system on your Raspberry-Pi.

All required packages such as Maven, Ant, Git should be installed in the system.

Installation Guide
Please follow below steps to setup OpenCV and JavaCV on Raspberry Pi

Step-1: Update Raspbian

At the start you need compiler tools so update and upgrade the Raspberry Pi (Raspbian Linux) with latest available packages

Get all the compiler tools and video libraries using the below commands:
Open the terminal window and fire following commands one by one

sudo apt-get update sudo apt-get upgrade

Step-2: Install Dependencies

Open CV requires few external dependencies for it to be able to run, therefore you need to install all the required dependencies in the following order:
Open the terminal window and fire following commands one by one

Note:
If some of the libraries fail to download, then try to look for a different version of the library using apt-cache search as some of these libraries are not in Raspbian repository anymore or are available with newer versions
For e.g. :

sudo apt-cache search ffmpeg

Step 3: Download OpenCV Source Code and Install

Now we have our dependencies installed, let’s grab the OpenCV archive version 2.4.13 from the official OpenCV repository.

(Note: As future versions of OpenCV are released, you can replace 3.1.0, 3.2.0 with the latest version number, but this guide is focused around version 2.4.13).

• Uncompress the downloaded source file
  Move into uncompressed directore and create a new directory named “release” inside it
  Run below commands to configure, compile and install OpenCV.Please note that compilation will take approximately 4-5 hours, so grab yourself a nice pizza and relax !

•  cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local .. make sudo make install sudo idconfig

After cmake is done successfully, the jar file will be under release/bin/ and release/lib/ folder. Since we build version 2.4.13, the generated file will be named OpenCV-2413.jar and libOpenCV_java.so.

Both generated files will required in build path for any java based project with OpenCV. While libOpenCV_java.so is required for any C/C++ based project with OpenCV.

Note: If you get any Linker error or java.library.path issue ,you need to provide path for generated OpenCV jar file with java -jar cmd. using –Djava.library.path= ‘path to OpenCV libs’ or you can export LD_LIBRARY_PATH system variable with lib path. you can also add export command to .bashcr file as well

Step 4 : Compile JavaCV / javaCPP

Now that we have compiled and installed OpenCV 2.4.13 on our Raspberry Pi, we can compile and install JavaCV and JavaCPP on Raspberry Pi. In order to do that, you need to have Java, Maven and Git installed on the Raspberry Pi.

The compilation of JavaCV and javaCPP does not run with all versions of Java (such as Oracle JDK 8). It runs correctly with OpenJDK 7 which needs to be installed temporarily for this purpose if it’s not already installed. There might be multiple versions of Java installed. It needs to be ensured that OpenJDK 7 is the default selected Java version. Please run following commands on your terminal window to install OpenJDK.

Install OpenJDK 7 and select OpenJDK 7 as default Java version:

For JavaCV / javaCPP Maven is needed. The source will be downloaded using Git.

Once the source is downloded reset the sources to the old version 0.7 of JavaCV and javaCPP as they are the recent working ones for our purpose. Two modifications to the source files need to be performed to run on the Raspberry Pi and these will be done with a sed command. Afterwards the compilation can be started.

Run the following commands in terminal window: (Follow steps in order)

And that’s it. You have successfully installed JavaCV/JavaCPP on Raspberry Pi. Next step is to verify your installation.

Step 5: Testing your OpenCV / JavaCV Installation

Open your favorite editor on Raspberry Pi (VI/NANO/PageLeaf).

Create a new java file as shown below and save the file.

Note: Raspbian ships latest Oracle JDK version with it. Please use command ‘whereis java’ to find out the location of the Oracle JDK and make it default java version now onwards.
Now compile this file using Oracle JDK version >= 7. Type following commands into your terminal to compile and run this java code.

The java code output must be as below:

Congrats! You have a brand new, fresh install of OpenCV / JavaCV on your Raspberry Pi. And I’m sure you’re just itching to leverage your Raspberry Pi to build some awesome computer vision apps.

Conclusion

Our comprehensive approach can help implement various IoT solutions such as color detection, face recognition system, motion tracking, Object identification and Mobile robotics.

So, what’s next?

In our upcoming blog we aim to write about complete guide for automating air conditioner and television appliances for implementing features through gesture recognition.