For more details, here is a relevant tutorial video made by original author who explains who this works.

Here gives the source code of this video demonstration.

You can download the full code at:
1) Full Demo Solution (Code + video 43Mb)
http://db.tt/MukGcwB
2) Full Demo Code
http://db.tt/GkT6J9k

First we gather a data set of all the hand-shapes we wish to recognise. A naive approach to recognizing a new image D would be to simply compare it with all the images stored in the data set and find the target image T with the closest match. But because there are so many images in the data set this will take far too long. We can reduce the time by using a multi-scale approach. We divide up the data set into groups of images, which are similar to one another by blurring the images at different levels so that small differences between similar images will be eroded.

More information please refer here http://www.ercim.eu/publication/Ercim_News/enw46/shamaie.html

About openCV

A library of programming functions for real time computer vision,released under a BSD?license, and free for academic and commercial use (a). Current version is openCV 2.1, this version has been installed and compiled under my VS 2008 environment but with some errors, may not be very easy to configure this version. While the stabilities of openCV 1.0 have been firmly proved already and that is easier to install and configure. Install guides can be found as ?openCV install guide in openCV wiki,?installing openCV in windows, vc2008 install openCV 2.0/2.1(in Chinese)?and get started with openCV.

A easy start of ‘hello world‘ application using openCV can be very helpful. Under most occasions, openCV is used within windows console applications with a black command window and openCV’s own interfaces (highgui) to display images and draw patterns, as the ‘hello world’ example shows. As well there are some other interface components included in openCV library such as sliders, mouse movement events and key input events. Using these provided features of openCV, object detection could be done via capturing webcam images, analysing them and finally output the consequences.

One recommendation of learning openCV online is Yahoo openCV group. It is a group with huge number of active members posting hundreds of messages each day, may be a little massive, but a good place to raise questions and meet other people, highly recommended.

About detections & recognitions

Detections are core functions of openCV. Color knobs, objects, motions, gestures and faces are subjects of openCV detections and recognitions.

Color knob detections

detecting colors using openCV is basic, there could be several ways to achieve the color detections. For instance, as the color detection used in my hand recognition project, the target color is divided into three perspectives of H, B, and V. By combining these parameters, theoretically it is available to detect any color. But the fault of HSV method is that the background colours are usually too sensitive to be specified. So does another method which uses simple brightness threshold to get target colours, but it is limited as well. A better way of detecting a range of colours is to use ‘camshift’ which requires a source image which represents colour information for target colours. There is an example of using ‘camshift’ here.

Object detections

Object detection is kind of similar as colour extraction since to recognise the objects they should be detected first. Here is an example of using Actionscript 3.0 to tracking objects via webcam. In openCV it is simple as well. Tracking the objects, then using circle or line detection methods to extract rough object shapes. It could be very accurate of detecting object angles which are demonstrated here, and another object detection example illustration here. Detecting an array of object is nearly the same as detecting solo object, except some extra work is needed to loop all objects at one time and draw them out, this could be seem here as detecting combined objects.

Motion detections

There is an example of motion detections in openCV’s example folders, which is called motemlp.exe and a source file motempl.c. The picture on the left shows the motion directions and sub components’ directions are being marked clearly.

Gesture and face detections
Face detections are maturely applied in digital cameras presently. It is dependant on haar trainings and classifiers therefore the detection results seem robust. As well, a face detection example with source code can be found in openCV’s example folder. In this review face detection would not be emphasised too much as there are millions of tutorials telling how to detect the face in either static or dynamic circumstances.

Recognitions of hand gestures is the challenging part.  On current stage, hand gestures especially finger tips are still tough due to the natural shapes of hands are ambiguous for computer to identify which is which. Therefore some simple and intuitive gestures are commonly adopted to relatively improve the accuracy of recognitions. Recognition gestures requires two factors: a. correct contours and b. predefined gesture patterns. The first factor is reachable but the second requires more work to cheer up. Concerning the second factor, there have been some theoretical papers raising solutions of detecting hand gestures. Toughly, putting these theoretical methods into practical use and generate kind of applications providing open source codes still has long way to go. A progressive project can be referred here as hand detection and image descriptions.

Useful toolkits

HandVu

ARToolkit

I have experienced that and that was awful. Therefore i decide to share these solutions by providing functional codes as below. The solution is easy, after adding new media player relevant classes then, using codes below to replace those generated automatically by system. The key part of the change is from ‘public:CWND’ to ‘public:COleDispatchDriver’.

CWMPControls.cpp

CWMPControls.h

CWMPMedia.cpp

CWMPMedia.h

CWMPPlaylist.cpp

CWMPPlaylist.h

CWMPSettings.cpp

CWMPSettings.h

ocx1.cpp

ocx1.h

Step 1: Visual C++ wizard, choose MFC App.

Step 2: Visual C++, choose single document, to create main MFC app frame.

Step 3: Add openCV libs in project properties.

Step 4: Add variables in Doc class: add a variable m_image, include highgui.h head file in the first beginning, and the variable CImage   m_image.

Step 5: Add two virtual functions to open and save image file, as shown below.

Step 6: Add codes in View class to show the image

Step 7: Compile the app, then an image can be open through open menu, here comes the beauty Lena.

—

There are several vital parameters in using the function cvsnakeimage, particularly parameters relating to curve energy which concerns judgements of finger tips. E.g.

A copy of full source code of demonstrating the use of the function could be found here cvSnakeImage for hand detection

secret powers from Mechanics of Destruction on Vimeo.

Relevant posts & comments please refer to  ‘Hand gesture detection and recognition using openCV’， and related comments here.

A full paper about this can be found here titled as ‘tracking a hand manipulating objects‘

As described by the authors, a method of individual local tracker is used to achieve extractions. To achieve the goal as the left picture shows, it is required to build a 3-dimensional frame to skip the covering of objects and overlapped parts of hands. Seems the authors do not narrowly focus on color image segmentation which has been commonly adopted by computer vision researchers, but they integrate an estimation of features as well as 2.5-dimensional maps. That means probably extra dimension may required. In previous posts, we have successfully located contours of hands yet the shape recognition is still under investigation. And this new method may inspire us something further in hands tracking and recognition.

Linking DLLs:

• To permanantly include necessary dll files, add “C:\Program Files\OpenCV\bin” to PATH by visiting Advanced tab in System of Windows (the locate of directory might be different).
• One can just copy necessary dll files into project directory with source files.
• It might be required to restart Visual C++ when execution of instance failes after successful build.

Customize Global Options:

• Open the Visual C++ .Net Application. In the menu bar, select Tools->Options
• In the listing, choose Projects->VC++ Directories.
• First, select Library files from the “Show Directories for” List Box.
• Click the Insert New icon, and locate the folder where you have installed opencv.
• Consider that it is installed in “C:/Program Files/OpenCV”.
• In the Library files list, locate and add:

“C:\Program Files\OpenCV\lib”

• Now choose Include files in the list box, and locate and add the following directories:

“C:\Program Files\OpenCV\cv\include”
“C:\Program Files\OpenCV\cxcore\include”
“C:\Program Files\OpenCV\otherlibs\highgui”
“C:\Program Files\OpenCV\cvaux\include”
“C:\Program Files\OpenCV\otherlibs\_graphics\include”

• Next, choose source files in the list box, and locate and add the following directories:

“C:\Program Files\OpenCV\cv\src”
“C:\Program Files\OpenCV\cxcore\src”
“C:\Program Files\OpenCV\cvaux\src”
“C:\Program Files\OpenCV\otherlibs\highgui”
“C:\Program Files\OpenCV\otherlibs\_graphics\src”

• Now click OK in the Options dialog.
• You have successfully configured the global settings.

Create New Project:

• Within Developer Studio create new application:
• Select from menu “File”->”New…”->”Projects” tab.
• Choose “Win32 Application” or “Win32 console application” – the latter is the easier variant and both the sample projects have this type.

• Type the project name and choose location
• Click Ok.. In the Application Wizard, Just click Finish.

• After the above steps done Developer Studio will create the project folder (by default it has the same name as the project), <project name>.vcproj file, Solution <project name>.sln and, Three Source files: <project name>.cpp, stdafx.cpp and stdafx.h. StdAfx files are precompiled header files, which can be very useful if you want to reduce the compilation time.
• For example, consider that we have created a new “Hello” Project. Open the Hello.cpp file, and include the OpenCV-related #include directives:

#include <cv.h>
#include <cxcore.h>
#include <highgui.h>

• Note that these should be included after stdafx.h or you may get build errors.
• Now Type some OpenCV code, and Build the Solution by pressing the F7 Key. There should be linker errors.
• Add dependency projects into workspace
• Choose from menu: “Project” -> “Properties”.
• Choose “Linker” tab -> “Input” category -> “Additional Dependencies:”. Add the paths to all necessary import libraries (cxcore[d].lib cv[d].lib highgui[d].lib cvaux[d].lib cvcam[d].lib)
  • Note: The debug versions are available only when you Build the Visual C++ .NET solution provided with the OpenCV installation.

• If the build process complains about a missing ‘windows.h’ header file, then you’ll need to install the latest version of the Microsoft Windows SDK.
• . That’s it!.. Now Build and Run the application using F5 key and enjoy OpenCV!

.

Post quoted from: openCV Wiki, all copyrights reserved by original authors.

I’m using OpenCV for some tests, and for some reasons (freelance gigs and Symbian SDK) using MS Visual Studio. As new and shiny OpenCV 2.0 is out I decided to switch to it. As it happen, one absolutely have to read buried in the download section readme, before doing anything.
The thing is, OpenCV 2.0 doesn’t include lib files for VS. They have to be built by user.
So here is step by step retelling of readme:
1. Rename your old OpenCV installation to save it, just in case
2. Download and install OpenCV 2.0a
3. Download and install CMake
4. Reboot (or CMake wouldn’t work)
5. Go to C:\Program Files\CMake 2.6\bin and run cmake-gui.exe
6. In the “Where is the source code” field choose your new OpenCV directory (C:\OpenCV)
In “Where to build the binaries” choose directory for VS compiled OpneCV (C:\OpenCV\VS2005)
7. press Configure button and choose VS2005 (or whatever) as building enviroment
8. Press Generate and VS project will be generated in the C:\OpenCV\VS2005
9. Launch solution and build it. For debug build some projects require debug python libraries. As riseriyo pointed in comments if you have Python installed other than 2.6 that can cause problem.
10. Copy *.lib from C:\OpenCV\vs2005\lib\release (or debug) to C:\OpenCV\lib
Copy *.dll from C:\OpenCV\vs2005\bin\release to C:\OpenCV\bin
11. Now, reconfigure your application project. Include directories now “C:\OpenCV\include\opencv” instead of “C:\OpenCV\include
12. All libraries renamed from *.lib to *200.lib (cv.lib to cv200.lib) or *200d.lib for debug. Rename them, or change project settings.

The post was posted originally in ‘Mirror Image‘, all copyrights reserved by original authors.

As far as in my mind, there are three approaches that people have taken to detect hands using openCV, though not all of them have been tested by myself (a little shame). One is to use accurate HAAR-classifiers to locate and detect gestures which is considered as quite stable way but costing very much. Another is to use adaptive skin detection algorithms combined with motion analysis which seems easier to do than previous one. And the last is to use human skin colour segmentation to detect the contours of hands combined with hand convexity detection to recognise hands’ gestures [please refer to: hand gesture recognition using openCV].

HAAR-classifier database is rarely provided with open source projects. But we found some helpful papers which may give some thoughts of classifier training and test.

About hands database:

A colour hand gesture database for evaluating and improving algorithms on hand gesture and posture recognition

Application of classifiers:

Real-time hand tracking using a set of cooperative classifiers based on HAAR-like features

Alternative skin detection based approach gesture recognition through angle space:

Gesture recognition through angle space

As these are mostly asked questions about using openCV to detect and recognise all kinds of subjects such as colours or hands, a review which includes tutorials, progresses and examples (if possible) would be published shortly…

Conclusively there are two ways of objects recognition, which are

• combining simple shapes’ recognition to detect complicated objects;
• using pre-trained data to match objects if they are as set.

The ways we used to detect circles and lines by openCV belong to the first category, and training needed recognition ways such as faces detection and hands detection belong to the second category. Recognition can be quite stable after good data training, but it may not recognize other features without pre-defined, so does the rules of faces detection with openCV.

But if recognition is used to design other applications such as objects controlling or multi-touch demonstrations, it could not be better choice adopting training way to set up recognition functionalities. As the picture shown above, i abstract the recognition part from ARtoolkit which is a popular open source library in internet. The training and recognition test results shows that  it can recognize nearly all the patterns pre-trained.

Quite much work has been doing to improve accuracy of pattern recognition, to enable computer applications being sufficiently intelligent facing this dynamic world. ARtoolkit is one of them. By training computers to recognize patterns as shown above, i have opportunities to shift my attention from recognition to applications, which means, representation forms of recognition affect interaction.

By extracting recognition codes from ARtoolkit, it seems easier to train a computer to recognize patterns like letters and other graphics, and its recognizing results seem pretty good. Based on that, creating appropriate applications become urgent.

The reason of that shortage is due to the lacking of appropriate algorithms. In AS3 objects can not be respectively recognized through colors or shapes before. Fortunately through combination of some kinds of image processing functions, we could still be able to find out the exact way of representing objects under webcam, just as processed images shown below.

This progress provides AS3 more potential to build up interactive prototypes through webcam and normal surfaces with kind of semantic objects laid on. Thus the AS3′s abilities of building up flexible interfaces may be well used in afterward works.

One very interesting aspect of human-computer interaction is methodologies and their effectiveness. There is a view bore in my mind that until interaction became invisibly natural, then interaction did its job. Comparing to thousand years’ history of human lives, computer interaction faces its challenge to integrate itself into natural lives.

We use object recognition to simulate original daily domestic interaction like reading mails and leaving notes. Once computers can recognize specific objects as different meanings relating to its huge stored information, we believe that would be a further step towards natural and efficient interaction.

From shallow to deep MFC.pdf

VC++.pdf

While i was starting MFC in visual studio 2008, i got a fatal error in debugging as below:

Your version of the windows SDK is earlier than 6.0. Try setting the ‘WINVER’ and ‘_win32_winnt’ definitions in your project to less than 0×0600.


I tried several ways to fix that issue while nothing happened, also i investigated the documents in internet by Google neither any helpful information was provided. There indeed existed a solution which was as blew:

stdafx.h- add these codes in the header
#ifndef WINVER                  // Specifies that the minimum required platform is Windows Vista.
#define WINVER 0×0500          // Win2k
#ifndef _WIN32_WINNT            // Specifies that the minimum required platform is Windows Vista.
#define _WIN32_WINNT 0×0500    // Win2k

While that solution did not make any differences. And i tried another way- to download the Windows Platform SDK and installed it, and the most important, update the including library files to at least version 6, then try it again, everything was done.

This is a screenshot of demonstration hands detection using the code hsv.cpp. Comparing with the old version within black background and green contours lines, this version actually is based on that. Besides, two functions are added in this new version: using rectangle to locate the hands location, and using eslipe to illustrate the angle of hands motion.

hsv.cpp source code

Related post may interest you

Hand gesture detection and recognition using openCV

Some posts with prototype codes are occasionally requested to share the relevant codes. Due to this, i make a code collection in this post which could be convenient for all the readers. But please note that some codes still need additional image files or haar files that you should pay attention to.

haarFaceDetect
If you can not download the source code file, please refer to below
http://www.andol.info/download/haarFaceDetect.cpp

This is a prototype designed to test face detection using haar training algrithms. Needs image with faces and relevant haar trained xml file.

hsv
If you can not download the sourse code file, please refer to here below
http://www.andol.info/download/hsv.cpp
This is a prototype designed to detect hands ( not hands gesture ) in the dynamic motion images from web-cam. It uses color in a specific range to detect the hands, and then filters all the noises out to get the biggest contours of hands. Several backgrounds are tested by this codes.

HoughCircle
If you can not download the source code file, please refer to here below:
http://www.andol.info/download/HoughCircle.cpp
Static circle objects in the image are quite simple to grab. However, when it comes to combining recognition containing squares and circles, i have to admit that squares are easier to detect.

Recently, it is being applied to consumer electronic products because companies consider it an important key to their success.

However, there is a difference in the concept of usability between the two applications. Unlike the software user interfaces, the image/impression felt by the users are as important as the performance for a consumer electronic product to be successful. ”

As the post ‘10 videoes show some interesting AR applications‘ showed, integrating augmented reality technologies into interactive scenerial building is increasingly enhancing the experiences of interaction, both in the side of realastic world objects recognition and virtual applications.

These 2 pictures showed above are a demonstration of AR technology application with ARToolKit. Especially, i changed the default pattern for recognition and printed my own name on the card instead. After i trained the codes to recognize the new pattern, obviously, i got the satisfactory consequence. The codes recognized that new pattern rapidly and accurately, as well as using openGL to create practical coordinates and virtual 3D cube.

Although this isn’t the newest technology today, it indeed explored a new path to archive another advanced interaction method.

During the past days, i was trying to get the computer recognizing blue and red colors to archive my prototype application. I tested two methods to distinguish the colors recognition effects, that were value-fixed approach and value-dynamic approach. 

What i was doing is to build a simple, flexible and adaptable intellegent colors recognition prototype which could work under varities of spatial environments. First, a fixed value of color was set. Any colors match this value would be recognized as specific one. It is simple and easy, although it is dependentable on the environment light. ( if the environment is controlable, this method would be good one. ) Second approach is that, i devided the picture into 3 channels which are H, S, and V. And then, from H ( Hue), i defined a range of color as one color and other ranges for another. This experiment looks not so difficult and more robust that previous one, however, there’s still one thing which should be cautious: light temperature. Light tempreature changes, color’s hue exceeds.

So, before there invents more advantaged devices, recognizing which colors from which colors is relatively limited.

After several weeks’ hard working, finally, the memory leaking problem is fixed. 

In the last post which listed nearly all kinds of variaties declare and delete methods, here i add several tips for memory check while it is leaking:

1) int, double, float … : these kinds of variaties don’t matter so much;

2) IplImage* : this really matters, especially when cvCloneImage is conducted, if the destination variaty was not cleared, the memory would fly up.

3) CvMemStorage: be careful, to delete this data you have to use not ‘cvReleaseData’ but ‘cvReleaseMemStorage’, one declaration, one delete operation.

It can be a daunting task to hunt exactly where memory needs to be released. So I’ve compiled this short list of places where you should look out for memory leaks.

Create it, then Release it

If you create something, make sure you release it before “returning”. This is probably the very first thing you should check when fixing memory leak problems with OpenCV. For example, if you do a cvCreateImage, make sure you do a cvReleaseImage. There are many things you can create. Here are some functions that “create” and their corresponding “release” functions

One warning though: If you create something and want to return it, don’t release it. Lets say a function that creates a checkerboard image and returns it. If you release the image before returning it, you’re freeing all memory that stores the image data. And when you try accessing memory that isn’t yours, you get a crash.

Release returned structures

This is the second thing you should check for. Often, once you return a structure (say, an image).. you forget about it.

Multiple Memory Allocations

This is the third thing you should check for: Allocating memory, and then changing the pointer itself. Here’s some example code:

IplImage* image = cvCreateImage(whatever);
image = CreateCheckerBoard(whatever);
…
cvReleaseImage(&image);

This function creates a memory leak. First, you allocate some memory for image. Then, you call the function CreateCheckerBoard. This function itself creates new memory. And image now points to this new memory. The memory created in the first step is lost forever. No variable points to it. A memory leak. To fix this, you need to modify the code like this:

IplImage* image = NULL;
image = CreateCheckerBoard(whatever);
…
cvReleaseImage(&image);

If you return a sequence, release its storage

There are many instances where you use the CvSeq data structure. And often you might want to return this structure for further use. If you release its storage (a CvMemStorage structure) within the function itself, you’d free the memory where the sequence is stored. And then you’d try and access it in the calling function. Again, crash.

A temporary fix would be to just erasing the cvReleaseMemStorage statement… but that would mean lots of memory.
To fix this, you don’t release the memory in the function itself. You release it in the calling function like this:

cvReleaseMemStorage(&thesequence->storage);

storage is a member of the CvMemStorage structure that always points to the memory where its stored.

Again, this is just an example. There are more structures where a similar situation could arise.

Dependence on other structures

I quite recently discovered this memory leak. To explain this, I’ll use an example: Lets say you find out the contours of an image. OpenCV would return a “linked list” type structure calledCvSeq. You decide to access the third element of this linked list. OpenCV returns a pointer to the third element. All going great till this moment.

Now you decide to save all the points of this contour (the third element) in a data structure of your own. Since this is an array of points, you do something like:

mystructure->points = thirdcontour->points;

You set the pointer to equal to the thirdcontour. This is the bug. If you release the storage of the sequence (which you should), mystructure has a bad pointer. To fix this, allocate new memory to mystructure->points and then copy contents ofthirdcontour->points… something like this:

mystructure->points = (CvPoint*)malloc(sizeof(CvPoint) * thirdcontour->total);
memcpy(mystructure->points,thirdcontour->points,sizeof(CvPoint)*thirdcontour->total);

This creates new memory for your structure and then copies each element there. Once you’ve done this, you can release the storage of the sequence without fear.

How to get the gestures needed?
Generally, in order to detect hand and extract gestures from dymanic background environments, some algrithms such as color segment and some kinds of ‘mean’ algrithms are neccessary, indeed. And, as far as i have learned until now, i found 2 methods to effectively extract target gestures from background images. Those methods are color segment, motion detection and more advanced sensing like infrared detecting integrating with motion or color. This step is quit easy to archive because of the limited algrithms and hand characters which include color, motion and what? Nothing! Probably temperature, but that is not so universal.

What chatacters could be used for recognizing gestures?
  So, if the gestures are extracted from the complicated background images, the next step would be recognize them. Recognition is not as simple as detection which you just do some filter work. It needs several unique characters from the detected images. Most of the time, these characters probably contain: 1) angle – detecting the rotation   2) size – detecting the area   3)  peak and valley which is specific for the open hand  4) moving speed and direction. 

Related Posts: Hand guesture recognition using openCV

Clicking the pictures and the detailed previews will display which show the hand contours extracting from video capture. These pictures illustrate the results of color based segmentation for hand gesture detection ( not recognition yet ). it is a  real time programme,  however, in this demonstration, the programme still depends on environments’ light , because of the color based segmentation method. If, combinned with motion detection and background substraction, it supposes to be robuster.

Related Post:  Color recognition using openCV

Recognising colors in computer vision using openCV is not so complicated as it is expected. Actually, getting the capture image from webcam or some other advanced cameras, converting the color from RGB to HSV or HSI( not sure it is right ), and extracting the specific color(s) from the converted image, that is done. After that, some filters like CVSMOOTH, CANNY and CVTHRESHOLD can be applied to take off the noises in the previous image.

Give some codes here :

IplImage* src = cvCreateImage( sz, 8, 3 );

IplImage* hsv_image = cvCreateImage( sz, 8, 3);

IplImage* hsv_mask = cvCreateImage( sz, 8, 1);

IplImage* hsv_edge = cvCreateImage( sz, 8, 1);

CvScalar  hsv_min = cvScalar(0, 80, 80, 0);

CvScalar  hsv_max = cvScalar(20, 150, 255, 0);

/*

——————————————*/

src = cvQueryFrame( capture);

cvCvtColor(src, hsv_image, CV_BGR2HSV);

cvInRangeS (hsv_image, hsv_min, hsv_max, hsv_mask);

cvSmooth( hsv_mask, hsv_mask, CV_MEDIAN, 13, 0, 0, 0 );

cvCanny(hsv_mask, hsv_edge, 1, 3, 5);

In this post, as the progress goes, the hand detection technologies using openCV is introduced. As the pictures inllustrated below, this detection method is independent  with distance and background ( just not the background full of hands ), and the main segment method is color abstract which means getting the hand color pixels filtered. As the result tested within several environments, it workd well except slightly noise varying. But the noise is easy to take off.

The tested source code file is offered underneath the illustrations, feel free to use it.

SOURCE FILE HERE:  handDetection.CPP
Related post which may interest you:
Hand gesture detection and recognition

In this post, i will present you the new progress of object recognition using openCV—angle recognition.

Actually, this is just a small step further than the last one which is rectangles’ recognition. From the four outlines of rectangle as well as the four corner points, we can get two relative points to calculat the angle between them. As inllustrated below, i chosed two points which are most left and right. Once the position of these two points are captured, the SINE value of the angle can be calculated like this

SIN (angle) = abs(p1.y-p2.y)/sqrt(
(p1.x-p2.x)*(p1.x-p2.x)+(p1.y-p2.y)*(p1.y-p2.y))

So, that’s it.

First-chance exception at 0x7c90eb74 in camera1.7.exe: 0xC0000008: An invalid handle was specified.

And, the debugging pointer was on the left side of the code:

capture = cvCaptureFromCAM(0);

Well, this line ran smoothly in Code:Blocks, why at here it went wrong.

However, i think, openCV’s default functions which are included in its library maybe probably have this abilities. As a result, today i found a openCV function called

CvConvexityDefect

-Structure describing a single contour convexity detect

and there is  a illustration picture on the side of the instruction like below:

So, until now, colors*shape*angle=>infinite, the future looks bright to code some application of interactive objects recognition. But these founded a problem that is memory leaking in C++. I am newbie of C++ and i used Actionscript to make codes which were automatically recyling all the memory trash. But now, the trash memory have to be delete manually, terrible nightmare to me.

However, when turns to rectangle recognition, it seems work right but not so much. Look carefully, there are multilines drawed on one rectangle detected, and the rectangle number displayed on the left side was totally wrong—too much rectangles, that was impossible because as i see that i just put 3 rectangles under the camera. As a result, i was sucked here and couldnt move forward.

If there’s any good idea, welcome to leave a comment or contact me.

In color recognition, it can detect any color, but there’s still a little bug that,  the detectation color should be preset before recognizing the objects in the destination image. I am not sure whether there’s any color filter in openCV like Actionscript3 which can just directly filter the necessary color objects.

Source code colorrecg.pdf

There was a problem often happened in the prototype demonstration. When turned off the demonstrating computer and camera and then turned it on, there popped a interesting problem that was the camera was reset and the preset zoom area disappeared.

So, the best idea is to design a lable to tell prototype which part is target work place and which parts are not. After one day’s researching, finally, the solution is presented here (the picture above)— two circle dots are used to specify the invisible border of work place, out part of these two dots’ rectangle, there would not be work place any more, vice versa.

From black screen to graphic screen, this is big step over openCV. After the compiling problem was solved, the progress of objects recognition is much faster than ever, although there still are some unknow bugs. Mainly, the routine of interactive objects recognition prototype developing is from object dectetion to multi-objects recognition and at last it would be added artificial factors.

[The figure above illustrates prorotype recognizing multi-objects and displaying the number of objects detected.]

After several times’ failure, finally i fixed the problem of C++ compiling. Admittedly, that was under pressure. This new stage starting from openCV will mainly focuses on new interaction method and effectiveness evaluation.

This is the interface of prototype for central heating control.

Note: please turn off your speaker or use earphone when test this prototype.

If you have MICPHONE connected with your computer while you browsing this page, it is a big chance to blow to your micphone to driven the windmill spin. Have fun.

Well, this is the second version of my windmill which name is Blowit which i gave it just moment ago. In this version, the power of driven has been improved and one interactive feature was added which is -micphone driven. So, it will be much mature prototype to experient this interaction way. However, i am still wondering to where i can use this prototype as it is designed to check the sound and use the audio level.

Related Reading:

Click-driven windmill

The first picture is original image  captured from webcam, it is little blur and grey which is not easy to identify. The second picture is the one processed by flash:as3, obviously, it can recognize the text and part of background which have high contrast to light color.

What i want to continue to do is to develop the color recognization and shape recognization, as well as adding more intellegent application in it. I have to admit that this is just the start of this interesting interaction way, once the further version was completed, i hope perhaps it would shock you.

‘Central heating control panel’ prototyping is going to end, especially after two evaluations of usability.The picture above is only interface of this interactive prototype which is designed to change users’ attitude and behavior towards energy consumption and green-house effect CO2 emission. We have tested the usability of this device to ask several participants learning it and teaching me back to check the degree of that. Comparing with the former version V1.2, it is in progress.

This example is driven by mouse-click, and i am trying to design a new version which can be driven by voice. If that is done, we can blow the microphone to spin the windmill.

For weather report module.

For weather report module.

a prototype to improve communication among people when in elavator.

the more louder the sound is, the higher bares display->if the bar pattern changed to other creative one, it would be more interesting.