The video vibration is annoying, because as we know it, this causes big barriers to programmers from recognising objects/people accurately. However, the thing is, when the vibrations are amplified, i mean hugely amplified with a set of algorithms, there are amazing results happenning, as demonsted in this picture.

If this is used in realistic application, such like video chat or remote hospital, one can see the other’s heart beating, which indicates whether they are anxious of exciting.

For more information, refer to the original source of the project: http://people.csail.mit.edu/mrub/vidmag/

The release of HAAR training classifier for hand gesture detection

This is the last post of the GSoC hand gesture detection series, and the trained HAAT classifier is finally released here.
This is only a short announcement, but the classifier in .xml file can be reached here in https://github.com/yandol/GstHanddetect.
The release contains two .xml classifier files, respectively they are for CLOSED PALM and FIST detection.

Hope this is useful.

A previous article has described a method to detect and recognise hand gestures using OpenCV, actually the c# wrapper of OpenCV called emgu CV. Details of this method can be found here a method of detecting and recognising hand gestures using Opencv. But Tongo’s project wrapped in emgu CV may not be easy to configure, so this article gives a new example transfering thet project to plain c++ OpenCV environment.

The example codes of the project, as well as the configurations, are described bellow.

Structure of previous hand gesture detection and recognition project

To make a clear tutorial of how the previous project did the hand gesture detection and recognition, the functions used in the project are listed below, through which I hope it can make a clear picture of this project’s algorithm flow.

1) reading video files. The project did not read video images from live webcam, rather it provided a video file as the source of video images. Similar functions were used to query video image one by one by using the function – grabber.QueryFrame().

2) extracting contours and hulls of hand gestures. In this part, the method used HSV colour ranges to extract hand colours – the range can be adjusted dynamically according to background environments. A set of functions were applied to improve the accuracy of colour detection, such as cvErode and cvDilate. Here is a clip of source code in the project.

3) selecting the right contours. The skin detection generated many contours, including face shapes with similar colours as hands. So, the right contours need to be pick up from all these contours. In this project, the author used a simple method by comparing the size of contours – usually the hand in the image had the biggest contour size.

4) if the biggest contour existed, then counting the finger number from detected contours. During this progress some basic markings such as drawing a rectangle or eclipse are helpful to identify the right object. Counting the finger was realised via a function of defect detection. There were obvious defects between finger contours with up and down shapes.

Transferring previous project to c++

Based on the understanding of hand gesture detection and recognition algorithm flow described above, it is possible to map the flow in c++ projects. The c++ project used similar OpenCV functions to realise the detection and recognition, and it followed the same flow of detection. Below is a screenshot of the c++ project detecting and recognition simple finger gestures as numbers. The source code of this project is also included in the bottom of this post and the download page as well.

Code example

See the example codes in download page.

Forewords

The first try of HAAR training was based on the tutorial mentioned in the previous post (see Collection of own hand gesture samples – the call for pictures, training method, tool and results). The aim was to go through all steps of HAAR training and see:

1) How positive and negative samples should be captured and setup for training?
2) How large should the scale of sample be? This is, what exactly are these differences between 1000 samples’ training and 9000 samples’ training in terms of accuracy and effectiveness.

The trial training

Full details of the trial training can be found in the previous post A failed example of hand gesture recognition using openCV haartraining classifiers.

As mentioned in the old post, the size of samples were crucial to the training. There is a size for best training results, which is around 20X20. Bigger sizes of course are helpful to improve the detection accuracy but the memory of training machine can easily running out.

Lessens learnt – good and bad

As described in the failed example of HAAR training, the sample pictures should not contain too much background information. For example, if detecting a open palm the backgrounds between fingers were also captured and trained as a part of palm, which led to very low detection accuracy. Rather, the sample pictures should be captured in a manner of such for face detection – with more important features but less backgrounds.

Although the failed example had nearly 10,000 samples its results were still poor. This does not require the increase of training samples. In another run of the failed example of training, after training closed palm gestures, 800 positive samples gave a result with approximately 70% accuracy, and the robustness seemed to be possibly improved through using more samples in different scenarios.

The call for hand gesture samples

Quite long time ago a call for hand gesture samples was made in here, unfortunately this call received few responses making contributions to the samples. Guess if I post free databases of hand gestures for high quality HAAR training then the situation would be quite different – kidding.

The quality of samples matters!!!

Of course high quality samples can improve the training results, but the real situation is that it is not easy to take thousands of hand gesture photos in a short time and these photos need to be in a series and include target detection features. Plus cropping these features out of photos manually, this is overloaded. So that is why we need some hand gesture databases to do the job. In my project I had a chance to use Cambridge’s hand gesture database, these samples were in 80×60 size and had most hand gestures. However the backgrounds were black and the features were not very well cropped, so I have to give up, and instead, starting to collect my own samples. I wish someday I could publish these as another hand gesture database sharing with the public.

Based on my experiences of training so far, the samples from webcam used to have good enough quality for the training. Normal webcams can produce 320×240 pictures, and nowadays 640×480 is quite common. With the webcam, then with a short programme using openCV to grab webcam streams and save these as specifically sized samples, the sample collection work is done. An example of using openCv to collect sample pictures can go here Writing a simple hand gesture picture sampler for classifier training. Also, in Naotoshi’s tutorial, another tool was provided to speed the manual cropping of sample pictures, the link is here image cropper

Negative samples ALSO matter

Negative samples are those pictures without target features, so theoretically any background pictures off the hands can be used as negative samples. Therefore this is easier to collect. What I intend to mention here is, I made a stupid mistake in my project and that led to very bad detection results, no matter how much positive samples were collected and cropped. I resized the background pictures to 40×30 size, and the positive samples were with 20×20 size. Based on that, the detection results generated unbelievably high false detection rate, as I mentioned in a previous post here A failed example of hand gesture recognition using openCV haartraining classifiers.

And after I found out and changed these negative samples to normal sizes like 320×240, bingo~ I got quite good improvements, and I could see the differences of detection accuracy each time I added positive and negative samples for training.

From the picture of face detection in openCV website…

This post is the first of series articles discussing openCV HAAR classifier training and use, as it was implemented in GSoc 2012 project. Here in this post there is no complex algorithms presented, instead, this focuses on requirements and techniques for successful classifier training using openCV.

Samples used in openCV face detection

3,000 negative pictures, 5,000 positive frontal face pictures were used and resulted in openCV’s good accuracy of face detection. These 5k pictures were from 1,000 individuals. Put aside how difficult it was to collect so many pictures (which I will talk about in another post), this number is believed to be able to achieve satisfied results due to the fact that the most common features should be covered, such like the dark eyes, bright nose and the relative positions amongst these parts. Special features, such like the colours of eyes, the height of nose and the width of face cheeks, are less worthy of training.

Downloads of face samples, as mentioned in Naotoshi’s tutorials, can be referred in http://note.sonots.com/SciSoftware/haartraining.html#x15ebd98.

How these samples were used

openCV never told how these samples were used. As openCV only provides xml files, the only thing clear is how many stages used in the training with these samples. Neither were the cropping of pictures nor ultimate picture sizes provided. As far as concerned,

Kuranov et. al. [3] states as 20×20 of sample size achieved the highest hit rate. Furthermore, they states as “For 18×18 four split nodes performed best, while for 20×20 two nodes were slightly better. Thus, -w 20 -h 20 would be good

And, how the picture cropping affects the quality of training will be discussed in a later post.

Method of training

To save some words, the detailed procedures of sample training as well as parameter introductions please refer to Naotoshi’s tutorial. Here I introduce .bat files used in my training. These files save good time to deal with command lines.

create_samples.bat

haartraining.bat

and, cascadeconvert.bat (opencv_cascadeconvert.exe is not included in openCV installation)

Coming soon…

Due to the end of this year’s Google Summer of Code 2012 is approaching soon, the development of hand gesture detection plugin for Gstreamer is also going to be packed. As one of important outputs of this project, herewith I forecast the good results of natural hand gesture detection using openCV HAAR training classifiers, mainly including the gestures of fist and closed palm.

Relevant posts regarding to this project, for example, how the call for hand gesture pictures was passively responded and how the classifier was going when insufficient samples were provided, will be published shortly in a series. Topics will cover:

1. Analysis of openCV’s face detection – the sample, method and performance
2. Collection of own hand gesture samples – the call for pictures, training method, tool and results
3. First trial of classifier training – the good and bad
4. Improved classifier training – the changes and lessens learned
5. Dead try – the keys to successful classifier training (samples and parameters)

Before the example – why haar classifiers for hand gesture detection

As mentioned in one of early articles such like hand gesture detection and recognition using openCV, there are two main ways to detect hand gestures, including skin colour segmentation and haar classifier training, although recently some feature-detecting algorithms are used to detect hand gestures such like swift. Skin colour segmentation is a dead end – hand colours can be white, or black, or any other colours – depends on the varieties of environmental light conditions. So using a skin colour model (as in The colour range for HSV skin extraction) with a specific colour range can do hand gesture recognition quite well in strict scenarios but this still faces tough challenges for other use.

The source codes

it can be downloaded from here SOURCE CODES OF HAND GESTURE DETECTION AND RECOGNITION USING OPENCV, relevant files/codes are for requests.

The question raised about hsv colour values

Several days ago I got an email from a developer who asked why there were differences in the hsv threshold values between my example and the theoretical values from a paper. Well, my answer is that my example was designed to fit with the specific environment the application was implemented, which could be somehow smaller or larger than the theoretical range.

A simple demonstration

As recently I am working on a hand posture/gesture recognition project, I am curious about how the range of the hand colour would be different from the previous example. So I wrote a little piece of code and followed the hsv value from a paper which talks about the statistical skin colour ranges between different populations. A range of H for asian population was used, as it was specified to be between 0-50, and the range of S was between 0.23-0.68.

I had a quick run-out, and the results were showed as the first picture said, which seemed quite nice. But, as I moved my hands close to the window with rich sunshine, the palms became white and the colour range cannot recognise accurately any more.

Conclusions

Using hsv colour to extract skin areas in the picture is, indeed, an efficient way, based on current dominant computers it is a good way to have real-time results.

using hsv is not a very stable way that can be relied on to detect hand gestures, as demonstrated clearly above, the contextual environments had significant influence on the accuracy of the results.

Fortunately, only if, the environment the applications aim to use in, the hsv can be a good way to achieve your goal by carefully configuring the colour range.

This example aimed to provide a good classifier that was able to recognise a flat contract palm in videos, but the results, as demonstrated in the haar performance evaluation, turned out quite disappointed. Here I list the details of this example, including the first stage of sampler extract, creating samples, training samples, and the final evaluation.

Note that this example was a very typical in terms of hand gesture recognition – it followed good procedural flows of haartraining and evaluation, but with low level accuracy of hand gesture recognition in video streams.

What this example aimed to do?

So far when I was typing these words there were still very few functions provided by openCV to be able to recognise natural hand gestures accurately, some laboratory demos may should improved recognition performance though.

For this reason, this example was proposed to provide a good hand gesture recognition function that was based on openCV haartraining. As a good starting point to detect complicated hand gestures, the example focused on flat contract palm gesture.

Progresses

The example started from collecting a good number of gesture pictures, as required by haartraining, both positive pictures and negative pictures were collected with 4,000-5,000 pictures each category. A simple picture sampler was described in one prior article here haartraining sample picture collector.

haartraining progresses:

all haartraining done until here.

//

Outcomes

Since the original outcomes of the haartraining were .txt files in respective folders, it needed a converter to produce .xml file for test use. The txt2xml converter can be found her haar converter.

Finally the test programme to see how the trained haar classifier worked with video streams. The test codes can be found here hand gesture haar test.

and the final outcomes:

Progresses in training a classifier for hand gesture recognition is never an easy job, one of the difficulties that needs to overcome is the collection of sample images, both positive and negative. Collecting thousands of sample pictures for each hand gesture is a huge task that requires enormous time and energy on this. Also, inviting so many users to give hand gesture pictures seems a big challenge.

Need a quick and dirty solution?

A easy way

Inspired by cambridge’s hand gesture database, in which there are a series of pictures having very similar hand gestures, My guess is that their sample work probably used video cameras as gestures sample sources. Indeed, it is quick and and easy – suppose one second’s video could generate 25 frames that means, 400 seconds (= 6 minutes 40 seconds) could give 10,000 hand gesture pictures, and the number is still growing.

The automation tool

It is wise to take good and many samples from videos, but is not so smart if doing the sampling work one frame by one frame – this should be done automatically!!!

For this reason, a simple programme was written in couple minutes. Using several basic functions in c++ and opencv 2.4, the source code can be found here – hand gesture sampler, or in download page in http://download.andol.info/hand%20gesture%20sampler.cpp.

Test

This little programme is able to control if taking samples when monitoring the webcam. By taking snapshots from webcam video streams, it makes taking the hand gesture samples much easier.

Give me your hands!!!

As a new hand gesture recognition project is launched recently, there is a strong need to collect a big number of good quality hand pictures – it is a pity I have only two hands. So I need your hands, do not worry, I am not using that for criminal purposes nor for dangerous operations, just need some pictures to train my programme to be capable to recognise beautiful hands like yours.

And the most important thing is, you will be paid to contributing your hand or other people’s high quality pictures of hand. The requirements are simple as follows.

1. a positive hand picture should be given with clear backgrounds, e.g. hands on white backgrounds;

2. a positive hand picture should be given together with a negative picture which does not have the hand in the picture;

3. note the resolution of pictures, should be at least 320*240;

4. try to cover as many as possible hand gestures;

5. zip and send the pictures to my email gsoc[at]andol.info, your payment will be subject to the quality of pictures provided.

Thanks in advance.

A new project launching of hand gesture recognition in gstreamer – as the proposal to developing a gstreamer plugin for accurate hand gesture recognition in multimedia streams has been accepted successfully, more practical activities are going to start soon. The expected results of this project is to develop a gstreamer plugin that is capable to support hand gesture index in video streams, also, recognising real time hand gestures in live video stream, thus enabling developers to build up advanced applications such like video playing control using fingers.

Why mapping hand gestures to system mouse pointer controls?

Hand gesture recognition, including hand gestures and movements, is becoming increasingly attractive. Many people are currently working on it to achieve natural interaction. One important step between hand gesture recognition and natural-interaction applications is to map detected hand gestures to system mouse pointer control. In one of previous articles here.

In this article an simple example is provided to demonstrate how to map detected hand gestures’ movements to system mouse pointer position using c#.

An example

Because my current work is focusing on c#, this example is written in c#. For other programme languages, such as c++ and c, the principle is similar, the detail functions may be different though.

Below is the source codes of this example, comments are given to explain what does each line mean.

Step further – more operations with mouse pointer control

There are other functions specified to take over system mouse pointer control.

It is not working to use
mouse_event(MouseEventFlag.LeftDown, 100,100,0,UIntPtr.Zero)
to trigger a mouse left button click at [10,10].

To realise that, there needs two steps.
1) moving pointer to right position [10,10].
2) triggering a left button click event, by using
mouse_event(MouseEventFlag.LeftDown, 0, 0, 0, UIntPtr.Zero).

When I was building up a new silverlight project based on existing silverlight 3d template, I met a problem that said ‘error  : Unable to read the project file ‘OverlappingTabs.cspro … C:\Program Files\MSBuild\Microsoft\Silverlight\v4.0\Microsoft.Silverlight.CSharp.targets” was not found. Confirm that the path in the declaration is correct, and that the file exists on disk.’

This problem happened because of the support of silverlight, as the visual studio 2010 did not have the required silverlight supporting files installed. To fix this problem, just setup ‘Microsoft® Silverlight® 5 Tools for Visual Studio® 2010 SP1‘.

This post demonstrates the breaking down of xtion pro into parts. For more pictures reading the whole post.

When I was writing a simple example of OpenCV below, an repeating error happened as ‘VIDIOC_QUERYMENU:Invalid argument’.

Is this because the Qt for drawing the output of openCV?

The code

The code that caused the above error is like this:

The error happened when initialising the webcam, but the webcam was working fine even though the errors were given in the console window.

The fix

From a post from stackoverflow, this problem was

with Qt handling its own event loop and thus the event loop of OpenCV is starved and never get run.

The way to get them to work together is quite simple: Display your OpenCV as a QPixmap (by convert your image to QImage then use QLabel to display it). Then add this QLabel to your QWidget. Your QWidget can either be embedded or become the main widget of your QApplication.

But it may not be so easy to get rid of, as other posts also mentioned that

You can’t mix openCV’s control of the event loop and qt’s app.exec

Either use the Qt flavour of the cvNamedWindow, or simple grab images from openCV and display them in a QLabel. Or better inherit from QWidget and write your own QImage painter

void OpencvWidget::paintEvent(QPaintEvent*)
{
//m_win is the window size
QPainter p(this);
p.drawImage(m_win,m_image,m_image.rect());

}

The best way to fix ‘VIDIOC_QUERYMENU:Invalid argument’

I have faced a similar problem recently (displaying webcam information using OpenCV and Qt as the final GUI). The best way that I find out to play the video (which is basically a set of images) is to use a GLWidget.

This is a brief summary of the work that has been done previously, including the ubuntu 11.04 install, Xtion depth camera setting, openCV 2.3.1 installing, and the final openNI install and configure.

Because this is an overview of the past work, the source codes are not included here.

Ubuntu 11.04 on pandaboard

The installation of Ubuntu 11.04 in the pandaboard did spend me some time. Since there was no hard drive in the pandaboard the SD card was specified to play that role. This means the whole Ubuntu 11.04 system should be written to the SD card. Especially, the becaue the pandaboard was based on ARM cup, it needed a special version of ubuntu – a pre-install ubuntu version for OMAP4. Downloading the system image, then the system install stared.

The hard part I met was not in the downloading, but the image system writing. Since the image was formated in RAW image, normal image burning software e.g. nero was not able to burn such image. Although the official instruction in ubuntu provided the instructions of burning the system image within three different systems windows, linux, and mac.

The instruction of windows system was not working. A disk writer software called ‘disimagewriter’ did not work functionally, at least in my and my coleagues’ computers. And currently, there was no other software that could do the same thing. I had a macbook but unfortunately the official instructions were not lucky to me.

Finally I found a shortcut. I downloaded a software to mount the RAW image as a system drive, and then used another software to copy the whole disk drive to another. In other words, I copied the RAW image -based drive to the SD card drive. It turned out successfully. The details can be reached here.

After an installation the ubuntu now was running on the pandaboard – feels pretty good, but the wifi signal sucked.

Webcams and the Xtion depth camera

Install openCV 2.3.1 in Ubuntu 11.04 with pandaboard

Ubuntu systems have good supports for webcams, but most of these are based on normal x86/x64 systems, not for ARM systems. I was worried about this problem, as the cameras to use with the system were quite special – not only the rgb webcams, but also the depth camera from ASUS Xtion pro camera.

A good way to use the cameras and to get proper drivers is to install some universal drivers, such as v4l, and ffmpeg.

During the install of openCV in the system, these libraries were particularly important.

Install openNI, NITE and depth camera driver for natural interaction app development

Install openNI for ARM-linux was quite easy, as there were three key components ready to download in the openni.org .

Downloading the unstable versions – openNI binary for ARM, sensor openNI for ARM, unzipping the packages and using the sudo ./install.sh’ command to complete the install.

* be aware of that, there is currently no openNI middleware package for ARM -based linux system, which means, some of the advanced features of openNI such like ‘user generators’ and ‘skeleton generators’ could not be used. The middleware for linux x86 was tested, but with no luck. The error’ one or more of the following nodes could not be enumerated‘ was actually caused by that factor.

Anyway, I ve got rgb and depth images to use. Hopefully the middleware for arm-linux comes out sooner.

The summary

It is hard to describe all details in a short post. As posted in the previous articles, the lessens learnt from the whole procedures are valuable, and more important, I wish these would be shared to give guidances to other people’s work.

ERROR – ‘One or more of the following nodes could not be enumerated:’

STATUS: not solved yet – still a big headache

Xn::UserGenerator and OpenNI

Install OpenNI in Linux with Arm is never a easy job – few people work on it – most of the work is based on windows and kinect rather than ubuntu and xtion and the arm board, and little experience is shared over the internet. So when I tried to run the usertracker example from the sample folder, I got this error with no luck or idea.

The thing is that, I installed OpenNI, NITE and driver, that referred to the install work of others – sorry not to mention a few here. The simpleviewer example worked fine, the image and depth maps were successfully captured.

But, the problem happened when – creating usergenerators, like:

Help needed

ImageGenerator and DepthGenerator have been tested with well working. Any other generators do not work, and keep giving the same error.

Any ideas would be very much appreciated!

When I moved the openCV development platform from the home to the office, a wired thing happened – the programme that I debugged smoothly in the home did not work at all in the office – even the simplest openCV camera view example, and the error said that – ‘HIGHGUI ERROR: V4L: index 0 is not correct‘ – but nothing of openCV configurations were changed at all!

So, where was the error from and, how to fix that wired error?

What caused this error?

Firstly I searched google for ‘HIGHGUI ERROR: V4L: index 0 is not correct’. This issue should be relevant with V4L.

Some pages the search results suggested to remove the v4l in Ubuntu, and then reinstall it again cleanly.

Some others suggested that re-configure and compile the openCV 2.3.1 might be solving the problem – as there was a file called ‘cvcap_libv4l.cpp’ in the source compiling folder, and a bug function was called in that file – find a line with ‘mman‘ function, and it supposed to be ‘libv4l_mman‘.

But, reconfiguring and re-compiling the openCV? That would take ages!

The unexpected solution

Before starting the re-compiling thing, be aware of this – the camera, yes, the camera!

Changing another camera may help? Why not bother to have a dead try?!

Oh-my— my camera was not plugged in!!!

Plugged in the camera, bingo~ everything worked fine.

Appendix – reading images from the webcam

This is a simple example of capturing live images from webcams using the new installed OpenCV 2.3.1 in the Ubuntu. The relevant information of OpenCV 2.3.1 install can see my previous posts -

Installing openCV in Ubuntu with pandaboard part 1
Installing openCV in Ubuntu with pandaboard part 2
Installing openCV in Ubuntu with pandaboard part 3 – the network proxy
Installing openCV in Ubuntu with pandaboard part 4 – compiling openCV 2.3.1
Installing openCV in Ubuntu with pandaboard part 5 – running openCV 2.3.1 examples, and
Tutorial: A Comprehensive Guide to Installing and Configuring OpenCV 2.3.1 on Ubuntu

This simple example includes lines of source codes using the new OpenCV API, for your convenience.

The source code

The source file compiling

Save the source file as ‘readwebcam’ for example, go to the directory containing the file.

Open the terminal, and type in the command like:

Then run it:

Here it is! Now let’s start rocking the hand gesture recognition!

This post describes an error and its solution in ffmpeg make for openCV install. It is based on Osman’s comprehensive troubleshooting of openCV 2.3.1 installing and configuring, but adds an additional error – ‘[libavutil/libavutil.so.51] Error 1‘ and the solution.

This error happens when running ‘make’ command after the ffmpeg configuration. It is a fatal error which directly stops the make process and fails the ffmpeg install.

why ‘[libavutil/libavutil.so.51] Error 1′ happens

Osman’s configurations for ffmpeg install are like this:

This was used in my first ffmpeg install, and was added with another line as:

But the result turned out this error – [libavutil/libavutil.so.51] Error 1

The solution

It was suggested over the internet that this problem was caused by the lack of ‘libvpx’ – this was essential to be used with ‘–enable-shared’.

So, clean the previous make using ‘make clean’ or ‘make distclean’, then re-configure the install as:

Then ‘make’ – bingo~ errors gone!

Thanks to the comprehensive troubleshooting guide to opencv installation posted by Osman, some install errors could be referred and fixed. But some errors are not included in that guide, as the chance of the error is various.

The point of this post is to add another minor error that encountered in my v4l making for opencv installing.

‘QMainWindow: No such file or directory’

‘No such file or directory’, this type of error used to be caused by the lack of some libraries not installed yet. The potential ways to fix these are to install the lacked libs, of course. But it may take a little time to check which libs are lacked, as the error message used to tell limited information.

In this example, ‘QMainWindow’ turned out to be relevant with libqt4-dev after a quick search in the internet. So, using the command below could easily solve that problem.

Carrying on the installation until it finishes. Then redo the v4l make, binggo~ the problem’s gone!

The initial outcomes of the OpenNI installation

A previous post has described an overall progress of installing OpenNI in Linux-arm -based pandaboard with Xtion pro. This post is a sequence of the instsallation, which demonstrates the success of OpenNI installation, and the working of Xtion pro with Linux-arm Ubuntu.

Subsequent developments

The first depth image in this post reminds me a potential breakthrough of hand detection – the fingers could be easily extracted from the backgrounds, no matter how complicated these are. Combined with the OpenCV 2.3.1 installed days ago, the hand detection could be developing some interesting applications.

About OpenNI

OpenNI has shown some lights to natural interactions by using depth map images together with RGB images from cameras. Natural human gestures, e.g. hand moving, limbs movements are all being developed fast. Microsoft Kinect is a typical example that demonstrated the huge potential of the OpenNI in terms of game applications. While, more work could be done with OpenNI, with other devices that could capture depth images, e.g. ASUS’s Xtion pro. In the OpenNI community supported by manufacturers e.g. Primesense, Microsoft and Asus more information could be found in openni.org.

About installing OpenNI

Many tutorials can be search out in the internet, there vary from different operation systems to different installation packages and to various installation errors and solutions as well. Mostly, these focus on three popular systems – windows, linux and mac os. In the windows systems, of course, the installation is relatively the easiest as a good population of developers are working on it. In the mac os, it is ok to install the OpenNI as it is intuitive and with good support from the community. In the Linux system the situation is more complicated as different sub systems were involved. Firstly, different versions of Linux and their configurations, secondly, the Linux running on a broad range of devices with diverse system structures.

And here in this post, it is about installing OpenNI in a Linux version – Ubuntu 11.04 Natty in a ARM -based device – the pandaboard.

The Installation steps

Jatin Sharma posted an article talking about how to install OpenNI on Beagleboard with Kinect – just seem this as a similar version of pandaboard and Xtion pro. That post is referable to the installations described in this article. So I would ‘borrow’ some of the procedures to better demonstrate my installations of OpenNI.

1. certain packages are required pior to the installations of OpenNI. Try the commands below to do fetch and install these packages.

* Jatin mentioned an error in ‘libglut3-dev’, as this lib has been replaced by ‘freeglut3-dev’, so you can directly try the later one. But actually I installed the ‘libglut3-dev’ successfully in Synaptic Package Manager, as well as ‘freeglut3-dev’. But never mind, as long as the required package is installed and working properly.

2. fetch the OpenNI package

Making a new directory to hold this package would be a good idea, that gives you much convenience. Let’s say to make it ‘~/Documents/OpenNI’

using the command below to get the package of OpenNI:

*depends on the versions, the contents of the OpenNi folder may be different.But the common thing is to get into the ‘Build’ folder. Mine is under ‘~/Documents/OpenNI/Platform/Linux/Build’.

* Please be aware of Jatin’s information that

Edit the make file in ~/kinect/OpenNI/Platform/Linux-x86/Build
In the latest version of OpenNI, the ARM platform has been included by the name Linux-ARM. However if we go into ~/kinect/OpenNI/Platform/Linux-ARM/Build/ and try to build it you’ll face errors thus better would be to use the old Linux-x86 platform files with some modifications.

Open ~/kinect/OpenNI/Platform/Linux-x86/Build/CommonMakefile and comment out these lines:

CFLAGS += -malign-double
and
ifeq ($(SSE_GENERATION), 2)
CFLAGS += -msse2
else
ifeq ($(SSE_GENERATION), 3)
CFLAGS += -msse3
else
($error “Only SSE2 and SSE3 are supported”)
endif
endif

We need to do this because ‘ -malign-double’ and ‘-msse3’ flags are valid only for x86 platform and will not work for gcc on ARM. Commenting out these statements makes the code usable for ARM.

Make and Make install

Future work

This installation has not yet been validated if the OpenNI samples are working or not. The sample validation and programming part will be published in another post.

Hi my friend,

Thanks for your continuous supports, interests, and valuable comments to ANDOL.INFO, I would remember all of these, and make the coming year better.

Merry Xmas

and

Happy New Year 2012!!!

Best wishes,

Andol Li

The last tutorial about installing opencv in ubuntu with pandaboard 4 – compiling the opencv 2.3.1 did not complete the final compile steps, this post aims to complete that.

Continue the compiling of openCV 2.3.1

I assume you have finished the openCV compiling using cmake ., make, and make install commands in the terminal. Bear in my mind there is one thing I need to remind – be aware of the system date and time, as I left that incorrect and made some unnecessary compiling mistakes.

The system configurations

After the ‘make install’ commands was executed, the system configurations needed to be set up.

1. use the command below to set the opencv’s lib links in opencv.conf file.

adding the following link to the end of the file: /usr/local/lib

2. need to load the configuration file by using

3. open another file and add the configuration file path, adding the highlighted contents as in the below picture.

4. that’s done! Let’s launch an example to have a look if it works.

Running the example

1. make sure the build-essential is installed for opencv programme compiling, if not, using this:

sudo apt-get install build-essential

2. open the terminal, and go to the place holding the sample files, e.g. opencv-2.3.1/samples/c

3. change the target file mod, if you receive permission errors:

4. get the file compiled ready:

this command invokes the GNU C compiler to compile the file contours.c and make it an executable file called ‘contours’.

5. run the executable file contours:

dang-dang….. here it is…

Before the compiling

I was about to write a post about compiling the newest openCV 2.3.1 in Ubuntu with pandaboard, but because of the proxy network problem in Ubuntu software update and the right solution to the proxy network settings, this post was postponed.

So this post will be focusing on compiling openCV 2.3.1 in Ubuntu, with the right proxy networking settings and the synaptic package manager in working states.Sebastian Montabone has made a tutorial about installing openCV in Ubuntu, but that used openCV 2.0 and was not up-to-date. So, this tutorial aims to add an update of the newest version of openCV 2.3.1 installation.

Software updating

The compiling starts from software updating, which aims to prepare all necessary installation packages in the system. To make these packages needed for installations clear, I post these in forms of terminal commands, so you can type these commands one by one to see if they are installed already, or need updates.

Please be aware that, this post assumes the openCV 2.3.1 installation package has been downloaded in the local folder called ‘Download’, and has been unzipped into a folder called ‘OpenCV-2.3.1′.

Compiling steps

1. open the terminal, go to the directory where the openCV folder is located in, in this example, it is ‘/home/cw418/Downloads/OpenCV-2.3.1′.

2. type the cmake command in the terminal. please be aware there is a small ‘.’ after the cmake command.

3. after the cmake command is executed, type new make command.

This may take a while, so you can go and have a cup of tea first.
The tutorial continues on the next post – installing openCV 2.3.1 in ubuntu with pandaboard part 5.

The problem of networking proxy in ubuntu for openCV installation

In the last post instsalling openCV in Ubuntu with pandaboard a networking proxy problem was raised. The problem is relevant to software updates and installations in Ubuntu, before and after openCV installation. There were two general suggestions made in the previous post – finding a new network environment, or, properly setting up the proxy parameters.

The solution to ubuntu proxy settings

For the second suggestion, it seems more difficult than the first one, as it needs to be aware of different applications’ networking settings. This is my empirical experiences that took me nearly four days to get the right place.

1. I assume You have successfully got the proxy address, which used to be like this username:password@wwwproxy.domain.com:8080. This would be useful in the later setting.

2. There are two types of network proxy settings – application wide and whole system wide. I made mistakes in this stage so several days were wasted.

3. Click the power button on the top right corner, select the ‘System Settings’ from the popping up menu.

4. Go to ‘Network Proxy’ to start configuring the proxy parameters – if you have automatic scripts for proxy configurations then use that, if not, then fill in the proxy parameters in proper columns. Make sure you are aware of the bottom setting options – Applying system wide… click that button and confirm the passwords popping up for system authorisation.

5. Open firefox and check the network setting to see if it uses system configurations, if yes, see if firefox can open websites.

6. If firefox works, then go to ‘synaptic package manager’, go to ‘settings/preferences/network’, make sure the option is selected as ‘direct connection to the internet’, apply it.

7. Go and check the package manager updates, it should be working now.

Some errors caused by the incorrect proxy settings

Some typical errors caused by incorrect proxy settings include

- ‘failed to fetch’ error when trying to update the packages using apt-get update
- ’404 not found’ errors in reloading synaptic package manager

How to install openCV in Ubuntu without networks

In the last post of installing openCV in Ubuntu with pandaboard, the failure of software update of openCV was discussed. But if there was no alternative to switch to other working networks, then the only way to install openCV in Ubuntu is as the way showed in this post.

- first, downloading the OpenCV-2.3.1a.tar.bz2 from openCV wiki, selecting ‘other announcements’ then going to ‘for mac and linux’, the page will be directed to sourceforge in this page: http://sourceforge.net/projects/opencvlibrary/files/opencv-unix/.

- copying the downloaded file to Ubuntu system, let’s make it in ‘Download’ folder. Then starting the manual installation – this part is rarely mentioned in tutorials over Internet.

How to install openCV in Ubuntu 11.04 manually

Once the openCV installation file has been ready in the folder, firstly it needs to unzip – using the commands in terminal:

- tar -xvjpf OpenCV-2.3.1a.tar.bz2

The untar progress will be running like this for a short while:

Untar openCV 2.3.1a in Downloads

After the openCV installation package is successfully untared in the Download folder, the source files can be seen. This is not the ready-to-use version, as the source files still need compiling.

–
To continue – part 3 – Compiling the openCV 2.3.1 within Ubuntu in pandaboard.
a networking-functional version of installing can be reached in Sebastian Montabone’s post ‘Installing OpenCV 2.2 in Ubuntu 11.04‘.

OpenCV at ubuntu with pandaboard

At the first beginning of post, I would like to remind a tiny, but sometimes may huge, problem in installing openCV at ubuntu with pandaboard – the network proxy setting. This tiny issue could be the fatal problem when it was not aware of. I had tried hundreds of times to ‘apt-get update’ but nothing worked out. After I searched the problem of ‘failed to fetch…’, I was not well informed about how severe that problem was. Until I tied to change the Ubuntu Software Center’s networking settings, I noticed there was an tap used for networking proxy purposes.

How serious is this issue

It is common that computers setting in universities are behind a boss proxy, it means all computers inside the university intranet have to get through the university server – data flows, firewalls and connection routines of course. The problem here is, the ubuntu system seems to recognise such super proxy not very well, to be honest, poorly. And what surprised me was that there was little posts in the internet that provided relevant information about how to set up the network connection before starting software update of installations.

As a result, the plan to install JAVA in ubuntu with pandaboard for openCV project development was non-doubtfully postponed – the reason was for the above.

How to overcome opencv at ubuntu installation

Well, the fastest and simplest way directing at the solution of this issue is to change to another network environments, or, use other network connection ways that does not need proxy settings. A good solution is already there – the wireless network. However, I have to mention that, the wifi signal received by the pandaboard was so poor – in most occasions there was only one signal bar – so tiny that I was always worried about if if would fade out. And the results proved that as well – the connection was unstable and slow.

Ok, no more chatty words, to address the network problem, either a wireless network is needed, or a open network connection that does not hold by a proxy is useful as well.

So, I am going to take the pandabaord home and get new software installed, until then I will be back to post ongoing openCV developments.

Some problematic screenshots

If you feel this post not providing the solution to the problem, please make yourself free to post any comments below.

Useful openCV mailing list

Someone in the comments asked where to get so much information about openCV learning and developing, well, one of my primary source to get openCV related information – as well as most people’s issues they are facing recently is the openCV mailing list – a good way to receive archived threads and discussions from lots groups such as yahoo openCV groups and google openCV groups, and even stackoverflow q&a forum, or maybe the canonical openCV wiki is also a good source.

Benefits to subscribe these sources’ mailing list

Well, it is obvious that if all of these groups and forums are subscribed by one reader, then the overwhelming information will keep him super hyper busy – all together these may generate up to thousands of openCV related items – including all kinds of problems, all levels of issues, and all possibilities of new openCV breakthroughs.

How to subscribe openCV mailing list

The simplest way to subscribe the openCV mailing list is to go to the groups or forum, make a registration, and probably there is a slot called ‘enter email address‘ waiting for your email address to fill in, after confirming the subscription – probably through emails the openCV mailing list will be like a loyal servant collecting and sending everyday or if you wish every week’s openCV discussions to your email, enjoy.

Opencv mailing list’s risks?

Well, if I have to say, the only risk is the too much information I receive everyday, I even have to set the mailing list subscription emails into a special category.

@丕子‘s comment reminded me to broaden the review area to cover multi-objects detection. Apologise for not making clear what is about to review in the second part, as a continue review of people counting using openCV, as in part 1 the review did not tell a clear content structure.
@zhijie thanks for your kind wishes.

Challenges for people counting using openCV

In part 1 the review mentioned some difficulties to apply openCV for people counting, including camera positions, specific detection feature (faces) and overlapping objects.

There are some other challenges as well – multiple people detection and tracking. This even concerns previous challenges, like the object overlapping. Someone suggested to use infrared photodetectors to avoid such issue, while that does not make sense here in using openCV and camera.

Methods of people counting using openCV

Andrey‘s suggestions gave three ideas from simple to advance. Simply enough, it was advised to calculate differences between successive frames of video stream and this detected line-crossing event by determining motion masks. A little more complicated, the idea was to use ‘running average‘ approach, and the most advanced one, was to integrate background subtraction technique to enhance the accuracy. The running average can be reached here openCV cvRunningAVG. However, there is still a problem – what if people gather together as a group then enter the room? In other words, how to keep tracking objects that may be combined and separated, unpredictably.

Counting grouped people using openCV

A short paper was recommended by someone in overstackflow.com, about human tracking by fast mean shift model seeking. Due to the limitation of time, the post is not demonstrating the details, but a picture is presented for demonstration. The paper can be reached via the link provided on the bottom of the post. This paper shows tracks behind people’s movements, grouping and separating have little influence on it.

Other ways of people counting using openCV

Other approaches were also introduced to deal with people group gathering problem. Someone (sorry I forgot the source) used track ID to mark different states of people. Every individual was given a unique track ID, which was binded with pixel-based contours. When two contours emerged together, the two track IDs would follow the common contour, until these people entered the room and were counted by the system.

A short summary

Neither people counting using openCV in part 1 and 2 is posted with some new cutting edge techniques, with which the people counting could be overwhelmingly improved. Rather, the state of people counting using openCV is very briefly introduced, with some concrete methods for people counting use. Wish these could be sparking some new thoughts.

Looking back the methods for people counting using openCV, some achievements have been made, especially in terms of counting a general number of people within specific settings. However, the future developments still face some problems, such as accurate grouped people counting, dwelling people counting and, probably, super density people counting – will it be possible to do that by counting heads?

Any comments are warmly welcome, including criticisms, as this review is widely open to anyone who is interested in people counting using openCV.

people counting openCV.pdf

Why using openCV for people counting?

There are alternatives to count people in a public place, for example using microcontrollers and lasers and arduino to design a cool and accurate system to count people in and out a room. But why does it bother to use webcams and, more importantly what the post focuses on, openCV?

1. it is easy to set-up. NOTHING is easier to mount a camera over the door and connected with power and data cables. Setting up a laser system? Think about the DIY work.

2. a lot of developers and programmers are working on that – making openCV more suitable and efficient to count people. A simple example is the blob extracting app.

The state of art of openCV for people counting

Blob extracting using openCV or flash (as3) have been researched by many developers. The accuracy has reached a high level, but limited to single objects – objects should not be overlapped – if people in the camera could be simply seem as objects.

Codeproject.com shows a new post discussing recognising people even though they are overlapped to some extent, as the picture below shows. The post compares different algorithms from simple threshold to image differences and erosion .

A pedestrian detector comes with recent versions of openCV (>=2.2), in modules/objdetect/src/hog.cpp, and samples/cpp/peopledetect.cpp. Unfortunately this new example has not be well documented officially.

If the camera for people counting is mounted statically, a substraction method is suggested by evident’s work. A relevant paper is linked http://portal.acm.org/citation.cfm?id=1561072&preflayout=flat.

Face detection for people counting using openCV is also concerned, as an example is described in opencv wiki – face detection using openCV. But the drawbacks are clear – this method needs faces for people counting. In other words, top-view cameras do not fit.

The rest part of review is published in the second part – a review of people counting using openCV part 2.

What webcams/motion sensing cameras are available for OMAP4 platforms?

As the Ubuntu 11.04 Natty has been successfully installed in the pandaboard, the next step of development work is to plug in webcams and motion sensing cameras (kinect and xtion). About installing necessary software such as openCV and openNI a new post will be dedicated to that. This post, mostly, desperately, is about seeking available supports to install motion sensing camera – especially the ASUS Xtion camera in OMAP4 pandaboard, which currently does not have many documents related to camera use in OMAP4 pandaboard.

In pandaboard’s website a compatible webcam Logitech webcam Pro 9000 is listed as one of accessories recommended for purchases. However this webcam is exclusively used in Android systems in pandaboard, in other words, there is no source found about Ubuntu based cameras.

Why cameras are not universally compatible?

My guess, to the incompatibility of cameras in pandaboard may be caused by the CPU – OMAP and normal Intel CPUs may have different structures and command sets. So they need different drivers to make cameras work. The problem is – where, and how.

Can you help?
There are some forum discusses seeking supports to camera drivers in OMAP4 platforms, pandaboard@googlegroups.com has quite many threads about this, but provides no canonical answers.

So this post is not about solving problems, instead, it calls for supports from anyone who may have useful information or experiences, to find a way to using universal cameras in OMAP4 pandaboard.

Intalling ubuntu in pandaboard for opencv development

Last post Doing openCV in Pandaboard described a odd mistake happened during the first boot of ubuntu system after the RAW image burning. The screen was frozen on the first booting, did not provide detailed information about any reasons.

I searched the internet and found two tools suitable to do the job – mounting the RAW image as a virtual disk in my windows xp pc, then burning the virtual disk image to the SD card. Two respective free-software was download – mount image pro for RAW image mounting as a virtual disk, and, minitool partition wizard for disk image copying/burning. These tools have been zipped, and can be reached via the link provided on the bottom of the post

At this time I installed these two tools, and successfully mounted the RAW image as a virtual disk 2, turned out it had three partitions indeed, the last one was not allocated/formatted yet, the first one was nearly 70 M and the second one was the main partition which had over 2 G contents. But the installation instruction in ubuntu official webpage said the disk space would be resized automatically. So I inserted the SD card and found it was recognised as disk 3. Here came the key operation – copying the disk 2 to disk 3, and leave the rest space as not resized.

Wait and wait and wait……

The copying was finally over, as the sd adapter did not flash any more. Then copying the MOL and u-boot.bin files to the first partition to guide the system boot.

Taking out the sd card from the host pc, and inserted it in the pandaboard. Exciting moment was finally coming…

Screenshots of ubuntu intallation on pandaboard

It was successful! Now the openCV development could start from here

The image mounting tool and disk copying tools can be reached via the link below, note the copyrights belonging to original publishers.

tools – mounting RAW image of ubuntu and burning disk copies

Pandaboard and ubuntu and openCV

In last post a brief show-off was given, with some snapshots of the new arriving pandaboard, but the post only had a quick mention of doing opencv jobs in pandaboard. Hence, I draft a new post to describe the environment of using opencv in pandaboard. Mostly it is about installing operation system (ubuntu 11.04) in the pandaboard in which to enable opencv development.

Issues of burning RAW image to SD card – ubuntu and openCV

Many tutorials have been given by bloggers, including video tutorials in youtube.com. Since the pandaboard used sd/mmc card memory as main storage, the ubuntu needs to be burned in the sd card to boot the board. Buring ubuntu in SD card had a simple glance at the official tutorial to install ubuntu in the pandaboard. However, that tutorial introduced an invalid software to burn RAW image in to the SD card, as it – win32 image writer kept popping ‘error 8′. All three versions had been tested, still gave same errors, and were finally given up.

Ubuntu RAW image’s contents – the head scratches

I looked into the RAW image extracted from the .gz file, found it consisted of three files, specifically, partitions, respectively named as ’0′, ’1.img’, and ’2′. So it seemed I need to project the three-partition image to the SD card. In windows xp, virtual image reading, disk partition copying – what tools could fix this headache?

Stucked ubuntu booting – any ideas?
I partitioned the SD card into three parts, as the RAW image did, same size, same files and same partition orders. Then copied related files to three partitions, burned the ’1.img’ to the second partition – only got cold black screen. Then I changed the first partition’s setting as active partition – windows used to boot from active partition. Try again, OOps, it worked this time…wait………stucked…lots errors…no booting ;{

(to be continued)

In a early post openCV was used to segment natural hand gestures from complicated backgrounds in real environments, as the picture above showed (see the original post hand gesture detection and recognition using openCV).

The picture above is a screenshot from a recent paper, about using Ataboost with SIFT (scale invariant feature transform), to detect natural hand gestures. The SIFT is used in the paper to reduce the background noise in the training stage, and so experimental results demonstrated in the paper show the approach performs with high accuracy.

So, the the adaboost learning algorithm is used after the SIFT background noise reducing. Relevant functions corresponding to these two algorithms have not been checked in openCV, to see if there is any working functions to realise such algorithms. However, more details about the use of Adaboost and SIFT, the paper hand posture recognition using Adaboost with sift for human robot interaction is reachable in the download page.

The secret to successfully disable annoying autosaving and revisions in WordPress

Searching out the workpress official site there are a number of posts talking about how to stop the autosaving and revision wordpress. If searching the term in google then the answers are numberous. Typically there are two types of solutions. One is of course to use plugins to do the job, and the other one is to modify some settings manually. The hand-coding solution used to have two lines:

But pasting these two lines in the wp-config.php file does not work, many users complain about that.

The reason is that these two lines are put in wrong places. Putting these lines on the last of the configure file does not make it work, as the wp-settings.php has been loaded already. So, the successful secret is that, put these two lines before the code loading the setting file. Have a try, it works in my blog.

Haarcascades training (haartraining) is seemed an quick tool to achieve accurate hand gesture detection and recognition. The face and body detection examples included in openCV’s installation example folders (\opencv\data\haarcascades\) demonstrate how fast the haarcascades files help to do the job. More information about how to train the haarcascades files can go to sonots.com.

Many face-image databases have been provided for haarcascades training, e.g. http://www.face-rec.org/databases/. But hand images are less contributed for haarcascades training. At least, these are more difficult than face images to find in the internet. I was also asked by many blog readers for the haarcascades training file to support their hand detection or recognition related projects.

So I post an example of using haarcascades training file 1256617233-1-haarcascade_hand.xml for hand gesture detection.

The source codes can be downloaded from (or go to the download page):

http://download.andol.info/1256617233-1-haarcascade_hand.xml
http://download.andol.info/1256617233-2-haarcascade-hand.xml
http://download.andol.info/haarcascades-based%20hand%20detection.cpp

Ok, this is the story. If you are a returning reader of my blog you probably would be noticing the improvement of website speed – it is in a rocket now. Looking back two weeks ago, the blog was hosted in Godaddy, known as the slow daddy. Until recent, even I have noticed and reported the unbearable slowness for weeks, I am still not able to open my blog and post new projects and codes.

Well, the story has a happy ending. The slow daddy’s hosting plan is going to expire in a month time, so I switch to bluehost with my broken heart. Based on tonight’s performance, the speed is amazingly satisfied.

So, bluehost K.O. gadaddy.

Farewell, slow daddy’s wild girls ;(

The preparation work
When I googled ‘pandaboard ubuntu’ I was firstly given the item of ‘ubuntu on pandaboar’. Following the link, finally, I was led to a ‘OMAP Ubuntu’ instruction page – it tells where I should start, to download the system image, and to burn it in a SD card.

Then downloading the right version of Ubuntu for OMAP4 platform. A number of copy versions are provided in the download page, in which the right one is ‘Texas Instruments OMAP4 preinstalled netbook image’, the download file after clicking the link is named like this ‘ubuntu-11.04-preinstalled-netbook-armel+omap4.img.gz’.

After the preinstalled system image downloading, need to unzip it, using winrar or winzip or other proper unzip tools.

(To continue – burning the downloaded image in a SD card)

Just received the pandaboard, to show off (;)) before I start to install Ubuntu on it, and to run openCV to see what will happen. New post soon, about installing Ubuntu on this board, then installing and running openCV on it. More posts will be published soon.

Transferring the openCV from c++ to Java is what i am doing recently. Comparing with opencv c++ it feels more natural to programme in Java, because of the way of Object-oriented programming and some of Java’s advantages.

But how can the openCV be used in Java seamlessly?

The easiest way is to take advantage of existing Java openCV projects, such as JavaCV (http://code.google.com/p/javacv/) or openCV in Java (http://ubaa.net/shared/processing/opencv/). These wrappers do make the use of openCV relatively easier, as the face detection example below.

import hypermedia.video.*;
import java.awt.Rectangle;

OpenCV opencv;

void setup() {

    size( 320, 240 );

    opencv = new OpenCV(this);
    opencv.capture( width, height );
    opencv.cascade( OpenCV.CASCADE_FRONTALFACE_ALT );    
    // load the FRONTALFACE description file
}

void draw() {

    opencv.read();
    image( opencv.image(), 0, 0 );

    // detect anything ressembling a FRONTALFACE
    Rectangle[] faces = opencv.detect();

    // draw detected face area(s)
    noFill();
    stroke(255,0,0);
    for( int i=0; i<faces.length; i++ ) {
        rect( faces[i].x, faces[i].y, faces[i].width, faces[i].height ); 
    }
}
[These lines of codes are from 

http://ubaa.net/shared/processing/opencv/opencv_detect.html]

Another advantage of using openCV in Java is that it can be integrated into mobile systems such as the Android. With Eclipse the wrapper classes can be used to programme intuitively. For example:

import static com.googlecode.javacv.cpp.opencv_core.*;
import static com.googlecode.javacv.cpp.opencv_imgproc.*;
import static com.googlecode.javacv.cpp.opencv_highgui.*;

public class Smoother {
    public static void smooth(String filename) { 
        IplImage image = cvLoadImage(filename);
        if (image != null) {
            cvSmooth(image, image, CV_GAUSSIAN, 3);
            cvSaveImage(filename, image);
            cvReleaseImage(image);
        }
    }
}

The source codes are referred from http://www.hackchina.com/r/59225/main.cpp__html

DETAILS

space: 10G (or more as request)
monthly bandwidth: unlimited (though you can use bandwidth unlimitedly, please be aware of other sites’ performances)
database: one mysql datebase sized at 250M
program environment: php4/5one
domain: your personal domain can be bound to your hosting space
ftp account provided for file managements
price: £10 pa

REQUIREMENTS

1. ideally for personal blog or storage space only, no bbs or sns-like sites, nor BT sites.
2. all maintaining work will done by me, if any problem encountered there will be fast response as possible.
3. do not use this for illegal purposes against laws

if this may interest you or have any detail enquiries please contact andol yandol[at]hotmail.com with no hesitation.

As asked many times about how to use openCV to detect body movements or other object movements, i borrow an example from openCV installation samples, which regards to motion detections particularly in motion directions. Some relevant posts have been posted before here, but to declare the use of openCV samples, this artical is posted. The picture demonstrating above has clearly showed the detection results which is, from left up to right down, just as the object moves, which is actually my head and part of shoulder.

Though this example is far too simple to put in practical use, this can be still useful for further applications as long as the codes are improved, at least the detection rules are remained. Here below is the list of source code which can be used directly as a .cpp file, check it out:

To download the source code, please refer to directory “OpenCV\samples\c\motempl.c“.

I found this by chance when i was searching something about body gesture recognitions. But this open library does attract me because it provides some features which can be very useful to detect and recognise objects, such like the picture above shows. Anyway i just provide a brief introduction to this library, and for more information please refer the official site here http://www.aforgenet.com/framework/samples/.

Shortly after further using and comparing with openCV in some object detections and recognitions, i will post another review post on this. Here is the brief introductions below.

AForge.NET framework is represented not only by a set of libraries and corresponding documentation, but also consists of collection of different sample applications, which are aimed to demonstrate usage of different framework’s API. The samples collection together with the provided framework’s documentation serves a good source of information and allows people to start working quickly with the framework and apply it for solving their own tasks in their different applications.

All the sources of AForge.NET framework’s sample applications are available in framework’s installation package, which contains all the components required to start working with the framework. Also binaries of all applications are available for downloading directly from the site, so each sample could be tested without installing the framework (look for downloads links on pages describing samples).

Farewell, 2010!

Wish every friend happy new year 2011!

As the new release of openCV version 2.0, the projects which were published in this site may generate many issues while trying to build the code in a project. To address this type of issue, please check the version of openCV, as this may be a easy problem if the version triggers issues.

This is a quick reminder that noticing the openCV version while compiling the codes, hope this helps.

More information about the differences between version 1.2 and 2.0, i suggest the openCVWiki and another site openCV Reference.

1. Multitouch and gestures defined

The term multitouch refers to the ability not only to detect physical touches and movements on a screen, but to detect and track multiple touches and movements simultaneously. Touch events are similar to mouse events, except that you can receive and track more than one of them at once, and touch events do not support mouse-specific concepts such as hovering.

Gestures are the synthesis of multiple touch events into a single event. Examples of gestures include “pinching” an image to scale it, or “swiping” to delete something from a list. Some platforms explicitly support the concept of gestures, reducing the amount of work that a developer needs to do in order to detect and react to them, and some platforms require developers to capture multiple touch events and synthesize them into gestures themselves. The Flash Platform automatically synthesizes the most common gestures across different platforms, but also provides developers with the APIs necessary to synthesize their own.

Although multitouch technology has been around for many years, it was the popularity of the Apple iPhone that really introduced the concept to the mainstream. The advantages of interacting with devices directly, rather than through buttons or using a stylus, has become so apparent that the behavior is making its way into desktop computing, as well. Windows 7 supports multitouch right out of the box, HP has been selling touch-enabled TouchSmart computers since 2007, and Microsoft launched the gesture- and touch-centric Microsoft Surface in 2008. Additionally, Apple introduced their multitouch trackpad with the MacBook Air, and has since incorporated the technology into their entire laptop line. Apple’s newest mouse—the Magic Mouse—even has limited gesture support. Touch, multitouch, gesture, and haptic-based computing is becoming so prominent that almost all new high-end, hand-held devices support one or more of these interaction models.

…

For full information please refer this original article http://www.adobe.com/devnet/flash/articles/multitouch_gestures.html

For more details, here is a relevant tutorial video made by original author who explains who this works.

Here gives the source code of this video demonstration.

You can download the full code at:
1) Full Demo Solution (Code + video 43Mb)
http://db.tt/MukGcwB
2) Full Demo Code
http://db.tt/GkT6J9k

First we gather a data set of all the hand-shapes we wish to recognise. A naive approach to recognizing a new image D would be to simply compare it with all the images stored in the data set and find the target image T with the closest match. But because there are so many images in the data set this will take far too long. We can reduce the time by using a multi-scale approach. We divide up the data set into groups of images, which are similar to one another by blurring the images at different levels so that small differences between similar images will be eroded.

More information please refer here http://www.ercim.eu/publication/Ercim_News/enw46/shamaie.html

About openCV

A library of programming functions for real time computer vision,released under a BSD?license, and free for academic and commercial use (a). Current version is openCV 2.1, this version has been installed and compiled under my VS 2008 environment but with some errors, may not be very easy to configure this version. While the stabilities of openCV 1.0 have been firmly proved already and that is easier to install and configure. Install guides can be found as ?openCV install guide in openCV wiki,?installing openCV in windows, vc2008 install openCV 2.0/2.1(in Chinese)?and get started with openCV.

A easy start of ‘hello world‘ application using openCV can be very helpful. Under most occasions, openCV is used within windows console applications with a black command window and openCV’s own interfaces (highgui) to display images and draw patterns, as the ‘hello world’ example shows. As well there are some other interface components included in openCV library such as sliders, mouse movement events and key input events. Using these provided features of openCV, object detection could be done via capturing webcam images, analysing them and finally output the consequences.

One recommendation of learning openCV online is Yahoo openCV group. It is a group with huge number of active members posting hundreds of messages each day, may be a little massive, but a good place to raise questions and meet other people, highly recommended.

About detections & recognitions

Detections are core functions of openCV. Color knobs, objects, motions, gestures and faces are subjects of openCV detections and recognitions.

Color knob detections

detecting colors using openCV is basic, there could be several ways to achieve the color detections. For instance, as the color detection used in my hand recognition project, the target color is divided into three perspectives of H, B, and V. By combining these parameters, theoretically it is available to detect any color. But the fault of HSV method is that the background colours are usually too sensitive to be specified. So does another method which uses simple brightness threshold to get target colours, but it is limited as well. A better way of detecting a range of colours is to use ‘camshift’ which requires a source image which represents colour information for target colours. There is an example of using ‘camshift’ here.

Object detections

Object detection is kind of similar as colour extraction since to recognise the objects they should be detected first. Here is an example of using Actionscript 3.0 to tracking objects via webcam. In openCV it is simple as well. Tracking the objects, then using circle or line detection methods to extract rough object shapes. It could be very accurate of detecting object angles which are demonstrated here, and another object detection example illustration here. Detecting an array of object is nearly the same as detecting solo object, except some extra work is needed to loop all objects at one time and draw them out, this could be seem here as detecting combined objects.

Motion detections

There is an example of motion detections in openCV’s example folders, which is called motemlp.exe and a source file motempl.c. The picture on the left shows the motion directions and sub components’ directions are being marked clearly.

Gesture and face detections
Face detections are maturely applied in digital cameras presently. It is dependant on haar trainings and classifiers therefore the detection results seem robust. As well, a face detection example with source code can be found in openCV’s example folder. In this review face detection would not be emphasised too much as there are millions of tutorials telling how to detect the face in either static or dynamic circumstances.

Recognitions of hand gestures is the challenging part.  On current stage, hand gestures especially finger tips are still tough due to the natural shapes of hands are ambiguous for computer to identify which is which. Therefore some simple and intuitive gestures are commonly adopted to relatively improve the accuracy of recognitions. Recognition gestures requires two factors: a. correct contours and b. predefined gesture patterns. The first factor is reachable but the second requires more work to cheer up. Concerning the second factor, there have been some theoretical papers raising solutions of detecting hand gestures. Toughly, putting these theoretical methods into practical use and generate kind of applications providing open source codes still has long way to go. A progressive project can be referred here as hand detection and image descriptions.

Useful toolkits

HandVu

ARToolkit

I have experienced that and that was awful. Therefore i decide to share these solutions by providing functional codes as below. The solution is easy, after adding new media player relevant classes then, using codes below to replace those generated automatically by system. The key part of the change is from ‘public:CWND’ to ‘public:COleDispatchDriver’.

CWMPControls.cpp

CWMPControls.h

CWMPMedia.cpp

CWMPMedia.h

CWMPPlaylist.cpp

CWMPPlaylist.h

CWMPSettings.cpp

CWMPSettings.h

ocx1.cpp

ocx1.h

Step 1: Visual C++ wizard, choose MFC App.

Step 2: Visual C++, choose single document, to create main MFC app frame.

Step 3: Add openCV libs in project properties.

Step 4: Add variables in Doc class: add a variable m_image, include highgui.h head file in the first beginning, and the variable CImage   m_image.

Step 5: Add two virtual functions to open and save image file, as shown below.

Step 6: Add codes in View class to show the image

Step 7: Compile the app, then an image can be open through open menu, here comes the beauty Lena.

—

There are several vital parameters in using the function cvsnakeimage, particularly parameters relating to curve energy which concerns judgements of finger tips. E.g.

A copy of full source code of demonstrating the use of the function could be found here cvSnakeImage for hand detection

Revision 30/03/2012: The source code can be found here http://download.andol.info/cvsnakeimage.cpp

secret powers from Mechanics of Destruction on Vimeo.

Relevant posts & comments please refer to  ‘Hand gesture detection and recognition using openCV’， and related comments here.

—Some thoughts of <An Education>

Iceland people joke themselves as ‘disaster exporters’ while after last bankruptcies it follows volcano eruption which involves the whole Europe and is known world widely.

Millions of people are stranded in Europe, and its negative waves continue, types of vegetables like mushrooms and tomatoes in Huddersfield’s Sainsbury are almost  in short now.

I’d better get some canned food just in case. ..

It is but a photograph of a rail station in Huddersfield.

A full paper about this can be found here titled as ‘tracking a hand manipulating objects‘

As described by the authors, a method of individual local tracker is used to achieve extractions. To achieve the goal as the left picture shows, it is required to build a 3-dimensional frame to skip the covering of objects and overlapped parts of hands. Seems the authors do not narrowly focus on color image segmentation which has been commonly adopted by computer vision researchers, but they integrate an estimation of features as well as 2.5-dimensional maps. That means probably extra dimension may required. In previous posts, we have successfully located contours of hands yet the shape recognition is still under investigation. And this new method may inspire us something further in hands tracking and recognition.

A great bar called ‘The Moorlands’, we had buffet lunch there and the atmosphere in the bar is really good, classical Yorkshire country. Scratching a car while we were leaving, sorry we just ran away.

The sky there was charming, with huge number of pieces of clouds in clearly blue background, that’s lovely. Looking up the glide planes flew freely just above peaks, jealous.

Leaving the peak zone, we went to Chatsworth. Pictures blow show a piece of iceberg of  deluxe Britain noble life. Paintings, sculptures, and numerous exhibitions fill the rooms, incredibly…

And finally, a picture of my standing on the top of a hill, though it was chilly and windy i was the first arriving.

When i was young i used to watch cars, tractors and boats going and coming everywhere, that is still in my memories. What attracted me so much was the wondering of where were these vehicles going and what were they doing.

Even today i can still image  scenes of my childhood as a little boy watching these vehicles until them gradually disappearing in far distance. And another strange habit which still remains is the preferences of smelling exhaust gas, not the gasoline’s but the diesel oil’s. Today when i saw the boat cruising in the narrow canal and sensed the smells, again i was wondering where it was going…

When i told friends that i had been UK for sixty weeks already i was often asked how could i calculate weeks staying. Even further if my further explanations are lacked, probably i would be misunderstood that the days in UK must be very tough due to the behaviour ‘calculation of weeks’ seems days being either idle or difficult for me.

I have no idea how should i provide them an appropriate explanation of my research life. Superficially it is a little boring and repeating, but indeed, it is challenging, exciting and enjoyable.

Linking DLLs:

• To permanantly include necessary dll files, add “C:\Program Files\OpenCV\bin” to PATH by visiting Advanced tab in System of Windows (the locate of directory might be different).
• One can just copy necessary dll files into project directory with source files.
• It might be required to restart Visual C++ when execution of instance failes after successful build.

Customize Global Options:

• Open the Visual C++ .Net Application. In the menu bar, select Tools->Options
• In the listing, choose Projects->VC++ Directories.
• First, select Library files from the “Show Directories for” List Box.
• Click the Insert New icon, and locate the folder where you have installed opencv.
• Consider that it is installed in “C:/Program Files/OpenCV”.
• In the Library files list, locate and add:

“C:\Program Files\OpenCV\lib”

• Now choose Include files in the list box, and locate and add the following directories:

“C:\Program Files\OpenCV\cv\include”
“C:\Program Files\OpenCV\cxcore\include”
“C:\Program Files\OpenCV\otherlibs\highgui”
“C:\Program Files\OpenCV\cvaux\include”
“C:\Program Files\OpenCV\otherlibs\_graphics\include”

• Next, choose source files in the list box, and locate and add the following directories:

“C:\Program Files\OpenCV\cv\src”
“C:\Program Files\OpenCV\cxcore\src”
“C:\Program Files\OpenCV\cvaux\src”
“C:\Program Files\OpenCV\otherlibs\highgui”
“C:\Program Files\OpenCV\otherlibs\_graphics\src”

• Now click OK in the Options dialog.
• You have successfully configured the global settings.

Create New Project:

• Within Developer Studio create new application:
• Select from menu “File”->”New…”->”Projects” tab.
• Choose “Win32 Application” or “Win32 console application” – the latter is the easier variant and both the sample projects have this type.

• Type the project name and choose location
• Click Ok.. In the Application Wizard, Just click Finish.

• After the above steps done Developer Studio will create the project folder (by default it has the same name as the project), <project name>.vcproj file, Solution <project name>.sln and, Three Source files: <project name>.cpp, stdafx.cpp and stdafx.h. StdAfx files are precompiled header files, which can be very useful if you want to reduce the compilation time.
• For example, consider that we have created a new “Hello” Project. Open the Hello.cpp file, and include the OpenCV-related #include directives:

#include <cv.h>
#include <cxcore.h>
#include <highgui.h>

• Note that these should be included after stdafx.h or you may get build errors.
• Now Type some OpenCV code, and Build the Solution by pressing the F7 Key. There should be linker errors.
• Add dependency projects into workspace
• Choose from menu: “Project” -> “Properties”.
• Choose “Linker” tab -> “Input” category -> “Additional Dependencies:”. Add the paths to all necessary import libraries (cxcore[d].lib cv[d].lib highgui[d].lib cvaux[d].lib cvcam[d].lib)
  • Note: The debug versions are available only when you Build the Visual C++ .NET solution provided with the OpenCV installation.

• If the build process complains about a missing ‘windows.h’ header file, then you’ll need to install the latest version of the Microsoft Windows SDK.
• . That’s it!.. Now Build and Run the application using F5 key and enjoy OpenCV!

.

Post quoted from: openCV Wiki, all copyrights reserved by original authors.

I’m using OpenCV for some tests, and for some reasons (freelance gigs and Symbian SDK) using MS Visual Studio. As new and shiny OpenCV 2.0 is out I decided to switch to it. As it happen, one absolutely have to read buried in the download section readme, before doing anything.
The thing is, OpenCV 2.0 doesn’t include lib files for VS. They have to be built by user.
So here is step by step retelling of readme:
1. Rename your old OpenCV installation to save it, just in case
2. Download and install OpenCV 2.0a
3. Download and install CMake
4. Reboot (or CMake wouldn’t work)
5. Go to C:\Program Files\CMake 2.6\bin and run cmake-gui.exe
6. In the “Where is the source code” field choose your new OpenCV directory (C:\OpenCV)
In “Where to build the binaries” choose directory for VS compiled OpneCV (C:\OpenCV\VS2005)
7. press Configure button and choose VS2005 (or whatever) as building enviroment
8. Press Generate and VS project will be generated in the C:\OpenCV\VS2005
9. Launch solution and build it. For debug build some projects require debug python libraries. As riseriyo pointed in comments if you have Python installed other than 2.6 that can cause problem.
10. Copy *.lib from C:\OpenCV\vs2005\lib\release (or debug) to C:\OpenCV\lib
Copy *.dll from C:\OpenCV\vs2005\bin\release to C:\OpenCV\bin
11. Now, reconfigure your application project. Include directories now “C:\OpenCV\include\opencv” instead of “C:\OpenCV\include
12. All libraries renamed from *.lib to *200.lib (cv.lib to cv200.lib) or *200d.lib for debug. Rename them, or change project settings.

The post was posted originally in ‘Mirror Image‘, all copyrights reserved by original authors.

I was told the origin of the Easter by my supervisor though i have been knowing that for quite long. Anyway it is good to have holidays like that.

Happy Easter!

I have been interested in playing badminton for years, because surprisingly i did not get many chances to play and practice that. But funny enough we start playing badminton in office recently though it is officially not allowed. Fortunately even in such small space in the office i am having the fun of being relax. Thank you guys with whom i play badminton, definitely we need to apply a space for entertainments, and going out of the deadly boring office.

Another thing, i am going to learn swimming this summer. A good teacher has been invited who promises that i will never ever be drawn during learning. That makes me confident.

Here we go…

Days ago i saw a post talking about ‘paging wordpress posts without using plug-ins‘ like pagenav. The way to integrate such codes into themes is easy, but with a little bug. 
First put this paging function in the theme’s functions.php

function kriesi_pagination($query_string){
global $posts_per_page, $paged;
$my_query = new WP_Query($query_string .”&posts_per_page=-1″);
$total_posts = $my_query->post_count;
if(empty($paged))$paged = 1;
$prev = $paged – 1;
$next = $paged + 1;
$range = 2; // only edit this if you want to show more page-links
$showitems = ($range * 2)+1;

$pages = ceil($total_posts/$posts_per_page);
if(1 != $pages){
echo “<div class=’pagination’>”;
echo ($paged > 2 && $paged+$range+1 > $pages && $showitems < $pages)? “<a href=’”.get_pagenum_link(1).”‘>最前</a>”:”";
echo ($paged > 1 && $showitems < $pages)? “<a href=’”.get_pagenum_link($prev).”‘>上一页</a>”:”";

for ($i=1; $i <= $pages; $i++){
if (1 != $pages &&( !($i >= $paged+$range+1 || $i <= $paged-$range-1) || $pages <= $showitems )){
echo ($paged == $i)? “<span class=’current’>”.$i.”</span>”:”<a href=’”.get_pagenum_link($i).”‘ class=’inactive’ >”.$i.”</a>”;
}
}

echo ($paged < $pages && $showitems < $pages) ? “<a href=’”.get_pagenum_link($next).”‘>下一页</a>” :”";
echo ($paged < $pages-1 && $paged+$range-1 < $pages && $showitems < $pages) ? “<a href=’”.get_pagenum_link($pages).”‘>最后</a>”:”";
echo “</div>n”;
}
}

Then put this reference code in the place where page navigation is needed to show.

<?php kriesi_pagination($query_string); ?>

And finally add some styles in the theme’s style.css file.

.pagination{
line-height:23px;
}
.pagination span, .pagination a{
font-size:12px;
margin: 2px 6px 2px 0;
background:#fff;
border:1px solid #e5e5e5;
color:#787878;
padding:2px 5px 2px 5px;
text-decoration:none;
}
.pagination a:hover{
background: #8391A7;
border:1px solid #fff;
color:#fff;
}
.pagination .current{
background: #fff;
border:1px solid #8d8d8d;
color:#393939;
font-size:12px;
padding:2px 5px 2px 5px;
}

Here comes a problem. When that is put in a new page to navigate posts, it does not work. And after carefully checking, the problem gets located in the function calling where the  ’query_string’ is used. If it is going to be used in a page with its ID equals 99, then the ‘query_string’ should be like this:

kriesi_pagination(“cat=99″);

or else, the page navigation would not show up due to the post number is zero.

As far as in my mind, there are three approaches that people have taken to detect hands using openCV, though not all of them have been tested by myself (a little shame). One is to use accurate HAAR-classifiers to locate and detect gestures which is considered as quite stable way but costing very much. Another is to use adaptive skin detection algorithms combined with motion analysis which seems easier to do than previous one. And the last is to use human skin colour segmentation to detect the contours of hands combined with hand convexity detection to recognise hands’ gestures [please refer to: hand gesture recognition using openCV].

HAAR-classifier database is rarely provided with open source projects. But we found some helpful papers which may give some thoughts of classifier training and test.

About hands database:

A colour hand gesture database for evaluating and improving algorithms on hand gesture and posture recognition

Application of classifiers:

Real-time hand tracking using a set of cooperative classifiers based on HAAR-like features

Alternative skin detection based approach gesture recognition through angle space:

Gesture recognition through angle space

As these are mostly asked questions about using openCV to detect and recognise all kinds of subjects such as colours or hands, a review which includes tutorials, progresses and examples (if possible) would be published shortly…

It is not a movie about love and wedding though it begins and ends there, it is about courage, the courage between the dream and the realism, the courage which is also needed even in the dream. Poking arms and waking up from the dream seems not so usable sometimes rather, taking the sword and facing the challenge would be the best choice for Alice.

I do think the movie trying to tell  a truth that, some things are destined, but even destinations need believes, insisting and courage.

There is an exception of myself to predict where love story movies going to, don’t know why, maybe because don’t know how. It is about love and life, realism and dream, girls and boys, but not right or wrong.

“Don’t worry about me, i am not alone here…”

St. Paul’s hall in sunset.