Just value your mind

A method of detecting and recognising hand gestures using OpenCV (2)

hand detection opencv

A previous article has described a method to detect and recognise hand gestures using OpenCV, actually the c# wrapper of OpenCV called emgu CV. Details of this method can be found here a method of detecting and recognising hand gestures using Opencv. But Tongo’s project wrapped in emgu CV may not be easy to configure, so this article gives a new example transfering thet project to plain c++ OpenCV environment.

The example codes of the project, as well as the configurations, are described bellow.

Structure of previous hand gesture detection and recognition project

To make a clear tutorial of how the previous project did the hand gesture detection and recognition, the functions used in the project are listed below, through which I hope it can make a clear picture of this project’s algorithm flow.

1) reading video files. The project did not read video images from live webcam, rather it provided a video file as the source of video images. Similar functions were used to query video image one by one by using the function – grabber.QueryFrame().

grabber = new Emgu.CV.Capture(@”.\..\..\..\M2U00253.MPG”);

2) extracting contours and hulls of hand gestures. In this part, the method used HSV colour ranges to extract hand colours – the range can be adjusted dynamically according to background environments. A set of functions were applied to improve the accuracy of colour detection, such as cvErode and cvDilate. Here is a clip of source code in the project.

public override ImageDetectSkin(ImageImg, IColor min, IColor max)
{
Image currentYCrCbFrame = Img.Convert();
Image skin = new Image(Img.Width, Img.Height);int y, cr, cb, l, x1, y1, value;int rows = Img.Rows;
int cols = Img.Cols;
Byte[, ,] YCrCbData = currentYCrCbFrame.Data;
Byte[, ,] skinData = skin.Data;for (int i = 0; i < rows; i++)
for (int j = 0; j < cols; j++)
{
y = YCrCbData[i, j, 0];
cr = YCrCbData[i, j, 1];
cb = YCrCbData[i, j, 2];

cb -= 109;
cr -= 152;
x1 = (819 * cr – 614 * cb) / 32 + 51;
y1 = (819 * cr + 614 * cb) / 32 + 77;
x1 = x1 * 41 / 1024;
y1 = y1 * 73 / 1024;
value = x1 * x1 + y1 * y1;
if (y < 100)
skinData[i, j, 0] = (value < 700) ? (byte)255 : (byte)0;
else
skinData[i, j, 0] = (value < 850) ? (byte)255 : (byte)0;

}
StructuringElementEx rect_6 = new StructuringElementEx(6, 6, 3, 3, Emgu.CV.CvEnum.CV_ELEMENT_SHAPE.CV_SHAPE_RECT);
CvInvoke.cvErode(skin, skin, rect_6, 1);
CvInvoke.cvDilate(skin, skin, rect_6, 2);
return skin;

}

3) selecting the right contours. The skin detection generated many contours, including face shapes with similar colours as hands. So, the right contours need to be pick up from all these contours. In this project, the author used a simple method by comparing the size of contours – usually the hand in the image had the biggest contour size.

Double Result1 = 0;
Double Result2 = 0;
while (contours != null)
{
Result1 = contours.Area;
if (Result1 > Result2)
{
Result2 = Result1;
biggestContour = contours;
}
contours = contours.HNext;
}

4) if the biggest contour existed, then counting the finger number from detected contours. During this progress some basic markings such as drawing a rectangle or eclipse are helpful to identify the right object. Counting the finger was realised via a function of defect detection. There were obvious defects between finger contours with up and down shapes.

#region defects drawing
for (int i = 0; i < defects.Total; i++)
{
PointF startPoint = new PointF((float)defectArray[i].StartPoint.X,
(float)defectArray[i].StartPoint.Y);PointF depthPoint = new PointF((float)defectArray[i].DepthPoint.X,
(float)defectArray[i].DepthPoint.Y);PointF endPoint = new PointF((float)defectArray[i].EndPoint.X,
(float)defectArray[i].EndPoint.Y);LineSegment2D startDepthLine = new LineSegment2D(defectArray[i].StartPoint, defectArray[i].DepthPoint);

LineSegment2D depthEndLine = new LineSegment2D(defectArray[i].DepthPoint, defectArray[i].EndPoint);

CircleF startCircle = new CircleF(startPoint, 5f);

CircleF depthCircle = new CircleF(depthPoint, 5f);

CircleF endCircle = new CircleF(endPoint, 5f);

//Custom heuristic based on some experiment, double check it before use
if ((startCircle.Center.Y < box.center.Y || depthCircle.Center.Y < box.center.Y) && (startCircle.Center.Y < depthCircle.Center.Y) && (Math.Sqrt(Math.Pow(startCircle.Center.X – depthCircle.Center.X, 2) + Math.Pow(startCircle.Center.Y – depthCircle.Center.Y, 2)) > box.size.Height / 6.5))
{
fingerNum++;
currentFrame.Draw(startDepthLine, new Bgr(Color.Green), 2);
//currentFrame.Draw(depthEndLine, new Bgr(Color.Magenta), 2);
}

currentFrame.Draw(startCircle, new Bgr(Color.Red), 2);
currentFrame.Draw(depthCircle, new Bgr(Color.Yellow), 5);
//currentFrame.Draw(endCircle, new Bgr(Color.DarkBlue), 4);
}
#endregion

Transferring previous project to c++

Based on the understanding of hand gesture detection and recognition algorithm flow described above, it is possible to map the flow in c++ projects. The c++ project used similar OpenCV functions to realise the detection and recognition, and it followed the same flow of detection. Below is a screenshot of the c++ project detecting and recognition simple finger gestures as numbers. The source code of this project is also included in the bottom of this post and the download page as well.

Code example

See the example codes in download page.






QR code for this post, SCAN ME
QR:  A method of detecting and recognising hand gestures using OpenCV (2)

Posts may be of your interests

27 Comments On “ A method of detecting and recognising hand gestures using OpenCV (2)”

  1. gns says:

    what is name of the algortihm which you used for skin recognition. please could you prefer an article about this?

    • Andol says:

      @gns
      the algorithms are as the follows:
      1. change rgb colour image to hsv colour image
      2. split hsv colours into different channels
      3. according to skin colour’s range, extract colour results from channels

      cheers

  2. gns says:

    ok thanks..got it… but, because of name of class i think it’s YCBCr color space..where is the YCbCr color space?

  3. gns says:

    couldu you prefer to me an article? i need it so much..YCbCr algorithm is that:
    Y = 0.299R + 0.587G + 0.114B (5)
    Cb= ( -0.169R 0.331G 0.5B)
    Cr = (0.5R -0.419G -0.081B)
    HSV is very different from code..why?

  4. Andol says:

    @gns
    using HSV may have different results from using YCrCb colour space. so far YCrCb has the best skin detection results among all colour spaces I have tested.
    So, if you are interested in YCrCb colour space, then the code given above should be providing some clues.
    hope this helps.

    cheers

  5. Superpiffer says:

    I’m sorry but… Where’s the code? The download link points to a blank page.

  6. Pengin says:

    http://www.andol.info/downloads/
    the above link is not avaiable, and I m doing the related research, would u please provide the C++ version of the ? or just email to me. thanks!

  7. shankar says:

    Hi i’m very impress with your work.
    where can i find the documentation for this project?
    if u did can u upload it?

  8. Ahmad Azri Mohd says:

    @andol
    i’m trying to execute this program of yours as a reference for my final year project. but i did not work.

    does the spec of PC/laptop must be similar to yours in order for the program to work because my friend told me that it was actually necessary; the processor, windows version, etc.

    can you give the the instruction on how to input all your coding (from the http://db.tt/MukGcwB ).

    or i need to make a new program coding similar to yours instead of executing your program? because there is a problem when trying to configure.

    • Andol says:

      generally the configurations need to be:

      1) 32bit windows xp/7
      2) openCV version may need to be the same – different versions may cause imcompatibilities
      3) input video size should be the same – this can be changed in the code where initial video image is captured

      pretty much of these.

      cheers

  9. Yogesh K says:

    how to read video images from live webcam???how to move opengl model using hand gestures using opencv???

  10. Yogesh K says:

    How to handle ” type initializer for ‘Emgu.CV.CvInvoke’ threw an exception.”

  11. MinSoo K says:

    I really want to see how to openCV C++ version code is working.

    but unfortunately, I can’t find the source in the above link(http://www.andol.info/download).

    would you plz let me know how to get the c++ version source?

  12. yogesh k says:

    i want to perform transformation on opengl model using hand gestures.so plz tell me how to integrate opengl and emgu cv code..

  13. shankar says:

    Can u tell me the name of the algorithms u used for the recognition?
    Do u have the design and implementation report?
    unfortunately, I can’t find the source in the above link(http://www.andol.info/download).

  14. Harsh Oswal says:

    I am working on c# and on executing it gives error on this line
    “grabber = new Emgu.CV.Capture(@”.\..\..\..\M2U00253.MPG”);”
    how do i solve this i have tried to replace it by “grabber = new Emgu.CV.Capture(0);” and “grabber = new Emgu.CV.Capture(1);” but none of this work

  15. Andrea says:

    Hello Andol,

    first of all, my compliments for the project. I used the c# project, and the hand recognition is really good. I was searching for a c++ version, and i was really happy to find here a version of the previous one in c#. I’m working on a personal program of hand detection in c++, and i’d like to try this version of the software since the c# worked very well.
    But i have a problem.. I can’t find the source code in the download page..
    How can i find the c++ version of the project?
    Thanks in advance,

    Regards

  16. Thanks for this wonderful content. I really enjoyed it a lot.

    What you have done is pretty inspirational.
    |
    This is really great content. I would like to use your style
    to improve my own content on my website.

    my web site … learn how to salsa (http://safeshare.tv)

  17. Avec notre site web, vous pouvez télécharger Minecraft 1.7.9 sans payer.

    C’est la dernière version que l’on vous donne.

    Vous pourrez télécharger à haute vitesse
    via notre lien!

    my homepage: dayz free download

  18. ӏ have been browsing on-line greater than tɦreе hoսrs
    today, but I never found any interesting artiϲle like
    yours. It is lovely price sufficient for me. In my view, if all webmasters and bloggerѕ made
    excellent content ɑs you ԁid, the internet will be a lot more helρful than ever
    before.

Leave a comment

You can use these XHTML tags:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Copyrights 2006-2014 © Andol Li | Theme Tree2, re-designed by Andol Li| Powered by WordPress
WWW.ANDOL.INFO- 90 queries in 0.694 seconds. Go to TOP