OpenCV quick and dirty tutorial

Introduction to Perception and Robotics - IPR 2011

Ana Huaman

February 13, 2011

Ana Huaman

OpenCV quick and dirty tutorial

What you will learn today...

You will learn what is OpenCV and to use it for your projects. Specically we will see: Create a window to display images and video Get image and video from your webcam Get video stream from a remote webcam via http (MiGIO) Use lters to manipulate your image Blob detection Basic face detection Pretty basic Motion tracking

Ana Huaman

OpenCV quick and dirty tutorial

Before beginning...

ACHTUNG! All material of this tutorial (slides and C++ code) are on the Wiki

Ana Huaman

OpenCV quick and dirty tutorial

What is OpenCV?

Open Source Computer Vision in C/C++ Intended for real-time applications OS independent

If you can, buy this book

Ana Huaman

OpenCV quick and dirty tutorial

OpenCV Features

Image and video I/O Image processing:

Filtering Edge detection Corner detection Color conversion Histograms...

Motion analysis:
Tracking Motion segmentation...

Object recognition Basic Graphical User Interface

Event handlers Scroll-bars...

Many, many more...

Ana Huaman OpenCV quick and dirty tutorial

OpenCV Modules

(aka Those headers you normally put on top of your le) cv Main OpenCV functions cvaux Auxiliary (experimental) functions cxcore Data structures and linear algebra support highgui Graphical User interface functions
Template #include #include #include #include <stdio.h> <cv.h> <cxcore.h> <highgui.h>

int main(int argc, char* argv[]) { /** Your code here */ return 0; }

Ana Huaman

OpenCV quick and dirty tutorial

Getting your hands dirty

OpenCV Basics

Ana Huaman

OpenCV quick and dirty tutorial

Basics 1: Create a window and display an image

IplImage* img = 0; img = cvLoadImage("Lama1.png"); cvNamedWindow("Display Image"); cvWaitKey(0); cvReleaseImage(&img); cvDestroyWindow("Display Image");

Ana Huaman

OpenCV quick and dirty tutorial

Basics 2: Display video stored in your machine

cvNamedWindow("Display Video"); cvCapture* capture = cvCreateFileCapture(2); IplImage* frame; while(1) { frame = cvQueryFrame(capture); cvShowImage("Display Video", frame); char c = cvWaitKey(33); if(c == 27) break; } cvReleaseImage(&img); cvDestroyWindow("Display Video");

Ana Huaman

OpenCV quick and dirty tutorial

Basics 3: Capture video input

cvNamedWindow("Capture Video"); cvCapture* capture=cvCreateCameraCapture(2); IplImage* frame; while(1) { frame = cvQueryFrame(capture); cvShowImage("Capture Video", frame); char c = cvWaitKey(33); if(c == 27) break; } cvReleaseImage(&img); cvDestroyWindow("Capture Video");

Ana Huaman

OpenCV quick and dirty tutorial

Basics 4: Draw random stu in your image

cvLine(src1, pt1,pt2, blue, thickness, connectivity); cvCircle(src1, pt1, radius, red, thickness, connectivity); cvRectangle(src1, pt15, pt16, cyan, -1, connectivity); cvPutText(src1, text, pt1, &font1, purple); cvFillConvexPoly(src1, pts_poly, 12, yellow);

Ana Huaman

OpenCV quick and dirty tutorial

Basics 5: Make your GUI fancy

int main(int argc, char* argv[]) { cvCreateTrackbar("Switch", name, &g_switch_value, 1, switch_callback); /** ... More code ... */ while(1) { if(colorInt == 0) {cvCircle(src1, pt2, radius, green,2,8);} else {cvCircle(src1, pt2, radius, orange,2,8);} cvShowImage(name, src1); /** ... More code ... */ } /** ... More code ... */ } void switch_callback(int position) { if(position == 0) {colorInt = 0;} else {colorInt = 1;} }
Ana Huaman OpenCV quick and dirty tutorial

Basics 6: Mouse Handler

int main(int argc, char* argv[]) { cvSetMouseCallback( name, my_mouse_callback, (void*) image); /** ... More code ... */ } void my_mouse_callback(int event, int x, int y, int flags, void* param) { switch(event){ case CV_EVENT_MOUSEMOVE: /** Do something */ case CV_EVENT_LBUTTONDOWN: /** Do something */ case CV_EVENT_LBUTTONUP /** Do something */ } }

Ana Huaman

OpenCV quick and dirty tutorial

Making our life a little messier...

Filtering

Ana Huaman

OpenCV quick and dirty tutorial

Filtering 1: Gaussian Blur

/** Create an IplImage* out /** Perform a cvSmooth(img, image for the output */ = cvCreateImage(cvGetSize(img), IPL_DEPTH_8U, 3); Gaussian blur */ out, CV_GAUSSIAN, 15,15);

(a) Input

(b) Output

Figure: Gauss Blur

Ana Huaman

OpenCV quick and dirty tutorial

Filtering 2: Canny Detection

/** Perform canny edge detection */ cvCanny(out, out, 10,100,3);

(a) Input Rovio

(b) Output Rovio

(c) Input Pleo

(d) Output Pleo

Figure: Canny Results

Ana Huaman OpenCV quick and dirty tutorial

What you were waiting for...

Useful stu for your Projects

Blob Detection Face Detection

Ana Huaman

OpenCV quick and dirty tutorial

Blob Detection

A Blob is a 8-connected component of pixels, that is,a group of pixels connected for some common characteristic (color) There are libraries around to do this, we are using cvBlobsLib You are welcome to try whatever works for you. However, be aware that I can only help you with the above library. Be also aware: For some reason, your TA was not able to run the latest version of the library (v8.3) with her setup (OpenCV 2.2 + Visual Studio 2010). She is using the v6.1. with a small x (I put it online too, to make your life easier than hers). If you can make work the latest version...share your knowledge with us please!

Ana Huaman

OpenCV quick and dirty tutorial

Why would I want to detect blobs?

Imagine you want to track your Pleo using the overhead camera. A relatively simple way to do that is to track him (her) by color (roughly green) and use that as feedback for your controller... If you are going for a goal (as you will in Project 2), you know that it is not static...it may move away (fruits are alive) or you can move it accidentally. Hence, you need to know! Long story short: Use blobs as your eyes to detect changes in the environment!

Ana Huaman

OpenCV quick and dirty tutorial

Blob Detection: Recipe

There are a few steps you must follow in order to get your Blobs Visualize what you want to detect. Let us say it is a red ball. So, you want to detect a red object Filter your image: Prepare the image to be processed (use cvSmooth and transform it to HSV format with cvCvtColor(img, hsv img, CV RGB2HSV) Segmentate your image: Separate what you want, use a lter Detect the blobs: Use functions in cvBlobsLib:CBlobResult Filter the blobs according to your needs (size, area, perimeter..): Use CBlobs.lter(...) Track them (follow their movement): This is the fun part!

Ana Huaman

OpenCV quick and dirty tutorial

Blob Detection: Example

PS.- In the latest OpenCVDemo on the Wiki you can nd a sample code to detect the red color on an image and in a short video included. This code is also on the TA prism website. You may observe how the red coat of Rovio is detected (it is painted in blue) and the center is marked with a yellow circle. These snapshots are of the DemoCode (option b) running. You may try it!

(a) Input Rovio

(b) Output Rovio

Figure: After detecting the red coat...

Ana Huaman

OpenCV quick and dirty tutorial

Blob Detection: Snippet

This is what generated the gure in the previous slide!
// Smooth input image using a Gaussian filter, assign HSV, BW image cvSmooth(input, img, CV_GAUSSIAN, 7, 9, 0 ,0); cvCvtColor(img, hsv_img, CV_RGB2HSV); /** Makes image hsv_img the HSV of img */ /** Here goes some filtering code...check Blobs1.cpp for complete code */ // Detect Blobs using the mask and a threshold for area blobs = CBlobResult(i1, NULL, 0, true); blobs.Filter(blobs, B_INCLUDE, CBlobGetArea(), B_GREATER, 800); blobs.GetNthBlob(CBlobGetArea(), blobs.GetNumBlobs()-1, blobArea); // example for (int i = 1; i < blobs.GetNumBlobs(); i++ ) { blobArea = blobs.GetBlob(i); CvPoint centrum = cvPoint(BlobEllipse.center.x, BlobEllipse.center.y); /** Drawing with Blue the big blob and with a yellow circle the center */ if(blobArea.Area() > 150) { blobArea.FillBlob(input, cvScalar(255, 0, 0)); cvCircle(input, centrum, 20, CV_RGB(250, 250, 0),2, 1);} }

Ana Huaman

OpenCV quick and dirty tutorial

Additional in Blobs: Why HSV?

In case you wonder. As Mike explained, HSV is a format easier to use to detect colors. Remember? Hue (H): Represents the color as an angle from 0 to 360 (Watch out!) Saturation (S): Indicates the richness of color (range of grey). Also called the degree of purity of the color. Value(V): Indicates the brightness of the color

Ana Huaman

OpenCV quick and dirty tutorial

Okay, new topic: Face Detection

Actually this is not new...you are detecting an object again! To detect something means to nd a group of pixels which share some characteristics in common In the case of a face, we can talk about color, geometric characteristics, edges... ...OpenCV does this for you (at some extent, not perfect)

Ana Huaman

OpenCV quick and dirty tutorial

Face Detection - Snippet

/** Create storage to save face info */ CvMemStorage* storage = cvCreateMemStorage(0); /** Give to OpenCV a file with "face characteristics" to detect */ CvHaarClassifierCascade* cascade = (CvHaarClassifierCascade*) cvLoad("AdditionalStuff/haarcascade_frontalface_alt2.xml"); double scale = 1.3; static CvScalar colors[] = {{{0,255,255}}, {{0,128,255}},{{0,255,255}},{{255,0,255}}}; /** Detect objects (faces) */ cvClearMemStorage(storage); /** DETECTION! Faces no smaller than 20x30..approx. */ CvSeq* objects = cvHaarDetectObjects(img, cascade, storage, 1.1, 4, 0, cvSize(20,30)); CvRect* r; /** Loop through objects and draw boxes around faces */ for(int i =0; i<(objects ? objects->total :0); i++) { r = (CvRect*) cvGetSeqElem(objects, i); cvRectangle( img, cvPoint(r->x, r->y), cvPoint( r->x + r->width, r->y + r->height), colors[i%8],2); }

Ana Huaman

OpenCV quick and dirty tutorial

Face Detection: Snippet in action

Our input:

Ana Huaman

OpenCV quick and dirty tutorial

Face Detection: Snippet in action

Result: What we get (try the Demo!):

Ana Huaman

OpenCV quick and dirty tutorial

Okay: Face Detection in Video

To detect a face (complex object) takes time; you may not notice when you detect it in a snapshot, but if you do it continuously... Check this output of the OpenCV demo, and check the original le (under /video in the code we gave you on the Wiki). You will see how slow this is in comparison Improve it!

Ana Huaman

OpenCV quick and dirty tutorial