20110220 computer vision_eruhimov_lecture01
Download as PPTX, PDF2 likes1,688 views
1) Computer vision techniques such as stereo vision and object detection can be used for applications in robotics. 2) Stereo vision involves calculating depth from two images using techniques like block matching and disparity maps. It can be implemented efficiently on GPUs. 3) Object detection methods include feature detection using SIFT or SURF descriptors followed by matching and geometry validation using techniques like RANSAC. These allow objects to be detected and their pose estimated.
20110220 computer vision_eruhimov_lecture01
- 1. Computer vision for roboticsVictor EruhimovCTO, itseezhttp://www.itseez.com
- 2. Why do we need computer vision?Smart video surveillanceBiometricsAutomatic Driver Assistance SystemsMachine vision (Visual inspection)Image retrieval (e.g. Google Goggles)Movie productionRobotics
- 3. Vision is hard! Even for humans…
- 5. AgendaCamera modelStereo visionStereo vision on GPUObject detection methodsSliding windowLocal descriptors ApplicationsTextured object detectionOutlet detectionVisual odometry
- 9. Homography
- 10. Perspective-n-Points problemP4PRANSAC (RANdomSAmple Consensus)
- 11. Stereo: epipolar geometryFundamental matrix constraint
- 12. Stereo RectificationAlgorithm steps are shown at right:Goal:Each row of the image contains the same world points“Epipolar constraint”Result: Epipolar alignment of features:12All: Gary Bradski and Adrian Kaehler: Learning OpenCV
- 13. Stereo correspondenceBlock matchingDynamic programmingInter-scanline dependenciesSegmentationBelief propagation
- 14. Stereo correspondence block matchingFor each block in left image:Search for the corresponding block in the right image such that SSD or SAD between pixel intensities is minimum
- 15. Pre- and post processingLow texture filteringSSD/SAD minimum ambiguity removalUsing gradients instead of intensitiesSpeckle filtering
- 16. Stereo Matching
- 17. Parallel implementation of block matchingThe outer cycle iterates through disparity valuesWe compute SSD and compare it with the current minimum for each pixel in a tileDifferent tiles reuse the results of each other17
- 19. Optimization conceptsNot using texture – saving registers1 thread per 8 pixels processing – using cacheReducing the amount of arithmetic operationsNon-parallelizable functions (speckle filtering) are done on CPU19
- 20. Performance summaryCPU (i5 750 2.66GHz), GPU (Fermi card 448 cores)Block matching on CPU+2xGPU is 10 times faster than CPU implementation with SSE optimization, enabling real-time processing of HD images!
- 21. Full-HD stereo in realtimehttp://www.youtube.com/watch?v=ThE7sRAtaWU
- 22. Applications of stereo visionMachine visionAutomatic Driver AssistanceMovie productionRoboticsObject recognitionVisual odometry / SLAM
- 23. Object detection
- 25. Cascade classifierimagefacefacefaceStage 1Stage 2Stage 3Not faceNot faceNot faceReal-time in year 2000!
- 26. Face detection
- 27. Object detection with local descriptorsDetect keypointsCalculate local descriptors for each pointMatch descriptors for different imagesValidate matches with a geometry model
- 30. SIFT descriptorDavid Lowe, 2004
- 31. SURF descriptor4x4 square regions inside a square window 20*s4 values per square region
- 32. More descriptorsOne way descriptorC-descriptor, FERNS, BRIEFHoGDaisy
- 34. Ways to improve matchingIncrease the inliers to outliers ratioDistance thresholdDistance ratio threshold (second to first NN distance)Backward-forward matchingWindowed matchingIncrease the amount of inliersOne to many matching
- 35. Random Sample ConsensusDo n iterations until #inliers > inlierThresholdDraw k matches randomlyFind the transformationCalculate inliers countRemember the best solutionThe number of iterations required ~
- 37. Scaling upFLANN (Fast Library for Approximate Nearest Neighbors)In OpenCV thanks to Marius MujaBag of WordsIn OpenCV thanks to Ken ChatfieldVocabulary treesIs going to be in OpenCV thanks to Patrick Mihelich
- 38. ProjectsTextured object detectionPR2 robot automatic pluginVisual odometry / SLAM
- 40. Object detection exampleIryna Gordon and David G. Lowe, "What and where: 3D object recognition with accurate pose," in Toward Category-Level Object Recognition, eds. J. Ponce, M. Hebert, C. Schmid, and A. Zisserman, (Springer-Verlag, 2006), pp. 67-82. Manuel Martinez Torres, Alvaro ColletRomea, and Siddhartha Srinivasa, MOPED: A Scalable and Low Latency Object Recognition and Pose Estimation System, Proceedings of ICRA 2010, May, 2010.
- 41. Keypoint detectionWe are looking for small dark regionsThis operation takes only ~10ms on 640x480 imageThe rest of the algorithm works only with keypoint regionsItseez Ltd. http://itseez.com
- 42. Classification with one way descriptorIntroduced by Hinterstoisser et al (Technical U of Munich, EcolePolytechnique) at CVPR 2009A test patch is compared to samples of affine-transformed training patches with Euclidean distanceThe closest patch together with a pose guess are reconstructedItseez Ltd. http://itseez.com
- 43. Keypoint classification examplesOne way descriptor does the most of the outlet detection job for us. Few holes are misclassifiedGround holePower holeNon-hole keypoint from outlet imageBackground keypointItseez Ltd. http://itseez.com
- 44. Object detectionObject pose is reconstructed by geometry validation (using geomertic hashing)Itseez Ltd. http://itseez.com
- 45. Outlet detection: challenging casesShadows
- 47. Partial occlusionsItseez Ltd. http://itseez.com
- 48. PR2 plugin (outlet and plug detection)http://www.youtube.com/watch?v=GWcepdggXsU
- 49. Visual odometry
- 51. More fun