Smart Camera as Embedded System
- 1. Smart Cameras as Embedded Systems B. Swarnalatha M.Tech VLSI 12651D5710 Adams Engineering College Paloncha. ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710
- 2. Overview of Seminar Image Video Digital Camera & Smart Camera Detection and recognition algorithms Low level processing High level processing Towards and Embedded System Requirements Components Core Processor TriMedia Advancements in VLSI required to implement Embedded Smart camera ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710
- 3. IMAGE An image may be defined as a two-dimensional function f(x, y) where x and y are spatial (plane) coordinates, and the amplitude of f at any pair of coordinates (x, y) is called the intensity or gray level of the image at that point. A small region in the image is shown in matrix. 10 12 35 54 34 23 201 2 10 12 4 5 6 7 8 9 9 9 0 87 6 8 0 7 68 8 9 09 6 5 87 88 7 9 9 8 8 8 8 8 8 0 8 8 89 9 90 0 0 5 00 8 54 4 55 6 76 7 4 6 99 3 65 7 7 89 7 6 7 6 6 6 78 9 166 6 4 44 4 5 55 43 2 45 6 54 67 45 7 ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710 3 8 77 6 54 87 5 7 98 54
- 4. ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710
- 5. Digital Camera and Smart Camera Digital Cameras capture only images as digital files that users can upload to their Computer , manipulate with software and distribute electronically. Eg. Nikon Smart Cameras capture high-level descriptions of the scene and analyze What they see. These devices could support a wide variety of applications including human and animal detection, surveillance, motion analysis, and facial identification Eg. Windows for KINECT Digital Camera ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710 SMART Digital Camera
- 6. Detection and Recognition Algorithms Low Level Processing •Region Extraction •Contour Following •Ellipse fitting •Graph Matching High Level Processing • Hidden Markov Models •Classifiers ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710
- 7. DETECTION AND RECOGNITION ALGORITHMS Low level processing Region Extraction The subject to be identified for gesture recognition in the video frames are extracted. Fig2. (b) • Contour following Grouping of pixels into contours that geometrically define the regions. Fig2. (c) • Ellipse fitting To correct for deformations in image processing caused by clothing, objects in the frame, or some body parts blocking others, an algorithm fits ellipses to the pixel regions to provide simplified part attributes. Fig2. (d) • Graph matching Meaningful feature vectors are extracted from the modeled body parts with ellipses. Figure2 ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710
- 8. High level Processing The high level processing component , which can be adopted to different Applications, compares the motion pattern of each body part. Example, a pointing gesture could be recognized as a command to “go to the next slide” in a smart meeting room or “open the window” in a smart car , Where as a smart security camera might interpret the gesture as suspicious or threatening. ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710
- 9. Human Detection and Activity Recognition Algorithm ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710
- 10. Towards an Embedded Smart Camera Requirements Frame rate: The embedded smart camera system must process a certain amount of frames per second to properly analyze motion and provide useful results Latency: The amount of time takes to produce a result from the processed frames Components • 100 MHz Philips TriMedia TM-1000 as video processor. • Hi8 Cameras • Shared memory interface • Host computer • Debugging algorithms and programs • Networked system for connecting multiple cameras Smart Meeting Room ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710
- 11. TriMedia Processor TM-1000 The Philips TriMedia TM1000 family of devices have a higher performance Very Long Instruction word (VLIW ) core. ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710
- 12. TriMedia Processor TM-1000 Features • a 5-issue VLIW architecture with a 32-bit word size; • 27 functional units, offering a choice of operation types in each slot in the instruction; • any operation can be guarded to provide conditional execution without branching; • instruction set and functional units optimized with respect to media processing; • a single multi-ported register file with bypass network, allowing 1-cycle latency operations; • 32 kB, 8-way instruction cache; • 16kB, 8-way, quasi-dual ported, data cache; • a variable-length (compressed) instruction set design ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710
- 13. Advancements in VLSI required to implement a smart camera embedded system Algorithmic challenges Choosing best computationally efficient algorithm with minimum memory usage. Library Functions The library function of the processor should provide special function to provide Instruction level-parallelism Eg1. INONZERO: This instruction takes two input operands. If the first is non zero, the destination is set to the value of the second operand; otherwise it is set to zero. Eg2. IABS: This instruction can provide absolute values ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710
- 14. Advancements in VLSI required to implement a smart camera embedded system Control-to-data transformation The data transformation can be controlled using a processor with more functional units. Designing SINGLE-INSTRUCTION MULTIPLE-DATA (SIMD) Processors to achieve real time performance. Existing processors like (80xx, INTEL ) are pixel plane processors. These processor can perform arithmetic and logical operations with limited no of operands. Embedding single-instruction multiple-data (SIMD) processors into sensors is critical to improve the real time performance. Processor architectures are to be designed to take multiple data in one instruction to increase the computation power required to meet the real time challenges. ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710
- 15. Reference: Wayne Wolf, Burak Ozer, Tiehan Lv “Smart Camera as Embedded Systems”. IEEE Trans. Vol. 45, 2002. Copy right IEEE ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710
- 16. THANK YOU ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710