Scribd
Upload
30 views

ECE4490 FinalPresentation

This document discusses correcting distortion caused by the rolling shutter effect in CMOS cameras. It presents an architecture that finds global motion between frames using Lucas-Kanade optical flow, transforms images to correct for distortion, and implements this using OpenMP for parallelization across multiple threads. Evaluation shows the system achieves real-time framerates for 720p video with increasing performance from coarser to finer granularity of parallelization. Limitations include the need for more sophisticated motion estimation and the problem containing serial regions limiting parallel speedup.

Uploaded by

Mike Branson
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
30 views

ECE4490 FinalPresentation

This document discusses correcting distortion caused by the rolling shutter effect in CMOS cameras. It presents an architecture that finds global motion between frames using Lucas-Kanade optical flow, transforms images to correct for distortion, and implements this using OpenMP for parallelization across multiple threads. Evaluation shows the system achieves real-time framerates for 720p video with increasing performance from coarser to finer granularity of parallelization. Limitations include the need for more sophisticated motion estimation and the problem containing serial regions limiting parallel speedup.

Uploaded by

Mike Branson
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 13

CMOS Rolling Shutter

Distortion Correction
Dale Taylor (dat38) Matt Hagen (mrh53)

Monday, May 3, 2010


Real World Problem

+ =

CMOS Camera Distorted


Camera Motion Video

Monday, May 3, 2010


CMOS Sensor
1
2
3
...

Reads data sequentially, line by line

Monday, May 3, 2010


Distortion
Parallelogram

Vertical Stretch

Trapezoid

Monday, May 3, 2010


Solution: Architecture

Frame 0 Find Global Transform


Motion (LK) Image

Fixed Frame
Frame 1

OpenMP -> Easily Changeable Granularity

Monday, May 3, 2010


Lucas Kanade Optical Flow

• Optical Flow - pattern of apparent motion of objects


in a visual scene caused by the relative motion
between an observer and the scene

• dI/dx*Vx + dI/dy*Vy = -dI/dt

• LK assumes constant flow within a block of an image

• Pixels within a region form a least squares system (2


variables, n^2 equations)

Monday, May 3, 2010


Global Motion
• Select blocks from a set of
frames

• Downsample region so
motion is only a few pixels
and LK holds

• Compute luminance partial


derivates (x,y,t)

• Solve least squares system


(A’Av = bA’) to find motion
vector (-2.5,0.1)

Monday, May 3, 2010


Affine Transform
• Use Vx to find skew

• Skew Factor = Vx * 1/fps Modeled Skew

• [ 1, skew ; 0, 1 ]

• New pos = transform * old pos


Original Frame
• Crop the image so the frame is
filled

Corrected Frame
Monday, May 3, 2010
Granularity
• Coarse - parallel over frames

• Medium

• LK - parallel over flow regions

• Transform - parallel over rows

• Fine

• LK - parallel ops in downsample, least


squares, and derivative computation

Monday, May 3, 2010


Results
240
Coarse Granularity
Med Granularity
Fine Granularity
180
FPS (720p video)

120

60

0
1 2 4 8 16

Num Threads (8 x 2.8Ghz)


Monday, May 3, 2010
Difficulties / Limitations
• Block size is important, more sophisticated
methods needed for wide variety of input

• Smaller operations only parallelize to a factor of 4

• Problem has many serial regions, different than


trying to parallelize a standalone algorithm

• AVI / Jpeg decompression took some trial and


error

• Only works for certain types of motion

Monday, May 3, 2010


Questions

Monday, May 3, 2010


References

• J.-B. Chun et al., Supressing Rolling Shitter Distortion of CMOS Image Sensors by
Motion Vector Detection

• C.-K. Liang, Analysis and Compensation of Rolling Shutter Effect, IEEE Transactions
on Image Processing,Vol 17, No 8, August 2008

• "Optical Flow." Wikipedia, The Free Encyclopedia. Wikimedia Foundation, Inc. 15


April 2010.

• "Lucas-Kanade Optical Flow Method." Wikipedia, The Free Encyclopedia.


Wikimedia Foundation, Inc. 3 May 2010.

Monday, May 3, 2010

You might also like