Raskar Npar04final
Download as PPT, PDF2 likes273 views
The document proposes a gradient-based image fusion technique to enhance nighttime videos with context from daytime imagery. It uses gradient fields from multiple images to reconstruct an output that maintains subtle details while mixing dissimilar inputs. This allows enhancing night videos with daytime backgrounds or creating surreal videos. The technique solves a Poisson equation to find an image matching the mixed gradients. It handles issues like boundary conditions and color shifts. Examples demonstrate context enhancement of traffic camera footage and surreal time-lapse effects.
Raskar Npar04final
- 1. Image Fusion for Context Enhancement and Video Surrealism Adrian Ilie UNC Chapel Hill Ramesh Raskar Mitsubishi Electric Research Labs, (MERL) Jingyi Yu MIT
- 3. Dark Bldgs Reflections on bldgs Unknown shapes
- 4. ‘ Well-lit’ Bldgs Reflections in bldgs windows Tree, Street shapes
- 5. Background is captured from day-time scene using the same fixed camera Night Image Day Image Context Enhanced Image
- 6. Mask is automatically computed from scene contrast
- 7. But, Simple Pixel Blending Creates Ugly Artifacts
- 9. Pixel Blending Our Method : Integration of blended Gradients
- 10. Outline Context Enhancement Gradient-based Fusion Video Enhancement Surrealism
- 11. Nighttime image Daytime image Gradient field Importance image W Final result Gradient field Mixed gradient field G 1 G 1 G 2 G 2 x Y x Y I 1 I 2 G G x Y
- 12. Reconstruction from Gradient Field Problem: minimize error I’ – G| Estimate I’ so that G = I’ Poisson equation I’ = div G Full multigrid solver I’ G X G Y
- 13. Why Gradient-based Approach Comparison of intensity values are important Maintain gradients to capture local variations Directly solve for desired gradients Maintain subtle details Mix dissimilar images No need for precise segmentation
- 14. Comparison Average Subtle details are lost Pixel-wise blending Sharp transitions
- 15. Issues Boundary conditions Color shifts
- 16. Boundary Conditions Assumed Neumann condition at borders, I’ · N = 0 , Enforced by haloing image with blacks
- 17. Color Shift Mixing dissimilar images Goal: final image appearance matches input images at corresponding pixels I final (x,y) = c 1 I poisson (x,y) + c 2 Solve W i (x,y) I original (x,y) = c 1 I poisson (x,y) + c 2 Each color channel reconstructed separately
- 19. Outline Context Enhancement Gradient-based Fusion Video Enhancement Surrealism
- 22. Overview of Process Enhanced video Note: exit ramp, lane dividers, buildings not visible in original night video, but clearly seen here. Day time image: By averaging 5 seconds of day video Original night time traffic camera 320x240 video Mask frame (for frame above): Encodes pixel with intensity change Input Output
- 23. Algorithm Frame N Daytime image TimeAveraged importance mask Gradient field Final result Processed binary mask Gradient field Frame N-1 Mixed gradient field
- 24. Outline Context Enhancement Gradient-based Fusion Video Enhancement Related Work Surrealism
- 25. Related Work Spatio-temporal Composition Duchamp ( Nude descending a staircase ) Freeman 2002 Fels 1999, Klein 2002, Cohen 2003 Gradient-based Techniques Multi-spectral: Socolinsky 1999 Shadow removal: Weiss 2001 High dynamic range: Fattal 2002 Image editing: Perez 2003 Some at Siggraph’04
- 26. Rene Magritte, ‘Empire of the Light’ Surrealism
- 27. Outline Context Enhancement Gradient-based Fusion Video Enhancement Surrealism
- 28. Time-lapse Mosaics Maggrite Stripes time
- 31. t
- 32. Sunrise at Night
- 33. BiSolar System
- 34. Discussion User Experience More effective in conveying scene context ‘ Dreamy’ appearance Nonrealistic : False conditions Applications Tools for artists Surveillance Amusement park rides Performance ~1 sec/frame for 320x240 ~ 3 min for 4Mpixel image
- 35. Image Fusion for Context Enhancement Nonrealistic but comprehensible context Fusion using multiple images Enhancing night images with day bgrnd Gradient-based fusion Video surrealism tools t
Editor's Notes
- #2: <Title> Hello. My name is Mike Brown, and I am here to present a paper by Adrian Ilie, Ramesh Raskar and Jingyi Yu on Gradient-Domain Context Enhancement for Fixed Cameras. I will try to answer your questions, but I suggest that you address the more specific ones to the authors themselves, by email.