Scribd
Upload
45 views

Human Vision: Jitendra Malik U.C. Berkeley

This document summarizes human vision and visual processing in the brain. It discusses how the eye contains photoreceptors that detect light and transmit signals to the retina. The retina contains ganglion cells with receptive fields that respond to light in certain areas. These signals are then sent to the LGN and visual cortex, where neurons in V1 become selectively tuned to orientation due to their receptive field properties. The document outlines how feedforward processing in the ventral visual stream leads to increasing specificity and complexity in object recognition areas like IT cortex. It concludes by discussing how computer vision algorithms were inspired by properties of visual processing in the brain.

Uploaded by

Elisée Ndjabu DH
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
45 views

Human Vision: Jitendra Malik U.C. Berkeley

This document summarizes human vision and visual processing in the brain. It discusses how the eye contains photoreceptors that detect light and transmit signals to the retina. The retina contains ganglion cells with receptive fields that respond to light in certain areas. These signals are then sent to the LGN and visual cortex, where neurons in V1 become selectively tuned to orientation due to their receptive field properties. The document outlines how feedforward processing in the ventral visual stream leads to increasing specificity and complexity in object recognition areas like IT cortex. It concludes by discussing how computer vision algorithms were inspired by properties of visual processing in the brain.

Uploaded by

Elisée Ndjabu DH
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 45

Human

 vision    

Jitendra  Malik  
U.C.  Berkeley  
Visual  Areas  
Mathema;cal  Abstrac;on  
The  photoreceptor  mosaic:  
rods  and  cones  are  the  eye’s  pixels  
Cones  and  Rods  

After dark adaptation, a single rod can respond to a


single photon
The  three  cone  types  have  different  
spectral  sensi;vity  func;ons  
ON  and  OFF  cells  in  re;nal  ganglia  
Recep;ve  
Fields  

Figure 6.16 Receptive fields


The receptive field of a receptor is simply the area of the visual field from which
light strikes that receptor. For any other cell in the visual system, the receptive
field is determined by which receptors connect to the cell in question.
The  recep;ve  field  of  a  re;nal  ganglion  cell  can  be  
modeled  as  a  “Difference  of  Gaussians”  
Convolving  an  image  with  a  filter  
Each  output  unit  gets  the  weighted  
sum  of  image  pixels  
Each  output  unit  gets  the  weighted  
sum  of  input  units  

We can think of this weighting function


as the receptive field of the output unit
Visual  Processing  Areas  
 
Macaque  Visual  Areas  
Orienta;on  Selec;vity  in  V1  
Recep;ve  fields  of  simple  cells  
(discovered  by  Hubel  &  Wiesel)  
The  1D  Gaussian  and  its  deriva;ves  
Oriented  Gaussian  Deriva;ves  in  2D  
Oriented  Gaussian  First  and  Second  Deriva;ves  
Modeling  simple  cells  
•  Elongated  direc;onal  
Gaussian  deriva;ves  
•  Gabor  filters  could  be  
used  instead  
•  Mul;ple  orienta;ons,  
scales  
Recep;ve  fields  of  complex  cells  
Orienta;on  Energy  

2 2
•  OE = ( I ∗ f odd ) + ( I ∗ f even )
•  Can be used to model complex cells, as this is
insensitive to phase
•  Multiple scales
Hypercolumns  in  visual  cortex  
Macaque  Visual  Areas  
Rolls  et  al  (2000)  model  of  ventral  stream  
Object  Detec;on  can  be  very  fast  
•  On  a  task  of  judging  animal  vs    no  
animal,  humans  can  make  mostly  
correct  saccades  in  150  ms  (Kirchner  &  
Thorpe,  2006)  
 
–  Comparable  to  synap;c  delay  in  the  re;na,  
LGN,  V1,  V2,  V4,  IT  pathway.    
–  Doesn’t  rule  out  feed  back  but  shows  feed  
forward  only  is  very  powerful  
•  Detec;on  and  categoriza;on  are  
prac;cally  simultaneous  (Grill-­‐Spector  
&  Kanwisher,  2005)  
Feed-­‐forward  model  of  the  ventral  stream  
Intrinsic  &  Extrinsic  Connec;vity  of  the  Ventral  Stream  
(Kravitz,  Saleem,  Baker,  Ungerleinder,  Mishkin,  TICS,  2013)  
What  can  we  learn?  
•  Neurons  show  increasing  specificity  higher  in  the  
visual  pathway  
•  V1  simple  and  complex  cells  are  orienta;on-­‐tuned  
•  Convolu;on  with  a  linear  kernel  followed  by  simple  
non-­‐lineari;es  is  a  good  model  for    computa;on  in  
re;na,  LGN  and  V1,  but  beyond  that  we  do  not  
have  sa;sfactory  computa;onal  models  
•  Good  designs  of  visual  systems  are  likely  to  be  
hierarchical  and  “mostly”  feedforward  
Neuroscience  &  Computer  Vision  Features  

•  Hubel  &  Wiesel’s  finding  of  orienta;on  selec;ve  


simple  and  complex  cells  in  V1  inspired  features  such  
as  SIFT  and  HOG.  
•  A  feed-­‐forward  view  of  processing  in  the  ventral  
stream  with  layers  of  simple  and  complex  cells  led  to  
the  neocognitron  and  subsequently  convolu;onal  
networks.  
•  We  now  know  that  the  ventral  stream  is  much  more  
complicated  with  bidirec;onal  as  well  as  feedback  
connec;ons.  So  far  this  has  not  been  exploited  much  
in  computer  vision  
Convolu;onal  Neural  Networks  
(LeCun  et  al)  
Convolu;onal  Neural  Networks  (LeCun  
et  al)  
Training  mul;-­‐layer  networks  

You might also like