Next-Generation Stanford Shading System

Next-Generation Stanford Shading System

Fourier Slice Photography Ren Ng Stanford University Conventional Photograph Light Field Photography Capture the light field inside the camera body

Hand-Held Light Field Camera Medium format digital camera Camera in-use 16 megapixel sensor Microlens array

Light Field in a Single Exposure Light Field in a Single Exposure Light Field Inside the Camera Body Digital Refocusing

Digital Refocusing Questions About Digital Refocusing What is the computational complexity? Are there efficient algorithms?

What are the limits on refocusing? How far can we move the focal plane? Overview Fourier Slice Photography Theorem

Fourier Refocusing Algorithm Theoretical Limits of Refocusing Previous Work

Integral photography Lippmann 1908, Ives 1930 Lots of variants, especially in 3D TV Okoshi 1976, Javidi & Okano 2002

Closest variant is plenoptic camera Adelson & Wang 1992 Fourier analysis of light fields Chai et al. 2000

Refocusing from light fields Isaksen et al. 2000, Stewart et al. 2003 Fourier Slice Photography Theorem In the Fourier domain, a photograph is a

2D slice in the 4D light field. Photographs focused at different depths correspond to 2D slices at different trajectories. Digital Refocusing by RayTracing x u

Lens Sensor Digital Refocusing by RayTracing x u

Imaginary film Lens Sensor Digital Refocusing by RayTracing x

u Imaginary film Lens Sensor Digital Refocusing by RayTracing x

u Imaginary film Lens Sensor Digital Refocusing by RayTracing x

u Imaginary film Lens Sensor Refocusing as Integral Projection

u x x u Imaginary film Lens

Sensor Refocusing as Integral Projection u x x u

Imaginary film Lens Sensor Refocusing as Integral Projection u x

x u Imaginary film Lens Sensor

Refocusing as Integral Projection u x x u Imaginary film

Lens Sensor Classical Fourier Slice Theorem Integral Projection 1D Fourier

Transform 2D Fourier Transform Slicing Classical Fourier Slice Theorem Integral

Projection 1D Fourier Transform 2D Fourier Transform Slicing

Classical Fourier Slice Theorem Integral Projection 1D Fourier Transform 2D Fourier Transform

Slicing Classical Fourier Slice Theorem Integral Projection Spatial Domain Fourier Domain

Slicing Classical Fourier Slice Theorem Integral Projection Spatial Domain Fourier Domain

Slicing Fourier Slice Photography Theorem Integral Projection Spatial Domain Fourier Domain

Slicing Fourier Slice Photography Theorem Integral Projection 4D Fourier Transform

Slicing Fourier Slice Photography Theorem Integral Projection 2D Fourier Transform

4D Fourier Transform Slicing Fourier Slice Photography Theorem Integral Projection

2D Fourier Transform 4D Fourier Transform Slicing Fourier Slice Photography Theorem

Integral Projection 2D Fourier Transform 4D Fourier Transform Slicing

Photographic Imaging Equations Spatial-Domain Integral Projection Fourier-Domain Slicing Photographic Imaging Equations Spatial-Domain Integral Projection Fourier-Domain Slicing

Photographic Imaging Equations Spatial-Domain Integral Projection Fourier-Domain Slicing Theorem Limitations Film parallel to lens

Everyday camera, not view camera Aperture fully open Closing aperture requires spatial mask Overview

Fourier Slice Photography Theorem Fourier Refocusing Algorithm

Theoretical Limits of Refocusing Existing Refocusing Algorithms Existing refocusing algorithms are expensive O(N4)

All are variants on integral projection where light field has N samples in each dimension

Isaksen Vaish Levoy Ng et et et et

al. al. al. al. 2000 2004 2004 2005

Refocusing in Spatial Domain Integral Projection 2D Fourier Transform 4D Fourier

Transform Slicing Refocusing in Fourier Domain Integral Projection Inverse

2D Fourier Transform 4D Fourier Transform Slicing Refocusing in Fourier Domain

Integral Projection Inverse 2D Fourier Transform 4D Fourier Transform

Slicing Asymptotic Performance Fourier-domain slicing algorithm Pre-process: O(N4 log N)

Refocusing: O(N2 log N) Spatial-domain integration algorithm

Refocusing: O(N4) Resampling Filter Choice Triangle filter (quadrilinear) Kaiser-Bessel filter

(width 2.5) Gold standard (spatial integration) Overview Fourier Slice Photography Theorem

Fourier Refocusing Algorithm Theoretical Limits of Refocusing Problem Statement Assume a light field camera with

An f /A lens N x N pixels under each microlens If we compute refocused photographs from these light fields, over what range

can we move the focal plane? Analytical assumption Assume band-limited light fields Band-Limited Analysis Band-Limited Analysis

Band-width of measured light field Light field shot with camera Band-Limited Analysis Band-Limited Analysis

Band-Limited Analysis Photographic Imaging Equations Spatial-Domain Integral Projection Fourier-Domain Slicing Results of Band-Limited Analysis Assume a light field camera with

An f /A lens N x N pixels under each microlens From its light fields we can

Refocus exactly within depth of field of an f /(A N) lens In our prototype camera Lens is f /4

12 x 12 pixels under each microlens Theoretically refocus within depth of field of an f/48 lens Light Field Photo Gallery Stanford Quad Rodins Burghers of Calais

Palace of Fine Arts, San Francisco Palace of Fine Arts, San Francisco Waiting to Race Start of the Race

Summary of Main Contributions Formal theorem about relationship between light fields and photographs

Computational application gives asymptotically fast refocusing algorithm Theoretical application gives analytic solution for limits of refocusing Future Work

Apply general signal-processing techniques Cross-fertilization with medical imaging Thanks and Acknowledgments

Collaborators on camera tech report Marc Levoy, Mathieu Brdif, Gene Duval, Mark Horowitz and Pat Hanrahan Readers and listeners Ravi Ramamoorthi, Brian Curless, Kayvon Fatahalian, Dwight Nishimura,

Brad Osgood, Mike Cammarano, Vaibhav Vaish, Billy Chen, Gaurav Garg, Jeff Klingner Anonymous SIGGRAPH reviewers Funding sources

NSF, Microsoft Research Fellowship, Stanford Birdseed Grant Questions? Start of the race, Stanford University Avery Pool, July 2005

Recently Viewed Presentations

  • Different types of capital goods in agriculture and the ...

    Different types of capital goods in agriculture and the ...

    la seule différence étant que les PPC sont estimés à l'aide de la valeur marchande . des actifs = > les PPC sont estimés comme recettes sacrifiées de la vente des actifs (élevage, arbre, etc.) sur le marché plutôt que...
  • Rheo-SANS at the NIST Center for Neutron Research

    Rheo-SANS at the NIST Center for Neutron Research

    As the beam traverses from the outer edge of the cup through the gap into the bob, the transmission changes. By looking at the transmission vs. position (see right) we can determine the proper alignment of the tangential slit. The...
  • 9.5 - The Small and Large Intestine

    9.5 - The Small and Large Intestine

    Egestion. The indigestible components of food such as cellulose help us feel full longer, can be important in the maintenance of healthy body weight, and help to retain some water in the large intestine which is important in the elimination...
  • Science Assessment of CAMx and CMAQ: I. Comparison of Model ...

    Science Assessment of CAMx and CMAQ: I. Comparison of Model ...

    Science Assessment of CAMx and CMAQ: I. Comparison of Model Inputs and Algorithm Comparisons ... Batang 돋움 Wingdings Default Design SigmaPlot 8.0 Graph Microsoft Excel Chart Simulation of Houston-Galveston Airshed Ozone Episode with EPA's CMAQ Slide 2 Emissions Inventory Transport...
  • How are we feeling today?  a British cartoon

    How are we feeling today? a British cartoon

    Churchill, Roosevelt and Stalin are shown as doctors (note Stalin checking the pulse). Doctors are professional, clever, assured at caring for patients (so 'Trust me, I'm a doctor'). Denotation Connotation Meaning This is a propaganda cartoon assuring the British people...
  • Southern Connecticut State University Core-CT Time and Labor

    Southern Connecticut State University Core-CT Time and Labor

    Southern Connecticut State UniversityCore-CT . Time and Labor Employee Self Service: Running Attendance Report ... as needed, by clicking the down arrow next to the default value. You can add a UserID to send this report to by clicking on...
  • Spiceland Chapter 11 - University of Texas at El Paso

    Spiceland Chapter 11 - University of Texas at El Paso

    Sum-of-the-Years' Digits (SYD) Sum-of-the-Years' Digits (SYD) On January 1, we purchase equipment for $50,000 cash. The equipment has a service life of 5 years and an estimated residual value of $5,000. Using SYD, compute depreciation expense for the first two...
  • A birds eye view of Sequoy ah Yankee

    A birds eye view of Sequoy ah Yankee

    Omniscient (all knowing) Limited Third Person Point of View: The narrator tells the inner thoughts and feelings of only one character. "Angela felt cold as she waited on the corner for the bus. After about ten minutes, the bus came...