Lecture 1: Images and image filtering

Lecture 1: Images and image filtering

CS5670: Computer Vision Noah Snavely Lecture 15: Structure from motion Readings Szeliski, Chapter 7.1 7.4

Structure from motion Given many images, how can we a) figure out where they were all taken from? b) build a 3D model of the scene?

This is (roughly) the structure from motion problem Structure from motion Reconstruction (side)

Input: images with points in correspondence (ui,j,vi,j) Output structure: 3D location xi for each point pi motion: camera parameters Rj , tj possibly Kj Objective function: minimize reprojection error

(top) pi,j = Input

Camera calibration & triangulation Suppose we know 3D points And have matches between these points and an image How can we compute the camera parameters?

Suppose we have know camera parameters, each of which observes a point How can we compute the 3D location of that point? Structure from motion

SfM solves both of these problems at once A kind of chicken-and-egg problem (but solvable) Photo Tourism

First step: how to get correspondence? Feature detection and matching Feature detection Detect features using SIFT [Lowe, IJCV 2004]

Feature detection Detect features using SIFT [Lowe, IJCV 2004] Feature matching Match features between each pair of images

Feature matching Refine matching using RANSAC to estimate fundamental matrix between each pair Correspondence estimation

Link up pairwise matches to form connected components of matches across several images Image 1 Image 2

Image 3 Image 4 Image connectivity graph

(graph layout produced using the Graphviz toolkit: http://www.graphviz.org/) Structure from motion X4 X1

X3 g(R, T, X) X2

X5 minimize X7

non-linear least squares X6 p1,1 p1,3

p1,2 Camera 1 R1,t1

Camera 3 Camera 2 R2,t2 R3,t3

Problem size What are the variables? How many variables per camera? How many variables per point? Trevi Fountain collection

466 input photos + > 100,000 3D points = very large optimization problem Structure from motion Minimize sum of squared reprojection errors:

predicted image location observed image location

indicator variable: is point i visible in image j ? Minimizing this function is called bundle adjustment

Optimized using non-linear least squares, Levenberg-Marquardt e.g. Is SfM always uniquely solvable?

Is SfM always uniquely solvable? No Incremental structure from motion

Incremental structure from motion Incremental structure from motion Incremental structure from motion

Photo Explorer Demo Questions?

SfM Failure cases Necker reversal Structure from Motion Failure cases Repetitive structures

SfM applications

3D modeling Surveying Robot navigation and mapmaking Visual effects (Match moving) https://www.youtube.com/watch?v=RdYWp70P_kY

Applications Photosynth Applications Hyperlapse https://www.youtube.com/watch?v=SOpwHaQnRSY

Recently Viewed Presentations

  • Sport in Society: Issues & Controversies

    Sport in Society: Issues & Controversies

    Sports in Society: Issues & Controversies Race and Ethnicity: Are They Important in Sports? Defining Race & Ethnicity Race refers to a category of people regarded as socially distinct because they share genetic traits believed to be important by people...
  • Plans for Reconstruction Andrew Johnson Easy Plan Keep

    Plans for Reconstruction Andrew Johnson Easy Plan Keep

    Arial Times New Roman Wingdings Gungsuh Kirsty Orbit Slide 1 Slide 2 Slide 3 ` Carpetbagger Carpetbag Scalawags White Conservatives - KKK ...
  • 2018 AHEA Conference (Red Deer) - consultmcgregor.com

    2018 AHEA Conference (Red Deer) - consultmcgregor.com

    2018 AHEA Conference (Red Deer) Followership. Sue L. T. McGregor PhD IPHE. www.consultmcgregor.com. Not everyone is a leader. Leaders need followers. Leaders and followers take part in a common enterprise - they are dependent on each other! Their futures .
  • WE MAKE A DIFFERENCE  Boys & Girls Clubs

    WE MAKE A DIFFERENCE Boys & Girls Clubs

    Times Arial Times New Roman Wingdings Verdana Verdana-Bold Curtain PowerPoint Presentation PowerPoint Presentation WHAT WE DO CONT. WHAT WE DO CONT: THE PROGRAMS The Facts Our Mission What the Club Provides for the Kids Why We're Special Why We're Special...
  • SAQ Marking Meeting - 7th Oct 2011

    SAQ Marking Meeting - 7th Oct 2011

    Arial Calibri Times New Roman Office Theme Slide 1 Slide 2 Slide 3 Reading Slide 5 Useful websites Guidelines - NICE SIGN CEM BTS Paediatrics Other Other Exam Prep Resources Slide 15 Part B - Two hours Slide 17 Part...
  • The Motorcycle Diaries- Che Guevara - Weebly

    The Motorcycle Diaries- Che Guevara - Weebly

    In Valparaiso, Che treats a woman suffering from a heart condition (aggravated by asthma). It is in this encounter that Guevara makes a sudden discovery. He learns of his 'powerlessness' in the situation and a desire to change the injustices...
  • Broadband Service Switzerland - ACN Inc.

    Broadband Service Switzerland - ACN Inc.

    ACN Line Rental Product Training Portugal Contents Opportunity & Benefits Service Offering Customer Sign-Up Process Important Information Provisioning Process Opportunity & Benefits Population: 10.6 million Fixed lines in use: 4.22 million Benefits Increase in customers with a bundled line rental...
  • Dividing A Nation - Weebly

    Dividing A Nation - Weebly

    James K. Polk was a strong supporter of Manifest Destiny. ... There shall be neither slavery, nor involuntary servitude in any territory on the continent of America which shall hereafter be acquired by or annexed to the United States…..except for...