香港中文大学：《CMSC5719 Seminar》课程教学资源（讲义）Lecture 06 3D computer vision techniques

3D computer vision techniques KH Wong 3D computer vision techniques v.4b2

3D computer vision techniques v.4b2 1 3D computer vision techniques KH Wong

Seminar Title:3D computer vision techniques. Abstract In this talk,the ideas of obtaining 3D information of objects (or called 3D reconstruction)using different techniques are discussed.Currently,the most popular one is the image based method that uses 2D cameras for 3D reconstruction;in particular reconstruction based on one-image,two-image and multiple- image are discussed.Moreover,batch and sequential treatments of input data are studied.I will also talk about novel techniques, such as using multiple cameras and laser based methods to obtain 3D information.And I will discuss how 3D computer vision is used in film and game production.Finally naked-eye 3D display technologies will be mentioned. 3D computer vision techniques v.4b2 2

3D computer vision techniques v.4b2 2 Seminar Title: 3D computer vision techniques. ⚫ Abstract In this talk, the ideas of obtaining 3D information of objects (or called 3D reconstruction) using different techniques are discussed. Currently, the most popular one is the image based method that uses 2D cameras for 3D reconstruction; in particular reconstruction based on one-image, two-image and multiple￾image are discussed. Moreover, batch and sequential treatments of input data are studied. I will also talk about novel techniques, such as using multiple cameras and laser based methods to obtain 3D information. And I will discuss how 3D computer vision is used in film and game production. Finally naked-eye 3D display technologies will be mentioned

Overview(part1) ●Introduction ● From 2D to 3D Camera systems/calibration Feature extraction/correspondence Reconstruction algorithms .2 views,3 views,N views Real-time algorithms/Kalman filter ●Previous projects Virtual viewer/Projector camera systems 。Keystone correction ●Novel setups Multiple cameras/Camera array Obtain 3D directly ● Structured light 。Laser approach Kinect approach Photometric stereo 3D computer vision techniques v.4b2 3

3D computer vision techniques v.4b2 3 Overview (part1) ⚫ Introduction ⚫ From 2D to 3D ⚫ Camera systems/calibration ⚫ Feature extraction/correspondence ⚫ Reconstruction algorithms ⚫ 2 views, 3 views , N views ⚫ Real-time algorithms/Kalman filter ⚫ Previous projects ⚫ Virtual viewer/ Projector camera systems ⚫ Keystone correction ⚫ Novel setups ⚫ Multiple cameras/ Camera array ⚫ Obtain 3D directly ⚫ Structured light ⚫ Laser approach ⚫ Kinect approach ⚫ Photometric stereo

Overview(part 2) Applications Photos from tourists(photo tourism) http://phototour.cs.washington.edu/ ●3 D displays Possible future research Classification based on 3D information ●( Content search 3D based on 3D keys Merging with sound information 3D computer vision techniques v.4b2

3D computer vision techniques v.4b2 4 Overview (part 2) ⚫ Applications ⚫ Photos from tourists (photo tourism) http://phototour.cs.washington.edu/ ⚫ 3D displays ⚫ Possible future research ⚫ Classification based on 3D information ⚫ Content search 3D based on 3D keys ⚫ Merging with sound information

Motivation ●Ve live in a3 D world We see 2D images but perceive the world in 3D Intelligent robot should have this 3D reconstruction capability 3D computer vision techniques v.4b2 5

3D computer vision techniques v.4b2 5 Motivation ⚫ We live in a 3D world ⚫ We see 2D images but perceive the world in 3D ⚫ Intelligent robot should have this 3D reconstruction capability

How to obtain 3D information? ●Cameras-2D ·Range sensors-3D 3D computer vision techniques v.4b2 6

3D computer vision techniques v.4b2 6 How to obtain 3D information? ⚫ Cameras-2D ⚫ Range sensors-3D

Challenges Obtain 3D information for tasks in a 3D world. .2D-to-3D reconstruction from a camera 3D directly-laser range sensor,kinect sensor ●Novel sensors Camera array/multiple camera One pixel camera light field camera 3D computer vision techniques v.4b2 7

3D computer vision techniques v.4b2 7 Challenges ⚫ Obtain 3D information for tasks in a 3D world. ⚫ 2D-to-3D reconstruction from a camera ⚫ 3D directly— laser range sensor, kinect sensor ⚫ Novel sensors ⚫ Camera array/ multiple camera ⚫ One pixel camera ⚫ light field camera

2D-to-3D reconstruction (feature based method) Camera (perspective projection) Features-extraction and correspondences Methods One-image method Two-image (Stereo)method Three-image method ●N-image method ·Bundle adjustment Kalman filter 3D computer vision techniques v.4b2 8

3D computer vision techniques v.4b2 8 2D-to-3D reconstruction (feature based method) ⚫ Camera (perspective projection) ⚫ Features-extraction and correspondences ⚫ Methods ⚫ One-image method ⚫ Two-image (Stereo) method ⚫ Three-image method ⚫ N-image method ⚫ Bundle adjustment ⚫ Kalman filter

●●● http://upload.wikimedia.org/wikipedia/en/8/81/Pinhole-camera.png Camera:3D to 2D projection Pinhole ①amera Perspective model u=F*X/Z(nonlinear relation) V=F*Y/Z Virtual World Screen or CCD center sensor Z F F Real Screen Thin lens Or 3D computer vision techniques v.4b2 or a pin hole CCD sensor

3D computer vision techniques v.4b2 9 Camera: 3D to 2D projection Perspective model u=F*X/Z (nonlinear relation) v=F*Y/Z F Z Y v World center F Thin lens or a pin hole Virtual Screen or CCD sensor Real Screen Or CCD sensor Pinhole Camera http://upload.wikimedia.org/wikipedia/en/8/81/Pinhole-camera.png

Perspective Womd●● Projective Coordinates Zw Yw Xw Rc,Tc Model Matt=l V-axIs image XYZ A Xc-axis Camera Coordinates. Zc-axis (u,V) Principal axis←---- 0= (Image center, 0,0,0) 0x0) (Camera “1 center) u-axis F-focal length Yc-axis (0,0)of image plane 3D computer vision techniques v.4b2 10

3D computer vision techniques v.4b2 10 Perspective Projective ⚫ Model M at t=1 c (Image center, ox ,oy ) F=focal length image Oc= (0,0,0) (Camera center) Xc-axis Zc-axis Yc-axis v-axis u-axis X,Y,Z (u,v) (0,0) of image plane Camera Coordinates. World Coordinates Zw Yw Xw Rc,Tc Principal axis