3D computer vision techniques v.4b2 1

3D computer vision techniques KH Wong ●●●●● 3D computer vsion techniques v 4

3D computer vision techniques v.4b2 1 3D computer vision techniques KH Wong

●●● Seminar Title: 3D computer ●●●● ●●●●● ●●●● ●●●●● vision techniques ●●●● ● Abstract In this talk, the ideas of obtaining 3D information of objects(or called 3D reconstruction)using different techniques are discussed. Currently, the most popular one is the image based method that uses 2D cameras for 3D reconstruction; in particular econstruction based on one-image two-image and multiple image are discussed. Moreover, batch and sequential treatments of input data are studied. I will also talk about novel techniques such as using multiple cameras and laser based methods to obtain 3D information. And I will discuss how 3D computer vision is used in film and game production. Finally naked-eye 3D display technologies will be mentioned 3D computer vsion techniques v 4b2

3D computer vision techniques v.4b2 2 Seminar Title: 3D computer vision techniques. ⚫ Abstract In this talk, the ideas of obtaining 3D information of objects (or called 3D reconstruction) using different techniques are discussed. Currently, the most popular one is the image based method that uses 2D cameras for 3D reconstruction; in particular reconstruction based on one-image, two-image and multiple￾image are discussed. Moreover, batch and sequential treatments of input data are studied. I will also talk about novel techniques, such as using multiple cameras and laser based methods to obtain 3D information. And I will discuss how 3D computer vision is used in film and game production. Finally naked-eye 3D display technologies will be mentioned

●●● ●●●● ●●●●● ●●●● ●●●●● Overview (part 1) ●●●● Introduction From 2D to 3D Camera systems/ calibration Feature extraction/correspondence Reconstruction algorithms 2ⅵiews,3vews, N views Real-time algorithms/ Kalman filter Previous projects Virtual viewer/ Projector camera systems Keystone correction Novel setups Multiple cameras/ camera array Obtain 3D directly Structured light Laser approach Kinect approach ● Photometric stereo 3D computer vsion techniques v 4

3D computer vision techniques v.4b2 3 Overview (part1) ⚫ Introduction ⚫ From 2D to 3D ⚫ Camera systems/calibration ⚫ Feature extraction/correspondence ⚫ Reconstruction algorithms ⚫ 2 views, 3 views , N views ⚫ Real-time algorithms/Kalman filter ⚫ Previous projects ⚫ Virtual viewer/ Projector camera systems ⚫ Keystone correction ⚫ Novel setups ⚫ Multiple cameras/ Camera array ⚫ Obtain 3D directly ⚫ Structured light ⚫ Laser approach ⚫ Kinect approach ⚫ Photometric stereo

●●● ●●●● ●●●●● ●●●● ●●●●● Overview(part 2) ●●●● ● Applications o Photos from tourists(photo tourism http:/phototour.cs.washington.edu/ ●3 D displays o Possible future research Classification based on 3d information o Content search 3d based on 3d keys Merging with sound information 3D computer vsion techniques v 4b2

3D computer vision techniques v.4b2 4 Overview (part 2) ⚫ Applications ⚫ Photos from tourists (photo tourism) http://phototour.cs.washington.edu/ ⚫ 3D displays ⚫ Possible future research ⚫ Classification based on 3D information ⚫ Content search 3D based on 3D keys ⚫ Merging with sound information

●●● ●●●● ●●●●● ●●●● ●●●●● Motivation ●●●● o We live in a 3d world We see 2D images but perceive the world in 3D e Intelligent robot should have this 3d reconstruction capability 3D computer vsion techniques v 4

3D computer vision techniques v.4b2 5 Motivation ⚫ We live in a 3D world ⚫ We see 2D images but perceive the world in 3D ⚫ Intelligent robot should have this 3D reconstruction capability

●●● ●●●● ●●●●● ●●●● How to obtain 3D information? 3:8 ● Cameras-2D Range sensors 3D 3D computer vsion techniques v 4

3D computer vision techniques v.4b2 6 How to obtain 3D information? ⚫ Cameras-2D ⚫ Range sensors-3D

●●● ●●●● ●●●●● ●●●● ●●●●● Challenges ●●●● o obtain 3d information for tasks in a 3d world o 2D-to-3D reconstruction from a camera 3d directly- laser range sensor, kinect sensor ● Novel sensors Camera array/ multiple camera ° One pixel camera ight field camera 3D computer vsion techniques v 4

3D computer vision techniques v.4b2 7 Challenges ⚫ Obtain 3D information for tasks in a 3D world. ⚫ 2D-to-3D reconstruction from a camera ⚫ 3D directly— laser range sensor, kinect sensor ⚫ Novel sensors ⚫ Camera array/ multiple camera ⚫ One pixel camera ⚫ light field camera

●●● 2D-to-3D reconstruction ●●●● ●●●●● ●●●● ●●●●● (feature based method) ●●●● Camera(perspective projection Features-extraction and correspondences ● Methods ° One-image method TWO-image(Stereo) method ° Three-image method N-image method Bundle adjustment ● Kalman filter 3D computer vsion techniques v 4

3D computer vision techniques v.4b2 8 2D-to-3D reconstruction (feature based method) ⚫ Camera (perspective projection) ⚫ Features-extraction and correspondences ⚫ Methods ⚫ One-image method ⚫ Two-image (Stereo) method ⚫ Three-image method ⚫ N-image method ⚫ Bundle adjustment ⚫ Kalman filter

●●● http://upload.wikimediaorg/wikipedia/en/8/81/pinhole-camera.png3... ●●●● ●●●●● Camera: 3D to 2D projection ●●●● Pinhole Camera Perspective model u=FX/Z(nonlinear relation V=FRY/Z Virtual World Screen or cCd center Y sensor Screen Or Thin lens CC or a pin he sensor

3D computer vision techniques v.4b2 9 Camera: 3D to 2D projection Perspective model u=F*X/Z (nonlinear relation) v=F*Y/Z F Z Y v World center F Thin lens or a pin hole Virtual Screen or CCD sensor Real Screen Or CCD sensor Pinhole Camera http://upload.wikimedia.org/wikipedia/en/8/81/Pinhole-camera.png

●●● ●●●● Perspective ●●●● Wod● Projective Coordinates Zw XW Rc. TC Modelmat t=l image XYZ v-aXIs XC-axis Camera u Coordinates Zc-axis Principal axis≤- c(Image center,(0,0, 0) Camera center u-aXIs F=focal length Yc-axis (0,0)of image plane 3D computer vsion techniques v 4b

3D computer vision techniques v.4b2 10 Perspective Projective ⚫ Model M at t=1 c (Image center, ox ,oy ) F=focal length image Oc= (0,0,0) (Camera center) Xc-axis Zc-axis Yc-axis v-axis u-axis X,Y,Z (u,v) (0,0) of image plane Camera Coordinates. World Coordinates Zw Yw Xw Rc,Tc Principal axis