《图像处理与计算机视觉 Image Processing and Computer Vision》课程教学资源（PPT课件讲稿）Chapter 08 Stereo vision

Image processing and computer VISIon Chapter 8: Stereo vision Stereo vO. a

Image processing and computer vision Chapter 8: Stereo vision Stereo v0.a 1

3-D computer vision, an overview Find 3-d structure using 2-D images One camera: Possible but limited results, e.g. by identify vanishing lines Two cameras: Stereo: use Epipolar geometry( this chapter) Three cameras Trifocal tensor method (very advanced not discussed here cameras Factorization(Chapter 9), fast method, moderate accuracy Bundle adjustment chapter 11), slow but accurate Stereo vO. a

3-D computer vision, an overview • Find 3-D structure using 2-D images • One camera: – Possible but limited results, e.g. by identify vanishing lines • Two cameras: – Stereo: use Epipolar geometry (This chapter) • Three cameras: – Trifocal tensor method (very advanced not discussed here) • N cameras: – Factorization (Chapter 9), fast method, moderate accuracy – Bundle adjustment (chapter 11), slow but accurate. Stereo v0.a 2

Intro Essential Mat. I Fundamental Mat. I Epipolar Geom. I Corresp. I Reconst In this chapter you will learn Stereo camera setup Essential matrix (E)for describing the geometry between two cameras of known intrinsic parameters Fundamental matrix (f) for describing the geometry between two cameras of unknown intrinsic parameters Epipolar geometry parameters and characteristics The correspondent problem 3D Reconstruction using stereo vision Stereo vO. a

Intro. | Essential Mat. | Fundamental Mat. | Epipolar Geom. | Corresp. | Reconst. In this chapter you will learn • Stereo camera setup • Essential matrix (E)for describing the geometry between two cameras of known intrinsic parameters • Fundamental matrix (F) for describing the geometry between two cameras of unknown intrinsic parameters • Epipolar geometry: parameters and characteristics • The Correspondent problem • 3D Reconstruction using stereo vision Stereo v0.a 3

Intro. Essential Mat. Fundamental Mat. Epipolar Geom. Corresp. Reconst Part 1: A simple approach 3-D reconstruction from stereo mages Assumption: the cameras are paralle/ (2 principal axes are parallel) and the camera shift is only in the horizontal direction Stereo vO. a

Intro. | Essential Mat. | Fundamental Mat. | Epipolar Geom. | Corresp. | Reconst. Part 1: A simple approach 3-D reconstruction from stereo images Assumption : the cameras are parallel (2 principal axes are parallel) and the camera shift is only in the horizontal direction Stereo v0.a 4

Intro Essential Mat. I Fundamental Mat. I Epipolar Geom. I Corresp. I Reconst Introduction to stereo vision Objectives: Basic idea of stereo vision Stereo reconstruction by epipolar geometry Stereo camera pair calibration find Fundamental matriX F) Construct the 3d ( graphic) model from 2 images Inside a computer e.g. Graphic 3-Dobⅰect model in a game Stereo vO. a

Intro. | Essential Mat. | Fundamental Mat. | Epipolar Geom. | Corresp. | Reconst. Introduction to Stereo Vision • Objectives: – Basic idea of stereo vision – Stereo reconstruction by epipolar geometry • Stereo camera pair calibration (find Fundamental matrix F) • Construct the 3D (graphic) model from 2 images Stereo v0.a 5 e.g. Graphic model in a game Inside a computer 3-D object

Intro Essential Mat. I Fundamental Mat. I Epipolar Geom. I Corresp. I Reconst a simple stereo system Assume cameras are aligned horizontally left Right Object CameraCamera Px(x,y, z) (No vertical disparity) Apoint in 3D(Px) Principle Principle axiS axIs eft image Right image plane plane Left Right Image : XL Image plane plane Focal Left Length Stereo: d Rig optical Baseline(B)optical Center Center B Baseline) O(left o(rig Left camera center (reference point) Horizontal (reference Disparity=XL-XR point) Stereo vO. a

Intro. | Essential Mat. | Fundamental Mat. | Epipolar Geom. | Corresp. | Reconst. A simple stereo system Assume cameras are aligned horizontally (No vertical disparity) • Stereo v0.a 6 A point in 3D (Px) Right image plane Left image plane Left optical Center O(left) (reference point) Right optical Center O(right) Stereo: Baseline (B) xL xR Focal Length f Object Px(x,y,z) z Left camera center (reference point) Horizontal Disparity=xL -xR B (Baseline) Left Camera Principle axis Right Camera Principle axis Left Image plane Right Image plane

Intro. Essential Mat. I Fundamental Mat. Epipolar Geom. I Corresp. I Reconst Example assume cameras are aligned horizontallyno vertical motion of camera= no vertical disparity Left image Right image White crosses are overlaid on images to show the positions of features They are not 换| in the original pictures a corner feature is found in a 10x10 window(w) Find the correspondence at (XR yR) centered at the Horizontal disparity=XL-XR left image(XL,yu (overlay a cross) Thestereosequencevideohttp://www.youtube.com/watch?veursyjokuils Stereo vO. a

Intro. | Essential Mat. | Fundamental Mat. | Epipolar Geom. | Corresp. | Reconst. Example Assume cameras are aligned horizontally(No vertical motion of camera= no vertical disparity) • Stereo v0.a 7 Left image Right image Find the correspondence at (xR,yR) Horizontal disparity=XL -XR A corner feature is found in a 10x10 window (w) centered at the left image (xL ,yL ) (overlay a cross) White crosses are overlaid on images to show the positions of features. They are not in the original pictures. The stereo sequence video : http://www.youtube.com/watch?v=Ursyjokuils

Intro Essential Mat. I Fundamental Mat. I Epipolar Geom. I Corresp. I Reconst Triangular calculation Left Right bject Camera Camera Px(x,y, z) Principle Principle By similar triangle axIs axIs w.r.t left camera lens center z (x-b Left Right Image X Image lane plane elimate x→ f·b Focal Leng th b baseline) Left camera center By similar triangle, (reference point) Horizontal w.r.t right camera lens center Disparity=X -XE Stereo vO. a One major problem is to locate x' and x' r the correspondence problem

Intro. | Essential Mat. | Fundamental Mat. | Epipolar Geom. | Corresp. | Reconst. • , ( ) elimate ( ' l ' r ' ' l r x x = z f x - b x = z f f b x z = x - x ) •  Stereo v0.a 8 x’l x’r Focal Length f Object Px(x,y,z) z Left camera center (reference point) Horizontal Disparity=xL -xR b (Baseline) Left Camera Principle axis Right Camera Principle axis Left Image plane Right Image plane By similar triangle, w.r.t left camera lens center By similar triangle, w.r.t right camera lens center Triangular calculation One major problem is to locate x’l and x’r The correspondence problem x

Intro. Essential Mat. I Fundamental Mat. I Epipolar Geom.| Corresp.|Reconst Stereo vision example step1: feature extraction(e.g. harris operator for the left and right image find features and locate their windows c:\Otest-imagestabe 7\ labe7-000 bmp c:0MisionlOtest-imagesllabe 7 abe7-007 bmp 50 100 200 250 250 300 50 400 400 50 200 300 400 500 Left image right image Features are shown by overlaid markers on images Stereo vO. a

Intro. | Essential Mat. | Fundamental Mat. | Epipolar Geom. | Corresp. | Reconst. Stereo vision example: step1:feature extraction (e.g. Harris operator) : for the left and right image find features and locate their windows • Stereo v0.a 9 Left image right image Features are shown by overlaid markers on images

Intro. Essential Mat. I Fundamental Mat. I Epipolar Geom. I Corresp. I Reconst Stereo vision example: step2 Correspondence problem example Find correspondence of f1 in the right image and determine which is the match (see chapter 4, features extraction and tracking csicnDtest-m 0e71abe7.000bmp clDMsionest mag Left image Right image fl: a small window f2 13 CL2=Cross correlate 13 Cross correlate f i with f2 f with f 3 Stereo vO. a

Intro. | Essential Mat. | Fundamental Mat. | Epipolar Geom. | Corresp. | Reconst. Stereo vision example: step2: Correspondence problem example: Find correspondence of f1 in the right image and determine which is the match (see chapter 4, features extraction and tracking) • Stereo v0.a 10 r1,2=Cross correlate f1 with f2 f1:a small window f2 f3 r1,3=Cross correlate f1 with f3 Left image Right image