Chapter 16 Image Matching

16.1 Introduction

16.1.1 Image Matching and Object Reconstruction object reconstruction: determination of object’s pose or shape image matching and object reconstruction needed for

• time-varying sequences for recognizing parts in motion • visual inspection of geometry of 3D parts

• medical diagnosis of beating hearts

• monitoring land use

• deriving topographic maps from satellite or aerial imagery • analysis of slices of computer tomography images

I^{1}, I^{2}, ..., I^{k}: object mapped into images T_{0}^{1}, T_{0}^{2}, ..., T_{0}^{k}: object to image mapping transformations T_{0}^{1}, T_{0}^{2}, ..., T_{0}^{k}: describe illumination, reflectance, sensing, ... assumption: transformations T_{0}^{k} are one to one assumption implicitly excludes transparent surfaces or occlusions p_{G}: unknown parameters specifying geometry of object p_{R}: unknown parameters specifying reflectance property of object p_{P}: unknown parameters specifying pose of camera k p_{A}: unknown parameters specifying atmospheric response mapping can be summarized as:

I^{k} = T_{0}^{k}(p_{G}, p_{R}, p_{P}, p_{A}, ...)

general setup of object reconstruction and image matching

(a) object mapped to I^{1}, I^{2} via transformations T_{0}^{1}, T_{0}^{2}

(b) setup for two perspective images =====Fig. 16.1=====

image I^{2} can be predicted from I^{1} by:

T_{2}^{1} = T_{0}^{1} · T_{2}^{0} = T_{0}^{1} · (T_{0}^{2})^{−1}(p_{G}, p_{R}, p_{P 1}, p_{P 2}, p_{A}, ...)

16.1.2 The Principle of Image Matching I′, I′′: two digital or digitized images or image patches I′, I′′: size 5 × 5 (point tracking) to 10000 × 10000 (satellite image registration) P′, P′′: points of images I′, I′′ (r′, c′), (r′′, c′′): coordinates of points P′, P′′ g′, g′′: intensities of points P′, P′′ T_{G}: specified geometric spatial-mapping function p_{G}: set or vector of unknown parameters if (r′, c′) corresponds to (r′′, c′′), their coordinates related by:

(r′, c′) = T_{G}(r′′, c′′; p_{G})

T_{I}: intensity-mapping function containing intensity relation p_{I}: unknown vector intensities on one image related to other image:

g′ = T_{I}(g′′; p_{I})

complete model of image matching:

g′(r′, c′) = T_{I}{g′′[r′, c′]; p_{I}} = T_{I}{g′′[T_{G}(r′′, c′′; p_{G})]; p_{I}}

T_{G}, T_{I}: may be deterministic, stochastic, and/or piecewise continuous

correspondence of P′, P′′: P′, P′′ relate to same point on object problem of image matching or correspondence consists of:

1. finding all corresponding points 2. determining parameters p_{G}, p_{I} of mapping functions T_{G}, T_{I} =====Fig. 16.2=====

a′, a′′: point attributes derived from intensity functions in neighborhood P′, P′′: points in a neighborhood around (r′, c′), (r′′, c′′)

P′ = P′(r′, c′; a′) and P′′ = P′′(r′′, c′′; a′′)

image matching solution achieved in three-step procedure

1. select appropriate image features in one or in both images e.g. interest operator to derive list of interesting points or edges

2. find corresponding point pairs (P′_{i}, P′′_{j}) with similarity and consistency with P′_{i} from first list and P′′_{j} from second list similarity: based on attributes and properties of intensity function consistency: based on degree to which spatial-mapping function fulfilled

3. interpolate parallaxes (r′′ − r′, c′′ − c′) between selected feature pairs

16.1.3 Image Matching Procedures properties of some correspondence algorithms for image matching =====Table 16.1===== similarity measures reflect the assumed intensity-mapping function • same intensity: g′ = T_{I}(g′′) = g′′: e.g. Horn and Schunck optic flow, minimum sum of absolute difference

• linear intensity mapping: g′ = ag′′ + b:

e.g. maximum cross-correlation

consistency measure or interpolation method on stereo images:

• (a) same size: surfaces parallel to image plane

• (b) contraction: locally tilted planar surface patch

• (c) expansion: smooth or piecewise smooth surface

• (d) difference: piecewise smooth with occlusions

=====Fig. 16.3===== precision of image matching and edge detection (all figures in pixels) =====Table 16.2===== =====Garfield 17:28=====

16.2 Intensity-Based Matching of One-Dimensional Signals g: replacing g′: intensity of first image f: replacing g′′: intensity of second image x: replacing r′: location of first image y: replacing r′′: location of second image n(x): observational noise component 1D intensity-based matching model:

g(x) = T_{I}{f[T_{G}(y; p_{G})]; p_{I}} + n(x)

16.2.1 The Principle of Differential Matching assumption: no intensity changes due to viewing direction x_{i}: location in first image û(x_{i}): unknown deformation at x_{i} to produce corresponding point point x_{i} in first image corresponds to point y_{i} in second image

y(x_{i}) = x_{i} − û(x_{i})

nonlinear model valid for all m observed values g(x_{i}), i = 1, ..., m:

g(x_{i}) = f[x_{i} − û(x_{i})] + n(x_{i}), i = 1, ..., m

n(x_{i}): additive noise; independent and identically distributed n(x_{i}): mean zero, variance σ_{n}^{2} assumed observation process: u(x)+ ⇒ x_{i} to the right of y(x_{i}) =====Fig. 16.4===== u_{0}(x_{i}): approximate values of unknown deformation Δ̂u(x_{i}): unknown correction for unknown value û(x_{i})

û(x_{i}) = u_{0}(x_{i}) + Δ̂u(x_{i})

abbreviations: g_{i} = g(x_{i}), Δ̂u_{i} = Δ̂u(x_{i}), n_{i} = n(x_{i}), u_{0i} = u_{0}(x_{i}), û_{i} = û(x_{i}) nonlinear model can be written as:

g_{i} = f(x_{i} − û_{i}) + n_{i} = f(x_{i} − u_{0i} − Δ̂u_{i}) + n_{i}, i = 1, ..., m

linearize f around point x_{i} − u_{0i} to obtain:

g_{i} = f(x_{i} − u_{0i}) − f′(x_{i} − u_{0i})Δ̂u_{i} + \frac{1}{2}f′′(x_{i} − u_{0i} − ξΔ̂u_{i})(Δ̂u_{i})^{2} + n_{i}

for some ξ ∈ [0, 1], and where f′(y) = df/dy, f′′(y) = d^{2}f/dy^{2} assumption: f′ does not vanish and second-order term negligible

Δg_{i} = Δg(x_{i}) = g(x_{i}) − f(x_{i} − u_{0i})
f′_{i} = f′(x_{i} − u_{0i})

linearized model with known: Δg_{i}, f′_{i}, unknown: Δ̂u_{i}

Δg_{i} = −f′_{i}Δ̂u_{i} + n_{i}, i = 1, ..., m

or explicitly

g(x_{i}) − f[x_{i} − u_{0}(x_{i})] = −\frac{df(y)}{dy}|_{y=x_{i}−u_{0}(x_{i})}Δ̂u_{i} + n(x_{i}), i = 1, ..., m

easily determine random variable Δ̂u_{i} assuming f′_{i} ≠ 0:

Δ̂u_{i} = −\frac{Δg_{i}}{f′_{i}}

or explicitly

Δ̂u_{i} = −\frac{g(x_{i}) − f[x_{i} − u_{0}(x_{i})]}{\frac{df(y)}{dy}|_{y=x_{i}−u_{0}(x_{i})}}

thus

û_{i} = u_{0i} + Δ̂u_{i}

principle of the applied differential approach to estimate local deformation =====Fig. 16.5=====

16.2.2 Estimating an Unknown Shift assumption: only uniform shift i.e. û(x) = û, y(x_{i}) = x_{i} − û u_{0}: initial approximation for u, thus û = u_{0} + Δ̂u, ∀x linearizing f around x_{i} − u_{0} for each i:

f[x_{i} − (u_{0} + Δ̂u)] = f(x_{i} − u_{0}) − f′(x_{i} − u_{0})Δ̂u

linearized model can be expressed as:

g(x_{i}) = f(x_{i} − u_{0}) − f′(x_{i} − u_{0})Δ̂u + n_{i}, i = 1, ..., m or in short

Δg_{i} = −f′_{i}Δ̂u + n_{i}, i = 1, ..., m

minimizing Ω = ∑_{i=1}^{m} n_{i}^{2} by choosing appropriate Δ̂u

\frac{∂Ω}{∂Δ̂u} = \frac{∂}{∂Δ̂u} m ; ∑ ; i=1(Δg_{i} + f′_{i}Δ̂u)^{2} = 2 m ; ∑ ; i=1 f′_{i}Δg_{i} + 2(∑ f′_{i}^{2})Δ̂u = 0

from which follows the estimate

Δ̂u = −\frac{∑_{i=1}^{m} f′_{i}Δg_{i}}{∑_{i=1}^{m} f′_{i}^{2}} = −\frac{∑_{i=1}^{m}[g(x_{i}) − f(x_{i} − u_{0})]f′(x_{i})}{∑_{i=1}^{m}[f′(x_{i})]^{2}}

16.2.3 Estimating Unknown Shift and Scale x_{0}: fixed reference point augment model by assuming transformation contains unknown scale:

y_{i} = s(x_{i} − x_{0}) − û = sx̄_{i} − û

in reduced coordinates

x̄_{i} = x_{i} − x_{0}

linear model reads as:

g(x_{i}) = f(sx̄_{i} − û) + n(x_{i}), i = 1, ..., m linearized model:

Δg_{i} = −f′_{i}Δ̂u + f′_{i}x̄_{i}Δ̂s + n(x_{i}), i = 1, ..., m

use Taylor’s expansion for linearized model and minimize ∑ n^{2}(x_{i}), thus

( \begin{matrix}∑ f′_{i}^{2} & − ∑ f′_{i}^{2}x̄_{i} \\ − ∑ f′_{i}^{2}x̄_{i} & ∑ f′_{i}^{2}x̄_{i}^{2} \end{matrix} ) ( Δ̂u ; Δ̂s ) = ( \begin{matrix}− ∑ f′_{i}Δg_{i} \\ ∑ f′_{i}x̄_{i}Δg_{i} \end{matrix} )

16.2.4 Compensation for Brightness and Contrast a: change in contrast b: change in brightness nonlinear model with brightness and contrast but without scale parameter:

g(x_{i}) = af(x_{i} − u) + b + n(x)

linearized model:

Δg_{i} = −a_{0}f′_{i}Δ̂u + f_{i}Δ̂a + b̂ + n_{i}, i = 1, ..., m

normal equation system for least-squares solution to minimize ∑ n^{2}(x_{i}):

( \begin{matrix}a_{0}^{2} ∑ f′_{i}^{2} & −a_{0} ∑ f′_{i}f_{i} & −a_{0} ∑ f′_{i} \\ −a_{0} ∑ f′_{i}f_{i} & ∑ f_{i}^{2} & ∑ f_{i} \\ −a_{0} ∑ f′_{i} & ∑ f_{i} & ∑ 1 \end{matrix} ) ( Δ̂u ; Δ̂a ; b̂ ) = ( \begin{matrix}−a_{0} ∑ f′_{i}Δg_{i} \\ ∑ f_{i}Δg_{i} \\ ∑ Δg_{i} \end{matrix} )

16.2.5 Estimating Smooth Deformations used for larger windows but transformation still smooth

16.2.6 Iterations and Resampling iterative estimation scheme: if initial approximations are crude

16.2.7 Matching of Two Observed Profiles both profiles corrupted by noise: reduce to methods developed so far

16.2.8 Relations to Cross-Correlation Techniques cross-correlation model: assumes a shift between two corresponding image =====Oldie 33:79=====

16.3 Intensity-Based Matching of Two-Dimensional Signals differential techniques for intensity-based matching expanded to 2D

16.3.1 The Principle and the Relation to Optical Flow nonlinear model:

g(r_{i}, c_{i}) = f(p_{i}, q_{i}) + n(r_{i}, c_{i}) i = 1, ..., m

with

( \begin{matrix}p \\ q \end{matrix} )_{i} = ( \begin{matrix}r \\ c \end{matrix} )_{i} − ( \begin{matrix}û(r, c) \\ v̂(r, c) \end{matrix} )_{i}

assumption: approximate values of unknown [û(r, c), v̂(r, c)] known related to optical flow equation: i.e. noise term omitted:

f[r + u(r, c) · dt, c + v(r, c) · dt, t + dt] = f(r, c, t)

linearize by Taylor’s expansion:

f(r, c, t) + \frac{∂f(r, c, t)}{∂r} · u(r, c) · dt + \frac{∂f(r, c, t)}{∂c} · v(r, c) · dt + \frac{∂f(r, c, t)}{∂t}dt = f(r, c, t)

let ∇f = (f_{r}, f_{c})^{T}, V = (u, v)^{T}, ḟ = ∂f/∂t, we get optical flow equation:

∇f · V + ḟ = f_{r}u + f_{c}v + f_{t} = 0

16.3.2 Estimating Constant-Shift Parameters simplest model for matching:

p = x − u, q = y − v

linearized model:

Δg_{i} = −f_{r_{i}} · Δ̂u − f_{c_{i}} · Δ̂v + n_{i}, i = 1, ..., m

normal equations to minimize ∑ n^{2}(x_{i}):

( \begin{matrix}∑_{i=1}^{m} f_{r_{i}}^{2} & ∑_{i=1}^{m} f_{r_{i}}f_{c_{i}} \\ ∑_{i=1}^{m} f_{r_{i}}f_{c_{i}} & ∑_{i=1}^{m} f_{c_{i}}^{2} \end{matrix} ) ( Δ̂u ; Δ̂v ) = − ( \begin{matrix}∑_{i=1}^{m} f_{r_{i}} · Δg_{i} \\ ∑_{i=1}^{m} f_{c_{i}} · Δg_{i} \end{matrix} )

16.3.3 Estimating Linear Transformations a_{k}, k = 1, ..., 8: eight unknown parameters model to match two small windows:

g_{1}(r_{i}, c_{i}) = f(r_{i}, c_{i}) + n_{1}(r_{i}, c_{i})

g_{2}(r_{i}, c_{i}) = h[f(p_{i}, q_{i})] + n_{2}(r_{i}, c_{i})

( \begin{matrix}p \\ q \end{matrix} )_{i} = ( \begin{matrix}a_{1} & a_{2} \\ a_{4} & a_{5} \end{matrix} ) · ( \begin{matrix}r \\ c \end{matrix} )_{i} + ( \begin{matrix}a_{3} \\ a_{6} \end{matrix} )

h(f) = a_{7} · f + a_{8}

linearized model:

Δg_{i} = f_{r_{i}} · r_{i} · Δ̂a_{1} + f_{r_{i}} · c_{i} · Δ̂a_{2} + f_{r_{i}}Δ̂a_{3} + f_{c_{i}} · r_{i} · Δ̂a_{4} + f_{c_{i}} · c_{i} · Δ̂a_{5}

+f_{c_{i}}Δ̂a_{6} + f_{i}Δ̂a_{7} + 1Δ̂a_{8} + n̄_{i}

to minimize ∑ n̄_{i}^{2}, normal equation matrix N:

( \begin{matrix}∑ f_{r}^{2}r^{2} & ∑ f_{r}^{2}rc & ∑ f_{r}^{2}r & ∑ f_{r}f_{c}r^{2} & ∑ f_{r}f_{c}rc & ∑ f_{r}f_{c}r & ∑ f_{r}fr & ∑ f_{r}r \\ & ∑ f_{r}^{2}c^{2} & ∑ f_{r}^{2}c & ∑ f_{r}f_{c}cr & ∑ f_{r}f_{c}c^{2} & ∑ f_{r}f_{c}c & ∑ f_{r}fc & ∑ f_{r}c \\ & & ∑ f_{r}^{2} & ∑ f_{r}f_{c}r & ∑ f_{r}f_{c}c & ∑ f_{r}f_{c} & ∑ f_{r}f & ∑ f_{r} \\ & & & ∑ f_{c}^{2}r^{2} & ∑ f_{c}^{2}rc & ∑ f_{c}^{2}r & ∑ f_{c}fr & ∑ f_{c}r \\ & & & & ∑ f_{c}^{2}c^{2} & ∑ f_{c}^{2}c & ∑ f_{c}fc & ∑ f_{c}c \\ & & & & & ∑ f_{c}^{2} & ∑ f_{c}f & ∑ f_{c} \\ & & & & & & ∑ f^{2} & ∑ f \\ & & & & & & & ∑ 1 \end{matrix} )

eight unknown parameters:

ŷ = ( Δ̂a_{1} Δ̂a_{2} Δ̂a_{3} Δ̂a_{4} Δ̂a_{5} Δ̂a_{6} Δ̂a_{7} Δ̂a_{8} )

right-hand side h:

h = ( \begin{matrix}∑_{i=1}^{m} f_{r_{i}}r_{i}Δg_{i} \\ ∑_{i=1}^{m} f_{r_{i}}c_{i}Δg_{i} \\ ∑_{i=1}^{m} f_{r_{i}}Δg_{i} \\ ∑_{i=1}^{m} f_{c_{i}}r_{i}Δg_{i} \\ ∑_{i=1}^{m} f_{c_{i}}c_{i}Δg_{i} \\ ∑_{i=1}^{m} f_{c_{i}}Δg_{i} \\ ∑_{i=1}^{m} f_{i}Δg_{i} \\ ∑_{i=1}^{m} 1Δg_{i} \end{matrix} )

solution: normal equation system Nŷ = h Δ̂a_{k}, k = 1, ..., 6: six corrections for geometric transformation Δ̂a_{7}, Δ̂a_{8}: corrections for radiometric parameter

16.3.4 Invariant Points (a),(b): corner points; scale difference cannot be determined (c),(d): circularly symmetric; rotation cannot be determined =====Fig. 16.8=====

16.4 An Interest Operator

16.4.1 Introduction interesting has several meanings, depending on context:

1. distinctness: distinguishable from immediate neighbors distinct points: corners, blobs, highly textured places, etc.

2. invariance: position and selection invariant w.r.t. geometric distortion invariance and distinctness: influence all subsequent steps in analysis

3. stability: position and selection invariant w.r.t. viewing stability: ensures interesting points in image correspond to object points corner points of polyhedra: stable T-junction: unstable since it results from occlusions stability: decisive for image-matching, 3D reconstruction =====Nalwa, A Guided Tour of Computer Vision, p. 137=====

4. uniqueness: global separability, i.e. imagewide separability distinctness: aims at local separability uniqueness: to avoid locally distinct but repetitive features or points uniqueness: closest notion to interestingness

5. interpretability: requires extracted points to have meaning e.g. corners, junctions of lines, centers of circles, rings, etc. an interest operator with a three-step procedure:

1. selection of optimal windows: search for local maxima of average gradient magnitude

2. classification of the image function within the selected windows classification distinguishes between types of singular points e.g. corners, rings, spirals, isotropic texture

3. estimation of the optimal point within the window as the classification precise for corners and for centers of circular symmetric features

16.4.2 Estimating Corner Points

16.4.3 Evaluation and Classification of Selected Windows

16.4.4 Selection of Optimal Windows

16.4.5 Uniqueness of Selected Points uniqueness of selected points: based on similarity derived from attributes highest uniqueness measure: points with no features in common with other repetitive features show low uniqueness uniqueness measure represented by the area of the circles =====Fig. 16.11===== =====joke=====

16.5 Robust Estimation for Feature-Based Matching

16.5.1 The Principle of Feature-Based Matching three steps of feature-based matching:

1. selecting features by using some interest operator

2. finding correspondences by using similarity and consistency measure

3. interpolating between parallaxes by spatial-mapping function

individual steps for similarity measure:

1. similarity between extracted features:

form preliminary list of correspondences, including weights

2. mapping function hypothesis: found by robust estimation procedure, similar to relaxation techniques by enforcing one-to-one correspondence between image features 3. final parameters of mapping function: by maximum likelihood estimate allowing

rigorous evaluation of the match

α: parameters to be estimated β: observed measurement maximum likelihood (ML) estimate:

α_{ML} = arg max_{α} P (β|α)

P (α): prior information about parameters Bayesian estimate: maximum a posteriori (MAP) estimate:

α_{MAP} = arg max_{α} P (α|β) = arg max_{α} \frac{P (β|α)P (α)}{P (β)}

=====J. V. Beck and K. J. Arnold, Parameter Estimation in===== =====Engineering and Science, Wiley, New York, 1977.=====

16.5.2 The Similarity Measure

16.5.3 Heuristics for Selecting Candidate Pairs finding candidate pairs of features: first step after selecting image features finding candidate pairs: to reduce algorithmic complexity in final match finding candidate pairs: all types of a priori knowledge may be included heuristics and strategies for selecting candidate pairs:

1. expected parallax may be used to exclude unlikely feature pairs

2. similarity of features required to be above certain threshold

3. uniqueness of selected points used to reduce number of candidate pairs

16.5.4 Robust Estimation for Determining the Spatial-Mapping Function preliminary correspondences must be compared to see if consistent with model most general model applied: local smoothness constraint almost everywhere finite-element description of spatial-mapping function would be appropriate robust estimation: used only for finding good hypothesis for match images with mostly translation (10, −5.7), no rotation (identity matrix) =====Fig. 16.12=====

16.5.5 Evaluating the Final Result result of finding most likely parameters of mapping function must be evaluated:

• to be sure the solution is correct

• to have quantitative measure for parameter quality apply classical hypothesis tests to evaluate the result:

1. global check if data and model are consistent using sum of squared residuals

2. precision of estimated parameters can be determined

3. result termed reliable only if enough points used to determine mapping

4. projection of one image into other: decisive check on matching correctness

16.6 Structure from Stereo by Using Correspondence =====Experiment: close one eye and put pen cap back===== implication of ambiguous correspondences between image points =====Nalwa, A Guided Tour of Computer Vision, Fig. 7.3===== monotonic-ordering assumption: conjugate image points have same order violation of the monotonic-ordering assumption =====Nalwa, A Guided Tour of Computer Vision, Fig. 7.7===== occlusion as an impediment to stereo =====Nalwa, A Guided Tour of Computer Vision, Fig. 7.4===== scene recovered from a pair of stereo images =====Nalwa, A Guided Tour of Computer Vision, Fig. 7.9===== large windows: to establish global correspondence small windows: using global correspondence for local correspondence stochastic relaxation as a tool for correspondence establishment =====Nalwa, A Guided Tour of Computer Vision, Fig. 7.10===== assumption: interior orientation of cameras determined by calibration for reasons of stability: at least five groups of points used for calibration

16.6.1 Epipolar Geometry image matching tremendously simplified: if relative orientation known relative orientation known: 2D search reduced to 1D by epipolar geometry O′, O′′: projection centers b: baseline length f′, f′′: focal lengths H′, H′′: principal points assumed to be two image coordinate origins (u′, v′), (u′′, v′′): two image coordinate systems P (x, y, z): 3D object point P′(u′, v′), P′′(u′′, v′′): 2D image coordinates for the point collinearity condition: five points P, O′, O′′, P′, P′′ lie in one plane epipolar plane ϵ(P): the plane P, O′, O′′, P′, P′′ lie in epipolar lines e′(P), e′′(P): intersection lines of epipolar plane and image planes different points: have different pairs of epipolar lines all epipolar planes: form pencil of planes passing through baseline b = (O′O′′) epipoles F′, F′′: epipolar line intersection point epipoles F′, F′′: intersection of baseline and image planes in general: epipolar lines are not parallel epipolar geometry of a general image pair =====Fig. 16.13=====

epipolar plane ϵ(P): defined by P′, O′, O′′ given one image point: epipolar line e′′(P) fixed and P′′ sits on this line since epipolar line known: search is necessary in only one dimension epipolar line constraint: strongest constraint in image matching epipolar line constraint: used as soon as available

K′, K′′: principal points (x′, y′), (x′′, y′′): image coordinate system x′ parallel to x′′ epipolar lines e′(P), e′′(P) identical and parallel to baseline b image planes μ′, μ′′: identical and parallel to baseline b search space for P′′ given P′(x′, y′): line y = y′ epipolar geometry of a normal image pair =====Fig. 16.14=====

16.6.2 Generation of Normal Images to rectify image pairs to normal: for correspondence and 3D determination ν: new image plane relation between general image pair and normal image pair =====Fig. 16.15===== procedure for rectification to normal image pair:

1. Choose a plane ν parallel to baseline b. now focal length f̄ of normal images fixed Choose new image coordinate systems (x′, y′), (x′′, y′′) with origins K̄′, K̄′′ x′ parallel to x′′ K̄′, K̄′′: principal points of new images K̄′O′ parallel to K̄′′O′′ Choose a common pixel spacing in the normal images.

2. For each of the images:

(a) Choose four points well distributed over the new image.

(x_{i}, y_{i}), i = 1, ..., 4: coordinates of 4 points in normal images

(b) Project the four points into the original image.

Use known pose of the cameras here.

(u_{i}, v_{i}), i = 1, ..., 4: four coordinates in image planes μ′, μ′′ (c) Solve the equation system:

(gx_{i} + hy_{i} + 1) · u_{i} = ax_{i} + by_{i} + c

(gx_{i} + hy_{i} + 1) · v_{i} = dx_{i} + ey_{i} + f

(d) Rectify the original image. each pixel (x_{j}, y_{j}) in normal image: determines (u_{j}, v_{j}) in original image

16.6.3 Specializing the Image-Matching Procedures interest operator reduced to 1D: searching edges across epipolar lines

16.6.4 Precision of Three-Dimensional Points from Image Points wide-angle stereo: distance between two centers of projection large wide-angle stereo: more precise estimates for 3D positions wide-angle stereo: more difficult to establish correspondence adaptive matching window: size inversely proportional to intensity variance adaptive matching window: size inversely proportional to estimated disparity (a) original (b) 3 × 3 (c) 7 × 7 (d) adaptive matching window size =====Nalwa, A Guided Tour of Computer Vision, Fig. 7.8===== stereo image pair taken with a photogrammetric stereo camera =====Fig. 16.17===== =====Garfield 17:52=====