Chapter 9 Texture

9.1 Introduction formal approach or precise definition of texture does not exist texture discrimination techniques are for the most part ad hoc principal characteristic of texture: repetition of basic pattern texture: nonlocal property, characteristic of region larger than its size =====/usr/local/vision/images/t bark.im===== =====/usr/local/vision/images/t clouds.im===== =====/usr/local/vision/images/t fur r.im===== =====/usr/local/vision/images/t pebbles.im===== =====/usr/local/vision/images/t straw.im===== =====/usr/local/vision/images/t water r.im===== visual cliff by William Vandivert, 1960 =====V. S. Nalwa, A Guided Tour of Computer Vision, Fig. 6.1=====

(a) water (b) beach pebbles (c) raffia weave (d) brick wall =====V. S. Nalwa, A Guided Tour of Computer Vision, Fig. 6.2===== texture: repeating patterns of local variations in image intensity which are

too fine to be distinguished as separate objects at the observed resolution

for humans, texture is the abstraction of certain statistical homogeneities from a portion of the visual field that contains a quantity of information grossly in excess of the observer’s perceptual capacity

texture analysis issues

1. pattern recognition: given textured region, determine the class the region belongs to

2. generative model:

given textured region, determine a description or model for it

3. texture segmentation: given image with many textured areas, determine boundaries

(a) multispectral image (b) printout of machine classification results =====Gonzalez, Digital Image Processing, Fig. 9.13=====

texture: fineness, coarseness, contrast, directionality, roughness, regularity statistical texture-feature approaches use

• autocorrelation function

• spectral power density function

• edgeness per unit area • spatial gray level co-occurrence probabilities

• gray level run-length distributions

• relative extrema distributions

• mathematical morphology

Given a generative model and the values of its parameters, one can synthesize homogeneous image texture samples associated with the model and the given value of its parameters. image texture analysis: verification and estimation verification: verify given image texture sample consistent with model estimation: estimate values of model parameters based on observed sample examples of model-based techniques

• autoregressive, moving-average, time-series models (extended to 2D)

• Markov random fields

• mosaic models

texel: texture element: basic textural unit of some textural primitives qualitatively evaluated image texture properties • fineness

• coarseness

• smoothness

• granulation

• randomness

• lineation

• mottled

• irregular

• hummocky For any textural surface there exists a scale at which, when the surface is examined, it appears smooth and textureless (e.g. see from infinite distance). Then as resolution increases, the surface appears as a fine texture and then a coarse one, and for multiple-scale textural surfaces the cycle of smooth, fine, and coarse may repeat. texture: cannot be analyzed without frame of reference on scale, resolution texture: scale-dependent phenomenon =====Fig. 9.1===== to characterize texture

• characterize gray level primitive properties

• characterize spatial relationships between them =====Oldie 33:20=====

9.2 Gray Level Co-Occurrence first-order gray-level statistics: statistics of single pixels,

e.g. histogram, mean, median, variance second-order gray-level statistics: the combined statistics of the gray levels

of pairs of pixels in which each two pixels in a pair have a fixed relative position, e.g. co-occurrence

gray level spatial dependence: characterize texture by co-occurrence The gray level co-occurrence can be specified in a matrix of relative frequencies P_{ij} with which two neighboring pixels separated by distance d occur on the image, one with gray level i and the other with gray level j. co-occurrence matrix: symmetric, function of angle and distance between pixels =====Fig. 9.2=====

probability of horizontal, d pixels apart pixels P (i, j, d, 0^{∘}) = #{[(k, l), (m, n)]|k − m = 0, |l − n| = d, I(k, l) = i, I(m, n) = j} probability of 45^{∘}, d pixels apart pixels

P (i, j, d, 45^{∘}) = #{[(k, l), (m, n)]|[(k − m = d, l − n = −d) or (k − m = −d,

l − n = d)], I(k, l) = i, I(m, n) = j}

probability of vertical, d pixels apart pixels P (i, j, d, 90^{∘}) = #{[(k, l), (m, n)]||k − m| = d, l − n = 0, I(k, l) = i, I(m, n) = j} probability of 135^{∘}, d pixels apart pixels

P (i, j, d, 135^{∘}) = #{[(k, l), (m, n)]|[(k − m = d, l − n = d) or (k − m = −d,

l − n = −d)], I(k, l) = i, I(m, n) = j}

matrix symmetric: P (i, j, d, a) = p(j, i, d, a) =====Fig. 9.3===== entropy highest: when all entries in P_{ij} are equal entropy highest: image where no preferred gray-level pairs exist features calculated from the co-occurrence matrix =====Fig. 9.4===== gray-level co-occurrence matrix: widely used in remote sensing applications

for land-use classification

9.2.1 Generalized Gray Level Spatial Dependence Models for Texture simple generalization: consider more than two pixels at a time

9.3 Strong Texture Measures and Generalized Co-Occurrence generalized co-occurrence: consider co-occurrence between texture primitives texture primitive: connected set of pixels characterized by attribute list simplest texture primitive: pixel with gray level attribute more complicated primitive: connected set of pixels homogeneous in level primitive: characterized by size, elongation, orientation, average intensity

9.3.1 Spatial Relationships primitives constructed: list of primitives, center coordinates, attributes generalized co-occurrence matrix P

P (t_{1}, t_{2}) = \frac{#\{(q_{1}, q_{2}) ∈ S|f(q_{1}) = t_{1} and f(q_{2}) = t_{2}\}}{#S}

S ⊆ Q × Q: binary relation satisfying spatial relationship Q: set of all primitives on the image T: set of primitive properties t_{1}, t_{2}: properties which primitives have f: function assigning to each primitive in Q a property of T

9.4 Autocorrelation Function and Texture texture: relates to spatial size of gray level primitives on image gray level primitives of larger size: coarser texture gray level primitives of smaller size: finer texture autocorrelation function: describes the size of gray level primitives autocorrelation function

p(x, y) = \frac{\frac{1}{(L_{x}−|x|)(L_{y}−|y|)} ∫_{−∞}^{∞} ∫_{−∞}^{∞} I(u, v)I(u + x, v + y)dudv}{\frac{1}{L_{x}L_{y}} ∫_{−∞}^{∞} ∫_{−∞}^{∞} I^{2}(u, v)dudv} |x| < L_{x}, |y| < L_{y}

autocorrelation function p[k, l] for N × N image:

p[k, l] = \frac{\frac{1}{(N−k)(N−l)} ∑_{i=1}^{(N−k)} ∑_{j=1}^{(N−l)} f[i, j]f[i + k, j + l]}{\frac{1}{N^{2}} ∑_{i=1}^{N} ∑_{j=1}^{N} f^{2}[i, j]} 0 ≤ k, l ≤ N − 1

gray level primitives on image relatively large: texture coarse:

autocorrelation drops off slowly with distance gray level primitives on image relatively small: texture fine:

autocorrelation drops off quickly with distance autocorrelation function: measure of periodicity and scale of texture

9.5 Digital Transform Methods and Texture In the digital transform method of texture analysis, the digital image is typically divided into a set of nonoverlapping small square subimages Fourier transform: uses the complex sinusoid basic set

9.6 Texture Energy In the textural energy approach, the image is first convolved with a variety of kernels. Then each convolved image is processed with a nonlinear operator to determine the total textural energy in each pixel’s neighborhood.

9.7 Textural Edgeness autocorrelation, digital transforms: reference texture to spatial frequency texture: in terms of edgeness per unit area small neighborhoods: to detect microedges large neighborhoods: to detect macroedges =====Fig. 9.6=====

9.8 Vector Dispersion In the vector dispersion technique, the image texture is divided into mutually exclusive neighborhoods, and a sloped plane fit to the gray levels is performed for each neighborhood. =====Fig. 9.7=====

9.9 Relative Extrema Density a texture measure: number of extrema per unit area in any row of pixels, a pixel i is a relative minimum if

g(i) ≤ g(i + 1) and g(i) ≤ g(i − 1)

pixel i is relative maximum if

g(i) ≥ g(i + 1) and g(i) ≥ g(i − 1)

constant gray level run of pixels: minimum and maximum simultaneously

9.10 Mathematical Morphology granularity of the binary image F

G(d) = 1 − \frac{#F ∘H_{d}}{#F} = \frac{#F − #F ∘H_{d}}{#F}

#F: number of elements in F H_{d}: disk structuring element of diameter d G(d) measures the properties of grain pixels that cannot be contained in some translated structuring element of size d that is entirely contained in the grain and contains the given pixel G(d) measures the proportion of pixels participating in grains of a size smaller than d

scale-k volume of the blanket around a gray level intensity surface I V (k) = _{(r,c)}^{∑}(I ⊕_{k} H)(r, c) − (I ⊖_{k} H)(r, c)

⊕_{k}: k-fold dilation with the structuring element ⊖_{k}: k-fold erosion with the structuring element =====joke=====

9.11 Autoregression Models autoregression models for texture synthesis: next gray level value a_{N+1}: linear combination of synthesized and noise values

a_{N+1} = K−1 ; ∑ ; k=0 α_{k}a_{N−k} + L−1 ; ∑ ; l=0 β_{l}b_{N−l}

a_{k}: given starting sequence b_{k}: randomly generated noise image =====Fig. 9.8===== two-dimensional autoregressive model: better performance on general textures

a(i, j) = _{(k,l)∈N(i,j)}^{∑} α(i − k, l − j)a(k, l) + _{(k,l)∈N(i,j)}^{∑} β(i − k, l − j)b(k, l)

=====Fig. 9.9===== The power of the autoregression linear estimator approach is that it is easy to use the estimator in a mode that synthesizes textures from any initially given linear estimator. In this sense the autoregressive approach is sufficient to capture everything about a texture.

Its weakness is that the textures it can characterize are likely to consist mostly of microtextures

9.12 Discrete Markov Random Fields The Markov random field model for texture assumes that the texture field is stochastic and stationary and satisfies a conditional independence assumption. when the distributions are Gaussian, the discrete Gauss-Markov field can be written as an equation in which each pixel’s value is a linear combination of the values in its neighborhood plus a correlated noise term

I(r, c) = _{(i,j)∈N(0,0)}^{∑} I(r − i, c − j)h(i, j) + u(r, c)

function h: coefficients of linear combination model parameters h: computed from texture image with least-squares method u: joint set of possible correlated Gaussian random variables

9.13 Random Mosaic Models random mosaic models: constructed in two steps. first step: provides a means of tessellating a plane into cells second step: assigns a property value to each cell

9.14 Structural Approaches to Texture Models pure structural models: primitives in regular repetitive spatial arrangements to describe texture: describe primitives and placement rules Escher-inspired pattern with two of several possible candidate texels =====V. S. Nalwa, A Guided Tour of Computer Vision, Fig. 6.5=====

(a) simple texture formed by repeated placement of disks (b) corrupted =====R. Jain, Machine Vision, Fig. 7.3=====

9.15 Texture Segmentation The constraint is that each region has a homogeneous texture, such as that arising from a frontal view, and that each pair of adjacent regions is differently textured

9.16 Synthetic Texture Image Generation fractals: shapes that exhibit recursive self-similarity Every fractal can be recursively subdivided into smaller nonoverlapping shapes, each of which is a scaled-down version of the whole, either in a deterministic sense, or in a statistical sense fractals: shapes that exhibit recursive self-similarity =====V. S. Nalwa, A Guided Tour of Computer Vision, Fig. 6.4=====

9.17 Shape from Texture image texture gradients on oblique photography can be used to estimate surface orientation of the observed three-dimensional object assumption: no depth change and no texture changes within the observed area assumption: no subtextures results of integrated approach to identification of image texels =====V. S. Nalwa, A Guided Tour of Computer Vision, Fig. 6.9===== image textural variation as cue to three-dimensional shape =====V. S. Nalwa, A Guided Tour of Computer Vision, Fig. 6.6===== shape of golf ball recovered from the projective distortion of circles =====V. S. Nalwa, A Guided Tour of Computer Vision, Fig. 6.11=====

unknown plane where textural surface observed

Ax + By + Cz + D = 0

where A^{2} + B^{2} + C^{2} = 1 from perspective projection, 3D point (x, y, z) with projection (u, v)

u = f \frac{x}{z} v = f \frac{y}{z}

f: distance from the image plane to center of projection

x = \frac{uz}{f} y = \frac{vz}{f}

solving for z

z = \frac{−Df}{Au + Bv + Cf}

keep going on ... use similar triangle geometry =====Fig. 9.11=====

3D implications of discontinuities in image characteristics of spatial texture =====V. S. Nalwa, A Guided Tour of Computer Vision, Fig. 6.14=====

9.18 Summary texture: in terms of primitives and spatial relationships qualitatively: shape from texture can work quantitatively: techniques generally not dependable =====joke=====