Part IV
APPLICATIONS OF ICA
Independent Component Analysis. Aapo Hyv
¨
arinen, Juha Karhunen, Erkki Oja
Copyright

2001 John Wiley & Sons, Inc.
ISBNs: 0-471-40540-X (Hardback); 0-471-22131-7 (Electronic)
21
Feature Extraction by ICA
A fundamental approach in signal processing is to design a statistical generative
model of the observed signals. The components in the generative model then give a
representation of the data. Such a representation can then be used in such tasks as
compression, denoising, and pattern recognition. This approach is also useful from a
neuroscientific viewpoint, for modeling the properties of neurons in primary sensory
areas.
In this chapter, we consider a certain class of widely used signals, which we
call natural images. This means images that we encounter in our lives all the time;
images that depict wild-life scenes, human living environments, etc. The working
hypothesis here is that this class is sufficiently homogeneous so that we can build a
statistical model using observations of those signals, and then later use this model for
processing the signals, for example, to compress or denoise them.
Naturally, we shall use independent component analysis (ICA) as the principal
model for natural images. We shall also consider the extensions of ICA introduced
in Chapter 20. We will see that ICA does provide a model that is very similar to
the most sophisticated low-level image representations used in image processing and
vision research. ICA gives a statistical justification for using those methods that have
often been more heuristically justified.
391
Independent Component Analysis. Aapo Hyv

a
i
(x y )s
i
(21.1)
where the
s
i
are stochastic coefficients, different for each image
I (x y )
. Alterna-
tively, we can just collect all the pixel values in a single vector
x =(x
1
x
2
 ::: x
m
)
T
,
in which case we can express the representation as
x = As
(21.2)
just like in basic ICA. We assume here that the number of transformed components
equals the number of observed variables, although this need not be the case in general.
This kind of a linear superposition model gives a useful description on a low level
where we can ignore such higher-level nonlinear phenomena as occlusion.
In practice, we may not model a whole image using the model in (21.1). Rather,
we apply it on image patches or windows. Thus we partition the image into patches

0
)
2
)cos(2(x  x
0
)+ )+i sin(2(x  x
0
)+ )]
(21.3)
where
 
is the constant in the gaussian modulation function, which determines the
width of the function in space.
 x
0
defines the center of the gaussian function, i.e., the location of the function.
 
is the frequency of oscillation, i.e., the location of the function in Fourier
space.
 
is the phase of the harmonic oscillation.
Actually, one Gabor function as in (21.3) defines two scalar functions: One as its real
part and the other one as its imaginary part. Both of these are equally important, and
the representation as a complex function is done mainly for algebraic convenience.
A typical pair of 1-D Gabor functions is plotted in Fig. 21.1.
Two-dimensional Gabor functions are created by first taking a 1-D Gabor function
along one of the dimensions and multiplying it by a gaussian envelope in the other
dimension:
g
2d


.
An open question is what set of values should one choose for the parameters to
obtain a useful representation of the data. Many different solutions exist; see, e.g.,
[103, 266]. The wavelet bases, discussed next, give one solution.
21.1.3 Wavelets
Another closely related method of multiresolution analysis is given by wavelets
[102, 290]. Wavelet analysis is based on a single prototype function called the mother
wavelet
(x)
. The basis functions (in one dimension) are obtained by translations
(x + l)
and dilations or rescalings
(2
s
x)
of this basic function. Thus we use the
family of functions

sl
(x)=2
s=2
(2
s
x  l)
(21.5)
The variables
s
and
l