Doherty, J.F. “Channel Equalization as a Regularized Inverse Problem”
Digital Signal Processing Handbook
Ed. Vijay K. Madisetti and Douglas B. Williams
Boca Raton: CRC Press LLC, 1999
c

1999byCRCPressLLC
31
Channel Equalization as a
Regularized Inverse Problem
John F. Doherty
Pennsylvania State University
31.1 Introduction
31.2 Discrete-Time Intersymbol Interference Channel Model
31.3 Channel Equalization Filtering
Matrix Formulation of the Equalization Problem
31.4 Regularization
31.5 Discrete-Time Adaptive Filtering
Adaptive Algorithm Recapitulation
•
Regularization Proper-
ties of Adaptive Algorithms
31.6 Numerical Results
31.7 Conclusion
References
31.1 Introduction
In this article we examine the problem of communication channel equalization and how it relates
to the inversion of a linear system of equations. Channel equalization is the process by which the
effect of a band-limited channel may be diminished, i.e., equalized, at the sink of a communication
system. Although there are many ways to accomplish this, we will concentrate on linear filters and
adaptive filters. It is through the linear filter approach that the analogy to matrix inversion is possible.

It has been shown that a bandpass transmitted pulse train has an equivalent low pass representa-
tion [1]
s(t)=
∞

n=0
A
n
p(t − nT )
(31.2)
where
{
A
n
}
is the information bearing symbol set, p(t) is the equivalent low pass transmit pulse
waveform, and T is the symbol rate. The observed signal at the input of the receiver is
r(t) =
∞

n=0
A
n

+∞
−∞
p(t − nT )g(t − nT − τ)dτ + n(t)
(31.3)
where g(t) is the equivalent low pass bandlimited impulse response of the channel and the channel
noise, n(t), is modeled as white Gaussian noise. The optimum receiver filter, w(t), is the matched


+∞
−∞
n(t)w(t − τ)dτ is a filtered
c

1999 by CRC Press LLC
version of the channel noise. The input to the equalizer is a sampled version of Eq. (31.4), that is,
sampling at times t = kT produces
x(kT ) =
∞

n=0
A
n
h(kt − nT ) + ν(kT )
(31.7)
as the input to the discrete time equalizer. By normalizing with respect to the sampling interval and
rearranging terms, Eq. (31.7) becomes
x
k
= h
0
A
k

desired symbol
+
∞


d
L





=





x
T
1
x
T
2
.
.
.
x
T
L






T
(31.10)
where (·)
T
denotes the transpose operation. The received sample at time k is x
k
, which consists of the
channel output corrupted by additive noise. The elements of the N × 1 vector c
k
are the coefficients
of the equalizer filter at time k. The equalizer is said to be in decision directed mode when
˜
d
k
is taken
as the output of the nonlinear decision device. The equalizer is in training, or reference directed,
mode when
˜
d
k
is explicitly made identical to the transmitted sequence A
k
. In either case, e
k
is the
error between the desired equalizer output,
˜
d
k
, and the actual equalizer output, x


X
T
X

−1
X
T
(31.12)
c

1999 by CRC Press LLC
where X
#

=

X
T
X

−1
X
T
represents the Moore-Penrose (M-P) inverse of X. If one or more of the
eigenvalues of the matrix X
T
X is zero, then the Moore-Penrose inverse does not exist.
To investigate the behavior of the inverse, we will decompose the data matrix into the form X =
X

is the minimum value of the PSD, k
is the number of non-vanishing derivatives of S
R
at S
R min
, and N is the equalizer filter length. Any
spectral loss in the signal caused by the channel is directly translated into a corresponding decrease
in the minimum eigenvalue of the received signal. If λ
min
becomes small, but nonzero, the data
correlation matrix X
T
X becomes ill-conditioned and its inversion becomes sensitive to the noise.
The sensitivity is expressed in the quantity
δ

=


˜
c − c



c

≤
σ
2
n

σ
4
n

≈
σ
2
n
S
R min
(31.15)
The relation in Eq. (31.15) is an indicator of the potential numerical problems in solving for the
equalizer filter coefficients when the data is spectrally deficient.
We see that direct inversion of the data matrix is not recommendable when the channel has severe
spectral nulls. This situation is equivalent to stating that the original estimation problem d = Xc
is ill-posed. That is, the equalizer is asked to reproduce components of the channel input that are
unobservable at the channel output or are obscured by noise. Thus, it is reasonable to ascertain the
modes of the input dominated by noise and give them little weight, relative to the signal dominated
components, when solving for the equalizer filter coefficients. This process of weighting is called
regularization.
Regularization can be described by relying on a generalization of the M-P inverse that depends on
the singular value decomposition (SVD) of the data matrix
X = UV
T
(31.16)
where U is an L× N unitary matrix, V is an N × N unitary matrix,  = diag
(
σ
1
,σ

,σ
†
2
,...,σ
†
N

and
σ
†
i
=



σ
−1
i
σ
i
= 0
0 σ
i
= 0
(31.18)
c

1999 by CRC Press LLC