Appendix A

Representation of Signals
in a Signal Space

A.1 LINEAR VECTOR SPACES

In order to facilitate the geometrical representation of signals, we treat them as vectors.
Indeed, signals can be considered to behave like vectors, as will be shown in the sequel. For
that purpose we recall the properties of linear vector spaces. A vector space is called a linear
vector space if it satisfies the following conditions:

1: xþy¼yþx ðA:1Þ
2: x þ ðy þ zÞ ¼ ðx þ yÞ þ z ðA:2Þ
3: ðx þ yÞ ¼ x þ y ðA:3Þ
4: ð þ Þx ¼ x þ x ðA:4Þ

where x and y are arbitrary vectors and  and  are scalars.

In an n-dimensional linear vector space we define a so-called inner product as

4
X
n
x  y¼ xi yi ðA:5Þ
i¼1

where xi and yi are the elements of x and y, respectively. Two vectors x and y are said to be
orthogonal if x  y ¼ 0. The norm of a vector x is denoted by k x k and we define it by

sffiffiffiffiffiffiffiffiffiffiffiffi
pffiffiffiffiffiffiffiffiffi
4
X n
kxk¼ xx¼ x2i ðA:6Þ
i¼1

Introduction to Random Signals and Noise Wim C. van Etten

# 2005 John Wiley & Sons, Ltd ISBN: 0-470-02411-9
216 APPENDIX A: REPRESENTATION OF SIGNALS IN A SIGNAL SPACE

This norm has the following properties:

5: k x k 0 ðA:7Þ
6: k x k ¼ 0 () x ¼ 0 ðA:8Þ
7: k x þ y kk x k þ k y k ðA:9Þ
8: k x k ¼ jj  k x k ðA:10Þ

In general, we can state that the norm of a vector represents the distance from an arbitrary
point described by the vector to the origin, or alternatively it is interpreted as the length of
the vector. From Equation (A.9) we can readily derive the Schwarz inequality

jx  yj  k x k  k y k ðA:11Þ

A.2 THE SIGNAL SPACE CONCEPT

In this section we consider signals defined on the time interval ½a; b. As in the case of
vectors, we define the inner product of two signals xðtÞ and yðtÞ, but now the definition reads
Z b
4
hxðtÞ; yðtÞi ¼ xðtÞyðtÞ dt ðA:12Þ
a

Note that using this definition, signals behave like vectors, i.e. they show the properties 1 to
8 as given in the preceding section. This is readily verified by considering the properties of
integrals.
Let us consider a set of orthonormal signals fi ðtÞg, i.e. signals that satisfy the condition

4 1; i ¼ j;
hi ðtÞ; j ðtÞi ¼ ij ¼ ðA:13Þ
0; i 6¼ j

where ij denotes the well-known Kronecker delta. When all signals of a specific class can
exactly be described as a linear combination of the members of such a signal set, we call it a
complete orthonormal signal set; here we take the class of square integrable signals. In this
case each arbitrary signal of that class is written as

X
n
sðtÞ ¼ si i ðtÞ ðA:14Þ
i¼1

and the sequence fsi g can also be written as a vector s, where the si are the elements of the
signal vector s. Figure A.1 shows a circuit that reconstructs sðtÞ from the set fsi g and the
orthonormal signal set fi ðtÞg; this circuit follows immediately from Equation (A.14).
No limitations are placed on the integer n; even an infinite number of elements is allowed.
The elements si are found by
Z b
si ¼ hsðtÞ; i ðtÞi ¼ sðtÞi ðtÞ dt ðA:15Þ
a
 THE SIGNAL SPACE CONCEPT 217

φ1(t)

s1

φ2(t)

s2
s(t)

.....
.....
.....

φn(t)

sn

Figure A.1 A circuit that produces the signal sðtÞ from its elements fsi g in the signal space

φ1(t)

b
(.) dt s1
a

φ2(t)

s(t) b
(.) dt s2
a
.....

.....

.....

φn(t)

b
(.) dt sn
a

Figure A.2 A circuit that produces the signal space elements fsi g from the signal sðtÞ

Therefore, when we construct a vector space using the vector s, we have a geometrical
representation of the signal sðtÞ. Along the ith axis we imagine the function i ðtÞ; i.e. the set
fi ðtÞg is taken as the basis for the signal space. In fact, si indicates to what extent i ðtÞ
contributes to sðtÞ. From Equation (A.15) a circuit is derived that produces the elements fsi g
representing the signal sðtÞ in the signal space fi ðtÞg; the circuit is given in Figure A.2.
The inner product of a signal vector with itself has an interesting interpretation:

Z b X
hsðtÞ; sðtÞi ¼ s2 ðtÞ dt ¼ k s k2 ¼ s2i ¼ Es ðA:16Þ
a i
218 APPENDIX A: REPRESENTATION OF SIGNALS IN A SIGNAL SPACE

with Es the energy of the signal sðtÞ. From this equation it is concluded that the length of a
vector in the signal space equals the square root of the signal energy. For purposes of
detection it is important to establish that the distance between two signal vectors represents
the square root of the energy of the difference of the two signals involved.
This concept of signal spaces is in fact a generalization of the well-known Fourier series
expansion of signals.

Example A.1:

As an example let us consider the harmonic signal sðtÞ ¼ Refa exp½jð!0 t þ Þg, where
Refg is the real part of the expression in the braces. This signal is written as

sðtÞ ¼ a cos cos !0 t  a sin sin !0 t ðA:17Þ

pffiffiffi pffiffiffiffi pffiffiffi pffiffiffiffi
As the orthonormal signal set we consider f 2 cosð!0 tÞ= T ;  2 sinð!0 tÞ= T g and the
time interval ½a; b is taken as ½0; T, with T ¼ k p
 ffiffiffi2p=!
ffi 0 and
pffiffikffi integer.
pffiffiffiffi In this
pffiffiffisignal space,
the signal sðtÞ is represented by the vector s ¼ ½ T a cos = 2; T a sin = 2. In fact, we
have introduced in this way an alternative for the signal representation of harmonic signals in
the complex plane. The elements of the vector s are recognized as the well-known quadrature
I and Q signals.
&

A.3 GRAM–SCHMIDT ORTHOGONALIZATION

When we have an arbitrary set ffi ðtÞg of, let us say, N signals, then in general these signals
will not be orthonormal. The Gram–Schmidt method shows us how to transform such a set
into an orthonormal set, provided the members of the set ffi ðtÞg are linearly independent, i.e.
none of the signal fi ðtÞ can be written as a linear combination of the other signals. The first
member of the orthonormal set is simply constructed as

4 f1 ðtÞ
1 ðtÞ ¼ pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ðA:18Þ
hf1 ðtÞ; f1 ðtÞi

In fact, the signal f1 ðtÞ is normalized to the square root of its energy, or equivalently to the
length of its corresponding vector in the signal space.
The second member of the orthonormal set is constructed by taking f2 ðtÞ and subtracting
from this f2 ðtÞ the part that is already comprised of 1 ðtÞ. In this way we arrive at the
intermediate signal
4
g2 ðtÞ ¼ f2 ðtÞ  hf2 ðtÞ; 1 ðtÞi1 ðtÞ ðA:19Þ

Due to this operation the signal g2 ðtÞ will be orthogonal to 1 ðtÞ. The functions 1 ðtÞ and 2 ðtÞ
will become orthonormal if we construct 2 ðtÞ from g2 ðtÞ by normalizing it by its own length:

g2 ðtÞ
2 ðtÞ ¼ pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ðA:20Þ
hg2 ðtÞ; g2 ðtÞi
 THE REPRESENTATION OF NOISE IN SIGNAL SPACE 219

Proceeding in this way we construct the kth intermediate signal by

4
X
k1
gk ðtÞ ¼ fk ðtÞ  hfk ðtÞ; j ðtÞij ðtÞ ðA:21Þ
j¼1

and by proper normalization we arrive at the kth member of the orthonormal signal set

4 gk ðtÞ
k ðtÞ ¼ pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ðA:22Þ
hgk ðtÞ; gk ðtÞi

This procedure is continued until N orthonormal signals have been constructed. In case there
are linear dependences, the dimensionality of the orthonormal signal space will be lower
than N.
The orthonormal space that results from the Gram–Schmidt procedure is not unique; it
will depend on the order in which the above described procedure is executed. Nevertheless,
the geometrical signal constellation will not alter and the lengths of the vectors are invariant
to the order chosen.

Example A.2:

Consider the signal set f fi ðtÞg given in Figure A.3(a). Since the norm of f1 ðtÞ is unity, we
conclude that 1 ðtÞ ¼ f1 ðtÞ. Moreover, from the figure it is deduced that the inner product
h f2 ðtÞ; 1 ðtÞi ¼ 0 and h f2 ðtÞ; f2 ðtÞi ¼ 4, which means that 2 ðtÞ ¼ f2 ðtÞ=2. Once this is
known, it is easily verified that h f3 ðtÞ; 1 ðtÞi ¼ 12 and h f3 ðtÞ; 2 ðtÞi ¼ 0, from which it
follows that g3 ðtÞ ¼ f3 ðtÞ  1 ðtÞ=2. The set of functions fgi ðtÞg has been depicted in
Figure A.3(b). From this figure we calculate k g3 k2 ¼ 14, so that 3 ðtÞ ¼ 2g3 ðtÞ, and finally
from all those results the signal set fi ðtÞg, as given in Figure A.3(c), can be constructed. In
this example the given functions do not show linear dependence and therefore the set fi ðtÞg
contains as many functions as the given set f fi ðtÞg.
&

A.4 THE REPRESENTATION OF NOISE IN SIGNAL SPACE

In this section we will confine our analysis to the widely used concept of wide-sense
stationary, zero mean, white, Gaussian noise. A sample function of the noise is denoted by
NðtÞ and the spectral density is N0 =2. We construct the noise vector n in the signal space,
where the elements of this noise vector are defined by

ni ¼ hNðtÞ; i ðtÞi ðA:23Þ

Since this integration is a linear operation, these noise elements will also show a Gaussian
distribution and it will be clear that the mean of ni equals zero, for all i. When, besides these
data, the cross-correlations of the different noise elements are determined, the noise
220 APPENDIX A: REPRESENTATION OF SIGNALS IN A SIGNAL SPACE

f2(t)

f1(t) f3(t)
2

1 1
1
0 1 t 0 ½ t 0 ¼ ¾ 1 t

−2
(a)
g2(t)
g1(t) g3(t)
2

1
1 ½
1
1 t ½ 0 ¼ ¾
0 0 t −½ t

−2
(b)

φ1(t) φ2(t) φ3(t)

1 1 1
1 1
0 1 t 0 ½ t 0 ¼ ¾ t
−1 −1
(c)

Figure A.3 The construction of a set of orthonormal functions fi ðtÞg from a set of given functions
ffi ðtÞg using the Gram–Schmidt orthogonalization procedure

elements are completely specified. Those cross-correlations are found to be

Z b Z b 
E½ni nj  ¼ E NðtÞ i ðtÞ dt Nð
Þ j ð
Þ d

a a
2 b 3
ZZ
¼ E4 NðtÞ Nð
Þ i ðtÞ j ð
Þ dt d
5
a

ZZ
b

¼ E½NðtÞ Nð
Þi ðtÞj ð
Þ dt d
ðA:24Þ
a
In this expression the expectation represents the autocorrelation function of the noise, which
reads RNN ðt;
Þ ¼ ðt 
ÞN0 =2. This is inserted in the last equation to arrive at
ZZb
N0
E½ni nj  ¼ ðt 
Þi ðtÞj ð
Þ dt d

2
a
Z
N0 b
¼ i ðtÞj ðtÞ dt ðA:25Þ
2 a
 THE REPRESENTATION OF NOISE IN SIGNAL SPACE 221

Remembering that the set fi ðtÞg is orthonormal, the following cross-correlations result
8
< N0
; for i ¼ j
E½ni nj  ¼ 2 ðA:26Þ
:
0; for i 6¼ j

From this equation it is concluded that the different noise elements ni are uncorrelated and,
since they are Gaussian with zero mean, they are independent. Moreover, all noise elements
show the same variance of N0 =2. This simple and symmetric result is another interesting
feature of the orthonormal signal spaces as introduced in this appendix.

A.4.1 Relevant and Irrelevant Noise

When considering a signal that is disturbed by noise we want to construct a common signal
space to describe both the signal and noise in the same space. In that case we construct a
signal space to completely describe all possible signals involved. When we want to attempt
to describe the noise NðtÞ using that signal space, it will, as a rule, be inadequate to comp-
letely characterize the noise. In that case we split the noise into one part Nr ðtÞ that is pro-
jected on to the signal space, called the relevant noise, and another part Ni ðtÞ that is
orthogonal to the space set up by the signals, called the irrelevant noise. Thus, the relevant
noise is given by the vector nr with components
Z b
nr;i ¼ NðtÞi ðtÞ dt ðA:27Þ
a

By definition the irrelevant noise reads

4
Ni ðtÞ ¼ NðtÞ  Nr ðtÞ ðA:28Þ

Next we will show that the irrelevant noise part is orthogonal to the signal space. For this
purpose let us consider a signal sðtÞ and an orthonormal signal space fi ðtÞg. Let us suppose
that sðtÞ can completely be described as a vector in this signal space. The inner product of
the irrelevant noise Ni ðtÞ and the signal reads
Z b Z b
Ni ðtÞsðtÞ dt ¼ fNðtÞ  Nr ðtÞgsðtÞ dt
a a
Z b Z b
¼ NðtÞsðtÞ dt  Nr ðtÞsðtÞ dt
a a
Z b X Z b X
¼ NðtÞ sk k ðtÞ dt  nr; k k ðtÞsðtÞ dt
a k a k
Z b X X
¼ sk NðtÞk ðtÞ dt  nr; k sk
a k k
X X
¼ sk nr; k  nr; k sk ¼ 0 ðA:29Þ
k k
222 APPENDIX A: REPRESENTATION OF SIGNALS IN A SIGNAL SPACE

For certain applications, for instance optimum detection of a known signal in noise, it
appears that the irrelevant part of the noise may be discarded.

A.5 SIGNAL CONSTELLATIONS

In this section we will present the signal space description of a few signals that are often met
in practice. The signal space with indications of possible signal realizations is called a signal
constellation. In detection, the error probability appears always to be a function of Ed =N0 ,
where Ed is the energy of the difference of signals. However, we learned in Section A.2 that
this energy is in the signal space represented by the squared distance of the signals involved.
Considering two signals si ðtÞ and sj ðtÞ, their squared distance is written as
Z b
dij2 ¼ k si  sj k2 ¼ ½si ðtÞ  sj ðtÞ2 dt ¼ Ei þ Ej  2Eij ðA:30Þ
a

where Ei and Ej are the energies of the corresponding signals and Eij represents the inner
product of the signals. In specific cases where Ei ¼ Ej ¼ E for all i and j, Equation (A.30) is
written as

dij2 ¼ 2Eð1  ij Þ ðA:31Þ

with the cross-correlations ij defined by

4 si  sj
ij ¼ ðA:32Þ
k si k  k sj k

A.5.1 Binary Antipodal Signals

Consider the two rectangular signals

rffiffiffiffi
E
s1 ðtÞ ¼ s2 ðtÞ ¼ ; 0tT ðA:33Þ
T

and their bandpass equivalents

rffiffiffiffiffiffi
2E
s1 ðtÞ ¼ s2 ðtÞ ¼ cos !0 t; 0tT ðA:34Þ
T

with T ¼ n  2p=!0 and n integer.

The cross-correlation of those signals equals 1 and the distance between the two
signals is
pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi pffiffiffiffi
d12 ¼ 2Eð1  12 Þ ¼ 2 E ðA:35Þ
 SIGNAL CONSTELLATIONS 223

s2 s1

E E

Figure A.4 The signal constellation of the binary antipodal signals

The space is one-dimensional,

pffiffiffiffi since
p ffiffiffiffi the two signals involved are dependent. The signal vec-
tors are s1 ¼ ½ E and s2 ¼ ½ E. This signal set is called an antipodal signal constellation
and is depicted in Figure A.4.

A.5.2 Binary Orthogonal Signals

Next we consider the signal set

rffiffiffiffiffiffi
2E
s1 ðtÞ ¼ cos !0 t; 0tT ðA:36Þ
T
rffiffiffiffiffiffi
2E
s2 ðtÞ ¼ sin !0 t; 0tT ðA:37Þ
T

where either T ¼ np=!0 (with n integer) or T  1=!0, so that 12 ¼ 0 or 12  0,

respectively. Due to this property those signals
pffiffiffiffiare called orthogonal
pffiffiffiffisignals. In this case
the signal space is two-dimensional and s1 ¼ ½ E; 0, whilepsffiffiffiffiffi
2 ffi¼ ½0; E . It is easily verified
that the distance between the signals amounts to d12 ¼ 2E. The signal constellation is
given in Figure A.5.

s2
d 12 = 2E

s1

Figure A.5 The signal constellation of the binary orthogonal signals

224 APPENDIX A: REPRESENTATION OF SIGNALS IN A SIGNAL SPACE

s2 s3

s4 s2

E
E

s3 s1 s5 s1

s6 s8
s4 s7

(a) (b)

Figure A.6 The signal constellation of M-phase signals

A.5.3 Multiphase Signals

pffiffiffiffi
In the multiphase case all possible signal vectors are on a circle with radius E. This cor-
responds to the M-ary phase modulation. The signals are represented by
(rffiffiffiffiffiffi  )
2E 2p
si ðtÞ ¼ Re exp j!0 t þ jði  1Þ ; for i ¼ 1; 2; . . . ; M; 0  t  T ðA:38Þ
T M

with the same requirement for the relation between T and !0 as in the former section. The
signal vectors are given by
 
pffiffiffiffi ði  1Þ2p pffiffiffiffi ði  1Þ2p
si ¼ E cos ; E sin ðA:39Þ
M M

In Figure A.6 two examples are depicted, namely M ¼ 4 in Figure A.6(a) and M ¼ 8 in
Figure A.6(b). The case of M ¼ 4 can be considered as a pair of two orthogonal signals;
namely the pair of vectors ½s1 ; s3  is orthogonal to the pair ½s2 ; s4 . For that reason this is a
special case of the biorthogonal signal set, which is dealt with later on in this section. This
orthogonality is used in QPSK modulation.

A.5.4 Multiamplitude Signals

The multiamplitude case is a straightforward extension of the antipodal signal constellation.
The extension is in fact a manifold, let us say M, of signal vectors on the one-dimensional
axis (see Figure A.4). In most applications these points are equidistant.
 SIGNAL CONSTELLATIONS 225

(a) (b)

Figure A.7 The signal constellation of QAM signals: (a) rectangular distribution; (b) circular
distribution

A.5.5 QAM Signals

A QAM (quadrature amplitude modulated) signal is a signal where both the amplitude and
phase are modulated. In that sense it is a combination of the multiamplitude and multi-
phase modulated signal. Different constellations are possible; two of them are depicted in
Figure A.7. Figure A.7(a) shows a rectangular grid of possible signal vectors, whereas in the
example of Figure A.7(b) the vectors are situated on circles. This signal constellation is used
in such applications as high-speed telephone modems, cable modems and digital distribution
of audio and video signals over CATV networks.

A.5.6 M-ary Orthogonal Signals

The M-ary orthogonal signal set is no more no less than an M-dimensional extension of the
binary orthogonal signal set; i.e. in this case the signal space has M dimensions and all
possible signals are orthogonal to all others. For M ¼ 3 and assuming that all signals bear
the same energy, the signal vectors are
pffiffiffiffi
s1 ¼ ½ E; 0; 0
pffiffiffiffi
s2 ¼ ½0; E; 0
pffiffiffiffi
s3 ¼ ½0; 0; E ðA:40Þ

This signal set has

ffi been illustrated in Figure A.8. The distance between two arbitrary signal
pffiffiffiffiffi
pairs is d ¼ 2E.

A.5.7 Biorthogonal Signals

An M-ary biorthogonal signal set is constructed from an M=2-ary orthogonal signal set by
simply adding the negatives of all the orthogonal signals. It will be clear that the dimension
226 APPENDIX A: REPRESENTATION OF SIGNALS IN A SIGNAL SPACE

s2

2E

2E

s1

2E
s3

Figure A.8 The signal constellation of the M-ary orthogonal signal set for M ¼ 3

s2 s2

−s3

−s 1 s1
−s 1 s1

s3
−s 2 −s 2

(a) (b)

Figure A.9 The signal constellation of the M-ary biorthogonal signal set for (a) M ¼ 4 and (b) M ¼ 6

of the signal space remains as M=2. The result is given in Figure A.9(a) for M ¼ 4 and in
Figure A.9(b) for M ¼ 6.

A.5.8 Simplex Signals

To explain the simplex signal set we start from a set of M orthogonal signals ffi ðtÞg with the
vector presentation ff i g. We determine the mean of this signal set

1X M
f¼ fi ðA:41Þ
M i¼1
 PROBLEMS 227

s2
2E

2E

s2 s1 s1

s3

(a) (b)

Figure A.10 The simplex signal set for (a) M ¼ 2 and (b) M ¼ 3

In order to arrive at the simplex signal set, this mean is subtracted from each vector of the
orthogonal set:

si ¼ f i  f ; i ¼ 1; 2; . . . ; M ðA:42Þ

In fact this operation means that the origin is translatedpto 

ffiffiffiffiffithe
ffi point f . Therefore the distance
between any pair of possible vectors si remains d ¼ 2E. Figure A.10 shows the simplex
signals for M ¼ 2 and M ¼ 3. Note that the dimensionality is reduced by 1 compared to the
starting orthogonal set.
Due to the transformation the signals are no longer orthogonal. On the other hand, it will
be clear that the mean signal energy has decreased. Since the distance between signal pairs
are the same as for orthogonal signals, the simplex signal set is able to realize communication
with the same quality (i.e. error probability) but using less energy per bit and thus less power.

A.6 PROBLEMS

A.1 Derive the Schwarz inequality (A.11) from the triangular inequality (A.9).
A.2 Use the properties of integration and the definition of Equation (A.12) to show that
signals satisfy the properties of vectors as given in Section A.1.
A.3 Use Equation (A.19) to show that g2 ðtÞ is orthogonal to 1 ðtÞ.
A.4 Show that for binary orthogonal signals one of the two given relations between T and
!0 is required for orthogonality.
A.5 Calculate the distance between adjacent signal vectors, i.e. the minimum distance, for
the M-ary phase signal.
A.6 Calculate the energy of the signals from the simplex signal set, expressed in terms of
the energy in the signals of the M-ary orthogonal signal set.
A.7 Calculate the cross-correlation between the various signals of the simplex signal set.