Download the full version of the textbook now at textbookfull.

com

Theory and Computation of Tensors: Multi-

Dimensional Arrays 1st Edition Yimin Wei

https://textbookfull.com/product/theory-and-
computation-of-tensors-multi-dimensional-
arrays-1st-edition-yimin-wei/

Explore and download more textbook at https://textbookfull.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Multi dimensional Approaches Towards New Technology Ashish

Bharadwaj

https://textbookfull.com/product/multi-dimensional-approaches-towards-
new-technology-ashish-bharadwaj/

textbookfull.com

Compilation for Secure Multi party Computation 1st Edition

Niklas Büscher

https://textbookfull.com/product/compilation-for-secure-multi-party-
computation-1st-edition-niklas-buscher/

textbookfull.com

Simultaneous Systems of Differential Equations and Multi-

Dimensional Vibrations 1st Edition Luis Manuel Braga Da
Costa Campos
https://textbookfull.com/product/simultaneous-systems-of-differential-
equations-and-multi-dimensional-vibrations-1st-edition-luis-manuel-
braga-da-costa-campos/
textbookfull.com

Drugs in American Society Erich Goode

https://textbookfull.com/product/drugs-in-american-society-erich-
goode/

textbookfull.com
Ceramics and Society A Technological Approach to
Archaeological Assemblages Valentine Roux

https://textbookfull.com/product/ceramics-and-society-a-technological-
approach-to-archaeological-assemblages-valentine-roux/

textbookfull.com

The everything guide to spices for health a complete guide

to the natural health boosting benefits of everyday spices
1st Edition Michelle Robson-Garth
https://textbookfull.com/product/the-everything-guide-to-spices-for-
health-a-complete-guide-to-the-natural-health-boosting-benefits-of-
everyday-spices-1st-edition-michelle-robson-garth/
textbookfull.com

Fundamentals of Magnonics Sergio M. Rezende

https://textbookfull.com/product/fundamentals-of-magnonics-sergio-m-
rezende/

textbookfull.com

Case Studies in Immunology A Clinical Companion Raif Geha

https://textbookfull.com/product/case-studies-in-immunology-a-
clinical-companion-raif-geha/

textbookfull.com

From Smart City to Smart Region: Digital Services for an

Internet of Places 1st Edition Corinna Morandi

https://textbookfull.com/product/from-smart-city-to-smart-region-
digital-services-for-an-internet-of-places-1st-edition-corinna-
morandi/
textbookfull.com
Weapons of Math Destruction Cathy O’Neil

https://textbookfull.com/product/weapons-of-math-destruction-cathy-
oneil/

textbookfull.com
 Theory and
Computation
of Tensors
 Theory and
Computation
of Tensors
Multi-Dimensional Arrays

WEIYANG DING
YIMIN WEI

AMSTERDAM • BOSTON • HEIDELBERG • LONDON

NEW YORK • OXFORD • PARIS • SAN DIEGO
SAN FRANCISCO • SINGAPORE • SYDNEY • TOKYO
Academic Press is an imprint of Elsevier
Academic Press is an imprint of Elsevier
125 London Wall, London EC2Y 5AS, UK
525 B Street, Suite 1800, San Diego, CA 92101-4495, USA
50 Hampshire Street, 5th Floor, Cambridge, MA 02139, USA
The Boulevard, Langford Lane, Kidlington, Oxford OX5 1GB, UK

Copyright © 2016 Elsevier Ltd. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or by any means, electronic
or mechanical, including photocopying, recording, or any information storage and retrieval system,
without permission in writing from the publisher. Details on how to seek permission, further
information about the Publisher’s permissions policies and our arrangements with organizations such as
the Copyright Clearance Center and the Copyright Licensing Agency, can be found at our website:
www.elsevier.com/permissions.

This book and the individual contributions contained in it are protected under copyright by the
Publisher (other than as may be noted herein).

Notices
Knowledge and best practice in this field are constantly changing. As new research and experience
broaden our understanding, changes in research methods, professional practices, or medical treatment
may become necessary.

Practitioners and researchers must always rely on their own experience and knowledge in evaluating
and using any information, methods, compounds, or experiments described herein. In using such
information or methods they should be mindful of their own safety and the safety of others, including
parties for whom they have a professional responsibility.

To the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume
any liability for any injury and/or damage to persons or property as a matter of products liability,
negligence or otherwise, or from any use or operation of any methods, products, instructions, or ideas
contained in the material herein.

Library of Congress Cataloging-in-Publication Data

A catalog record for this book is available from the Library of Congress

British Library Cataloguing-in-Publication Data

A catalogue record for this book is available from the British Library

ISBN 978-0-12-803953-3 (print)

ISBN 978-0-12-803980-9 (online)

For information on all Academic Press publications

visit our website at https://www.elsevier.com/

Publisher: Glyn Jones

Acquisition Editor: Glyn Jones
Editorial Project Manager: Anna Valutkevich
Production Project Manager: Debasish Ghosh
Designer: Nikki Levy

Typeset by SPi Global, India

Contents

Preface v

I General Theory 1

1 Introduction and Preliminaries 3

1.1 What Arc Tensors? . 3
1.2 Basic Operations 6
1.3 Tensor Decompositions. 8
1.4 Tensor Eigenvalue Problems !O

2 Generalized Tensor Eigenvalue Problems 11

2.1 A Unified Framework. 11
2.2 Basic Definitions 13
2.3 Seventl Brlsic Properties 14
2.3.1 Number of Eigel1values . 14
2.3.2 Spectral R.adius . 15
2.3.3 Diagonalizable Tensor Pairs 15
2.3.4 Gcrshgori n Circle Theorem 16
2.3.5 Backward Error Analysis 19
2.4 Real Tensor Pairs. 20
2.4.1 The Crawford Number 21
2.4.2 Symmetric-Definite Tensor Pair s 22
2.5 Sign-Complex Spectral Radius 26
2.5.1 Definitions 26
2.5.2 Collatz-Wielalldt Formula 27
2.5.3 Properties for Single Tensors 29
2.5.4 The Componentwise Distance to Singularity. 31
2.5.5 Bauer-Fike TheOl'eul 33
2.6 An Illustrative Example 34

II Hankel Tensors 37

3 Fast Tensor-Vector Products 39

3.1 Hankel Tensors 39
3.2 Exponential Data Fit,ting 40
3.2.1 The One-Dimensional Case 40
3.2.2 The l\lultidimensiollal Case 42
3.3 Anti-Circulant Tensors 45
3.3.1 Diagonalization 46
3.3.2 Singular Vailles 47
viii CONTENTS

3.3.3 Block Tellsors . 49

3.4 Fast Hankel Tensor-Vector Product 50
3.5 Numerical Examples 53

4 Inheritance Properties 59
4.1 Inheritance Properties 59
4.2 The First luhcl'itance Property of Haukel Tensors. 61
4.2.1 A COllvolution Formula . 61
4.2.2 Lower-Order Implies Higher-Order 63
4.2.3 SOS Dccomposit.ioll of Strong Hankel Tensors. 65
11.3 The Second Inheritance Property of Hankel Tensors 66
4.3.1 Strong Hankel Tensors . . 66
4.3.2 A General Vandermolldc Decomposition of Hankel i\latrices 68
4.3.3 An Augmcnted Vandcnnonde Dccomposition of Hankel
Tensors 71
4.3.4 The Second Inherit.ance Propert.y of Hankel Tensors 75
4.4 The Third Inheritance Property of Hankel Tensors 77

III M-Tensors 79

5 Definitions and Basic Properties 81

5.1 Preliminaries 81
5.l.! Nonnegative Tensor 81
5.1.2 From Atf-l\fatl'ix to M-Tensol' 82
5.2 SpeclrR] Propert.ies of M-Tensors. 83
5.3 Semi-Positivity 84
5.3.1 Definitions 84
5.3.2 Semi-Positive Z-Tensol's 85
5.3.3 Proof of Theorem 5.7 87
5.3.4 General M-Tensors 89
5.4 l\lonotonicity 90
5.4.1 Defiuitions 90
5.4.2 Properties 90
5.4.3 A Count.er Example 93
5.4.4 A Nontrivial l\lollotone Z-Tensor 93
5.5 An Extension of M-Tensors 93
5.6 SummRtion 95

6 Multilinear Systems with M-Tensors 97

6.1 i\lotivat.ions 97
6.2 Triangular Equations. 99
6.3 M-Equatiolls and Beyond 102
6.3.1 M-Equations 102
6.3.2 Nonpositive R.ight-Hand Side 104
6.3.3 Nonhomog eneous Left-Hanel Side 105
6.3.4 Absolute M-Equations 106
6.3.5 Banded M-Equation . 107
6.4 Iterative i\lethods for M-Equations . 108
6.4.1 The Classical Iterations 109
CONTENTS ix

6.4.2 The Newton Method for Symmetric M-Equat. iolls 111

6.'1.3 Numerical Tests 112
6.5 Perturbation Analysis of M-Eql.lations ]]4
6.5,] Backwarrl. Errors of Triangular M-Equations 115
6.5.2 Condition Numbers 116
6.6 Inverse Iteration 118

Bibliography 125

Subject Index 135

 Preface
This book is devoted to the theory and computation of tensors, also called hyper-
matrices. Our investigation includes theories on generalized tensor eigenvalue
problems and two kinds of structured tensors, Hankel tensors and M-tensors.
Both theoretical analyses and computational aspects are discussed.
We begin with the generalized tensor eigenvalue problems, which are re-
garded as a unified framework of different kinds of tensor eigenvalue problems
arising from applications. We focus on the perturbation theory and the error
analysis of regular tensor pairs. Employing various techniques, we extend sev-
eral classical results from matrices or matrix pairs to tensor pairs, such as the
Gershgorin circle theorem, the Collatz-Wielandt formula, the Bauer-Fike the-
orem, the Rayleigh-Ritz theorem, backward error analysis, the componentwise
distance of a nonsingular tensor to singularity, etc.
In the second part, we focus on Hankel tensors. We first propose a fast algo-
rithm for Hankel tensor-vector products by introducing a special class of Hankel
tensors that can be diagonalized by Fourier matrices, called anti-circulant ten-
sors. Then we obtain a fast algorithm for Hankel tensor-vector products by em-
bedding a Hankel tensor into a larger anti-circulant tensor. The computational
complexity is reduced from O(nm ) to O(m2 n log mn). Next, we investigate the
spectral inheritance properties of Hankel tensors by applying the convolution
formula of the fast algorithm and an augmented Vandermonde decomposition of
strong Hankel tensors. We prove that if a lower-order Hankel tensor is positive
semidefinite, then a higher-order Hankel tensor with the same generating vector
has no negative H-eigenvalues, when (i) the lower order is 2, or (ii) the lower
order is even and the higher order is its multiple.
The third part is contributed to M-tensors. We attempt to extend the
equivalent definitions of nonsingular M -matrices, such as semi-positivity, mono-
tonicity, nonnegative inverse, etc., to the tensor case. Our results show that
the semi-positivity is still an equivalent definition of nonsingular M-tensors,
while the monotonicity is not. Furthermore, the generalization of the “nonneg-
ative inverse” property inspires the study of multilinear system of equations.
We prove the existence and uniqueness of the positive solutions of nonsingular
M-equations with positive right-hand sides, and also propose several iterative
methods for computing the positive solutions.
We would like to thank our collaborator Prof. Liqun Qi of the Hong Kong
Polytechnic University, who leaded us to the research of tensor spectral theory
and always encourages us to explore the topic. We would also like to thank
Prof. Eric King-wah Chu of Monash University and Prof. Sanzheng Qiao of
McMaster University, who read this book carefully and provided feedback during
the writing process.
This work was supported by the National Natural Science Foundation of
China under Grant 11271084, School of Mathematical Sciences and Key Labo-
ratory of Mathematics for Nonlinear Sciences, Fudan University.
Chapter 1

Introduction and
Preliminaries

We first introduce the concepts and sources of tensors in this chapter. Several
essential and frequently used operations involving tensors are also included. Fur-
thermore, two basic topics, tensor decompositions and tensor eigenvalue prob-
lems, are briefly discussed at the end of this chapter.

1.1 What Are Tensors?

The term tensor or hypermatrix in this book refers to a multiway array. The
number of the dimensions of a tensor is called its order, that is, A = (ai1 i2 ...im )
is an mth -order tensor. Particularly, a scalar is a 0th -order tensor, a vector is
a 1st -order tensor, and a matrix is a 2nd -order tensor. As other mathematical
concepts, tensor or hypermatrix is abstracted from real-world phenomena and
other scientific theories. Where do the tensors arise? What kinds of properties
do we care most? How many different types of tensors do we have? We will
briefly answer these questions employing several illustrative examples in this
section.

Example 1.1. As we know, a table is one of the most common realizations of

a matrix. We can also understand tensors or hypermatrices as complex tables
with multivariables. For instance, if we record the scores of 4 students on 3
subjects for both the midterm and final exams, then we can design a 3rd -order
tensor S of size 4 × 3 × 2 whose (i, j, k) entry sijk denotes the score of the i-th
student on the j-th subjects in the k-th exam. This representation is natural
and easily understood, thus it is a convenient data structure for construction
and query. However, when we need to print the information on a piece of paper,
the 3D structure is apparently not suitable for 2D visualization. Thus we need
to unfold the cubic tensor into a matrix. The following two different unfoldings
of the same tensor both include all the information in the original complex
table. We can see from the two tables that their entries are the same up to a
permutation. Actually, there are many different ways to unfold a higher-order
tensor into a matrix, and the linkages between them are permutations of indices.

Theory and Computation of Tensors. 3

http://dx.doi.org/10.1016/B978-0-12-803953-3.50001-0
Copyright © 2016 Elsevier Ltd. All rights reserved.
4 CHAPTER 1. INTRODUCTION AND PRELIMINARIES

Sub. 1 Sub. 2 Sub. 3

Mid Final Mid Final Mid Final
Std. 1 s111 s112 s121 s122 s131 s132
Std. 2 s211 s212 s221 s222 s231 s232
Std. 3 s311 s312 s321 s322 s331 s332
Std. 4 s411 s412 s421 s422 s431 s432

Table 1.1: The first way to print S.

Mid Final
Sub. 1 Sub. 2 Sub. 3 Sub. 1 Sub. 2 Sub. 3
Std. 1 s111 s121 s131 s112 s122 s132
Std. 2 s211 s221 s231 s212 s222 s232
Std. 3 s311 s321 s331 s312 s322 s332
Std. 4 s411 s421 s431 s412 s422 s432

Table 1.2: The second way to print S.

Example 1.2. Another important realization of tensors are the storage of color
images and videos. A black-and-white image can be stored as a greyscale matrix,
whose entries are the greyscale values of the corresponding pixels. Color images
are often built from several stacked color channels, each of which represents
value levels of the given channel. For example, RGB images are composed of
three independent channels for red, green, and blue primary color components.
We can apply a 3rd -order tensor P to store an RGB image, whose (i, j, k) entry
denotes the value of the k-th channel in the (i, j) position. (k = 1, 2, 3 represent
the red, green, and blue channel, respectively.) In order to store a color video,
we may need an extra index for the time axis. That is, we employ a 4th -order
tensor M = (mijkt ), where M(:, :, :, t) stores the t-th frame of the video as a
color image.
Example 1.3. Denote x = (x1 , x2 , . . . , xn )> ∈ Rn . As we know, a degree-1
polynomial p1 (x) = c1 x1 + c2 x2 + · · · + cn xn can be rewritten into p1 (x) = x> c,
where the vector c = (c1 , c2 , . . . , cn )> . Similarly, a degree-2 polynomial p2 (x) =
P n >
i,j=1 cij xi xj , that is, a quadratic form, can be simplified into p2 (x) = x Cx,
th
where the matrix C = (cij ). By analogy, if we denote an m -order tensor
C = (ci1 i2 ...im ) and apply a notation, which will be introduced in the next
section, then the degree-m homogeneous polynomial
n X
X n n
X
pm (x) = ··· ci1 i2 ...im xi1 xi2 . . . xim
i1 =1 i2 =1 im =1

can be rewritten as
pm (x) = Cxm .
Moreover, x> c = 0 is often used to denote a hyperplane in Rn . Similarly,
Cxm = 0 can stand for an degree-m hypersurface in Rn . We shall see in Section
1.2 that the normal vector at a point x0 on this hypersurface is nx0 = Cxm−1
0 .
Visit https://textbookfull.com
now to explore a rich
collection of eBooks, textbook
and enjoy exciting offers!
1.1. WHAT ARE TENSORS? 5

Example 1.4. The Taylor expansion is a well-known mathematical tool. The

Taylor series of a real or complex-valued function f (x) that is infinitely differ-
entiable at a real or complex number a is the power series
∞
X 1 (m)
f (x) = f (a)(x − a)m .
m=0
m!

A multivariate function f (x1 , x2 , . . . , xn ) that is infinitely differentiable at a

point (a1 , a2 , . . . , an ) also has its Taylor expansion
∞ ∞
X X (x1 − a1 )i1 . . . (xn − an )in ∂ i1 +···+in f (a1 , . . . , an )
f (x1 , . . . , xn ) = ··· ,
i1 =0 i =0
n
i1 ! · · · in ! ∂xi11 . . . ∂xinn
which is equivalent to
n
X ∂f (a1 , . . . , an )
f (x1 , . . . , xn ) = f (a1 , . . . , an ) + (xi − ai )
i=1
∂xi
n n
1 X X ∂ 2 f (a1 , . . . , an )
+ (xi − ai )(xj − aj )
2! i=1 j=1 ∂xi ∂xj
n n n
1 X X X ∂ 3 f (a1 , . . . , an )
+ (xi − ai )(xj − aj )(xk − ak ) + · · · .
3! i=1 j=1 ∂xi ∂xj ∂xk
k=1

Denoting x = (x1 , x2 , . . . , xn )> and a = (a1 , a2 , . . . , an )> , then we can rewrite

the second and the third terms in the above equation as
(x − a)> ∇f (a) and (x − a)> ∇2 f (a)(x − a),
where ∇f (a) and ∇2 f (a) are the gradient and the Hessian of f (x) at a, re-
spectively. If we define the mth -order gradient tensor ∇m f (a) of f (x) at a
as
∂ m f (a1 , a2 , . . . , an )
∇m f (a) i1 i2 ...im =


,
∂xi1 ∂xi2 . . . ∂xim
then the Taylor expansion of a multivariate function can also be expressed by
∞
X 1 m
f (x) = ∇ f (a)(x − a)m .
m=0
m!
Example 1.5. The discrete-time Markov chain is one of the most important
models of random processes [10, 28, 81, 114], which assumes that the future is
dependent solely on the finite past. This model is so simple and natural that
Markov chains are widely employed in many disciplines, such as thermodynam-
ics, statistical mechanics, queueing theory, web analysis, economics, and finance.
An sth -order Markov chain is a stochastic process with the Markov property,
that is, a sequence of variables {Yt }∞
t=1 satisfying

Pr(Yt = i1 | Yt−1 = i2 , . . . , Y1 = it ) = Pr(Yt = i1 | Yt−1 = i2 , . . . , Yt−s = is+1 ),

for all t > s. That is, any state depends solely on the immediate past s states.
Particularly, when the step length s = 1, the sequence {Yt }∞
t=1 is a standard first-
order Markov chain. Define the transition probability matrix P of an second-
order Markov chain as
pij = Pr(Yt = i | Yt−1 = j),
6 CHAPTER 1. INTRODUCTION AND PRELIMINARIES

which is a stochastic matrix, that is, pij ≥ 0 for all i, j = 1, 2, . . . , n and

Pn
i=1 pij = 1 for all j = 1, 2, . . . , n. The probability distribution of Yt is denoted
e
by a vector
(xt )i = Pr(Yt = i).
Then the Markov chain is modeled by xt+1 = P xt , thus the stationary probabil-
ity distribution x satisfies x = P x and is exactly an eigenvector of the transition
probability matrix corresponding to the eigenvalue 1.
For higher-order Markov chains, we have similar formulations. Take a second-
order Markov chain, that is, s = 2, as an example. Define the transition proba-
bility tensor P of a second-order Markov chain as

pijk = Pr(Yt = i | Yt−1 = j, Yt−2 = k),

Pn
which is a stochastic tensor, that is, P ≥ 0 and i=1 pijk = 1 for all j, k =
1, 2, . . . , n. The probability distribution of Yet in the product space can be re-
shaped into a matrix

(Xt )i,j = Pr(Yt = i, Yt−1 = j).

Then the stationary probability distribution in the product space X satisfies

n
X
X(i, j) = P (i, j, k) · X(j, k)
k=1

for all i, j = 1, 2, . . . , n. If we further assume that

Pr(Yt = i, Yt−1 = j) = Pr(Yt = i) · Pr(Yt−1 = j)

for all i, j = 1, 2, . . . , n and denote (xt )i = Pr(Yt = i), that is, Xt = xt x>
t , then
the stationary probability distribution x satisfies x = Px2 [74]. We shall see in
Section 1.2 that x is a special eigenvector of the tensor P.

From the above examples, we can gain some basic ideas about what tensors
are and where they come from. Generally speaking, there are two kinds of
tensors: the first kind is a data structure, which admits different dimensions
according to the complexity of the data; the second kind is an operator, where
it possesses different meanings in different situations.

1.2 Basic Operations

We first introduce several basic tensor operations that will be frequently referred
to in the book. One of the difficulties of tensor research is the complicated
indices. Therefore we often use some small-size examples rather than exact
definitions in this section to describe those essential concepts more clearly. For
more detailed definitions, we refer to Chapter 12 in [49] and the references
[18, 66, 98].

• To treat or visualize the multidimensional structures, we often reshape a

higher-order tensor into a vector or a matrix, which are more familiar to
1.2. BASIC OPERATIONS 7

us. The vectorization operator vec(·) turns tensors into column vectors.
Take a 2 × 2 × 2 tensor A = (aijk )2i,j,k=1 for example, then

vec(A) = (a111 , a211 , a121 , a221 , a112 , a212 , a122 , a222 )> .

There are a lot of different ways to reshape tensors into matrices, which
are often referred to as “unfoldings.” The most frequently applied one
is call the modal unfolding. The mode-k unfolding A(k) of an mth -order
tensor A of size n1 × n2 × · · · × nm is an nk -by-(N/nk ) matrix, where
N = n1 n2 . . . nm . Again, use the above 2 × 2 × 2 example. Its mode-1,
mode-2, and mode-3 unfoldings are
 
a111 a121 a112 a122
A(1) = ,
a211 a221 a212 a222
 
a111 a211 a112 a212
A(2) = ,
a121 a221 a122 a222
 
a111 a211 a121 a221
A(3) = ,
a112 a212 a122 a222
respectively. Sometimes the mode-k unfolding is also denoted as Unfoldk (·).
• The transposition operation of a matrix is understood as the exchange
of the two indices. But higher-order tensors have more indices, thus we
have much more transpositions of tensors. If A is a 3rd -order tensor,
then there are six possible

transpositions denoted as A<[σ(1),σ(2),σ(3)]> ,
where σ(1), σ(2), σ(3) is any of the six permutations of (1, 2, 3). When
B = A<[σ(1),σ(2),σ(3)]> , it means

biσ(1) iσ(2) iσ(3) = ai1 i2 i3 .

If all the entries of a tensor are invariant under any permutations of the
indices, then we call it a symmetric tensor. For example, a 3rd -order tensor
is said to be symmetric if and only if

A<[1,2,3]> = A<[1,3,2]> = A<[2,1,3]> = A<[2,3,1]> = A<[3,1,2]> = A<[3,2,1]> .

• Modal tensor-matrix multiplications are essential in this book, which are

generalizations of matrix-matrix multiplications. Let A be an mth -order
tensor of size n1 × n2 × · · · × nm and M be a matrix of size nk × n0k , then
the mode-k product A ×k M of the tensor A and the matrix M is another
mth -order tensor of size n1 × · · · × n0k × · · · × nm with
nk
X
(A ×k M )i1 ...ik−1 jk ik+1 ...im = ai1 ...ik−1 ik ik+1 ...im · mik jk .
ik =1

Particularly, if A, M1 and M2 are all matrices, then A ×1 M1 ×2 M2 =

M1> AM2 . Easily verified, the tensor-matrix multiplications satisfy that
1. A ×k Mk ×l Ml = A ×l Ml ×k Mk , if k 6= l,
2. A ×k M1 ×k M2 = A ×k (M1 M2 ),
8 CHAPTER 1. INTRODUCTION AND PRELIMINARIES

3. A ×k (α1 M1 + α2 M2 ) = α1 A ×k M1 + α1 A ×k M1 ,
4. Unfoldk (A ×k M ) = M > A(k) ,
5. vec(A ×1 M1 ×2 M2 · · · ×m Mm ) = (Mm ⊗ · · · ⊗ M2 ⊗ M1 )> vec(A),
where A is a tensor, Mk are matrices, and α1 , α2 are scalars.
• If the matrices degrade into column vectors, then we obtain another cluster
of important notations for tensor spectral theory. Let A be an mth -order
n-dimensional tensor, that is, of size n × n × · · · × n, and x be a vector of
length n, then for simplicity:
Axm = A ×1 x ×2 x ×3 x · · · ×m x is a scalar,
Axm−1 = A ×2 x ×3 x · · · ×m x is a vector,
m−2
Ax =A ×3 x · · · ×m x is a matrix.

• Like the vector case, an inner product of two tensors A and B of the same
size are defined by
n1
X nm
X
hA, Bi = ··· ai1 i2 ...im · bi1 i2 ...im ,
i1 =1 im =1

which is exactly the usual inner product of the two vectors vec(A) and
vec(B).
• The outer product of two tensors is a higher-order tensor. Let A and
B be mth -order and (m0 )th -order tensors, respectively. Then their outer
product A ◦ B is an (m + m0 )th -order tensor with
(A ◦ B)i1 ...im j1 ...jm0 = ai1 i2 ...im · bj1 j2 ...jm0 .
If a and b are vectors, then a ◦ b = ab> .
• We sometimes refer to the Hadamard product of two tensors with the
same size as
⊗ B)i1 i2 ...im = ai1 i2 ...im · bi1 i2 ...im .
(AHAD
The Hadamard product will also be denoted as A. ∗ B in the descriptions
of some algorithms, which is a MATLAB-type notation.

1.3 Tensor Decompositions

Given a tensor, how can we retrieve the information hidden inside? One reason-
able answer is the tensor decomposition approach. Existing tensor decomposi-
tions include the Tucker-type decompositions, the CANDECOMP/PARAFAC
(CP) decomposition, tensor train representation, etc. For those readers inter-
ested in tensor decompositions, we recommend the survey papers [50, 66]. Some
tensor decompositions are generalizations of the singular value decomposition
(SVD) [49].
The SVD is one of the most important tools for matrix analysis and compu-
tation. Any matrix A ∈ Rm×n with rank r has the decomposition
A = U ΣV > ,
1.3. TENSOR DECOMPOSITIONS 9

where U ∈ Rm×r and V ∈ Rn×r are column orthogonal matrices, that is,
U > U = Im and V > V = In , and Σ = diag(σ1 , σ2 , . . . , σr ) is a positive diagonal
matrix. Then σk are called the singular values of A. If U = [u1 , u2 , . . . , ur ] and
V = [v1 , v2 , . . . , vr ], then the SVD can be rewritten asinto
r
X
A= σi ui vi> ,
i=1

which represents the matrix A as a sum of several rank-one matrices. For an

arbitrary nonsingular matrix M ∈ Rr×r , denote L = U ΣM > ∈ Rm×r and
R = V M −1 ∈ Rn×r . Then
A = LR>
is a low-rank decomposition if r is much smaller than m and n. These three
equivalent formulas of the SVD are all extended to the higher-order tensors.
Let A be an mth -order tensor of size n1 × n2 × · · · × nm . The Tucker-type
decompositions of the tensor A has the form
A = S ×1 U1> ×2 U2> · · · ×m Um
>
,
where S is an mth -order tensor of size r × r × · · · × r, called the core tensor,
and Uk ∈ Rn×r for k = 1, 2, . . . , m. Note that the first formula A = U ΣV >
of the SVD can be rewritten into this form A = Σ ×1 U > ×2 V > . If there
are no restrictions on S and Uk , then we have an infinite number of Tucker-
type decompositions, most of which are no more informative. The higher-order
singular value decomposition (HOSVD), proposed in [33, 35], is a special Tucker-
type decomposition, which satisfies that
• the core S is all-orthogonal (hSik =α , Sik =β i = 0 for all k = 1, 2, . . . , m,
α 6= β) and ordered (kSik =1 kF ≥ kSik =2 kF ≥ · · · ≥ kSik =nk kF for all
k = 1, 2, . . . , m); and
• the matrices Uk are column orthogonal.
Actually, the matrix Uk consists of the left singular vectors of the mode-k un-
folding A(k) , and kSik =1 kF , kSik =2 kF , . . . , kSik =nk kF are exactly the singular
values of A(k) . One can easily verify that if a matrix is all-orthogonal then it
must be a diagonal matrix. Thus the 2nd -order version of the HOSVD is exactly
the SVD.
Nevertheless, the core tensor S of the HOSVD is never sparse in the higher-
order cases, thus complicated and obscure. If we restrict the core tensor to be
diagonal, that is, the entries except for sii...i (i = 1, 2, . . . , r) are all zero, then
we may relax the restriction on the number of terms r. Denote the diagonal
entries of S as σ1 , σ2 , . . . , σr , and Uk = [uk1 , uk2 , . . . , ukr ] for all k = 1, 2, . . . , m.
The CP decomposition of the tensor A is referred to as
r
X
A= σi u1i ◦ u2i ◦ · · · ◦ umi ,
i=1

where r might be larger than n. Each term u1i ◦ u2i ◦ · · · ◦ umi in the CP
decomposition is called a rank-one tensor. The least number of the terms, that
is,
n Xr o
R = min r : A = σi u1i ◦ u2i ◦ · · · ◦ umi ,
i=1
10 CHAPTER 1. INTRODUCTION AND PRELIMINARIES

is called the CP-rank of the tensor A. The computation or estimation of the

CP-ranks of higher-order tensors is NP-hard [54], and still a hard task in tensor
research.
The tensor train format (TT) [92, 91] for higher-order tensors extends the
low-rank decomposition of matrices. Let A be an mth -order tensor. Then it can
be expressed in the following format
r1 rm−1
X X
ai1 i2 ...im = ··· (G1 )i1 k1 (G2 )k1 i2 k2 · · · (Gm−1 )km−2 im−1 km−1 (Gm )km−1 im
k1 =1 km−1 =1

for all i1 , i2 , . . . , im , where G1 and Gm are matrices and G2 , . . . , Gm−1 are 3rd -
order tensors. If r1 , r2 , . . . , rm−1 are much smaller than the tensor size, then
the TT representation will greatly reduce the storage cost for the tensor.

1.4 Tensor Eigenvalue Problems

We have introduced the tensor-vector products in the previous sections. Given
an mth -order n-dimensional square tensor A, notice that x 7→ Axm−1 is a non-
linear operator from Cn to itself. Thus we should consider some characteristic
values of this operator. For this sake, we can define several tensor eigenvalue
problems.
First, we introduce a homogeneous tensor eigenvalue problem. If a scalar
λ ∈ C and a nonzero vector x ∈ Cn satisfy
Axm−1 = λx[m−1] ,
where x[m−1] = [xm−1
1 , xm−1
2 , . . . , xm−1
n ]> , then λ is called an eigenvalue of A
and x is called a corresponding eigenvector. Furthermore, when the tensor A,
the scalar λ and the vector x are all real, we call λ an H-eigenvalue of A and x a
corresponding H-eigenvector. For a real symmetric tensor A, its H-eigenvectors
are exactly the KKT points of the polynomial optimization problem
max/min Axm ,
s.t. xm m m
1 + x2 + · · · + xn = 1.

This was the original motivation when Qi [98] and Lim [76] first introduced this
kind of eigenvalue problem.
We also have other nonhomogeneous tensor eigenvalue definitions. For ex-
ample, if a scalar λ ∈ C and a nonzero vector x ∈ Cn satisfy
Axm−1 = λx with x> x = 1,
then λ is called an E-eigenvalue of A and x is called a corresponding E-
eigenvector [99]. Furthermore, when the tensor A, the scalar λ, and the vector x
are all real, we call λ an Z-eigenvalue of A and x a corresponding Z-eigenvector.
For a real symmetric tensor A, its Z-eigenvectors are exactly the KKT points
of the polynomial optimization problem
max/min Axm ,
s.t. x21 + x22 + · · · + x2n = 1.
We will introduce more definitions of tensor eigenvalues in Chapter 2, which
can be unified into a generalized tensor eigenvalue problem Axm−1 = λBxm−1 .
Further discussions will be conducted on this unified framework.
Chapter 2

Generalized Tensor
Eigenvalue Problems

Generalized matrix eigenvalue problems are essential in scientific and engineer-

ing computations. The generalized eigenvalue formula Ax = λBx is also a
very pervasive model, which covers the matrix polynomial eigenvalue problems.
There has been extensive study of both the theories and the computations of
generalized matrix eigenvalue problems, and one can refer to [6, 49, 113] for
more details. Applying the notations introduced in Chapter 1, the generalized
tensor eigenvalue problem is referred to finding λ and x satisfying

Axm−1 = λBxm−1 .

The detailed definition will be introduced later.

2.1 A Unified Framework

Since Qi [98] and Lim [76] proposed the definition of tensor eigenvalues inde-
pendently in 2005, a great deal of mathematical effort has been devoted to
the study of the eigenproblems of single tensors, especially for the nonnegative
tensors [18, 19]. Nevertheless, there has been much less specific research on the
generalized eigenproblems, since Chang et al. [19, 20] introduced the generalized
tensor eigenvalues. Recently, Kolda and Mayo [68], Cui et al. [30], and Chen
et al. [25] proposed numerical algorithms for generalized tensor eigenproblems.
Kolda and Mayo [68] applied an adaptive shifted power method by solving the
optimization problem
Axm
max kxkm .
kxk=1 Bxm

In Cui, Dai, and Nie’s paper [30], they impose the constraint λk+1 < λk − δ
when the k-th eigenvalue is obtained, so that the (k + 1)-th eigenvalue can also
be computed by the same method as the previous ones. Chen, Han, and Zhou
[25] proposed homotopy methods for generalized tensor eigenvalue problems.
It was pointed out in [19, 20, 30, 68] that the generalized eigenvalue frame-
work unifies several definitions of tensor eigenvalues, such as the eigenvalues and

Theory and Computation of Tensors. 11

http://dx.doi.org/10.1016/B978-0-12-803953-3.50002-2
Copyright © 2016 Elsevier Ltd. All rights reserved.
12 CHAPTER 2. GENERALIZED TENSOR EIGENVALUE PROBLEMS

the H-eigenvalues [98], that is,

Axm−1 = λx[m−1] , where x[m−1] = [xm−1

1 , xm−1
2 , . . . , xm−1
n ]> ,

the E-eigenvalues and the Z-eigenvalues [99], that is,

Axm−1 = λx with x> x = 1,

and the D-eigenvalues [103], that is,

Axm−1 = λDx with x> Dx = 1,

where D is a positive definite matrix. We shall present several other potential

applications of generalized tensor eigenvalue problems.
The first example is the higher-order Markov chain [28]. Li and Ng [74]
reformulated an (m − 1)st -order Markov chain problem into searching for a
nonnegative vector x em−1 = x
e satisfying P x e with x e2 + · · · + x
e1 + x en = 1, where
P is the transition probability tensor of the Markov chain. Let E be a tensor
such that

(Exm−1 )i = xi (x1 + x2 + · · · + xn )m−2 (i = 1, 2, . . . , n)

for an arbitrary vector x = (x1 , x2 , . . . , xn )> . Thus a nonnegative vector xe that

satisfies the homogenous equality P x em−1 = E x em−1 is exactly what is required.
The US-eigenvalue [85] from quantum information processing is another
special case of the generalized tensor eigenvalue. Let S be an mth -order n-
dimensional symmetric tensor. Then the US-eigenvalues are defined by

S̄xm−1 = λx̄,

with kxk2 = 1.
S x̄m−1 = λx,

Denote Se as an mth -order (2n)-dimensional tensor with

S(1
e : n, . . . , 1 : n) = S̄ and S(n
e + 1 : 2n, . . . , n + 1 : 2n) = S,

e = (x> , x̄> )> . When m is even, let J be a tensor such that

and x

yi+n (y1 yn+1 + y2 yn+2 + · · · + yn y2n )m/2−1 , i = 1, 2, . . . , n,


m−1
(J y )i =
yi−n (y1 yn+1 + y2 yn+2 + · · · + yn y2n )m/2−1 , i = n + 1, . . . , 2n,

for an arbitrary vector y ∈ C2n . Noticing that x e1 x

en+1 +e en+2 +· · ·+e
x2 x xn x
e2n = 1,
we rewrite the definition of the US-eigenvalue into Sex em−1 = λJ x em−1 .
Another potential application is from multilabel learning, where the hyper-
graphs are naturally involved [116]. The Laplacian tensor of a hypergraph is
widely studied in [29, 58, 73]. Denote L as the Laplacian tensor of the hyper-
graph induced by the classification structures. Then Lxm presents the clustering
score of the data points {x1 , x2 , . . . , xn } when m is even. Borrowing the idea of
graph embedding [127], we can derive a framework of multilabel learning

max Lxm ,
s.t. Lp xm = 1,
2.2. BASIC DEFINITIONS 13

where Lp is the Laplacian tensor of a penalty hypergraph, which removes some

trivial relations. Employing the method of Lagrange multipliers, we can trans-
form the optimization problem into a generalized tensor eigenvalue problem
Lxm−1 = λLp xm−1 .
Moreover, consider a homogenous polynomial dynamical system [43],
dBu(t)m−1
= Au(t)m−1 .
dt
We explored the stability of the above system in [43]. Similarly to the linear
case, if we require a solution which has the form u(t) = x·eλt , then a generalized
tensor eigenvalue problem Axm−1 = (m − 1)λBxm−1 is naturally raised from
this homogenous system.
The generalized tensor eigenvalue problem is an interesting topic, although
its computational solutions are not well-developed so far. Therefore we aim at
investigating the generalized tensor eigenvalue problems theoretically, especially
on some perturbation properties as a prerequisite of the numerical computations
in [68].

2.2 Basic Definitions

We will first introduce some concepts and notations involved. Let K1,2 denote
a projective plane [113], in which (α1 , β1 ), (α2 , β2 ) ∈ K × K are regarded as the
same point, if there is a nonzero scalar γ ∈ K such that (α1 , β1 ) = (γα2 , γβ2 ).
We can take K as the complex number field C, the real number field R, or the
nonnegative number cone R+ .
The determinant of a tensor is investigated by Qi et al. [98, 57]. The
determinant of an mth -order n-dimensional tensor A is the resultant [31] of
the system of homogeneous equations Axm−1 = 0, which is also the unique
polynomial on the entries of A satisfying that
• det(A) = 0 if and only if Axm−1 = 0 has a nonzero solution;
• det(I) = 1, where I is the unit tensor;
• det(A) is an irreducible polynomial on the entries of A.
Furthermore, det(A) is homogeneous of degree n(m − 1)n−1 . As a simple ex-
ample in [98], the determinant of a 2 × 2 × 2 tensor is defined by
 
a111 a112 + a121 a122 0
 0 a111 a112 + a121 a122 
det(A) = det 
a211 a212 + a221
.
a222 0 
0 a211 a212 + a221 a222
Now we can define the generalized eigenvalue problems of tensor pairs which
is similar to the matrix case [6, 49, 113]. Let A and B be two mth -order tensors
in Cn×n×···×n . We call {A, B} a regular tensor pair, if
det(βA − αB) 6= 0 for some (α, β) ∈ C1,2 .
Reversely, we call {A, B} a singular tensor pair, if
det(βA − αB) = 0 for all (α, β) ∈ C1,2 .
14 CHAPTER 2. GENERALIZED TENSOR EIGENVALUE PROBLEMS

In this chapter, we focus on the regular tensor pairs.

If {A, B} is a regular tensor pair and there exists (α, β) ∈ C1,2 with a nonzero
vector x ∈ Cn such that

(2.1) βAxm−1 = αBxm−1 ,

then we call (α, β) an eigenvalue of the regular tensor pair {A, B} and x the
corresponding eigenvector. When B is nonsingular, that is, det(B) 6= 0, no
nonzero vector x ∈ Cn satisfies that Bxm−1 = 0, according to [57, Theorem
3.1]. Thus β 6= 0 if (α, β) is an eigenvalue of {A, B}. We also call λ = α/β ∈ C
an eigenvalue of the tensor pair {A, B} when det(B) 6= 0. Denote the spectrum,
that is, the set of all the eigenvalues, of {A, B} as

λ(A, B) = (α, β) ∈ C1,2 : det(βA − αB) = 0 ,

or, for a nonsingular B, we also denote


λ(A, B) = λ ∈ C : det(A − λB) = 0 .

This chapter is devoted to the properties and perturbations of the spectrum

λ(A, B) for a regular tensor pair {A, B}.

2.3 Several Basic Properties

There are plenty of theoretical results about the regular matrix pairs [6, 49, 113].
In this section, we extend several of them to the regular tensor pairs. Note that
some of the generalized results maintain the forms similar to the matrix case,
while others do not.

2.3.1 Number of Eigenvalues

How many eigenvalues does a regular tensor pair have? This might be the first
question about the spectrum of a regular tensor pair. We have the following
result:
Theorem 2.1. The number of the eigenvalues of an mth -order n-dimensional
regular tensor pair is n(m − 1)n−1 counting multiplicity.
Proof. If {A, B} is an mth -order n-dimensional regular tensor pair, then there
exists such (γ, δ) ∈ C1,2 that det(δA−γB) 6= 0. Assume that (γ, δ) is normalized
to |γ|2 + |δ|2 = 1. We have the transformation, as the matrix case [115, Page
300], ( (
Ae = γ̄A + δ̄B, A = γ Ae + δ̄ B,
e
⇔
Be = δA − γB, B = δ Ae − γ̄ B.
e

Thus there is a one-to-one map between λ(A, B) and λ(A,

e B):
e
 
αe = αγ̄ + β δ̄, α=α eγ + βeδ̄,
⇔
βe = αδ − βγ, β=α eδ − βγ̄.
e

Since det(B)
e 6= 0, then the eigenvalues of {A,
e B}
e are exactly the complex roots
of the polynomial equation det(A − λB) = 0. By [57, Proposition 2.4], the
e e
Visit https://textbookfull.com
now to explore a rich
collection of eBooks, textbook
and enjoy exciting offers!
2.3. SEVERAL BASIC PROPERTIES 15

degree of the polynomial det(Ae − λB) e of λ is no more than n(m − 1)n−1 . We

shall show that the degree is exactly n(m − 1)n−1 .
n−1
Denote the coefficient of the item λn(m−1) in det(Ae − λB)
e as f (A,
e B).
e
From the definition of det(A − λB), we see that f (A, B) is actually independent
e e e e
of the entries of A.
e Thus f (A,e B)
e = f (O, B),
e where O denotes the zero tensor.
It is easy to verify that f (O, B) = det(B) 6= 0. Thus f (I, B)
e e e is nonzero, and so
is f (A, B). Therefore the polynomial det(A − λB) = 0 has n(m − 1)n−1 roots
e e e e
counting multiplicity.

2.3.2 Spectral Radius

Spectral radius is an important quantity for the eigenvalue problems. To define
the spectral radius of a regular tensor pair, we introduce a representation of the
points in R1,2 first. Let (s, c) be a point in R1,2 . Without loss of generality, we
can assume that c ≥ 0 and s > 0 when c = 0. Define an angle in (−π/2, π/2] as
s
θ(s, c) := arcsin √ .
s2 + c2
When c is fixed, θ(·, c) is an increasing function; When s is fixed, θ(s, ·) is a
decreasing function.
For a regular tensor pair {A, B}, we define the θ-spectral radius as

ρθ (A, B) := max θ(|α|, |β|).

(α,β)∈λ(A,B)

Define another nonnegative function about {A, B} as

θ kAxm−1 k, kBxm−1 k .


ψθ (A, B) := max
x∈Cn \{0}

It is apparent that ρθ (A, B) ≤ ψθ (A, B), since θ kAxm−1 k, kBxm−1 k = θ(|α|, |β|),
if (α, β) ∈ λ(A, B) and x is the corresponding eigenvector.
When det(B) 6= 0, we can define the spectral radius in a simpler way

ρ(A, B) := max |λ|.

λ∈λ(A,B)

Similarly, we define a nonnegative function about {A, B} as

kAxm−1 k
ψ(A, B) := max .
x∈Cn \{0} kBxm−1 k

It is easy to verify that tan ρθ (A, B) = ρ(A, B) ≤ ψ(A, B) = tan ψθ (A, B).
Furthermore, if B is fixed, then the nonnegative function ψ(·, B) is a seminorm
on Cn×n×···×n . When A, B are matrices and B = I, the result becomes the
familiar one that the spectral radius of a matrix is always no larger than its
norm.

2.3.3 Diagonalizable Tensor Pairs

Diagonalizable matrix pairs play an important role in the perturbation theory
of the generalized eigenvalues [113, Section 6.2.3]. A tensor A is said to be
16 CHAPTER 2. GENERALIZED TENSOR EIGENVALUE PROBLEMS

diagonal if all its entries except the diagonal ones aii...i (i = 1, 2, . . . , n) are
zeros. We call {A, B} a diagonalizable tensor pair if there are two nonsingular
matrices P and Q such that

C = P −1 AQm−1 and D = P −1 BQm−1

are two diagonal tensors. Let the diagonal entries of C and D be {c1 , c2 , . . . , cn }
and {d1 , d2 , . . . , dn }, respectively. If (ci , di ) 6= (0, 0) for all i = 1, 2, . . . , n,
then {A, B} is a regular tensor pair. Furthermore, (ci , di ) are exactly all the
eigenvalues of {A, B}, and their multiplicities are (m − 1)n−1 .
It should be pointed out that the concept of “diagonalizable tensor pair”
is not so general as the concept of “diagonalizable matrix pair” [113, Section
6.2.3]. For instance, if matrices A and B are both symmetric, then the matrix
pair {A, B} must be diagonalizable. However, this is not true for symmetric
tensor pairs. Hence we shall present a nontrivial diagonalizable tensor pair to
illustrate that it is a reasonable definition.
Example 2.1. Let A and B be two mth -order n-dimensional anti-circulant
tensors [39]. According to the result in that paper, we know that

C = Fn∗ A(Fn∗ )m−1 and D = Fn∗ B(Fn∗ )m−1

are both diagonal, where Fn is the n-by-n Fourier matrix [49, Section 1.4].
Therefore an anti-circulant tensor pair must be diagonalizable.

2.3.4 Gershgorin Circle Theorem

The Gershgorin circle theorem [121] is useful to bound the eigenvalues of a
matrix. Stewart and Sun proposed a generalized Gershgorin theorem for a
regular matrix pair (see Corollary 2.5 in [113, Section 6.2.2]). We extend this
famous theorem to the tensor case in this subsection. Define the p-norm of a
tensor as the p-norm of its mode-1 unfolding [49, Chapter 12.4]

kAkp := kA(1) kp .

Particularly, the tensor ∞-norm has the form

n X
X n n
X
kAk∞ = max ··· |aii2 ...im |.
i=1,2,...,n
i2 =1 i3 =1 im =1

Notice that for a positive integer k, we have kx[k] k∞ = kx⊗k k∞ , where x[k] =
[xk1 , xk2 , . . . , xkn ]> is the componentwise power and x⊗k = x ⊗ x ⊗ · · · ⊗ x is the
Kronecker product of k copies of x [49, Section 12.3]. Denote M A := A ×1 M >
and AM m−1 = A×2 M · · ·×m M for simplicity. Then we can prove the following
lemma for ∞-norm:
Lemma 2.2. Let {A, B} and {C, D} = {CI, DI} be two mth -order n-dimensional
regular tensor pairs, where C and D are two matrices. If (α, β) ∈ C1,2 is an
eigenvalue of {A, B}, then either det(βC − αD) = 0 or

(βC − αD)−1 β(C − A) − α(D − B)

 
≥ 1.
∞
2.3. SEVERAL BASIC PROPERTIES 17

Proof. If (α, β) ∈ C1,2 satisfies that det(βC − αD) 6= 0, then it holds for an
arbitrary nonzero vector y ∈ Cn that (βC − αD)ym−1 6= 0, which is equivalent
to (βC − αD)y[m−1] 6= 0. Then the matrix βC − αD is nonsingular.
Since (α, β) is an eigenvalue of {A, B}, there exists a nonzero vector x ∈ Cn
such that βAxm−1 = αBxm−1 . This indicates that

β(C − A) − α(D − B) xm−1 = (βC − αD)xm−1 .

 

Thus from the expressions of C and D, we have

Gxm−1 := (βC − αD)−1 β(C − A) − α(D − B) xm−1 = x[m−1] .

 

Then following the definition of the tensor ∞-norm and

kG(1) vk∞ kG(1) x⊗(m−1) k∞ kGxm−1 k∞

kGk∞ = max ≥ = = 1,
v∈Cnm−1
\{0} kvk∞ kx⊗(m−1) k∞ kx[m−1] k∞

we prove the result.

Comparing with Theorem 2.3 in [113, Section 6.2.2], Lemma 2.2 specifies
that {C, D} has a special structure and the norm in the result must be the
∞-norm. Nevertheless, these restrictions do not obstruct its application to the
proof of the following lemma:
Lemma 2.3. Let {A, B} be an mth -order n-dimensional regular tensor pair.
Furthermore, assume that (aii...i , bii...i ) 6= (0, 0) for i = 1, 2, . . . , n. Denote
 X 
Di (A, B) := (α, β) ∈ C1,2 : |βaii...i − αbii...i | ≤ |βaii2 ...im − αbii2 ...im |
(i2 ,...,im )
6=(i,...,i)
Sn
for i = 1, 2, . . . , n. Then λ(A, B) ⊆ i=1 Di (A, B).
Proof. Take the diagonal matrices C = diag(a11...1 , a22...2 , . . . , ann...n ) and D =
diag(b11...1 , b22...2 , . . . , bnn...n ) in Lemma 2.2. Then from the assumption, we
know that {C, D} is a regular tensor pair.
When (α, β) ∈ λ(A, B) satisfies that det(βC − αD) = 0, it must hold that
βaii...i − αbii...i = 0 for some i. Thus it is obvious that (α, β) ∈ Di (A, B).
When (α, β) ∈ λ(A, B) does not satisfy that det(βC − αD) = 0, by Lemma
2.2, we have

(βC − αD)−1 β(C − A) − α(D − B)

 
∞
X βaii2 ...im − αbii2 ...im
= max ≥ 1.
i=1,2,...,n βaii...i − αbii...i
(i2 ,...,im )
6=(i,...,i)

Thus for some i, it holds that

X
|βaii...i − αbii...i | ≤ |βaii2 ...im − αbii2 ...im |,
(i2 ,...,im )
6=(i,...,i)

that is, (α, β) ∈ Di (A, B).

18 CHAPTER 2. GENERALIZED TENSOR EIGENVALUE PROBLEMS

Lemma 2.3 is alike in form to the Gershgorin circle theorem. However, it is

not the desired result, since there are still α and β on the right-hand side of the
inequality. To avoid this, we introduce the chordal metric in C1,2 [49, 113]:

|α1 β2 − β1 α2 |
chord (α1 , β1 ), (α2 , β2 ) = p p .
|α1 | + |β1 |2 |α2 |2 + |β2 |2
2

Moreover, we can easily prove by the Cauchy inequality that

X p
|βaii2 ...im − αbii2 ...im | ≤ |α|2 + |β|2
(i2 ,i3 ,...,im )
6=(i,i,...,i)
v
u !2 !2
u X X
×u |aii2 ...im | + |bii2 ...im | .
u
t
(i2 ,i3 ,...,im ) (i2 ,i3 ,...,im )
6=(i,i,...,i) 6=(i,i,...,i)

Finally, we obtain the Gershgorin circle theorem for regular tensor pairs.
Theorem 2.4. Let {A, B} be an mth -order n-dimensional regular tensor pair.
Suppose that (aii...i , bii...i ) 6= (0, 0) for all i = 1, 2, . . . , n. Denote the disks
n o
Gi (A, B) := (α, β) ∈ C1,2 : chord (α, β), (aii...i , bii...i ) ≤ γi

for i = 1, 2, . . . , n, where
v
u 2  2
u P P
(i2 ,i3 ,...,im ) |aii2 ...im | + (i2 ,i3 ,...,im ) |bii2 ...im |
u
u
6=(i,i,...,i) 6=(i,i,...,i)
γi = .
t
|aii...i |2 + |bii...i |2
Sn
Then λ(A, B) ⊆ i=1 Gi (A, B).
If B is taken as the unit tensor I, then Theorem 2.4 reduces to the Gershgorin
circle theorem for single tensors, that is, [98, Theorem 6(a)].
Furthermore, the Gershgorin circle theorem can have a tighter version when
the order of the tensor pair is no less than 3. A tensor is called semi-symmetric
[86] if its entries are invariable under any permutations of the last (m − 1)
indices. Then for an arbitrary tensor A, there is a semi-symmetric tensor Ae
such that Axm−1 = Ax e m−1 for all x ∈ Cn . Concretely, the entries of Ae are
1 X
aii2 ...im = aii02 i03 ...i0m ,
|π(i2 , . . . , im )| 0 0
e
0
(i2 ,i3 ,...,im )∈
π(i2 ,i3 ,...,im )

where π(i2 , i3 , . . . , im ) denotes the set of all permutations of (i2 , i3 , . . . , im ) and

|π(i2 , i3 , . . . , im )| denotes the number of elements in π(i2 , i3 , . . . , im ). Note that
aii...i = e aii...i and
X X
|aii2 ...im | ≥ |e
aii2 ...im |.
(i2 ,i3 ,...,im ) (i2 ,i3 ,...,im )
6=(i,i,...,i) 6=(i,i,...,i)

Hence we have a tight version of the Gershgorin circle theorem for regular tensor
pairs, that is, the disk Gei (A, B) in the following theorem must be no larger than
the disk Gi (A, B) in Theorem 2.4.
2.3. SEVERAL BASIC PROPERTIES 19

Theorem 2.5. Let {A, B} be an mth -order n-dimensional regular tensor pair.
Assume that (aii...i , bii...i ) 6= (0, 0) for all i = 1, 2, . . . , n. Let Ae and B e be the
m−1 m−1 m−1
semi-symmetric tensors such that Ax = Ax
e and Bx = Bxm−1 for
e
n
all x ∈ C . Denote the disks
n o
Gei (A, B) := (α, β) ∈ C1,2 : chord (α, β), (aii...i , bii...i ) ≤ γ


ei

for i = 1, 2, . . . , n, where
v
u 2  2
u P P
(i2 ,i3 ,...,im ) |e
aii2 ...im | + (i2 ,i3 ,...,im ) |bii2 ...im |
u e
u
6=(i,i,...,i) 6=(i,i,...,i)
γ
ei = .
t
|aii...i |2 + |bii...i |2
Sn
Then λ(A, B) ⊆ i=1 Gei (A, B).

2.3.5 Backward Error Analysis

Backward error is a measurement of the stability of a numerical algorithm [53].
It is also widely employed as a stopping criteria for iterative algorithms. We
propose the framework of the backward error analysis (see [52] for the matrix
version) on the algorithms for generalized tensor eigenproblems.
Suppose that we obtain a computational eigenvalue (b α, β)
b and the corre-
sponding computed eigenvector x b of a regular tensor pair {A, B} by an algo-
rithm. Then they can be regarded as an exact eigenvalue and eigenvector of
another tensor pair {A + E, B + F}. Define the normwise backward error of the
computed solution as
n o
ηδ1 ,δ2 (b
α, β,
bx b) := min k(E/δ1 , F/δ2 )kF : β(A
b + E)b xm−1 = αb(B + F)bxm−1 .

Here δ1 and δ2 are two parameters. When taking δ1 = δ2 = 1, the backward

error is called the absolute backward error and denoted as η[abs] (b
α, β,
bx b). When
taking δ1 = kAkF and δ2 = kBkF , the backward error is called the relative
backward error and denoted as η[rel] (b
α, β,
bx b).
m−1 b xm−1 . Then the two tensors E and
Denote the residual b bBb
r := α x − βAb
F must satisfy that βE b xbm−1 − α bm−1 = b
bF x r, which can be rewritten into an
underdetermined linear equation [53, Chapter 21]
  b ⊗(m−1) 
δ βx

E(1) /δ1 , −F(1) /δ2 · 1 ⊗(m−1) = br.
δ2 α
bx

Hence the least-norm solution is given by

b ⊗(m−1) †
 
δ βx
 
r 1 ⊗(m−1) ,
E(1) /δ1 , −F(1) /δ2 = b
δ2 α
bx

where A† denotes the Moore-Penrose pseudoinverse of a matrix A [49, Section

5.5.2]. Notice that v† = v∗ /kvk22 and kbx⊗(m−1) k2 = kb xkm−1
2 . Therefore we
derive the explicit form of the normwise backward error.
Random documents with unrelated
content Scribd suggests to you:
 I. In animal societies founded on nutrition, it is this function which
constitutes the social tie; the individuals composing it are attached
to one another in a permanent manner, from their birth onwards,
and the nutritive liquid circulates from one to another, thus
establishing a material community. It is found in the hydroid polypes,
the Bryozoa and the Tunicata. As examples of the superior forms, we
may quote the hydractinia, composed of individuals each of which
has its own special and exclusive function: some, that of feeding;
others, that of feeling and exploring; others, of defending the
colony; others, again, of reproducing it—the last-named being
divided into males and females. The siphonophora present an
analogous division of labour, and the community, over a metre in
length, suspended to a floating bladder, executes well co-ordinated
collective movements.
Is there—at any rate in the higher forms of these colonies—a
social instinct? Solidarity and reciprocity can indeed be perceived, in
an objective, material way, in the form of adherence, and vascular
communication; but nothing proves that there is anything more than
an organic solidarity and reciprocity. Perhaps, in circumstances such
as the nautical manœuvres just mentioned, in which a general
obedience to one directing individual has been ascertained, there is
a momentary consent—a certain unity of representation. To be
accurate, the terms individual and community are diverted from their
ordinary acceptation and used in an equivocal sense. Our notion of
the individual is that of an organised whole living independently by
itself: this no longer corresponds to the present case. Our notion of
a community is an assemblage of individuals, and as these are, in
the case under consideration, of a peculiar nature, it might thus be
contended with equal force that these aggregates deserve, or that
they do not deserve, the name of animal communities: it is a
question of the point of view. On the one hand, one may regard the
hydractinia or the siphonophora as a complex individual whose
organs are the fishing, the piloting, the reproductive, etc.,
individuals. On the other hand, it may be maintained that the food-
providers, pilots, etc., are true individuals whose aggregation forms
a society. In short, it is an undifferentiated state, in which individuals
and community are hardly to be distinguished from one another, and
are only two different aspects of the same whole. The social instinct,
also, if existing at all, is not yet differentiated from the conservative
instinct under its simplest forms—the search for food, defence,
attack. In fact, the two coincide. This stage has nothing more to
teach us. Let us now pass on to social forms whose psychology is
clearer.
II. These are the societies founded on reproduction—domestic
societies, or families, under their various forms. I prefer to begin
with these rather than with the gregarious state; first, on account of
their universality; then because they are the first to appear in
chronological order. Common opinion finds in them the first
manifestation of the social sentiments, their origin, their source, and
their moment of entry into the world. I reject this view in order to
adopt that which connects the social instinct with the gregarious
state.
If we take, one after another, the conditions of every aggregate
founded on reproduction, we shall find three stages: that of sexual
approach, that of maternal love, and lastly, but in the case of
animals only exceptionally, paternal love. The social instinct—i.e., the
more or less vague consciousness of at least a temporary solidarity
and reciprocity—does not, as we shall see, make its appearance at
any of these stages.
1. Sexual approach results from one particular instinct; it unites
two individuals only: can we consider it as the embryo of a society?
“Around sexuality are co-ordinated the altruistic instincts of which
the animal is capable.” This formula of Littré’s needs defining with
more precision. First, in the immense majority of cases the
connection is not lasting; the blind instinct satisfies itself, and all is
over. Higher up there are more permanent forms, such as polygamy
and polyandry; but these small communities founded on sex-
attraction are closed, and have no power of radiation or extension,
no future. Higher still we find monogamy, as among wolves, many
birds, etc.; but the monogamic aggregate is still more of a close
corporation than the others. Let us note, in passing, that these two
forms, polygamy and monogamy, are distributed through the animal
world in an apparently fortuitous manner, having no relation to the
intellectual development—as, for instance, the monogamy of the
stork and the polygamy of the monkey.
Finally, this first stage yields us no result, tending rather towards
social restriction than social extension.
2. Maternal love is of much greater importance. In domestic
societies it is the universal and permanent element, the vital bond.
This emotion is so widespread, so well known, we might say so trite,
that it seems to involve no mystery, and yet, if we descend into
animal psychology, we find nothing more enigmatical. The
development of sympathy and intelligence partly explains it in the
human species and the higher animals; but in the lower orders the
difficulty becomes extreme. Yet it shows itself among the annelids,
the crustacea, the mollusca, and even the echinoderms, which carry
their eggs about adhering to their bodies. Frequently it shows itself
as a feeling which, though vague, is tenacious, devoted, heroic. We
do not indicate all the difficulties of the question, as for instance:
How can an insect take such care of its eggs when it cannot
recognise its own form in a creature which in nowise resembles
itself, and has not even a living form?[169]
Most naturalists content themselves with ascertaining the fact,
without inquiring into its origin. Darwin declares that it is useless to
speculate on this subject. Others connect maternal affection with
parasitism—scarcely a legitimate hypothesis, since the parasite is the
enemy of its host and lives against his will at his expense. Romanes
seems to have recourse to the principle of serviceable variations; an
animal which takes care of its eggs or carries them about with it, has
a better chance of preserving them; and if this way of acting
becomes a fixed habit in its descendants, an instinct has been
established. This explanation reduces itself to chance and to the
hereditary transmission—an open question—of acquired
modifications.
Excluding the insects and those analogous cases which, as Espinas
has shown (op. cit., pp. 334-339), require a special explanation, it is
preferable to admit, with this author and Bain, the prominent part
played by contact. “It seems to me that there must be at the
foundation that intense pleasure in the embrace of the young which
we find to characterise the parental feeling throughout. The origin of
the pleasure may be as purely physical as in the love of the sexes;
... [there is] an initial satisfaction in the animal embrace, heightened
by reciprocation.”[170] “The female, at the moment when she gives
birth to little ones resembling herself, has no difficulty in recognising
them as the flesh of her flesh; the feeling she experiences towards
them is made up of sympathy and pity, but we cannot exclude from
it an idea of property which is the most solid support of sympathy.
She feels and understands up to a certain point that these young
ones which are herself at the same time belong to her; the love of
herself extended to those who have gone out from her changes
egoism into sympathy and the proprietary instinct into an
affectionate impulse. As sexual love implies the idea of mutual
ownership, so maternal love supposes that of subordinated
ownership. It is because this other self is so feeble that the interest
felt for it takes the form of pity.”[171] This last remark relates to an
emotional manifestation which Spencer regards as the source of
maternal love—tenderness for the weak. This seems to me rather
one of its elements than its sole basis. On the other hand, Bain
maintains that “an intensified attraction towards the weak is not
merely consistent with the gregarious situation, but seems to be
required by its varying exigencies.... An interest or solicitude about
weak members would be almost the necessary completion of the
social system” (Bain, op. cit., pp. 138, 139). This granted, maternal
love and social instinct would have an element in common, but they
nevertheless remain distinct and mutually independent.
I have insisted to some extent on maternal love, because it is one
of the most important manifestations of the emotional life. It is clear
that it belongs to the category of the tender emotions, of which it is
a well-determined form and remarkable by reason of its intensity;
but it is not the source of the social instinct, because it implies
neither solidarity nor reciprocity. It might be maintained that it is the
gate by which the feeling of benevolence made its entrance into the
world, and that its appearance is the earliest in date; but other
conditions are needed for the social instinct to reveal itself.
 3. The third stage, marked by the entrance of the father into the
domestic community, does not affect our conclusion. In the animal
world taken as a whole, paternal affection is rare and far from
permanent, and among the lower representatives of humanity the
feeling is a very weak one and the tie very loose. It exists, however,
and its origin is much more difficult to assign than in the case of
maternal love.[172] Though it may be maintained that in man it
originates in pride and the feeling of ownership (Bain), this
hypothesis is not applicable to animals; we cannot say, as in the
case of the mother, that there is a material and visible relation, so
that the offspring seems to be a separated portion of the parent. It
remains to establish the significance of sympathy for weakness, as a
primary cause of this feeling. We might add another element if we
admit, with Spencer, that the life in common of the father and
mother (paternal affection being only found where unions are
permanent) creates a current of affection in proportion to the
services rendered. Whatever origin we may assign to it, it adds
nothing to our discussion, and has no efficacity in arousing the social
instinct.
To sum up: what we find at the base of domestic aggregates is
tender emotion, the genesis of altruism, but restricted to a closed
group, without expansive force or elasticity.
III. The gregarious life—i.e., that of the animals who live in troops
or hordes—is founded on the attraction of like for like, irrespective of
sex, and for the first time manifests the true social tendencies,
through the habit of acting in common.
In its lowest degree it consists of accidental and unstable
assemblages which are, as it were, an attempt at life in common.
Every one knows that certain pelagic animals travel in vast numbers,
their course being determined by the temperature of the water or
the direction of the currents. We also know what happens in the
migrations of processionary caterpillars, of crickets, and more
especially of birds. Numerous species of animals assemble together
in the morning and evening to sing, utter their various cries, pursue
each other and gambol about, living dispersed at other times. This
shows, says Espinas, “a latent social tendency, always ready to show
itself when not combated by any other tendency.”
Higher still, we find assemblages of variable duration, but
voluntarily formed and maintained, in view of a common aim. They
have all the characteristics of a society—community of effort,
synergy, reciprocity of services. Darwin[173] has given many examples
of this: Pelicans fish in concert, and close in round their prey like a
living net; wolves and wild dogs hunt in packs, and help each other
to attack their victims. These communities are to some extent
accidental and unstable, and may come to an end in a final
competition for the sharing of the spoil. Much more stable are those
which have the common defence for their aim: rabbits warn one
another of danger; many mammals and birds place sentinels (it is
well known how difficult it is to approach a herd or drove of
animals); monkeys remove vermin or take out thorns from one
another’s skins, form a chain to cross the gap between two trees,
unite their forces to raise a heavy stone, and finally, gathered into
bands under the direction of a leader, they defend themselves
energetically and risk their lives to save their companions. We might
enumerate endless facts of this kind. No doubt, we have not yet
found the permanent organisation, the fixed division of labour, the
continuity, which are peculiar to the higher animal societies; but the
instability and intermittence of these social forms help us to
understand why they exist and whence they originate.
Social tendencies are derived from sympathy; they arise in
determinate conditions. The facts already given supply the answer to
the questions: how do they arise? what is their source? They arise
from the nature of things, from the conditions of the animal’s
existence; they are not based on pleasure, but on the unconscious
affirmation of the will to live; they are auxiliary to the instinct of
conservation. Society, as Spencer justly remarks, is founded on its
own desire—i.e., on an instinct.
The gregarious life, as this writer has shown in detail,
predominates among the herbivora and graminivora, who, as a rule,
being ill armed for strife and finding food in abundance, find it to
their advantage to live in herds.
 The contrary is the case with the carnivora; they are well armed,
and need ample space in which to hunt down their prey, so that it is
to their advantage to live in isolation, except in those cases already
mentioned, where they associate together for a difficult chase, or for
defence against a dangerous enemy.[174]
We may add that there are animals which, as they find it to their
advantage or otherwise, live alternately in communities or isolated.
“Certain sociable birds in Australia build bowers of branches, where
they assemble in great numbers during the day. In pairing-time the
society is broken up, and each couple retires by itself to construct its
own separate nest. While the temporary families last there is no
longer any assemblage, nor any life in common; it only begins again
when the young are able to try their wings. This is only one of
numberless examples which might be mentioned.”[175]
In short, gregarious life depends on stature, strength, means of
defence, kind and distribution of food, and mode of propagation.
Derived from necessity, this habit of life in common creates a
solidarity which is not mechanical and external, but psychological:
the sight, the touch, the smell of his companions constitute in each
individual a part of his own consciousness, of which he feels the
want in its absence; the distressed state and the lamentations of an
animal separated by chance from the herd are well known.
Here a disputed question suggests itself. It has been already
settled by implication in the course of the statements already made,
but it cannot be thus treated retrospectively and merely in passing.
For the moment I may confine myself to indicating it. If we compare
family societies and gregarious societies, what relation is there
between them? We find ourselves in presence of two opinions or
theories—one in favour of unity, the other of duality.
The first, the most ancient and widespread, derives social life from
domestic life. The family is the social molecule: by its increase are
formed aggregates of a more or less complex character, whose life in
common creates a solidarity and an exchange of services—i.e., the
conditions of a community.
The second admits two groups of irreducible feelings and
tendencies, mutually independent, though there are points of
contact. The social instinct is not derived from the domestic feelings,
while the latter are not derived from the social feelings. They are
distinct by nature, having their respective sources in the attraction of
like for like, irrespective of sex, and in the sexual appetite and the
development of the tender emotions.
More than this: some writers, especially zoologists, have
maintained that there is not merely dualism but antagonism. When
the feelings of domestic life are strong, social solidarity is lax or non-
existent. When social solidarity is close and rigorous, the family
tendencies are transitory, effaced, or nil—e.g., in ants and bees. The
case of the Australian birds shows us this antagonism in an
alternating form, the domestic and the social tendency
predominating by turns. No doubt this antagonism is not
irremediable and is compatible with various modifications and
compromises; but there is, in fact, a dualism not to be explained
away. I shall return later on to this question, only remarking in
anticipation that the dualist view seems to me the only admissible
one.
IV. The higher societies are those in which the animal world has
attained its loftiest degree of social development. In them we find
division of labour, solidarity, stability, and continuity through several
generations. Such are bees, wasps, ants, termites, beavers, etc. It is
not part of our subject to study them, since our only aim is to follow
the social tendencies to their highest point; but the problem already
suggested again arises: On what foundation do these higher
societies rest? Espinas, who admits the view of the family as the
source of social life, classes them among societies having
reproduction for their purpose. For my own part, I refer them to the
gregarious state, in which they mark the stage of the highest
perfection. Let me take this opportunity of pointing out the
inconveniences of a false position and the factitious difficulties
arising from it. The author draws out (pp. 370 et seq.) a detailed
comparison between the societies of bees and those of ants; he
demonstrates the superiority of the latter, who, according to
circumstances, dig, carve, build, hunt, store up food, reap harvests,
keep slaves and cattle, and when they carry on war against the
wasps (the warlike representatives of the bees) gain the victory. He
also clearly shows that this superiority is due to their terrestrial
habits, in which every contact, every march, leaves them a precise
indication of the nature of their surroundings. But he finds
something perplexing in this superiority. As a matter of fact, a hive is
a perfect domestic society, since the queen-bee—i.e., the common
mother—is the visible soul of social life among the bees. An ant-
heap is imperfect, “inferior” as a domestic society, as containing
several females. The apparent contradiction disappears, if we
consider that in both cases, especially in the second, the essential
element is the solidarity among the members, the mutual attraction
between similar beings, and that, consequently, we must refer them
to the gregarious, and not to the domestic type. For the rest, in
neither case does the family, in the true sense of the word, exist: it
is needless to demonstrate this at length, it is quite sufficient to note
the absence of maternal love. And so certain writers have made use
of this argument—as I have already said—to prove that such a high
development of the social tendencies has only been possible through
the suppression of the family tendencies.

II.

If we pass from animals to man, the situation remains the same,

and the tendency to social life, in spite of its manifold adaptations,
does not change its nature; it is always at bottom a solidarity and a
reciprocity of services, determined by the conditions of human
existence and variable as they. We need not come back to this; but
the question already hinted at—that of the relation between the
emotional manifestations serving as a basis to the family on the one
hand, and those which are the foundation of social life on the other
hand—presents itself anew. We cannot evade it, if we desire any
light on the origin of the social feelings.
If we assume the family as the primitive fact which, by its
increase, produced the clan, and afterwards, more complex
aggregates, such as tribes, connected with each other by the
memory of a common ancestor and at last subject to the authority
of a patriarch-king, the social development is simply an expansion of
the natural family. On this hypothesis, the domestic tendencies
(founded on reproduction) are primary; the social tendencies are
derivative and of secondary or tertiary formation.
If, on the contrary, we consider the smallest social groups (hordes,
clans, or whatever other name they may be called by) as existing by
themselves, independently of the domestic group, the tendency to
live in societies must be considered as irreducible and self-
determined; there is only one more general emotional phenomenon
whence it could be derived, viz., sympathy.
Evidently, this question cannot be settled a priori, but only by the
interpretation of facts. Now there is no lack of documents, supplied
by ethnology from observations on actually existing primitive
peoples, by the history of the remotest epochs, and by the literary
monuments of the earliest ages, which are the echo of prehistoric
times. There is no lack, either, of authorised works on the subject:
MacLennan, Bachofen, Tylor, Sumner Maine, Starcke, Westermarck—
to cite only a few at random. Although there is much disagreement,
both as to the facts and the interpretation of the facts, the
probability is very slight in favour of the priority of the family, very
great in favour of two distinct developments with inevitable points of
contact and interference.
Let us briefly recall the most generally admitted results of research
into the evolution of the family and the progress of social
development.
1. The evolution of the family has certainly not proceeded in all
places in the same way, a circumstance which always permits the
critic to oppose facts to the view he is combating. A disease inherent
in the human mind induces most writers to try and refer everything
to one formula, to impose on facts that perfect unity which, in such
matters, does not appear very probable. Those who assign the
greatest length of time to the evolution of the family admit three
stages: promiscuity, matriarchate, patriarchate.
The period of primitive promiscuity (Bachofen, MacLennan, Girard-
Teulon, etc.) is contested and rejected by many authorities. In any
case, it does not seem as if we could establish the rule without a
great number of exceptions. Not to speak, however, of archaic
institutions which have been interpreted in this sense, and as
survivals, there are still certain Tartar populations which approach
this stage. At Hawaii, the individual was related to the whole horde,
age alone determining the relationships: every one called all the old
people indiscriminately grandfather and grandmother; all those who,
as far as age went, might be his parents, father and mother; all
those of his own generation, brothers and sisters; and so on for sons
and daughters, grandsons and grand-daughters. These five terms
expressed all known degrees of kinship. We may note, in passing,
that a very weak psychological argument has been put forward in
order to disprove the existence of this period—viz., that the natural
jealousy of man would have rendered promiscuity impossible, at
least for any length of time. Those who have hazarded such
reasoning have been too ready to judge primitive man by civilised
standards. However this may be, such a mass, without individual
relationships, is rather a society than a family; or rather, it is an
undifferentiated state, which might be compared to the lowest form
of animal societies (the nutritive), which also is undifferentiated.
In the period of the matriarchate, which appears to have lasted for
a considerable time, the mother is the centre of the family. This
domestic form, coexisting with polygamy, polyandry, and even with
monogamy, has left so many traces, and is still met with in so many
different races and countries, from the ancient Egyptians and
Etruscans to the present natives of Sumatra and some regions of
Africa, that there is no dispute on the subject. The woman gives her
name to the children, kinship is reckoned, and the inheritance of
property (though not always that of political dignities) descends, in
the female line; the position of most importance is filled, not by the
father, but the uncle—the mother’s brother. The causes of the
matriarchal system have been much discussed. Did it originate in an
assumption that the true father was unknown, or in a common
opinion of his insignificance? Whatever view may be adopted, it
seems to me reasonable to compare the matriarchate with the
predominating system among animals—i.e., maternal societies
where the male is not admitted.
The patriarchate (agnatio) which makes the father the centre of
the family brings us down to the historic epoch, to which it was even
anterior in some parts of the globe. Its appearance is saluted in
lyrical terms by Bachofen as the triumph of ideas over matter: “By
the spiritual principle of paternity the chains of tellurism were
broken;” it was a conquest of mind over material nature—over what
can be seen and touched.[176] It is not known how it came about,
whether by adoption or by a pretence of childbirth. In any case, it
corresponds with the admission of the male into animal societies.
2. The development of social life is quite otherwise. It would be
foreign to our purpose to retrace its successive phases; let us
confine ourselves to the question of origin. What was primitive man?
On this point much has been written by way of argument and
conjecture. H. Spencer, in his Sociology (vol. i.), has made a
complete restoration from prehistoric documents, burial-mounds,
and more especially from the condition of contemporary savages.
Nothing proves that this picture will suit all classes; there have
existed not one primitive man, but primitive men differing
considerably, according to race and environment.
However far we go back, the first form of life in common seems to
be the horde, an unstable, unorganised aggregate, without
recognised kinships, drawn together instinctively in view of utility
and defence. But the true social unit, which arose at an early period
in various parts of the globe, is the clan (and analogous institutions),
a fixed, stable, coherent, closed aggregate, founded on religious or
other affiliation (but not on descent), independent of family
conditions: a man cannot belong to two clans at once, and in most
cases each of these groups is in a hostile attitude towards the rest.
How has this social molecule been able to aggregate to itself others,
and this closed organism to break its narrow limits, in order to
extend itself by increase and fusion? This is a somewhat obscure
question; perhaps by exogamy, i.e., the imperative custom which
forbade marriage within the group (yet in other groups the rule was
endogamy, i.e., the prohibition of marriage outside them); more
probably the great agent of assimilation and fusion was war,
followed by the assimilation of the conquered.
This simple comparison shows that the family and the clan are not
similar institutions: the first is an autonomous group belonging to a
master, and having for its end the enjoyment of property; the
second is a group of another nature having for its end the common
struggle for existence. “Where the interests defended by the family
are less important than those of the clan, the family is influenced by
the ideas which regulate the clan organisation; and this fact repeats
itself in all primitive societies when defence against an outside
enemy is the dominant necessity.”[177] The family group and the
social group have each sprung from different tendencies, from
distinct needs; each has its special, independent psychological
origin, and there is no possible derivation from one to the other.

III.

Life in common, even under the gregarious form, requires certain

ways of acting, and habits founded on sympathy and determined by
the concerted aim pursued by all. In order that it may become stable
and constitute a society, an element of fixity must be added—the
more or less clear consciousness of an obligation, of a rule, of what
has to be done or avoided. This is the appearance of the moral
sentiment. All conceptions of morality, coarse or refined, theoretical
or purely practical, agree on this point; divergences exist, in practice,
only as to the characteristics of the act reputed obligatory; in theory,
only as to its origin.
All real morality which has lived, i.e. governed a human society,
large or small, which has existed, not in the academic abstractions of
moralists, but in the concrete development of history, and has run its
complete course, passes through two principal periods.
One of these is instinctive, spontaneous, unconscious,
unreflecting, determined by the conditions of existence of a given
group at a given moment. It expresses itself in custom—a
heterogeneous mixture of beliefs and actions which, from the point
of view of reason and of a more advanced culture, we consider
sometimes as moral, sometimes as immoral, sometimes as unmoral,
i.e., puerile and futile, but all of which have been rigorously
observed.
The other is conscious, reflecting, many-sided, complex, like the
higher forms of social and moral life. It expresses itself in
institutions, written laws, religious or civil codes; and still more in the
abstract speculations of philosophical moralists. Then, the apogee
being reached, vague aspirations reach out towards a new, dimly
apprehended ideal, and the cycle begins over again.
Most constructors of a scientific system of morality have forgotten
or neglected the first period; but wrongly so, for it is the source of
the second. This, too, is the reason for the two opposite views held
with regard to the origin of moral development.
Some seek it in the order of knowledge, whence they deduce all
the rest; they suppose innate ideas, or an adaptation acquired
through a long process and fixed by heredity (Spencer), or the
consciousness of a categorical imperative, or the notion of utility; all
of which are intellectual solutions.
Others seek it in the order of instinct and feeling. They admit
tendencies, impulses implanted in us by nature, i.e., forming part of
our organisation, like thirst and hunger, whose satisfaction produces
pleasure and their non-satisfaction pain; this is the emotional view.
The two are not absolutely irreconcilable: each of them
corresponds to a different period of evolution; the emotional view to
the instinctive stage, the stage of moral chaos; the intellectualist
view to the reflective stage of rational organisation; but it is clear
that one alone can claim the mark of its origin. In other words, we
may say: in the moral consciousness there are two elements—
judgment and feeling. A judgment (approving or condemnatory) on
our own conduct and that of others is the result of a deeper process
—not an intellectual one—of an emotional process of which it is only
the clear and intelligible manifestation in consciousness. It would be
a psychological absurdity to suppose that a bare, dry idea, an
abstract conception without emotional accompaniments, and
resembling a geometrical notion, could have the least influence on
human conduct. No doubt, we must admit that the evolution is
rather that of moral ideas than of the moral sentiment, which, in
itself, is no more than a tendency to act—a predisposition; but an
evolution of purely speculative ideas, with no emotional
accompaniment, will have no results in the practical order. We may
note that the opposition between these two views is constantly
reflected in the history of moral theory. In England, where
psychology predominates, the doctrine of feeling has had numerous
champions, from Shaftesbury down to the present day. In Germany,
where metaphysics are predominant, the intellectualist doctrine,
since Kant, occupies the first place, except with Schopenhauer and
his adherents. It is quite natural that the metaphysicians,
intellectualists by temperament and by profession, should have
adopted this position.
For the rest we are concerned here with the moral sentiment, and
with that alone; the other elements of morality do not form part of
our study. It consists, at bottom, in movement or arrest of
movement, in a tendency to act or not to act; it is not, in its origin,
due to an idea or a judgment; it is instinctive, and herein lies its
strength. It is innate, not like an alleged archetype, infused into
man, invariable, illuminating him everywhere and always, but in the
same way as hunger and thirst and other constitutional needs. It is
necessary; it forces one to act (when not kept in check by counter-
tendencies), as the sight of water forces the duckling to plunge into
it. Thus we must say that the man who impulsively throws himself
into danger to save another is more thoroughly moral than he who
only does so after reflection; one must be blinded by intellectualist
prejudices to maintain the contrary. Natural morality is a gift—
theologians would say a grace; it is artificial, acquired morality,
which is measured by the quantity of resistance overcome. Finally,
like every other tendency, it results in satisfaction or dissatisfaction
(e.g., remorse).[178] In short, its innateness and its necessity place it
in the motor, not in the intellectual order.
These characteristics being determined, let us follow the progress
of its evolution. It presents two aspects: first, the positive,
corresponding to the genesis of the beneficent feelings, or active
altruism, an internal evolution—i.e., one of the primary feeling, in
and through itself; secondly, negative, corresponding to the rise of
the sense of justice, an external evolution—i.e., one produced under
the pressure of conditions of existence and coercive means.
I. We include under the name of beneficence, or active altruism,
such feelings as benevolence, generosity, devotion, charity, pity, etc.;
in short, those foreign or contrary to the instinct of individual self-
preservation. Their fundamental conditions are two psychological
facts already studied:
1. Sympathy, in the etymological sense, i.e., an emotional unison,
the possibility of feeling with another, and like him. Could a society
be based on this state alone? In extreme cases this might happen;
but such a society would be transitory, precarious, unstable: we
have found similar examples in the gregarious state, animal or
human. Stability requires stronger ties, that is to say, moral ones.
2. The altruistic tendency, or tender emotion, which exists in all
men, except in those to be referred to at the end of this chapter. It
belongs to our constitution, as much as the fact of having two eyes
or a stomach.
Now the question put to us is this: How is active altruism
developed, and by what psychological mechanism? How do
disinterested feelings arise from primitive egoism? Setting aside all
metaphysical solutions, such as Schopenhauer’s theory of universal
pity, compassion (Mitleid) for all beings, founded on a vague
consciousness of community of being and identity of origin—a
monistic conception,—I shall confine myself to a strictly
psychological explanation.
Benevolence arises from a particular form of activity accompanied
by pleasure: this vague and obscure formula will be explained
presently.
The fundamental tendency consists, in the first place, of
preserving, and then of extending one’s self, of being and well-
being, i.e., expending activity. Man may devote this activity to
things: he cuts, hacks, destroys, overthrows,—these are destructive
activities; he sows, plants, builds, and exercises preservative or
creative activities. He may apply it to animals or to men; he injures,
maltreats, destroys, or he cares for, helps, saves. Destructive activity
is accompanied by pleasure, but by a pathological one, since it is the
cause of evil. Preservative or creative activity is accompanied by
pure pleasure, leaving behind it no painful feeling; consequently, it
tends to repeat and increase itself: the object or the person which is
the cause of pleasure becomes a centre of attraction, the starting-
point of an agreeable association. To sum up, we have (1) a
tendency to the display of our creative activity; (2) the pleasure of
succeeding; (3) an object or living being to play a receptive part; (4)
an association between this being or object and the pleasure
experienced; whence a continually increasing attraction towards this
being or object. The conservative tendency in action and the law of
transference (see Part I., Chap. XII.) are the essential agents in the
rise of altruism.
This may be justified by several examples. If we reflect on the
preceding, it will be understood that benevolence may well be the
result of chance, and have, in its origin, no intentional character. A
man, without paying any special heed to it, happens to throw some
water on a plant which was drying up beside his door; next day he
chances to notice that it is beginning to revive; he repeats the
operation, intentionally; he becomes more and more interested in
the plant, grows attached to it, and would not like to be deprived of
it.[179] This is a very trivial, everyday occurrence, and there is no one
who has not experienced something of the sort; this is all the better,
as showing us the rise of the feeling in all its simplicity. If this
happens in the case of a plant, how much more easily in that of an
intelligent animal or a man!
It is an observed fact that a man attaches himself to another
rather in proportion to the services he renders than to those he
receives from him. There is, in general, a stronger current of
benevolence passing from the benefactor to his protégé than vice
versâ. Common opinion considers this illogical: from the point of
view of reason, it is so—not from that of feeling; and the preceding
analysis even shows that it must be so, because the benefactor has
put more of himself into the recipient of his bounty than the latter
can do to him. Thus, in many persons, gratitude needs to be
supported by reflection.
If we are ill-disposed towards any one, the best and surest
remedy against this incipient aversion is to render him some service.
Conversely, the person who refuses all our benefits and obstinately
avoids them becomes an object of indifference, or even hatred.
“During the proscriptions of Marius and Sulla,” says Friedmann,
“there were many sons who, out of fear, gave up their father, but it
was never known that a father had denounced his son; a fact that
somewhat startled the Roman moralists, who were unable to explain
it.” The explanation is involved in the constitution of the Roman
family, by which the father could confer many benefits on the son,
whereas the son was entirely dependent on the father, and could do
nothing for him.
Many other incidents might be cited to justify the accuracy of the
preceding analysis. Such is the mechanism by means of which our
emotional self succeeds in externalising, in alienating itself; but this
could not be done were there not at the origin and starting-point a
primary tendency, already studied under the name of tender
emotion. It is clear, also, that beneficence is a generic term
designating forms which vary according to circumstances: charity,
generosity, devotion, etc.
The extension and heightening of the feeling of beneficence have
taken place slowly, and owing to the work of certain men who
deserve to be called discoverers in morals. This expression may
sound strangely in some ears, because they are imbued with the
theory of an innate and universal knowledge of good and evil
imparted to all men at all times. If we admit, on the contrary,—as
observation teaches us to do,—not a ready-made, but a growing
morality, it must necessarily be the discovery of an individual or
group of individuals. Every one admits the existence of inventors in
geometry, in music, in the plastic and mechanical arts; but there
have also been men in moral disposition far superior to their
contemporaries, who have initiated or promoted reform in this
department. Let us note (for this point is of the highest importance)
that the theoretic conception of a higher moral ideal, of a step in
advance, is not sufficient; it needs a powerful emotion leading to
action, and, by contagion, communicating its own impulse to others.
The onward march is proportioned to what is felt, not to what is
understood.
Were the human race, in the beginning, cannibals? Some affirm
this, others deny it. What is certain is that the custom of eating
one’s fellow-men has existed in many places, and still exists in some.
It has been explained by scarcity of food, by superstitious beliefs, by
the intoxication of triumph in annihilating a vanquished enemy, by
the idea of assimilating his strength and courage, and by a variety of
other reasons; but it has not been sufficiently remarked that its
extinction has not always been due to the intervention of superior
races. It has sometimes taken place on the spot. In the Tahiti
Islands it had disappeared shortly before the arrival of Bougainville;
among the Redskins, and even among the Fijians, parties had been
formed in order to suppress not only cannibalism, but the tortures
inflicted on prisoners of war.[180] The promoters of this abolition,
whether individuals or groups, were certainly inventors. The
universality of human sacrifices is well known; they are found still
existing during the historic period, from China to Judæa, from
Greece to Gaul, from Carthage to Rome. How did they disappear?
On this point we have nothing but ignorance or legends, but they
could not have disappeared without the agency of man. Du Chaillu
cites a case in which reform is, so to speak, caught in the act—that
of an African chief who was the first to give orders that no slave
should be killed at his tomb.[181] Among the Aztecs, with their
bloodthirsty religion, a sect, formed before the arrival of the
Spaniards, had placed itself under the protection of a deity who
abhorred bloodshed. All the great ancient legislators, whether
historical or legendary—Manes, Confucius, Moses, Buddha,—we
might say all founders of religions, have been discoverers in morals;
whether the discovery originated with themselves alone or with a
collectivity as whose summary and embodiment they may be
regarded, matters little.
It would be easy to continue this historical demonstration, but the
above is sufficient to justify the term discoverers. From causes of
which we are ignorant, but analogous to those which produce great
poets or painters, there arise men of indisputable moral superiority
who feel what others do not feel, just as a great poet does
compared with ordinary men. And for one who has succeeded, how
many have failed for want of a favourable environment! A St.
Vincent de Paul among the Kanakas would be as impossible as a
Mozart among the Fuegians.
In primitive societies there has been a long struggle between the
strongest egoistic tendencies, with their dissolvent action, and the
weaker and intermittent altruistic tendencies, which have progressed
through the agency of some more enlightened individuals, and also
with the help of force, of which we still have to speak.
II. Let us now examine the development of the moral sentiment
under its negative and restrictive aspect—i.e., as the sense of
justice. Here the intellectual element evidently preponderates, and
its evolution involves the other.
“Justice,” says Littré, “has the same foundation as science.” One
rests on the principle of identity which governs the region of
speculation, the other rests on the principle of equivalence and rules
the sphere of action. Justice, in its origin, is a compensation for
damages. Its evolution starts from an instinctive semi-conscious
manifestation, rising by progressive steps to a universalist
conception. Let us mark the principal stages.
The first, and lowest, is neither moral nor social, but purely animal
and reflex—"a defensive reflex."[182] The individual who suffers
violence, who thinks himself attacked or injured, immediately reacts.
This is “the exasperated instinct of conservation,” or, to call it by its
true name, revenge. So the savage who, before Darwin’s eyes, broke
his son’s head for having dropped a store of shell-fish, the fruit of a
laborious day’s fishing. This defensive reflex frequently recurs in the
psychology of crowds; it is needless to give instances. It may seem
paradoxical to take revenge as a starting-point for the sense of
justice; but we shall see how it becomes mitigated and rationalised.
In fact, a second stage corresponds to revenge deferred through
premeditation, reflection, or some analogous cause. It tends towards
equivalence and reaches it under the form of retaliation, so frequent
in primitive communities. The idea of equality, tooth for tooth, eye
for eye, has won its way; the instinct has become intellectualised.
So far, the compensation claimed would appear to have only an
individual character; but it must very early have taken on a collective
character, by reason of the close solidarity uniting the members of
the small social aggregate—the clan or family. An all-powerful
opinion forces the injured party to pursue his revenge even when he
does not wish it; and when a vendetta is in force as between clan
and clan, the stage of collective responsibility appears, and the
notion of the compensation due is enlarged.
However, revenge restores, in the social aggregate, a state of war,
which has to be eliminated; hence a reaction on the part of the
community tending to suppress or attenuate it. This is the stage of
arbitration and peace-making. Many facts show that, in the
beginning, the decision of the umpires is without binding value, and
supported by no coercive means. It is a proof not so much of
culpability as of an indemnity to be paid to those concerned; the
criminal trial is as yet a civil action.
For this temporary and unsanctioned arbitration the social
development logically substitutes a permanent and guaranteed
arbitration, exercised by a chief, or an aristocracy, or the popular
assembly. Compensation becomes obligatory and is forcibly imposed.
The condemned person must submit or leave the community; if
refractory, he is excommunicated, and in primitive societies the
outlaw’s life is intolerable; we see the equivalent of it in modern
strikes. Let us also note the somewhat widely distributed custom of
a division of the indemnity imposed, one portion being assigned to
the injured party, the other to the state—i.e., the chief. The notion of
justice has taken on a definitely social character.
It only remains that it should become universal. It long remains
enclosed within the limits of the social group. All that contributes to
the material and moral welfare of the group is good, and conversely;
outside the group, all acts are unmoral. We find in history, and even
at the present time, many proofs of this dualism or duplication of the
individual, according as he is acting within his own social
environment or with regard to strangers. Such were the Germans of
Cæsar’s time.[183] In their earlier period, the Greeks considered
themselves as less under moral obligation towards the Barbarians,
and the Romans towards foreigners (hostes). It is especially owing
to the efforts of the philosophers—Socrates, Plato, Aristotle, the
Stoics—that justice ceased to be national and became universal. It
might be added that at that period when the notion of justice
remains a national one, it still varies within the group, according to
caste; it is not the same for priests and warriors, for free men and
for slaves, for aristocrats and for merchants. In the beginning,
particularism was the rule.
It is evident that, on the negative side, the evolution of moral life
has been especially due to the progress of intelligence; the
emotional element has only been incidental. Compared with the
sense of justice, the feeling of active benevolence, if not evolved
more quickly, at least appeared sooner, because nearer to instinct
and less dependent on reason. A certain philosopher (Kant, I
believe) was surprised that there is so much kindness and so little
justice among men. He did not observe as a psychologist, or else he
was led astray by intellectualist prejudice. This must be so, because
tenderness is innate and spontaneous, justice acquired and
deliberate; because one springs directly from an instinct, while the
other has to undergo various metamorphoses. If man is sociable and
moral, it is less because he thinks than because he feels in a certain
manner and tends in a certain direction.
To conclude: moral emotion is a very complex state. Those
sentimentalists in the last century, or in this, who have maintained
the hypothesis of a “moral sense,” have erroneously considered it as
a special sense with an innate faculty of discriminating good and
evil. It is not a simple act, but the sum of a set of tendencies. Let us
eliminate the intellectual elements, and enumerate its emotional
constituents only: (1) as basis, sympathy—i.e., a community of
nature and disposition; (2) the altruistic or benevolent tendency
manifesting itself under different forms (attraction of like to like,
maternal or paternal affection, etc.), at first weak, but gaining more
expansion by the restriction of the egoistic feelings; (3) the sense of
justice with its obligatory character—whose origin we have just
traced; (4) the desire of approbation or of divine or human rewards,
and the fear of disapprobation and punishments. As in the case of all
complex feelings, its composition must vary with the predominance
of one or other of its constituent elements; in one case it is
obligation (the Stoics), in another charity, in many the fear of public
opinion or of the law, of God or of the devil. It is impossible that it
should be constant and identical in all men.
IV.
The pathology of the moral sense cannot detain us long, its
detailed study belonging to the department of criminal anthropology.
Numerous works have been published on this subject within the last
half-century; there would be no advantage in presenting a bald
abstract of these. Lombroso’s view of the “born criminal,” with his
physiological, psychical, and social characteristics, has been violently
attacked, and sustained serious damage. Several successive theories
have attempted to explain the existence of this moral anomaly:
atavism, according to which the born criminal is a survival, a return
to primitive man, who is assumed to have been violent and
unsociable; infantilism, which has recourse, not to heredity, but to
arrested development, and alleges that the perversion which is
permanent in the criminal is normal, but transient, in the child; the
pathological view which connects the criminal type with epilepsy,
considered as the prototype of violent and destructive impulses; the
sociological view (the most recent), which attributes a preponderant
function to social conditions, and maintains that the criminal is “a
microbe inseparable from his environment.”[184] We need not enter
into a detailed examination of these hypotheses, which have given
rise to much passionate debate: one question alone concerns our
subject, that of moral insensibility—a condition described, long
before the days of criminal anthropology, under the names of moral
insanity (Prichard, 1835), folie morale, impulsive insanity, instinctive
monomania, etc., and which will serve to show once more the
independence and the preponderance of feeling in the moral life.[185]
“Moral insanity is a form of mental derangement in which the
intellectual faculties appear to have sustained little or no injury, while
the disorder is manifested principally or alone in the state of the
feelings, temper, or habit.” Such is the formula of Prichard, which
has been but little modified since. Translated into the language of
pure psychology, it signifies: a complete absence or perversion of
the altruistic feelings, insensibility to the representation of the
happiness or suffering of others, absolute egoism, with all its
consequences. By a self-evident analogy, this state has been called
one of moral blindness; and, like physical blindness, it has various
degrees. It has also been compared to idiocy. Reduced to the
vegetative and sensitive life, the idiot is, intellectually, opposed to
the genius, while the moral idiot is the antithesis of the great
benefactors of humanity (Schüle).
We may find numerous instances of moral insanity in works on
mental pathology and criminal anthropology.[186] It shows itself in
two forms: (1) the passive, or apathetic—i.e., that of pure
insensibility; if the temperament is cold and the circumstances
favourable, there is no violence to be feared; (2) the active, or
impulsive, where there is no check on the violence of the appetites.
Taken as a whole, it consists in: complete insensibility, absence of
pity, cold ferocity, absence of remorse after committing acts of
violence, or even murder. On this last point statistics and figures
have been given whose precision makes me somewhat suspicious;
[187]
for it is very difficult to penetrate so far into the consciousness
of a criminal as to be duped neither by the hypocrisy which
simulates remorse, nor by the boastfulness which feels but will not
acknowledge it. The absence of all maternal feeling, though rare,
has also been observed.
Moral insensibility is usually innate, and coincident with other
symptoms of degeneracy. Among several children of the same
family, brought up in the same surroundings, having received the
same care, a single one may differ from all the rest, be amenable
neither to gentleness nor to force, and manifest a precocious
depravity, which will only strengthen as he grows older.
This state may be acquired and momentary, its causes being
epilepsy, hysteria, apoplexy, paralytic dementia, senile decay, blows
on the head, etc. Krafft-Ebing, besides an observation made by
himself (loc. cit.), quotes from Wigan the case of a young man who,
in consequence of being struck on the head with a ruler, developed
complete moral insensibility. When, by means of the operation of
trephining, a splinter of bone pressing on the brain had been
removed, he returned to his former state. We have met with other
analogous cases in the course of this work.
The most difficult and fiercely debated point is whether this moral
anomaly is strictly instinctive and emotional in its origin, intellectual
activity being entirely unconnected with it. Most writers take the
affirmative view of this question, others deny it. The different modes
of mental activity are so interdependent, and their relations so close,
that it is difficult to solve the question definitely. We cannot refuse to
admit that the intellect sometimes suffers from a counter-shock; but
observation shows that most of these persons are well acquainted
with the requirements of morality, and have had the abstract ideas
of good, of evil, and of duty instilled into them by education, though
without the slightest influence on their conduct. They have moral
ideas, not moral feelings—i.e., a disposition to feel and act. The law
is to them nothing but a police regulation, which they are conscious
of having broken. Their intellect, often firm and clear, is only an
instrument for weaving skilful plots, or justifying themselves by
subtle sophisms.
It was worth our while to recall, if only in a cursory manner, the
nature of moral insensibility, in order to show the importance of the
emotional element. In these cases there is a lack of completeness,
and the deficit comes, not from the intellect, but from the character.
 CHAPTER IX.

THE RELIGIOUS SENTIMENT.

Importance of the subject—Its Divisions. First Period: origin of the religious
feeling—Primitive notions of the Infinite (Max Müller); Ancestor-worship
(H. Spencer)—Fetichism, animism; Predominance of fear—Practical,
utilitarian, social, but not moral character—Second period: (1) Intellectual
evolution; Conception of a Cosmic Order first physical, then moral—
Function of increasing generalisation; its stages; (2) Emotional evolution;
Predominance of love; addition of the moral sentiment—Third Period:
Supremacy of the rational element; Transformation into religious
philosophy; Effacement of the emotional element—Religious emotion is a
complete emotion—Manifold physiological states accompanying it; ritual,
a special form of the expression of emotion—The religious sentiment as a
passion—Pathology—Depressive forms: religious melancholy,
demonomania—Exalted forms: ecstasy, theomania.

It must be confessed that psychologists have not troubled

themselves greatly with the study of the religious sentiment.
Some omit it altogether, while others content themselves with a
brief reference in passing; they note the two essential elements
whence it is derived—fear, and tender emotion (love)—without
troubling themselves about the variable relations between these
two elements, or the multiform changes undergone by them in
the course of centuries, through the annexation of other
emotional states.[188] As we cannot deny its importance, this
abstention, or negligence, is not justifiable. To summon to our
aid an ill-understood respect, to maintain that one religion only is
true and all the others false, to allege that all are alike false,—
these and other analogous modes of reasoning are not in any
degree acceptable to psychology; for, even if we take up an
extreme position, and admit that all manifestations of the
religious sentiment are mere illusion and error, it remains none
the less true that illusion and error are psychic states, and
worthy of being studied as such by psychologists. To such, the
religious sentiment is a fact which they have simply to analyse
and to follow through its transformations without being
competent to discuss its objective value or its legitimacy. Thus
understood, the question bears on two principal points: primary
manifestations and their evolution, i.e., the different elements
which have constituted the religious sentiment during the various
stages of its existence.
In every religious belief there are of necessity two parts: an
intellectual element, a knowledge which constitutes the object of
belief, and an emotional state, a feeling which accompanies the
former and expresses itself in action. To any one deficient in the
second element, the religious feeling is unknown, inaccessible;
nothing remains to such persons but abstract metaphysical
conceptions. The study of the religious sentiment, in its evolution,
cannot dissociate these two elements; and it is the degree in which
the element of knowledge is present which renders a precise division
possible. I trace three periods: (1) that of perception and concrete
imagination, where fear and the practical, utilitarian tendencies are
predominant; (2) that of medium abstraction and generalisation,
characterised by the addition of moral elements; (3) that of the
highest concepts, where, the emotional element becoming more and
more rarefied, the religious feeling tends to be confounded with the
so-called intellectual feelings.

I.

As usual, authorities are not agreed on the question of origin.

Under what form did the religious sentiment first make its
appearance? We must first put aside two extremely systematic
answers, which, although differing in spirit, have this in common,
that they are both purely intellectualist.
 The first, a very ancient one, has found its latest and clearest
interpreter in Max Müller, who thinks that the notion of the Divine,
more especially under the form of the Infinite, preceded that of the
gods. Our senses give us the finite, but “beyond, behind, beneath,
and within the finite, the infinite is always present to our senses. It
presses upon us, it grows upon us from every side. What we call
finite in space and time, in form and word, is nothing but a veil or
net which we ourselves have thrown over the infinite.”[189] What,
then, is the infinite, he asks, but the object of all religions? The
religion of the infinite precedes and comprehends all others, and as
the infinite is implied by the senses (i.e., the limits to our sensory
perceptions imply an unlimited region beyond), it follows that
religion can, with as much right as reason, be called a development
of our sensory perceptions. The earliest religion consisted in the
adoration of various objects, taken, each in turn, and isolatedly, as
incarnations of the notion of the Infinite. This is what Max Müller
calls “Henotheism.” For him, polytheism and even fetichism are later
developments, resulting from the breaking up of the primitive unity,
and due to a disease of language: each name becomes a distinct
deity; words are raised to the dignity of things, having their life, their
attributes, and their legends: Nomina numina.
This last view, though it has had a certain vogue among linguists,
is worth nothing as a psychological explanation, for it is quite clear
that the word is only a starting-point or a vehicle for the process of
thought, which is the sole agent of the metamorphosis. If the
nomina become numina, it is by a disease of imagination or thought,
rather than of language.[190]
As for the principal thesis, the alleged primitive notion of the
Infinite, which is the source of henotheism, it is a metaphysical
hypothesis of extreme improbability. Primitive man, enclosed within
hard conditions of life, is practical and positive rather than a
dreamer; he does not naturally tend towards the Beyond. But a
better reason than this, and an entirely psychological one, is that he
is incapable of attaining to even a medium degree of abstraction and
generalisation. How could a savage, who cannot count up to four,
form any idea whatever of the Infinite? Evidently, this notion of the
Illimitable is far beyond him.
There is only one way of imparting a certain psychological
verisimilitude to Max Müller’s view, viz., to strip it of its intellectualist
character, and admit as its origin a feeling rather than a notion, a
craving, a tendency, rather than a cognition. Of these two factors,
which make up all religious belief, one intellectual, the other
emotional, which has the priority? Did the notion produce the
feeling, or the feeling excite the notion? Such is the problem which
lies at the heart of all debates on the origin of religious
manifestations. Some place it in the region of instinct: so Renan
when he compares religion in humanity to the nest-building instinct
of the bird. Others maintain that every feeling presupposes an
object. “At first sight this latter theory seems to have logic on its
side. It is clear that, in order to love or fear any being, one must
have conceived the notion of his existence. Yet, however
indispensable it may be to assume an intellectual operation at the
beginning of religion, we must recognise that the feelings set in
motion by this operation must have long preceded the most ancient
formulas of primitive theology.”[191] For my own part, I am inclined to
accept the priority of feeling, though unable to supply any
arguments based on fact; the period of origins being also that of
conjecture.
The second theory, that of Herbert Spencer, brings us down from
the notion of the Infinite to the extremely terrestrial mental life of
savages. It is well known that he reduces all primitive religions to
the cult of ancestors—to necrolatry. The primordial fact is the
conception of a spirit, or rather, of a double. The savage believes
that he has a Sosia, or, in other words, a principal ego and a
secondary ego. He infers the existence of this double from a great
number of facts, to him inexplicable: his shadow, his reflection in the
water, echoes, apparitions in dreams, fainting, trances, epilepsy, etc.
The world is thus, for him, full of wandering spirits which he tries to
propitiate. According to Spencer, fetichism and polytheism are only
aberrant forms of ancestor-worship, and he tries to prove this by a
series of arguments, through which we need not now follow him.
Imperturbable in his systematic deduction, he even asserts that he
can derive from the same root, by far-fetched and easily-
controverted arguments, the adoration of animals, plants, and
inanimate objects.[192] It is indisputable that a great number of
beliefs have sprung from this root, but this conception, which is
anthropomorphism carried to its extreme, is found to be too narrow
to include all the facts. Tylor and others have criticised it with some
vivacity, and I do not think that it now claims many adherents.
These two systematic hypotheses being put on one side, we may
remind the reader how religious development seems to have taken
place during this primitive period; for the march of evolution has not
been everywhere and always the same—a difficulty already pointed
out with regard to the social instinct. According to the best
authorities, the most frequent form has been the following.
The first stage is that of fetichism, polydemonism, naturism—
terms which in the history of religions are not quite synonymous, but
which answer to the same psychical condition, the adoration of some
object, living or not, which is perceived—i.e., apprehended as a
concrete form, at the same time body and soul—or rather animated
by a soul, judged to be benevolent or malevolent, useful or
injurious; for there is scant justification for the opinion that the
worshipper of a piece of wood or stone sees in it only a purely
material object.
The second stage is that of animism or spiritism, “a belief in spirits
having no substantial bond or necessary connection with
determinate natural objects.” The spirit is conceived as independent,
separable; it goes and comes, enters and departs; it is attributed not
only to men, but also to animals: the savage tries to deprecate the
wrath of the beast he has killed in hunting, the chief has his horses
and dogs buried with him. Psychologically, this stage corresponds to
a preponderance of the imagination over simple perception.
These primitive forms of religious belief originate in the tendency
of the savage, and the child, perhaps also of the higher animal, to
look upon everything as alive, to attribute desires, passions, will, to
everything that acts, to form his ideas of nature from what he knows
of his own nature. This anthropomorphism results from the