Get ebook downloads in full at ebookname.

com

An Introduction to Spatial Data Science with GeoDa

Volume 2 Clustering Spatial Data 1st Edition Luc
Anselin

https://ebookname.com/product/an-introduction-to-spatial-
data-science-with-geoda-volume-2-clustering-spatial-
data-1st-edition-luc-anselin/

OR CLICK BUTTON

DOWNLOAD EBOOK

Explore and download more ebook at https://ebookname.com

Instant digital products (PDF, ePub, MOBI) available
Download now and explore formats that suit you...

Spatial Data Quality 1st Edition Wenzhong Shi

https://ebookname.com/product/spatial-data-quality-1st-edition-
wenzhong-shi/

Hierarchical Modeling and Analysis for Spatial Data

Second Edition Banerjee

https://ebookname.com/product/hierarchical-modeling-and-analysis-
for-spatial-data-second-edition-banerjee/

Applied Spatial Analysis of Public Health Data 1st

Edition Lance A. Waller

https://ebookname.com/product/applied-spatial-analysis-of-public-
health-data-1st-edition-lance-a-waller/

Frank Zappa and the And 1st Edition Paul Carr

https://ebookname.com/product/frank-zappa-and-the-and-1st-
edition-paul-carr/
Happy Street Stella Maidment

https://ebookname.com/product/happy-street-stella-maidment/

How to Shit Around the World The Art of Staying Clean

and Healthy While Traveling 2nd Edition Edition Dr.
Jane Wilson-Howarth

https://ebookname.com/product/how-to-shit-around-the-world-the-
art-of-staying-clean-and-healthy-while-traveling-2nd-edition-
edition-dr-jane-wilson-howarth/

Tools of the Imagination Drawing Tools and Technologies

from the Eighteenth Century to the Present 1st Edition
Susan Piedmont-Palladino

https://ebookname.com/product/tools-of-the-imagination-drawing-
tools-and-technologies-from-the-eighteenth-century-to-the-
present-1st-edition-susan-piedmont-palladino/

The Nature of Natural History Marston Bates

https://ebookname.com/product/the-nature-of-natural-history-
marston-bates/

Pro Drupal Development 2nd Edition Vandyk

https://ebookname.com/product/pro-drupal-development-2nd-edition-
vandyk/
Santiago Calatrava Milwaukee Art Museum Quadracci
Pavilion scan 1st Edition Cheryl Kent

https://ebookname.com/product/santiago-calatrava-milwaukee-art-
museum-quadracci-pavilion-scan-1st-edition-cheryl-kent/
 An Introduction to Spatial
Data Science with GeoDa
Volume 2 – Clustering Spatial Data

This book is the second in a two-volume series that introduces the field of spatial data sci-
ence. It moves beyond pure data exploration to the organization of observations into meaningful
groups, i.e., spatial clustering. This constitutes an important component of so-called unsuper-
vised learning, a major aspect of modern machine learning.
The distinctive aspects of the book are both to explore ways to spatialize classic clustering meth-
ods through linked maps and graphs, as well as the explicit introduction of spatial contiguity
constraints into clustering algorithms. Leveraging a large number of real-world empirical il-
lustrations, readers will gain an understanding of the main concepts and techniques and their
relative advantages and disadvantages. The book also constitutes the definitive user’s guide for
these methods as implemented in the GeoDa open-source software for spatial analysis.
It is organized into three major parts, dealing with dimension reduction (principal components,
multidimensional scaling, stochastic network embedding), classic clustering methods (hierar-
chical clustering, k-means, k-medians, k-medoids and spectral clustering) and spatially con-
strained clustering methods (both hierarchical and partitioning). It closes with an assessment of
spatial and non-spatial cluster properties.
The book is intended for readers interested in going beyond simple mapping of geographical
data to gain insight into interesting patterns as expressed in spatial clusters of observations.
Familiarity with the material in Volume 1 is assumed, especially the analysis of local spatial au-
tocorrelation and the full range of visualization methods.

Luc Anselin is the Founding Director of the Center for Spatial Data Science at the University
of Chicago, where he is also Stein-Freiler Distinguished Service Professor of Sociology and the
College, as well as a member of the Committee on Data Science. He is the creator of the GeoDa
software and an active contributor to the PySAL Python open-source software library for spatial
analysis. He has written widely on topics dealing with the methodology of spatial data analysis,
including his classic 1988 text on Spatial Econometrics. His work has been recognized by many
awards, such as his election to the U.S. National Academy of Science and the American Academy
of Arts and Science.
Taylor & Francis
Taylor & Francis Group
http://taylorandfrancis.com
An Introduction to Spatial
Data Science with GeoDa
Volume 2 – Clustering Spatial Data

Luc Anselin
Designed cover image: © Luc Anselin

First edition published 2024

by CRC Press
2385 NW Executive Center Drive, Suite 320, Boca Raton FL 33431

and by CRC Press

4 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN

CRC Press is an imprint of Taylor & Francis Group, LLC

© 2024 Luc Anselin

Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot as-
sume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have
attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders
if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please
write and let us know so we may rectify in any future reprint.

Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or
utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including pho-
tocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission
from the publishers.

For permission to photocopy or use material electronically from this work, access www.copyright.com or contact the
Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. For works that are
not available on CCC please contact [email protected]

Trademark notice: Product or corporate names may be trademarks or registered trademarks and are used only for iden-
tification and explanation without intent to infringe.

ISBN: 978-1-032-71302-1 (hbk)

ISBN: 978-1-032-71316-8 (pbk)
ISBN: 978-1-032-71317-5 (ebk)

DOI: 10.1201/9781032713175

Typeset in Latin Modern font

by KnowledgeWorks Global Ltd.
Publisher’s note: This book has been prepared from camera-ready copy provided by the authors.
To Emily
Taylor & Francis
Taylor & Francis Group
http://taylorandfrancis.com
Contents

List of Figures xi

Preface xvii
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii

About the Author xix

1 Introduction 1
1.1 Overview of Volume 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Sample Data Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

I Dimension Reduction 5
2 Principal Component Analysis (PCA) 7
2.1 Topics Covered . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2 Matrix Algebra Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2.1 Eigenvalues and eigenvectors . . . . . . . . . . . . . . . . . . . . . . 9
2.2.2 Matrix decompositions . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.3 Principal Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3.1 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.3.2 Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.4 Visualizing principal components . . . . . . . . . . . . . . . . . . . . . . . . 19
2.4.1 Scatter plot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.4.2 Multivariate decomposition . . . . . . . . . . . . . . . . . . . . . . . 20
2.5 Spatializing Principal Components . . . . . . . . . . . . . . . . . . . . . . . . 21
2.5.1 Principal component map . . . . . . . . . . . . . . . . . . . . . . . . 22
2.5.2 Univariate cluster map . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.5.3 Principal components as multivariate cluster maps . . . . . . . . . . 22

3 Multidimensional Scaling (MDS) 25

3.1 Topics Covered . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.2 Classic Metric Scaling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.2.1 Mathematical Details . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.2.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.3 SMACOF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.3.1 Mathematical Details . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.3.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
3.4 Visualizing MDS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.4.1 MDS and Parallel Coordinate Plot . . . . . . . . . . . . . . . . . . . 39
3.4.2 MDS Scatter Plot with Categories . . . . . . . . . . . . . . . . . . . 39
3.4.3 3-D MDS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.5 Spatializing MDS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

vii
viii Contents

3.5.1 MDS and Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

3.5.2 MDS Spatial Weights . . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.5.3 MDS Neighbor Match Test . . . . . . . . . . . . . . . . . . . . . . . 44
3.5.4 HDBSCAN and MDS . . . . . . . . . . . . . . . . . . . . . . . . . . 46

4 Stochastic Neighbor Embedding (SNE) 47

4.1 Topics Covered . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
4.2 Basics of Information Theory . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.2.1 Stochastic Neighbors . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.3 t-SNE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
4.3.1 Cost Function and Optimization . . . . . . . . . . . . . . . . . . . . 50
4.3.2 Large Data Applications (Barnes-Hut) . . . . . . . . . . . . . . . . . . 51
4.4 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.4.1 Animation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
4.4.2 Tuning the Optimization . . . . . . . . . . . . . . . . . . . . . . . . 57
4.4.3 Interpretation and Spatialization . . . . . . . . . . . . . . . . . . . . 59
4.5 Comparing Distance Preserving Methods . . . . . . . . . . . . . . . . . . . 60
4.5.1 Comparing t-SNE Options . . . . . . . . . . . . . . . . . . . . . . . . 60
4.5.2 Local Fit with Common Coverage Percentage . . . . . . . . . . . . . . 61

II Classic Clustering 63
5 Hierarchical Clustering Methods 65
5.1 Topics Covered . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
5.2 Dissimilarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
5.3 Agglomerative Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
5.3.1 Linkage and Updating Formula . . . . . . . . . . . . . . . . . . . . . 69
5.3.2 Dendrogram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
5.4 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
5.4.1 Variable Settings Dialog . . . . . . . . . . . . . . . . . . . . . . . . . 76
5.4.2 Ward’s method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
5.4.3 Single linkage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
5.4.4 Complete linkage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
5.4.5 Average linkage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
5.4.6 Sensitivity Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

6 Partitioning Clustering Methods 85

6.1 Topics Covered . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
6.2 The K Means Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
6.2.1 Iterative Relocation . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
6.2.2 The Choice of K . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
6.2.3 K-means++ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
6.3 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
6.3.1 Digression: Clustering with Dimension Reduction . . . . . . . . . . . 94
6.3.2 Cluster Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
6.3.3 Cluster Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
6.3.4 Options and Sensitivity Analysis . . . . . . . . . . . . . . . . . . . . 98
6.4 Cluster Categories as Variables . . . . . . . . . . . . . . . . . . . . . . . . . . 101
6.4.1 Conditional Box Plot . . . . . . . . . . . . . . . . . . . . . . . . . . 102
6.4.2 Aggregation by Cluster . . . . . . . . . . . . . . . . . . . . . . . . . 102
Contents ix

7 Advanced Clustering Methods 105

7.1 Topics Covered . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
7.2 K-Medians . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
7.2.1 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
7.2.2 Options and Sensitivity Analysis . . . . . . . . . . . . . . . . . . . . 108
7.3 K-Medoids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
7.3.1 The PAM Algorithm for K-Medoids . . . . . . . . . . . . . . . . . . 109
7.3.2 Improving on the PAM Algorithm . . . . . . . . . . . . . . . . . . . 115
7.3.3 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
7.3.4 Options and Sensitivity Analysis . . . . . . . . . . . . . . . . . . . . 120

8 Spectral Clustering 121

8.1 Topics Covered . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
8.2 Spectral Clustering Logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
8.3 Clustering as a Graph Partitioning Problem . . . . . . . . . . . . . . . . . . 123
8.3.1 Graph Laplacian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
8.4 The Spectral Clustering Algorithm . . . . . . . . . . . . . . . . . . . . . . . 125
8.4.1 Adjacency matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
8.4.2 Clustering on the Eigenvectors of the Graph Laplacian . . . . . . . . 126
8.4.3 Spectral Clustering Parameters . . . . . . . . . . . . . . . . . . . . . 126
8.5 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
8.5.1 Cluster results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
8.5.2 Options and Sensitivity Analysis . . . . . . . . . . . . . . . . . . . . 129

III Spatial Clustering 131

9 Spatializing Classic Clustering Methods 133
9.1 Topics Covered . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
9.2 Clustering on Geographic Coordinates . . . . . . . . . . . . . . . . . . . . . 134
9.2.1 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
9.3 Including Geographical Coordinates in the Feature Set . . . . . . . . . . . . 136
9.3.1 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
9.4 Weighted Optimization of Geographical and Attribute Similarity . . . . . . 139
9.4.1 Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
9.4.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
9.5 Constructing a Spatially Contiguous Solution . . . . . . . . . . . . . . . . . 144
9.5.1 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146

10 Spatially Constrained Clustering – Hierarchical Methods 149

10.1 Topics Covered . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
10.2 Spatially Constrained Hierarchical Clustering (SCHC) . . . . . . . . . . . . 150
10.2.1 The Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
10.2.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
10.3 SKATER . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
10.3.1 Pruning the Minimum Spanning Tree . . . . . . . . . . . . . . . . . 157
10.3.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
10.4 REDCAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
10.4.1 Illustration – FullOrder-CompleteLinkage . . . . . . . . . . . . . . . 163
10.4.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
10.5 Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
x Contents

11 Spatially Constrained Clustering – Partitioning Methods 167

11.1 Topics Covered . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
11.2 Automatic Zoning Procedure (AZP) . . . . . . . . . . . . . . . . . . . . . . 168
11.2.1 AZP Heuristic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
11.2.2 Tabu Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
11.2.3 Simulated Annealing . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
11.2.4 ARiSeL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
11.2.5 Using the Outcome from Another Cluster Routine as the Initial
Feasible Region . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
11.2.6 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
11.2.7 Search Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
11.2.8 Initialization Options . . . . . . . . . . . . . . . . . . . . . . . . . . 178
11.3 Max-P Region Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
11.3.1 Max-p Heuristic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
11.3.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
11.3.3 Sensitivity Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

IV Assessment 189
12 Cluster Validation 191
12.1 Topics Covered . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
12.2 Internal Validity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
12.2.1 Traditional Measures of Fit . . . . . . . . . . . . . . . . . . . . . . . 193
12.2.2 Balance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
12.2.3 Join Count Ratio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
12.2.4 Compactness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
12.2.5 Connectedness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
12.2.6 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
12.3 External Validity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
12.3.1 Classic Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
12.3.2 Visualizing Cluster Match . . . . . . . . . . . . . . . . . . . . . . . . 200
12.4 Beyond Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202

Bibliography 205

Index 211
List of Figures

2.1 Clusters > PCA | MDS | t-SNE . . . . . . . . . . . . . . . . . . . . . . . . 8

2.2 Vectors in Two-Dimensional Space . . . . . . . . . . . . . . . . . . . . . . . 9
2.3 Italy Bank Characteristics Descriptive Statistics . . . . . . . . . . . . . . . 14
2.4 PCA Settings Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.5 PCA Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.6 Principal Components in the Data Table . . . . . . . . . . . . . . . . . . . 16
2.7 Variable Loadings Using the EIGEN Mehtod . . . . . . . . . . . . . . . . . 18
2.8 Principal Component Calculation . . . . . . . . . . . . . . . . . . . . . . . 18
2.9 Principal Component Scatter Plot . . . . . . . . . . . . . . . . . . . . . . . 19
2.10 SVD versus EIGEN Results . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.11 Principal Components and PCP . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.12 Box Map of Second Principal Component . . . . . . . . . . . . . . . . . . . 22
2.13 Principal Component Local Geary Cluster Map and PCP . . . . . . . . . . 23
2.14 Principal Component and Multivariate Local Geary Cluster Map . . . . . 23

3.1 Clusters > PCA | MDS | t-SNE . . . . . . . . . . . . . . . . . . . . . . . . 26

3.2 MDS Main Dialog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.3 MDS Scatter Plot – Default Settings . . . . . . . . . . . . . . . . . . . . . . 31
3.4 MDS and PCA Scatter Plot . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.5 MDS Dimensions and PCA . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.6 MDS Power Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.7 Iterative Majorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.8 MDS Scatter Plot for SMACOF . . . . . . . . . . . . . . . . . . . . . . . . 36
3.9 MDS Scatter Plot for SMACOF with Manhattan Distance . . . . . . . . . 37
3.10 MDS Classic vs SMACOF Dimensions . . . . . . . . . . . . . . . . . . . . 37
3.11 MDS and Parallel Coordinate Plot . . . . . . . . . . . . . . . . . . . . . . 38
3.12 MDS with a Categorical Variable . . . . . . . . . . . . . . . . . . . . . . . 39
3.13 3D MDS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.14 3D MDS after Rotation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.15 Linked Local Moran Cluster Map and MDS . . . . . . . . . . . . . . . . . 42
3.16 Common Connectivity K-6 Neighbors . . . . . . . . . . . . . . . . . . . . . 43
3.17 Local Neighbor Match Test for MDS . . . . . . . . . . . . . . . . . . . . . 45
3.18 Multivariate Local Neighbor Match Test . . . . . . . . . . . . . . . . . . . 45
3.19 HDBSCAN for MDS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

4.1 Clusters > PCA | MDS | t-SNE . . . . . . . . . . . . . . . . . . . . . . . . 48

4.2 t-SNE Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.3 t-SNE after full iteration – default settings . . . . . . . . . . . . . . . . . . 54
4.4 t-SNE 2D Coordinates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4.5 Inspecting interations of t-SNE algorithm: 250 and 300 . . . . . . . . . . . 56
4.6 Inspecting interations of t-SNE algorithm: 1000 and 3000 . . . . . . . . . . 56
4.7 t-SNE 2D Coordinates for theta = 0 . . . . . . . . . . . . . . . . . . . . . 57

xi
xii List of Figures

4.8 t-SNE 2D Coordinates for perplexity=50 . . . . . . . . . . . . . . . . . . . 58

4.9 t-SNE 2D Coordinates for perplexity=50, momentum=100 . . . . . . . . . 58
4.10 Nearest Neighbor Match Test for t-SNE . . . . . . . . . . . . . . . . . . . 59
4.11 Common Coverage Percentage measures of fit for different methods . . . . . 61

5.1 Clusters > K Means | K Medians | K Medoids | Spectral | Hierarchical . . 67

5.2 Single Linkage Hierarchical Clustering Toy Example . . . . . . . . . . . . . 72
5.3 Coordinate Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
5.4 Single Linkage – Step 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
5.5 Smallest nearest neighbors . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
5.6 Single Linkage – Step 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
5.7 Single Linkage – Step 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
5.8 Single Linkage – Step 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
5.9 Single Linkage – Step 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
5.10 Single Linkage – Step 6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
5.11 Single linkage iterations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
5.12 Single linkage dendrogram . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
5.13 CMAP SDOH Variables Descriptive Statistics . . . . . . . . . . . . . . . . 76
5.14 Hierarchical Clustering Variable Settings . . . . . . . . . . . . . . . . . . . 77
5.15 Initial Dendrogram – Ward’s Method . . . . . . . . . . . . . . . . . . . . . 77
5.16 Dendrogram – Ward’s Method, k=5 . . . . . . . . . . . . . . . . . . . . . . 78
5.17 Summary – Ward’s Method, k=5 . . . . . . . . . . . . . . . . . . . . . . . 78
5.18 Cluster Map – Ward’s Method, k=5 . . . . . . . . . . . . . . . . . . . . . . 79
5.19 Dendrogram – Single Linkage, k=5 . . . . . . . . . . . . . . . . . . . . . . 80
5.20 Summary – Single Linkage, k=5 . . . . . . . . . . . . . . . . . . . . . . . . . 81
5.21 Cluster Map – Single Linkage, k=5 . . . . . . . . . . . . . . . . . . . . . . . 81
5.22 Dendrogram – Complete Linkage, k=5 . . . . . . . . . . . . . . . . . . . . 82
5.23 Summary – Complete Linkage, k=5 . . . . . . . . . . . . . . . . . . . . . . 82
5.24 Cluster Map – Complete Linkage, k=5 . . . . . . . . . . . . . . . . . . . . 83
5.25 Dendrogram – Average Linkage, k=5 . . . . . . . . . . . . . . . . . . . . . 83
5.26 Summary – Average Linkage, k=5 . . . . . . . . . . . . . . . . . . . . . . . 84
5.27 Cluster Map – Average Linkage, k=5 . . . . . . . . . . . . . . . . . . . . . 84

6.1 Clusters > K Means | K Medians | K Medoids | Spectral | Hierarchical . . 86

6.2 K-means toy example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
6.3 Worked example – basic data . . . . . . . . . . . . . . . . . . . . . . . . . 88
6.4 Steps in the K-means algorithm . . . . . . . . . . . . . . . . . . . . . . . . 89
6.5 Squared distance to seeds . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
6.6 Step 1 – Summary characteristics . . . . . . . . . . . . . . . . . . . . . . . 90
6.7 Squared distance to Step 1 centers . . . . . . . . . . . . . . . . . . . . . . . 90
6.8 Step 2 – Summary characteristics . . . . . . . . . . . . . . . . . . . . . . . 90
6.9 Squared distance to Step 2 centers . . . . . . . . . . . . . . . . . . . . . . . 90
6.10 Step 3 – Summary characteristics . . . . . . . . . . . . . . . . . . . . . . . . 91
6.11 SDOH Census Tract Variables Descriptive Statistics . . . . . . . . . . . . . 93
6.12 Principal Components Variable Loadings . . . . . . . . . . . . . . . . . . . 94
6.13 K-Means Clustering Variable Settings . . . . . . . . . . . . . . . . . . . . . 95
6.14 Summary – K-Means Method, k=8 . . . . . . . . . . . . . . . . . . . . . . 96
6.15 PCP Plot of K-Means Cluster Centers . . . . . . . . . . . . . . . . . . . . 96
6.16 Cluster Map – K-Means Method, k=8 . . . . . . . . . . . . . . . . . . . . 97
6.17 K-Means – Evolution of fit with k . . . . . . . . . . . . . . . . . . . . . . . 98
6.18 K-Means – Elbow Plot WSS . . . . . . . . . . . . . . . . . . . . . . . . . . 99
List of Figures xiii

6.19 K-Means – Elbow Plot BSS/TSS . . . . . . . . . . . . . . . . . . . . . . . 99

6.20 Summary – K-Means Method, population bound 200,000, k=8 . . . . . . . 100
6.21 Cluster Map – K-Means Method, population bound 200,000, k=8 . . . . . 100
6.22 Conditional Box Plot by Cluster Category . . . . . . . . . . . . . . . . . . . 101
6.23 Dissolved Cluster Categories . . . . . . . . . . . . . . . . . . . . . . . . . . 102
6.24 Aggregated Cluster Categories . . . . . . . . . . . . . . . . . . . . . . . . . 102

7.1 Summary – K-Medians Method, k=8 . . . . . . . . . . . . . . . . . . . . . 106

7.2 Cluster Map – K-Medians Method, k=8 . . . . . . . . . . . . . . . . . . . 107
7.3 Manhattan Inter-Point Distance Matrix . . . . . . . . . . . . . . . . . . . . 109
7.4 BUILD – Step 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
7.5 Cluster Assignment at end of BUILD stage . . . . . . . . . . . . . . . . . . 110
7.6 PAM SWAP Case 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
7.7 PAM SWAP Case 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
7.8 PAM SWAP Cost Changes – Step 1 . . . . . . . . . . . . . . . . . . . . . . 112
7.9 PAM SWAP Cost Changes – Step 2 . . . . . . . . . . . . . . . . . . . . . . 113
7.10 PAM SWAP Cost Changes – Step 3 . . . . . . . . . . . . . . . . . . . . . . 114
7.11 PAM SWAP Cost Changes – Step 4 . . . . . . . . . . . . . . . . . . . . . . 115
7.12 PAM SWAP Final Assignment . . . . . . . . . . . . . . . . . . . . . . . . . 115
7.13 Summary, K-Medoids, k=8 . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
7.14 K-Medoids Cluster Map, k=8 . . . . . . . . . . . . . . . . . . . . . . . . . 119

8.1 Spirals Data Set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

8.2 K-Means and K-Medoids for Spirals Data . . . . . . . . . . . . . . . . . . 122
8.3 Spectral Clustering of Spirals Data . . . . . . . . . . . . . . . . . . . . . . 123
8.4 Cluster Characteristics for Spectral Clustering of Spirals Data – Default
Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
8.5 Cluster Characteristics for Spectral Clustering of Spirals Data – Mutual
KNN=3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
8.6 Cluster Characteristics for Spectral Clustering of Spirals Data – Mutual
KNN=3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
8.7 Cluster Characteristics for Spectral Clustering of Spirals Data – Gaussian
Kernel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
8.8 Cluster Characteristics for Spectral Clustering of Spirals Data – Gaussian
Kernel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

9.1 K-Means and K-Medoids on X-Y Coordinates . . . . . . . . . . . . . . . . 135

9.2 Spectral and Hierarchical on X-Y Coordinates . . . . . . . . . . . . . . . . 135
9.3 K-Means without and with X-Y Coordinates included . . . . . . . . . . . . 136
9.4 K-Means Cluster Characteristics . . . . . . . . . . . . . . . . . . . . . . . . 137
9.5 K-Means Cluster Characteristics with X-Y Coordinates . . . . . . . . . . . 138
9.6 Connectivity Check for Cluster Members . . . . . . . . . . . . . . . . . . . . 141
9.7 Cluster Map, K-Means with Weighted Optimization . . . . . . . . . . . . . 142
9.8 Cluster Characteristics, K-Means with Weighted Optimization . . . . . . . 143
9.9 Cluster Characteristics, K-Means with Weights at 0.25 . . . . . . . . . . . 144
9.10 Make Spatial Cluster 11 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
9.11 Make Spatial Cluster 10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
9.12 Make Spatial Cluster Map . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
9.13 Make Spatial Cluster Characteristics . . . . . . . . . . . . . . . . . . . . . 147

10.1 Clusters > SCHC | skater | redcap . . . . . . . . . . . . . . . . . . . . . . 150

xiv List of Figures

10.2 Arizona counties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

10.3 County identifiers and standardized variable . . . . . . . . . . . . . . . . . 152
10.4 AZ county queen contiguity (GAL) . . . . . . . . . . . . . . . . . . . . . . 152
10.5 SCHC complete linkage step 1 . . . . . . . . . . . . . . . . . . . . . . . . . 152
10.6 SCHC complete linkage step 2 . . . . . . . . . . . . . . . . . . . . . . . . . 153
10.7 SCHC complete linkage dendrogram, k=4 . . . . . . . . . . . . . . . . . . 153
10.8 SCHC complete linkage cluster map, k=4 . . . . . . . . . . . . . . . . . . 154
10.9 SCHC variable selection dialog . . . . . . . . . . . . . . . . . . . . . . . . . 154
10.10 SCHC cluster map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
10.11 SCHC cluster characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . 156
10.12 SKATER minimum spanning tree . . . . . . . . . . . . . . . . . . . . . . . 157
10.13 SKATER step 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
10.14 SKATER minimum spanning tree – first split . . . . . . . . . . . . . . . . 158
10.15 SKATER minimum spanning tree – final split . . . . . . . . . . . . . . . . 159
10.16 SKATER cluster map, k=4 . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
10.17 Ceará, first cut in MST . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
10.18 Ceará, SKATER cluster map, k=12 . . . . . . . . . . . . . . . . . . . . . . 160
10.19 Ceará, SKATER cluster characteristics, k=12 . . . . . . . . . . . . . . . . . 161
10.20 Ceará, constrained SKATER cluster map, k=7 with min population size . 162
10.21 Ceará, constrained SKATER cluster map, k=12 with min cluster size 10 . 162
10.22 REDCAP FullOrder-CompleteLinkage spanning tree . . . . . . . . . . . . 164
10.23 REDCAP step 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
10.24 Ceará, REDCAP cluster map, k=12 . . . . . . . . . . . . . . . . . . . . . . 165
10.25 Ceará, REDCAP cluster characteristics, k=12 . . . . . . . . . . . . . . . . 166

11.1 Clusters > AZP | max-p . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168

11.2 Arizona AZP initial feasible solution . . . . . . . . . . . . . . . . . . . . . 170
11.3 Arizona AZP initial Total Within SSD . . . . . . . . . . . . . . . . . . . . 170
11.4 Initial neighbor list . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
11.5 Step 1 neighbor selection (2) . . . . . . . . . . . . . . . . . . . . . . . . . . 170
11.6 Arizona AZP step 2 neighbor selection – 14 . . . . . . . . . . . . . . . . . . 171
11.7 Arizona AZP step 2 Total Within SSD . . . . . . . . . . . . . . . . . . . . . 171
11.8 Arizona AZP step 3 neighbor selection – 3 . . . . . . . . . . . . . . . . . . 172
11.9 Arizona AZP step 3 Total Within SSD . . . . . . . . . . . . . . . . . . . . 172
11.10 Ceará AZP clusters for p=12, local search . . . . . . . . . . . . . . . . . . 176
11.11 Ceará AZP clusters for p=12, tabu search . . . . . . . . . . . . . . . . . . 176
11.12 Ceará AZP clusters for p=12, simulated annealing search . . . . . . . . . . 177
11.13 Ceará AZP clusters for p=12, simulated annealing search with ARiSeL . . 178
11.14 Ceará AZP clusters for p=12, simulated annealing search with SCHC initial
regions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
11.15 Arizona counties population . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
11.16 Arizona max-p growth, start with 6 . . . . . . . . . . . . . . . . . . . . . . . 181
11.17 Arizona max-p growth phase – 6 and 2 . . . . . . . . . . . . . . . . . . . . 182
11.18 Arizona max-p growth phase – region 1 . . . . . . . . . . . . . . . . . . . . 182
11.19 Arizona max-p growth phase – regions 1 and 2 . . . . . . . . . . . . . . . . 182
11.20 Arizona max-p growth phase – pick 3 . . . . . . . . . . . . . . . . . . . . . 183
11.21 Arizona max-p growth phase – join 3 and 14 . . . . . . . . . . . . . . . . . 183
11.22 Arizona max-p growth phase – region 1, 2, 3 . . . . . . . . . . . . . . . . . 183
11.23 Arizona max-p growth phase – pick 12 . . . . . . . . . . . . . . . . . . . . 184
11.24 Arizona max-p growth phase – join 12 and 4 . . . . . . . . . . . . . . . . . 184
11.25 Arizona max-p enclaves and initial regions 1, 2, 3, 4 . . . . . . . . . . . . . 184
List of Figures xv

11.26 Arizona max-p assign enclaves . . . . . . . . . . . . . . . . . . . . . . . . . 185

11.27 Arizona max-p – feasible initial regions . . . . . . . . . . . . . . . . . . . . 186
11.28 Ceará max-p regions with population bound 422,619 . . . . . . . . . . . . 186
11.29 Ceará max-p regions with population bound 253,571 . . . . . . . . . . . . 187

12.1 Clusters > Cluster Match Map | Make Spatial | Validation . . . . . . . . 192
12.2 Hierarchical Clustering – Ward’s method, Ceará . . . . . . . . . . . . . . . 196
12.3 Internal Validation Measures . . . . . . . . . . . . . . . . . . . . . . . . . . 196
12.4 Internal Validation Result – Hierarchical Clustering . . . . . . . . . . . . . 197
12.5 Internal Validation Result – AZP with Initial Region . . . . . . . . . . . . 198
12.6 Adjusted Rand Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
12.7 Normalized Information Distance . . . . . . . . . . . . . . . . . . . . . . . 200
12.8 K-Means and SKATER overlap . . . . . . . . . . . . . . . . . . . . . . . . . 201
12.9 SCHC and REDCAP overlap . . . . . . . . . . . . . . . . . . . . . . . . . . 201
12.10 Cluster Match Map – SKATER and K-MEANS . . . . . . . . . . . . . . . 202
Taylor & Francis
Taylor & Francis Group
http://taylorandfrancis.com
Preface

In contrast to the materials covered in Volume 1, this second volume has no precedent in
an earlier workbook. Much of its contents have been added in recent years to the GeoDa
documentation pages, as the topics were gradually included into my Introduction to Spatial
Data Science course and implemented in GeoDa. At one point, the material became too much
to constitute a single course and was split off into a separate Spatial Clustering course. The
division of the content between the two volumes follows this organization.
In contrast to the first volume, where the focus is almost exclusively on data exploration,
here attention switches to the delineation of groupings of observations, i.e., clusters. Both
traditional and spatially constrained methods are considered. Again, the emphasis is on how
a spatial perspective can contribute to additional insight, both by considering the spatial
aspects explicitly (as in spatially constrained clustering) as well as through spatializing
classic techniques.
Compared to Volume 1, the treatment is slightly more mathematical and familiarity with the
methods covered in the first volume is assumed. As before, extensive references are provided.
However, in contrast to the first volume, several methods included here are new and have
not been treated extensively in earlier publications. They were typically introduced as part
of the documentation of new features in GeoDa.
The empirical illustrations use the same sample data sets as in Volume 1. These are included
in the software.
All applications are based on Version 1.22 of the software, available in Summer 2023. Later
versions may include slight changes as well as additional features, but the treatment provided
here should remain valid. The software is free, cross-platform and open source, and can be
downloaded from https://geodacenter.github.io/download.html.

Acknowledgments
This second volume is based on enhancements in the GeoDa software implemented in the
past five or so years, with Xun Li as the lead software engineer and Julia Koschinsky as
a constant source of inspiration and constructive comments. The software development
received institutional support by the University of Chicago to the Center for Spatial Data
Science.
Help and suggestions with the production process from Lara Spieker of Chapman & Hall is
greatly appreciated.
As for the first volume, Emily has been patiently living with my GeoDa obsession for many
years. This volume is also dedicated to her.
Shelby, MI, Summer 2023

xvii
Taylor & Francis
Taylor & Francis Group
http://taylorandfrancis.com
About the Author

Luc Anselin is the Founding Director of the Center for Spatial Data Science at the University
of Chicago, where he is also Stein-Freiler Distinguished Service Professor of Sociology and the
College. He previously held faculty appointments at Arizona State University, the University
of Illinois at Urbana-Champaign, the University of Texas at Dallas, the Regional Research
Institute at West Virginia University, the University of California, Santa Barbara, and The
Ohio State University. He also was a visiting professor at Brown University and MIT. He
holds a PhD in Regional Science from Cornell University.
Over the past four decades, he has developed new methods for exploratory spatial data
analysis and spatial econometrics, including the widely used local indicators of spatial
autocorrelation. His 1988 Spatial Econometrics text has been cited some 17,000 times. He
has implemented these methods into software, including the original SpaceStat software, as
well as GeoDa, and as part of the Python PySAL library for spatial analysis.
His work has been recognized by several awards, including election to the U.S. National
Academy of Sciences and the American Academy of Arts and Sciences.

xix
Taylor & Francis
Taylor & Francis Group
http://taylorandfrancis.com
1
Introduction

This second volume in the Introduction to Spatial Data Science is devoted to the topic
of spatial clustering. More specifically, it deals with the grouping of observations into a
smaller number of clusters, which are designed to be representative of their members. The
techniques considered constitute an important part of so-called unsupervised learning in
modern machine learning. Purely statistical methods to discover spatial clusters in data are
beyond the scope.
In contrast to Volume 1, which assumed very little prior (spatial) knowledge, the current
volume is somewhat more advanced. At a minimum, it requires familiarity with the scope
of the exploratory toolbox included in the GeoDa software. In that sense, it clearly builds
upon the material covered in Volume 1. Important principles that are a main part of the
discussion in Volume 1 are assumed known. This includes linking and brushing, the various
types of maps and graphs, spatial weights and spatial autocorrelation statistics.
Much of the material covered in this volume pertains to methods that have been incorporated
into the GeoDa software only in the past few years, so as to support the second part of an
Introduction to Spatial Data Science course sequence. The particular perspective offered is the
tight integration of the clustering results with a spatial representation, through customized
cluster maps and by exploiting linking and brushing.
The treatment is slightly more technical than in the previous volume, but the mathematical
details can readily be skipped if the main interest is in application and interpretation.
Necessarily, the discussion relies on somewhat more formal concepts. Some examples are the
treatment of matrix eigenvalues and matrix decomposition, the concept of graph Laplacian,
essentials of information theory, elements of graph theory, advanced spatial data structures
such as quadtree and vantage point tree, and optimization algorithms like gradient search,
iterative greedy descent, simulated annealing and tabu search. These concepts are not
assumed known but will be explained in the text.
While many of the methods covered constitute part of mainstream data science, the perspec-
tive offered here is rather unique, with an enduring attempt at spatializing the respective
methods. In addition, the treatment of spatially constrained clustering introduces contiguity
as an additional element into clustering algorithms.
Most methods discussed are familiar from the literature, but some are new. Examples
include the common coverage percentage, a local measure of goodness of fit between distance
preserving dimension reduction methods, two new spatial measures to assess cluster quality,
i.e., the join count ratio and the cluster match map, a heuristic to obtain contiguous results
from classic clustering results and a hybrid approach toward spatially constrained clustering,
whereby the outcome of a given method is used as the initial feasible region in a second
method. The techniques are the results of refinements in the software and the presentation
of cluster results, and have not been published previously. In addition, the various methods
to spatialize cluster results are mostly also unique to the treatment in this volume.

DOI: 10.1201/9781032713175-1 1
2 Introduction

As in Volume 1, the coverage here also constitutes the definitive user’s guide to the GeoDa
software, complementing the previous discussion.
In the remainder of this introduction, I provide a broad overview of the organization of
Volume 2, followed by a listing of the sample data sets used. As was the case for Volume 1,
these data sets are included as part of the GeoDa software and do not need to be downloaded
separately. For a quick tour of the GeoDa software, I refer to the Introduction of Volume 1.

1.1 Overview of Volume 2

Volume 2 is organized into four parts:
• Dimension reduction
• Classic clustering
• Spatial clustering
• Assessment
The first part reviews classic dimension reduction techniques, divided into three chapters,
devoted to principal components, multidimensional scaling and stochastic neighbor embed-
ding. In addition to the discussion of the classic properties, specific attention is paid to
spatializing these techniques, i.e., bringing out interesting spatial aspects of the results.
Part II covers classic clustering methods, in contrast to spatially constrained clusters, which
are the topic of Part III. Four chapters deal with, respectively, hierarchical clustering methods,
partitioning clustering methods (K-Means), advanced methods (K-Medians and K-Medoids)
and spectral clustering.
The chapters in Part III deal with methods to include an explicit spatial constraint of
contiguity into the clustering routine. The first chapter outlines techniques to spatialize
classic clustering methods, which involve soft spatial constraints. These techniques do not
guarantee a spatially compact (contiguous) solution. In contrast, the methods discussed in
the next two chapters impose hard spatial constraints. One chapter deals with hierarchical
approaches (spatially constrained hierarchical clustering, SKATER and REDCAP) and the
other with partitioning methods (AZP and max-p).
Part IV deals with assessment and includes a final chapter outlining a range of approaches
to validate the cluster results, both in terms of internal validity and external validity. It
closes with some concluding remarks.
As before, in addition to the material covered in this volume, the GeoDaCenter Github
site (https://geodacenter.github.io) contains an extensive support infrastructure. This
includes detailed documentation and illustrations, as well as a large collection of sample data
sets, cookbook examples and links to a YouTube channel containing lectures and tutorials.
Specific software support is provided by means of a list of frequently asked questions and
answers to common technical questions, as well as by the community through the Google
Groups Openspace list.
 Sample Data Sets 3

1.2 Sample Data Sets

As in Volume 1, the methods and software are illustrated by means of empirical examples
that are available directly from inside the GeoDa software. In Volume 2, only a subset of the
full slate of sample data are used.
These are
• Italian community banks (n=261)
– bank performance indicators for 2011-17 (used by Algeri et al., 2022)
– see Chapters 2 through 4
• Chicago CCA Profiles (n=77)
– socio-economic snapshot for Chicago Community Areas in 2020 (American Commu-
nity Survey from the Chicago Metropolitan Agency for Planning – CMAP – data
portal)
– see Chapter 5
• Chicago Census Tracts (n=791)
– socio-economic determinants of health in 2014 (a subset of the data used in Kolak
et al., 2020)
– see Chapters 6 and 7
• Spirals (n=300)
– canonical data set to test spectral clustering
– see Chapter 8
• Municipalities in the State of Ceará, Brazil (n=184)
– Zika and Microcephaly infections and socio-economic profiles for 2013-2016 (adapted
from Amaral et al., 2019)
– see Chapters 9 through 12
Further details are provided in the context of specific methods.
Taylor & Francis
Taylor & Francis Group
http://taylorandfrancis.com
 Part I

Dimension Reduction
Taylor & Francis
Taylor & Francis Group
http://taylorandfrancis.com
2
Principal Component Analysis (PCA)

The familiar curse of dimensionality affects analysis across two dimensions. One is the
number of observations (big data) and the other is the number of variables considered.
The methods included in Volume 2 address this problem by reducing the dimensionality,
either in the number of observations (clustering) or in the number of variables (dimension
reduction). The three chapters in Part I address the latter problem. This chapter covers
principal components analysis (PCA), a core method of both multivariate statistics and
machine learning. Dimension reduction is particularly relevant in situations where many
variables are available that are highly intercorrelated. In essence, the original variables are
replaced by a smaller number of proxies that represent them well in terms of their statistical
properties.
Before delving into the formal derivation of principal components, a brief review is included
of some basic concepts from matrix algebra, focusing in particular on matrix decomposition.
Next follows a discussion of the mathematical properties of principal components and their
implementation and interpretation.
A distinct characteristic of this chapter is the attention paid to spatializing the inherently
non spatial concept of principal components. This is achieved by exploiting geovisualization,
linking and brushing to represent the dimension reduction in geographic space. Of particular
interest are principal component maps and the connection between univariate local cluster
maps for principal components and their multivariate counterpart.
The methods are illustrated using the Italy Community Banks sample data set.

2.1 Topics Covered

• Understand the mathematics behind principal component analysis
• Compute principal components for a set of variables
• Interpret the characteristics of a principal component analysis
• Spatialize the principal components
• Investigate the connection between clustering of principal components and multivariate
clustering
GeoDa Functions
• Clusters > PCA
– select variables
– PCA parameters
– PCA summary statistics
– saving PCA results

DOI: 10.1201/9781032713175-2 7
8 Principal Component Analysis (PCA)

Toolbar Icons

Figure 2.1: Clusters > PCA | MDS | t-SNE

2.2 Matrix Algebra Review

Before moving on to the mathematics of principal components analysis, a brief review of
some basic matrix algebra concepts is included here. Readers already familiar with this
material can easily skip this section.
Vectors and matrices are ways to collect a lot of information and manipulate it in a concise
mathematical way. One can think of a matrix as a table with rows and columns, and a
vector as a single row or column. In two dimensions, i.e., for two values, a vector can be
visualized as an arrow between the origin of the coordinate system (0,0) and a given point.
The first value corresponds to the x-axis and the second value corresponds to the y-axis. In
Figure 2.2, this is illustrated for the vector:
 
1
v= .
2
The arrow in the figure connects the origin of a x − y scatter plot to the point (x = 1, y = 2).
A central application in matrix algebra is the multiplication of vectors and matrices. The
simplest case is the multiplication of a vector by a scalar (i.e., a single number). Graphically,
multiplying a vector by a scalar just moves the end point further or closer on the same slope.
For example, multiplying the vector v by the scalar 2 gives:
   
1 2
2× =
2 4
This is equivalent to moving the arrow over on the same slope from (1,2) to the point (2,4)
further from the origin, as shown by the dashed red line in Figure 2.2.
 Matrix Algebra Review 9

Figure 2.2: Vectors in Two-Dimensional Space

Multiplying a matrix by a vector is slightly more complex, but again corresponds to a simple
geometric transformation. For example, consider the 2 × 2 matrix A:
 
1 3
A= .
3 2
The result of a multiplication of a 2 × 2 matrix by a 2 × 1 column vector is a 2 × 1 column
vector. The first element of this vector is obtained as the product of the matching elements
of the first row with the vector, the second element similarly as the product of the matching
elements of the second row with the vector. In the example, this boils down to:
   
(1 × 1) + (3 × 2) 7
Av = = .
(3 × 1) + (2 × 2) 5
Geometrically, this consists of a combination of rescaling and rotation. For example, in
Figure 2.2, first the slope of the vector is changed, followed by a rescaling to the point (7,5),
as shown by the blue dashed arrows.
A case of particular interest is for any matrix A to find a vector v, such that when post-
multiplied by that vector, there is only rescaling and no rotation. In other words, instead of
finding what happens to the point (1,2) after pre-multiplying by the matrix A, the interest
focuses on finding a particular vector that just moves a point up or down on the same slope
for that particular matrix. As it turns out, there are several such solutions. This problem
is known as finding eigenvectors and eigenvalues for a matrix. It has a broad range of
applications, including in the computation of principal components.

2.2.1 Eigenvalues and eigenvectors

The eigenvectors and eigenvalues of a square symmetric matrix A are a special scalar-vector
pair, such that Av = λv, where λ is the eigenvalue and v is the eigenvector. In addition, the
different eigenvectors are such that they are orthogonal to each other. This means that the
product of two different eigenvectors is zero, i.e., vu vk = 0 (for u = k).1 Also, the sum of
squares of the eigenvector elements equals one. In vector notation, vu vu = 1.
1 The product of a row vector with a column vector is a scalar. The symbol  stands for the transpose of a
vector, in this case, a column vector that is turned into a row vector.
10 Principal Component Analysis (PCA)

What does this mean? For an eigenvector (i.e., arrow from the origin), the transformation
by A does not rotate the vector, but simply rescales it (i.e., moves it further or closer to the
origin), by exactly the factor λ.
For the example matrix A, the two eigenvectors turn out to be [0.6464 0.7630] and [-0.7630
0.6464], with associated eigenvalues 4.541 and -1.541. Each square matrix has as many
eigenvectors and matching eigenvalues as its rank, in this case 2 – for a 2 by 2 nonsingular
matrix. The actual computation of eigenvalues and eigenvectors is rather complicated, and
is beyond the scope of this discussion.
To further illustrate this concept, consider post-multiplying the matrix A with its eigenvector
[0.6464 0.7630]:    
(1 × 0.6464) + (3 × 0.7630) 2.935
=
(3 × 0.6464) + (2 × 0.7630) 3.465
The eigenvector rescaled by the matching eigenvalue gives the same result:
   
0.6464 2.935
4.541 × =
0.7630 3.465

In other words, for the point (0.6464 0.7630), a pre-multiplication by the matrix A just
moves it by a multiple of 4.541 to a new location on the same slope, without any rotation.
With the eigenvectors stacked in a matrix V, it is easy to verify that they are orthogonal
and the sum of squares of the coefficients sum to one, i.e., V  V = I (with I as the identity
matrix):
     
0.6464 −0.7630 0.6464 −0.7630 1 0
=
0.7630 0.6464 0.7630 0.6464 0 1
In addition, it is easily verified that V V  = I as well. This means that the transpose of V is
also its inverse (per the definition of an inverse matrix, i.e., a matrix for which the product
with the original matrix yields the identity matrix) or V −1 = V  .
Eigenvectors and eigenvalues are central in many statistical analyses, but it is important
to realize they are not as complicated as they may seem at first sight. On the other hand,
computing them efficiently is complicated, and best left to specialized programs.
Finally, a couple of useful properties of eigenvalues are worth mentioning.
The sum of the eigenvalues equals the trace of the matrix. The trace is the sum of the
diagonal elements. For the matrix A in the example, the trace is 1 + 2 = 3. The sum of the
two eigenvalues is 4.541 − 1.541 = 3.
In addition, the product of the eigenvalues equals the determinant of the matrix. For a
2 × 2 matrix, the determinant is ab − cd, or the product of the diagonal elements minus the
product of the off-diagonal elements. In the example, that is (1 × 2) − (3 × 3) = −7. The
product of the two eigenvalues is 4.541 × −1.541 = −7.0.

2.2.2 Matrix decompositions

In many applications in statistics and data science, a lot is gained by representing the
original matrix by a product of special matrices, typically related to the eigenvectors and
eigenvalues. These are so-called matrix decompositions. Two cases are particularly relevant
for principal component analysis, as well as many other applications: spectral decomposition
and singular value decomposition.
 Matrix Algebra Review 11

2.2.2.1 Spectral decomposition

For each eigenvector v of a square symmetric matrix A, it holds that Av = λv. This can
be written compactly for all the eigenvectors and eigenvalues by organizing the individual
eigenvectors as columns in a k × k matrix V, with k as the dimension of the matrix A.
Similarly, the matching eigenvalues can be organized as the diagonal elements of a k × k
diagonal matrix, say G.
The basic eigenvalue expression can then be written as

AV = V G.

Note that V goes first in the matrix multiplication on the right hand side to ensure that
each column of V is multiplied by the corresponding eigenvalue on the diagonal of G to yield
λv. Taking advantage of the fact that the eigenvectors are orthogonal, namely that V V  = I,
gives that post-multiplying each side of the equation by V  yields AV V  = V GV  , or

A = V GV  .

This is the so-called eigenvalue decomposition or spectral decomposition of the square

symmetric matrix A.
For any n × k matrix of standardized observations X (i.e., n observations on k variables),
the square matrix X  X corresponds to the correlation matrix. The spectral decomposition
of this matrix yields:
X  X = V GV  ,
with V as a matrix with the eigenvectors as columns, and G as a diagonal matrix containing
the eigenvalues. This property can be used to construct the principal components of the
matrix X.

2.2.2.2 Singular value decomposition (SVD)

The spectral matrix decomposition only applies to square matrices. A more general de-
composition is the so-called singular value decomposition, which applies to any rectangular
matrix, such as the n × k matrix X with (standardized) observations directly, rather than
its correlation matrix.
For a full-rank n × k matrix X (there are many more general cases where SVD can be
applied), the decomposition takes the following form:

X = U DV  ,

where U is a n × k orthonormal matrix (i.e., U  U = I), D is a k × k diagonal matrix and V

is a k × k orthonormal matrix (i.e., V  V = I).
While SVD is very general, the full generality is not needed for the PCA case. It turns out
that there is a direct connection between the eigenvalues and eigenvectors of the (square)
correlation matrix X  X and the SVD of X. Using the SVD decomposition above, and
exploiting the orthonormal properties of the various matrices, the product X  X can be
written as:
X  X = (U DV  ) (U DV  ) = V DU  U DV  = V D2 V  ,
since U  U = I.
It thus turns out that the columns of V from the SVD decomposition contain the eigenvectors
of X  X. In addition, the square of the diagonal elements of D in the SVD are the eigenvalues
12 Principal Component Analysis (PCA)

of the correlation matrix. Or, equivalently, the diagonal elements of the matrix D are the
square roots of the eigenvalues of X  X. This property can be exploited to derive the principal
components of the matrix X.

2.3 Principal Components

Principal components analysis has an eminent historical pedigree, going back to pioneering
work in the early twentieth century by the statistician Karl Pearson (Pearson, 1901) and the
economist Harold Hotelling (Hotelling, 1933). The technique is also known as the Karhunen-
Loève transform in probability theory, and as empirical orthogonal functions or EOF in
meteorology (see, for example, in applications of space-time statistics in Cressie and Wikle,
2011; Wikle et al., 2019).
The derivation of principal components can be approached from a number of different
perspectives, all leading to the same solution. Perhaps the most common treatment considers
the components as the solution of a problem of finding new variables that are constructed as
a linear combination of the original variables, such that they maximize the explained variance.
In a sense, the principal components can be interpreted as the best linear approximation to
the multivariate point cloud of the data.
The point of departure is to organize n observations on k variables xh , with h = 1, . . . , k,
as a n × k matrix X (each variable is a column in the matrix). In practice, each of the
variables is typically standardized, such that its mean is zero and variance equals one. This
avoids problems with (large) scale differences between the variables (i.e., some are very small
numbers and others very large). For such standardized variables, the k × k cross product
matrix X  X corresponds to the correlation matrix (without standardization, this would be
the variance-covariance matrix).2
The goal is to find a small number of new variables, the so-called principal components, that
explain the bulk of the variance (or, in practice, the correlation) in the original variables. If
this can be accomplished with a much smaller number of variables than in the original set,
the objective of dimension reduction will have been achieved.
Each principal component zu is a linear combination of the original variables xh , with h
going from 1 to k such that:

zu = a1 x1 + a2 x2 + · · · + ak xk

The mathematical problem is to find the coefficients ah such that the new variables maximize
the explained variance of the original variables. In addition, to avoid an indeterminate
solution, the coefficients are scaled such that the sum of their squares equals 1.
A full mathematical treatment of the derivation of the optimal solution to this problem
is beyond the current scope (for details, see, e.g., Lee and Verleysen, 2007, Chapter 2).
Nevertheless, obtaining a basic intuition for the mathematical principles involved is useful.
The coefficients by which the original variables need to be multiplied to obtain each principal
component can be shown to correspond to the elements of the eigenvectors of X  X, with the
2 The standardization should not be done mechanically, since there are instances where the variance

differences between the variables are actually meaningful, e.g., when the scales on which they are measured
have a strong substantive meaning (e.g., in psychology).
 Principal Components 13

associated eigenvalue giving the explained variance. Even though the original data matrix X
is typically not square (of dimension n × k), the cross-product matrix X  X is of dimension
k × k, so it is square and symmetric. As a result, all the eigenvalues are real numbers, which
avoids having to deal with complex numbers.
Operationally, the principal component coefficients are obtained by means of a matrix
decomposition. One option is to compute the spectral decomposition of the k × k matrix
X  X, i.e., of the correlation matrix. As shown in Section 2.2.2.1, this yields:

X  X = V GV  ,

where V is a k × k matrix with the eigenvectors as columns (the coefficients needed to

construct the principal components) and G a k × k diagonal matrix of the associated
eigenvalues (the explained variance).
A principal component for each observation is obtained by multiplying the row of standardized
observations by the column of eigenvalues, i.e., a column of the matrix V. More formally, all
the principal components are obtained concisely as:

XV.

A second and computationally preferred way to approach this is as a singular value decom-
position (SVD) of the n × k matrix X, i.e., the matrix of (standardized) observations. From
Section 2.2.2.2, this follows as
X = U DV  ,
where again V (the transpose of the k × k matrix V  ) is the matrix with the eigenvectors
of X  X as columns, and D is a k × k diagonal matrix, containing the square root of the
eigenvalues of X  X on the diagonal.3 Note that the number of eigenvalues used in the
spectral decomposition and in SVD is the same, and equals k, the column dimension of X.
Since V  V = I, the following result obtains when both sides of the SVD decomposition are
post-multiplied by V :
XV = U DV  V = U D.
In other words, the principal components XV can also be obtained as the product of the
orthonormal matrix U with a diagonal matrix containing the square root of the eigenvalues,
D. This result is important in the context of multidimensional scaling, considered in
Chapter 3.
It turns out that the SVD approach is the solution to viewing the principal components
explicitly as a dimension reduction problem, originally considered by Karl Pearson. The
observed vector on the k variables x can be expressed as a function of a number of unknown
latent variables z, such that there is a linear relationship between them:

x = Az,

where x is a k × 1 vector of the observed variables, and z is a p × 1 vector of the (unobserved)

latent variables, ideally with p much smaller than k. The matrix A is of dimension k × p
and contains the coefficients of the transformation. Again, in order to avoid indeterminate
solutions, the coefficients are scaled such that A A = I, which ensures that the sum of
squares of the coefficients associated with a given component equals one.
3 Since the eigenvalues equal the variance explained by the corresponding component, the diagonal elements

of D are thus the standard deviation explained by the component.

14 Principal Component Analysis (PCA)

Figure 2.3: Italy Bank Characteristics Descriptive Statistics

Instead of maximizing explained variance, the objective is now to find A and z such that
the so-called reconstruction error is minimized.4
Importantly, different computational approaches to obtain the eigenvalues and eigenvectors
(there is no analytical solution) may yield opposite signs for the elements of the eigenvectors.
However, the eigenvalues will be the same. The sign of the eigenvectors will affect the sign
of the resulting component, i.e., positives become negatives. For example, this can be the
difference between results based on a spectral decomposition versus SVD.
In a principal component analysis, the interest typically focuses on three main results. First,
the principal component scores are used as a replacement for the original variables. This
is particularly relevant when a small number of components explain a substantial share of
the original variance. Second, the relative contribution of each of the original variables to
each principal component is of interest. Finally, the variance proportion explained by each
component in and of itself is also important.

2.3.1 Implementation
Principal components are invoked from the drop-down list created by the toolbar Clusters
icon (Figure 2.1) as the top item (more precisely, the first item in the dimension reduction
category). Alternatively, from the main menu, Clusters > PCA gets the process started.
The illustration uses ten variables that characterize the efficiency of community banks, based
on the observations for 2013 from the Italy Community Bank sample data set (see Algeri
et al., 2022):
• CAPRAT: ratio of capital over risk weighted assets
• Z: z score of return on assets (ROA) + leverage over the standard deviation of ROA
• LIQASS: ratio of liquid assets over total assets
• NPL: ratio of non performing loans over total loans
• LLP: ratio of loan loss provision over customer loans
• INTR: ratio of interest expense over total funds
• DEPO: ratio of total deposits over total assets
• EQLN: ratio of total equity over customer loans
4 The concept of reconstruction error is somewhat technical. If A were a square matrix, one could solve

for z as z = A−1 x, where A−1 is the inverse of the matrix A. However, due to the dimension reduction, A is
not square, so something called a pseudo-inverse or Moore-Penrose inverse must be used. This is the p × k
matrix (A A)−1 A , such that z = (A A)−1 A x. Furthermore, because A A = I, this simplifies to z = A x
(of course, so far the elements of A are unknown). Since x = Az, if A were known, x could be found as Az,
or, as AA x. The reconstruction error is then the squared difference between x and AA x. The objective is
to find the coefficients for A that minimize this expression. For an extensive technical discussion, see Lee
and Verleysen (2007), Chapter 2.
 Another Random Document on
Scribd Without Any Related Topics
 The Project Gutenberg eBook of Fables of
Flowers for the Female Sex. With Zephyrus
and Flora, a Vision
 This ebook is for the use of anyone anywhere in the United
States and most other parts of the world at no cost and with
almost no restrictions whatsoever. You may copy it, give it away
or re-use it under the terms of the Project Gutenberg License
included with this ebook or online at www.gutenberg.org. If you
are not located in the United States, you will have to check the
laws of the country where you are located before using this
eBook.

Title: Fables of Flowers for the Female Sex. With Zephyrus and
Flora, a Vision

Author: John Huddlestone Wynne

Release date: June 23, 2021 [eBook #65677]

Language: English

Credits: Bryan Ness and the Online Distributed Proofreading

Team at https://www.pgdp.net (This file was produced
from images generously made available by The
Internet Archive/American Libraries.)

*** START OF THE PROJECT GUTENBERG EBOOK FABLES OF

FLOWERS FOR THE FEMALE SEX. WITH ZEPHYRUS AND FLORA, A
VISION ***
 FABLES OF FLOWERS,
FOR THE FEMALE SEX.
WITH
ZEPHYRUS and FLORA,
A VISION.
WRITTEN FOR THE AMUSEMENT OF

HER HIGHNESS THE PRINCESS ROYAL

BY THE AUTHOR OF THE CHOICE EMBLEMS, &C. &C.

Nature here
Wantons as in her prime, and plays at will
Her virgin fancies.
MILTON.

LONDON:
PRINTED FOR GEORGE RILEY, BOOKSELLER,
IN CURZON-STREET, MAY-FAIR.
AND SOLD BY
JOHN WILKIE, ST. PAUL’S-CHURCH-YARD.
MDCCLXXIII.
TO THE RIGHT HONOURABLE
LADY CHARLOTTE FINCH.

THESE NEW FABLES

WRITTEN FOR THE AMUSEMENT
OF
HER HIGHNESS,

CHARLOTTE,
PRINCESS ROYAL OF ENGLAND.

ARE MOST HUMBLY DEDICATED BY

HER LADYSHIP’S
MOST HUMBLE
AND OBEDIENT SERVANT,
THE AUTHOR.
 PREFACE.

W hen I survey the divine simplicity and blooming attractions, that

are displayed amongst the variegated tribes of the vegetable
creation, I cease to wonder, that Queens forego, for a while, the
compliments of a nation, or withdraw from the glitter of a court, to
be attended with the more splendid equipage of a bed of flowers;
where nothing seems wanting but the power of speech, to make
them become the most pleasing Monitors.
How far the Author of the following Fables, written for the
amusement of an exalted Personage, may have succeeded, in
descriptive fancy, as a poet: it is hoped, that, the moral and refined
admonitions which may be found to breathe, from the fragrant
bosom of a silver-robed Lily, or a blooming Jonquil, will throw a veil
over any poetical inaccuracies; for who can paint like Nature?
As to the novelty of the plan, I cannot but hold myself, in a great
measure, indebted to an ingenious Lady[1], well known in the
literary world. And can only say, that I have found both health and
recreation in the completion of it; by sharing some of the sweetest
hours of contemplation, among the lovely subjects of the following
pages.

THE AUTHOR.
TABLE of CONTENTS.
Page
ZEPHYRUS and FLORA 1
FABLE I.
The Hollyhock and Lily of the Vale 24
II.
The Aloe in Blossom 31
III.
The Rose and Hornet 37
IV.
The Sensitive Plant 42
V.
The Hawthorn and the Primrose 47
VI.
The White Rose and the Red 53
VII.
The Crocus 59
VIII.
The Anemone and the Passion Flower 64
IX.
The Lily and Narcissus 69
X.
The Ivy and Sweet Briar 73
XI.
The Violet Transplanted 77
XII.
The Tulip and the Amaranth 81
XIII.
The Honey Succkle and Youth 87
XIV.
Belinda and the Blue-Bell 92
XV.
The Larkspur and the Myrtle 97
XVI.
The Poppy and the Sun-Flower 102
XVII.
The Iris and the Rose 109
XVIII.
The Nasturtium and the Wall Flower 113
XIX.
The Traveller and Lapland Rose 119
XX.
The Deadly Night Shade 124
XXI.
The Crown Imperial and Hearts Ease 129
XXII.
The Water Lily 133
XXIII.
The Funeral Flowers and the Lover 137
XXIV.
The Field and Garden Daisy 142
XXV.
The Pinks and Arbutus 147
XXVI.
The Cockscomb and Sweet William 153
XXVII.
The Jasmine and Hemlock 157
XXVIII.
The Carnation and Southernwood 161
XXIX.
The Rosemary and Field Flower 165
 XXX.
The Judgment of the Flowers 169
 ZEPHYRUS
AND

FLORA:
A VISION.

I.

AS late I wander’d o’er the flow’ry plain,

Where Cambrian Cluyd pours his silver tide,
Amidst the pleasures of fair Plenty’s reign,
And blushing flow’rs and fruits on ev’ry side:

II.
Soft sigh’d the west winds, murm’ring o’er the dale,
Whose ev’ry charm rose fresher from the breeze;
The lofty hills more boldly kiss’d the gale,
Which skimm’d their tops, and shook the wavy trees.

III.
The sun descending, shot his golden beams
Askance, with many a cloud his ev’ning throne
Adorn’d; while mountains, woods, and lucent streams,
With the last blushes of his radiance shone.

IV.
Far stretching hence, Cambria’s rough heights I view,
Where Liberty long since forlorn retir’d,
Left fairer climes, and skies of brighter hue,
And, but at last, triumphantly expir’d:

V.
And wide around me wound the fertile vale[2],
Fit theme and subject of the poet’s song;
Whose num’rous beauties load the passing gale,
Whose breath repeats them, as it glides along.

VI.
Bright Phœbus sunk, dim twilight now succeeds,
Still gleaming dubious with uncertain ray,
While tremblingly among the vocal reeds
The ev’ning breezes still more faintly play.

VII.
Amid this beauteous, soft, and flow’ry scene,
On a high bank, all listless, I reclin’d;
 Whose shelving sides were crown’d with lively green,
By tufted trees and bord’ring flow’rs confin’d.

VIII.
Here, while the landscape faded on my sight,
Wild Fancy’s eye still brighter scenes supply’d;
I view’d not the last track of parting light,
Nor mark’d the fanning breezes as they dy’d.

IX.
At length, Imagination, roving maid,
Though gentle sleep had fetter’d all my pow’rs
In golden chains, my busy soul convey’d
To other landscapes and immortal bow’rs.

X.
Methought I stood amidst a garden fair,
Whose bounds no sight of mortal eye could trace,
Situate mid-way, betwixt earth, seas, and air,
Unmark’d by Time, uncircumscrib’d by Space.

XI.
Not half so sweet was that delightsome dale,
Which to my waking view appear’d so bright;
For here did never-ceasing suns prevail,
With mildest sweetness temp’ring heav’nly light.

XII.
Spring breath’d eternal glories o’er the land:
And gentlest winds, o’er fragrant lawns that blow,
Nurs’d beauteous buds unset by mortal hand,
And op’ning flow’rs that without planting grow.

XIII.
Serene the heav’ns, save where a cloudy shrine,
Big with cœlestial plenty, sail’d on high,
Show’r’d Spring’s own roses from her seat divine,
 And drew a purple radiance o’er the sky.

XIV.
Meanwhile, soft music echoing from each grove,
Tun’d to enchanting notes most soft and clear,
That breath’d the soul of harmony and love,
Thrill’d the rapt breast, and charm’d the list’ning ear.

XV.
And still the while, with voices loud and sweet,
The warbling birds in dulcet concert join’d,
The waters murm’ring flow with cadence meet,
“Low answer’d by the gently whisp’ring wind.”

XVI.
These themes of wonder silent I survey’d,
Attentive hanging on each dying sound;
Pleas’d with the glories which I saw display’d,
And scenes of joy and pleasure op’ning round.

XVII.
Yet still methought a certain want appear’d,
Of some to own this spot, so heav’nly fair,
Else were each charming flow’ret vainly rear’d,
“To bloom unnotic’d to the desart air:”

XVIII.
Else, were in vain these soft melodious strains,
Which the whole soul of harmony inspir’d,
Pour’d to the wild woods and the lonely plains,
Though worthiest still by all to be admir’d.

XIX.
Such were creation’s first imperfect hours,
When the gay heav’ns in early beauty shone,
And earth, bedeck’d with beasts, birds, plants, and flow’rs,
Spread all her bosom to the genial sun;
 XX.
Unfinish’d still the mighty work appear’d,
Till Man, the lord of all, was bid to rise;
With open brow his face divine who rear’d,
And sought with upright look his native skies.

XXI.
Thus as I paus’d, still louder swell’d the notes,
From ev’ry bush, and brake, and echoing hill;
While choirs cœlestial seem’d to tune their throats,
And, with glad voice the chearful chorus fill.

XXII.
Then, by some magic pow’r swift snatch’d away,
Ev’n to the midst of that delightful land,
I view’d at once all clad in bright array,
A thousand Genii of the gardens stand.

XXIII.
But far above all these a seat was plac’d,
Dress’d with each flow’r that ev’ry season knows,
Whose vary’d tints, in gem-like order, grac’d
The rural theatre which gradual rose.

XXIV.
For lo! the Genius of each blooming flow’r
Brought his own fav’rite with peculiar care,
To deck the arch of this inchanted bow’r,
And, bowing at the throne, he plac’d it there.

XXV.
A sight more beauteous ne’er did eye behold,
Than these bright tribes that glitter’d on the day;
And, rich in purple dyes and flaming gold,
Did their bright bosoms to the sun display.

XXVI.
Such was the throne;—but oh! what pen can trace
The heav’nly beauties of the matchless Two,
Who, glowing with each bright cœlestial grace,
Sat there aloft, conspicuous to the view!

XXVII.
The first, a youth of sweet and gentle mien,
With many a wreath and knotted garland crown’d;
Whose beauteous visage glow’d with charms serene,
And on whose shoulders purple wings were bound:

XXVIII.
These when he spreads, reviving Nature pours
Her copious treasures of immortal bloom;
Whilst through vast realms he scatters vernal stores,
And from his downy pinions shakes perfume.

XXIX.
His name was Zephyrus; and next him sat
The beauteous goddess of the blooming year,
The constant partner of his rural state,
To heav’n and earth, to gods and mortals dear;

XXX.
Flora, bright pow’r, who sheds a thousand sweets
O’er thousand lands, what time her gifts appear,
What time her consort with his kisses greets
Her coral lips, and wakes the rising year.

XXXI.
Her beauteous face was deck’d with youthful pride,
Her graceful form in flamy robes was dress’d;
And ev’ry charm wild Nature could provide,
Adorn’d her head, and beam’d upon her breast.

XXXII.
Beside the throne, rang’d in fair order, stood
 The various Seasons of the rolling year;
By all their train of months, weeks, days, pursu’d:
And all their various symbols flourish’d here.

XXXIII.
First came the Spring, led by the rosy Hours,
With all the Loves and Graces in her train;
Deck’d with her wreath of never-fading flow’rs,
Diffusing odours o’er the smiling plain.

XXXIV.
Next Summer came; his cheeks with ardour fir’d,
With his own blushing fruits and harvests crown’d;
Before whose face the infant Spring retir’d,
And with her roses strew’d the russet ground.

XXXV.
Stain’d with the grape’s press’d juice, with steadier pace,
Still looking backward on preceding time,
Ripe Autumn next succeeded in his place;
Scatt’ring rich fruits, the growth of ev’ry clime.

XXXVI.
Last Winter comes, with heavy step and flow,
A hoary captive bound in icy chains;
With haggard eyes, and mantle dipp’d in snow,
Who still of cold in Spring’s own realms complains.

XXXVII.
Not one of these, but from their various store
Some off’ring meet to lovely Flora pay;
Not one of these, but with that off’ring more,
And her soft reign most willingly obey.

XXXVIII.
Ev’n Winter’s self, with look averted, throws,
His thin-strewn flow’rets on the goddess’ shrine;
Ev’n his cold bosom for a moment glows,
When he beholds her radiant form divine.

XXXIX.
But now the Genii of each plant and flow’r,
Rang’d in fair order, wait her high commands;
And each, approaching her delicious bow’r,
In expectation of her verdict stands.

XL.
For many of the garden’s painted race,
And some that with their colours deck the field,
Rivals in wealth, in beauty, and in grace,
Had wag’d high wars, unknowing how to yield.

XLI.
All claim’d preferment, and each one could boast
Of some bright beauty or perfection dear,
Which should induce mankind to prize her most,
And to preferment make her title clear.

XLII.
And some, of empty shew and titles vain;
Alas! that Pride so many should deceive!
Claim’d o’er their kindred plants and flow’rs to reign:
And of their birthright others would bereave.

XLIII.
The Crown Imperial, and the spurious Flow’r
Which boasts of royal arms and royal mien[3];
The warlike Plant that claims immortal pow’r[4],
And that gay lady call’d the Meadow’s Queen.

XLIV.
All these, and more, that scorn’d a subject state,
Rose to the claim of high imperial sway:
Forgetting—to be good was to be great—
They rose to rule, unpractis’d to obey.

XLV.
Others again for beauty’s meed contend,
Chief amidst whom appear’d the Tulip race;
A painted tribe, born only to contend
For praise, where all is giv’n external grace.

XLVI.
Alcæa proud[5]; and lovely Venus’ joy,
That does from adverse winds its title claim[6];
The once conceited, self-admiring Boy[7],
Whose love prepost’rous gave a flow’r a name.

XLVII.
The proud Carnation dipp’d in brightest dyes,
Who still with thirst of praise and glory burns;
With her whose mirrour cheats deluded eyes[8],
And she that still to her lov’d Phœbus turns[9].

XLVIII.
There, with their num’rous chiefs of diff’rent hues,
The painted Cock’s Comb, and his lofty train,
Their beauties vaunting, to the rest refuse
To share the glories of their gaudy reign.—

XLIX.
The judges sat, each sep’rate claim was heard,
While some for rule, and some for praises, sought;
And some had been disgrac’d, and some preferr’d,
As in the goddess’ mind their various pleadings wrought

L.
But her lov’d consort, gently whisp’ring, said:
“What means my Queen, on these to cast her sight,
Who have but pride or lust of sway display’d,
Nor brought their real worth or virtues to the light?”

LI.
How many absent now, more fair than these,
With greater fragrance in lone valleys blow?
Or, if the garden’s flow’ry tribe more please,
Where do the Rose and lovely Vi’let glow?

LII.
The Lily where, and all that num’rous host,
Who claim true praise to innate virtue due;
Or do they merit least who loudest boast,
And with false glare impose upon the view?

LIII.
For sure, of all who feel my genial gale,
Or to the sun their fragrant breasts unfold,
The best and sweetest that on earth prevail,
Yet do I not in this fair court behold.

LIV.
He said; and Flora, rising from her throne,
Bade present search for ev’ry one be made:
Who, though their off’rings on her altar shone,
Their modest haste had from the court convey’d.

LV.
Strait they return’d:—The lovely blushing Rose,
The Lily ever chaste and ever fair,
The Vi’let sweet with purple tints that glows,
And Myrtle green, that scents the ambient air:

LVI.
With many more, grateful to sight and smell,
By bounteous heav’n with matchless charms endu’d;
That in the fragrant meads or gardens dwell,
 Or which wild wastes from human eyes seclude.

LVII.
These by their Genii now in modest guise,
Excus’d from pleading ’midst the mingled throng.
Claim’d but the tribute all allow’d their prize,
Nor sought their own just praises to prolong.

LVIII.
Yet, these once seen, abash’d their rivals stand;
And would have fled, but Flora this deny’d;
Who, rising graceful, with her out-stretch’d hand,
Thus briefly to th’ assembled pow’rs apply’d:

LIX.
“Genii of gardens, meads, and sylvan scenes,
Attendant still in Flora’s vernal train,
Say what this ardent, fond contention means,
Why strive you thus for pow’r, and strive in vain?

LX.
Are you not all beneath our sceptre blest;
Say, do not all confess our gentle sway?
Then seek not one to triumph o’er the rest,
But each in peaceful order still obey.

LXI.
So all the glories of my reign shall share,
So all be still in poets songs renown’d,
So shall my Zephyr still with gentlest air,
Wave o’er your beds, with bloom eternal crown’d.

LXII.
And you, who not for pow’r, but beauty’s charms,
For gaudy tints, still fiercely would contend;
What envious fire such gentle bosoms warms?
And where, alas! must the mad contest end?
 LXIII.
Each has her charms, and each peculiar worth,
To all in various portions duly giv’n,
By secret Nature working at its birth,
The lavish bounty of indulgent Heav’n.

LXIV.
Each has her charms:—but view the blushing Rose,
Behold the beauties of the Lily fair;
Few boast of equal excellence to those,
Yet with their modest merit none compare.

LXV.
These, therefore, we prefer; and though no Queen
Besides Ourselves we will to hold the reign;
Yet, for their true desart conspicuous seen,
We rank them foremost on the flow’ry plain.

LXVI.
Hear, and obey; and if aught else abide,
To raise dispute among your orders bright;
Still by true merit let the cause be try’d,
And specious shew yield to more solid right.”

LXVII.
She spoke;—the Seasons, and the winged Hours,
Confirm’d her voice; then breath’d a rich perfume,
Which Zephyr scatter’d wide o’er all the flow’rs,
And deck’d their leaves with more than mortal bloom.

LXVIII.
Then, his lov’d consort straining in his arms,
With gentlest touch salutes her swelling breast;
Who strait shone forth in more refulgent charms,
As Juno when by vernal Jove caress’d.

LXIX.
And sudden joining in a mazy dance,
The airy phantoms of the scene appear’d;
Some to the sprightly timbrel did advance,
While some their clear harmonious voices rear’d.

LXX.
But One among the rest, who view’d me stand
Intent, and gazing on the prospect near,
Came forth, and gently touch’d my trembling hand,
And bade me mark his words, and nothing fear:

LXXI.
“And seest thou not (said he) these vary’d flow’rs,
Contending still for beauty or for sway?
Such are the contests which employ man’s hours,
In life’s short, busy, transitory day.

LXXII.
For what is gaudy beauty’s short-liv’d bloom,
The pomp of pow’r, of riches, or of pride;
Soon bury’d in the undistinguish’d tomb,
Which all their boasted pomp at once must hide?

LXXIII.
Virtue alone survives, immortal maid!
Her truly amaranthine flow’r shall blow,
When all the rest are wrapt in dusky shade,
And laid in dark and dusty ruins low.

LXXIV.
Hear, and attend!—improve the moral strain,
So may’st thou sail safe through life’s dang’rous sea;
So from these scenes thou wisdom may’st attain,
And Flora prove Minerva’s self to thee.”

LXXV.
He ceas’d; and well I mark’d the prudent lore,
 And much revolv’d his saying in my mind;
Bent all the mystic moral to explore,
By this romantic, splendid scene design’d.

LXXVI.
But, the full concert swelling on my ear,
The bands of Sleep dissolv’d, away he flies;
At once the train of phantoms disappear,
And on my waking sight the vision dies.

LXXVII.
No longer now near Flora’s bow’r I stood,
But view’d with op’ning eyes the rising day;
Then down the Valley fair my path pursu’d,
And homeward took my solitary way.
FABLES OF FLOWERS.
 Fab. I.

The Holly Hock & Lily of the Vale

Fab. II.
The Aloe in Blossom
 FABLE I.
The HOLLYHOCK and the
LILY of the VALE.

I.

’TWAS early morn, Sol’s radiant beams

Illum’d the landscape round.
The dew-drops glitter’d on the day,
And gem-like deck’d the ground.

II.
Within the garden’s cultur’d walks
A Hollyhock there grew;
And there the Lily of the Vale
Kept humble distance due.

III.
Elate with pride, the gaudy flow’r
Expands its swelling breast;
And, joying in the vernal scene,
The Lily thus address’d:

IV.
“What dost thou here, mean paltry thing,
“Go blow in yonder field;
“Nor thus disgrace fair Flora’s tribes,
“That heav’nly beauties yield.

V.
“Go, with thy faint and sickly hue;
 “Some chearless vale adorn;
“But here intrude not on our reign,
“Nor drink the dew of morn.

VI.
“Whilst I with heighten’d colours glow,
“In Summer’s liv’ry gay;
“Imbibe the softest tints of light,
“And glitter on the day.

VII.
“Me yonder golden sun shall warm,
“At morn and noon-tide hour;
“And me his ev’ning beams attend,
“Like his own fav’rite flow’r.

VIII.
“Nor yonder Rose, nor Bacchus’ Plant,
“Which twining near me grows,
“Can boast more excellence than me,
“Or brighter dyes disclose.

IX.
“Hence thou! nor this fair spot profane,
“Where fairer flow’rets blow;
“Return again to shades obscure,
“And there neglected grow.”

X.
The Lily heard, with decent grace,
That scorn’d the boaster’s pride;
Then from her lone, unenvy’d bed
She thus in brief reply’d;

XI.
“From vaunting loud what fame is gain’d,
“To raise the boaster’s name;